A traffic sign detection method based on receptive field optimization and mixed convolution feature fusion

Abstract

The application discloses a traffic sign detection method based on receptive field optimization and mixed convolution feature fusion, and the method comprises the following steps: obtaining a traffic sign picture and constructing an original data set; dividing the original data set into a training set, a test set and a verification set; constructing a network model, wherein the network model comprises three parts of a feature extraction network, a feature fusion network and a prediction output network; setting an initial learning rate, a learning rate decay mode and a training frequency of the network model according to the training set, optimizing network parameters by using an SGD optimizer, and training the network model; inputting a picture to be detected into the trained network model to detect the traffic sign, and outputting specific position and category information of the traffic sign in the picture to be detected. The embodiment of the application is based on the optimization of receptive field and the fusion of mixed convolution features, efficiently improves the precision and speed of the model, improves the accuracy of multi-scale target detection, and can be widely applied to the technical field of intelligent traffic.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to a traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion. Background Art

[0002] Traffic sign recognition, as a crucial component of the transportation system, provides drivers and pedestrians with essential information about road conditions, reducing the risk of accidents. Traffic sign recognition technology not only helps drivers and autonomous vehicles accurately capture road information and provide precise detection results, but is also crucial for avoiding traffic accidents and ensuring driver safety. This technology uses computer vision and image processing to automatically detect and recognize traffic signs on the road, providing drivers with real-time, accurate traffic sign information and timely alerting them to road conditions, thereby reducing traffic accidents and improving driving safety and convenience. Traffic sign recognition technology plays a significant role in improving road safety, promoting the development of autonomous driving technology, enhancing the intelligence of driver assistance systems, and optimizing traffic management and planning. Furthermore, as a key component in autonomous driving systems, this technology helps autonomous vehicles understand road rules and make accurate driving decisions.

[0003] Traditional traffic sign recognition methods primarily rely on handcrafted features and classic machine learning algorithms. The process typically includes image preprocessing and region of interest (ROI) extraction, feature extraction (such as color, shape, and texture features), feature selection and dimensionality reduction, and finally, inputting the extracted features into a classifier for classification. While these methods can achieve good results in some simple scenarios, they are sensitive to environmental changes (such as illumination and occlusion) and perform poorly in complex environments and multi-class tasks, making them difficult to meet the needs of modern intelligent transportation systems. In recent years, the development of deep learning technology, particularly advanced general object detection techniques such as Faster R-CNN and YOLO, has gradually replaced traditional methods. However, to address the specific challenges of traffic sign detection, existing deep learning-based methods still need to improve both the accuracy and speed of traffic sign detection.

[0004] Shortcomings or deficiencies: While general object detection technology has been widely applied in various fields, it still faces numerous challenges in detecting small traffic signs captured by vehicle-mounted cameras, which have low resolution and limited information. A traffic sign may occupy only 30×30 pixels in a 2048×2048 pixel high-definition image. Therefore, efficient detection of small objects such as traffic signs remains a major challenge in the field of object detection.

[0005] To improve the performance of object detectors in detecting small objects, numerous studies in recent years have attempted to construct high-resolution feature maps to predict them and obtain richer detailed information. However, such methods often overlook the importance of aligning the receptive field size with the spatial region of small objects. Furthermore, top-down architectures with skip connections have been introduced to enhance small object detection capabilities. This strategy effectively improves detection accuracy by fusing low-level details at different scales with high-level semantic features. However, due to the complexity of the network structure, these methods increase the computational overhead during the training and testing phases, making real-time detection difficult. Therefore, finding a method to address the difficulties of small object detection, high computational cost, and poor real-time performance has become an urgent need. Summary of the Invention

[0006] The main purpose of the embodiments of the present invention is to propose a highly accurate and efficient traffic sign detection method based on receptive field optimization and hybrid convolutional feature fusion.

[0007] To achieve the above objectives, an embodiment of the present invention provides a traffic sign detection method based on receptive field optimization and hybrid convolutional feature fusion, comprising the following steps:

[0008] Obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset;

[0009] Divide the original dataset into a training set, a test set, and a validation set, and process them into a uniform size;

[0010] Constructing a network model, which includes three parts: a feature extraction network, a feature fusion network, and a prediction output network;

[0011] According to the training set, the initial learning rate, learning rate decay method, and number of training times of the network model are set, and the network parameters are optimized using the SGD optimizer to train the network model;

[0012] The image to be detected is input into the trained network model to detect traffic signs, and the specific location and category information of the traffic signs in the image to be detected are output.

[0013] In some embodiments, constructing the network model includes the following steps:

[0014] In the feature extraction network, the features of very small signboards are extracted through the P1 layer, and the features of signboards with gradually larger sizes are extracted through the P2 layer, the P3 layer, the P4 layer, and the P5 layer respectively.

[0015] In the feature fusion network, the deep feature information of the network is fused to obtain the supplementary layer information, thereby obtaining a high-resolution feature map with rich context information;

[0016] In the prediction output network, by using the YOLO series anchor-free detection method, multi-scale target prediction is performed on a high-resolution feature map with rich contextual information to obtain the specific location and category information of the traffic sign in the input image.

[0017] In some embodiments, in the feature extraction network, the network depth of each scale in the feature extraction network is adjusted according to the size distribution of traffic signs in the training set.

[0018] In some embodiments, the expression of each convolutional layer in the feature extraction network is:

[0019] P1 RF =λ×anchor tiny

[0020] P2 RF =λ×(anchor tiny +anchor mean ) / 2

[0021] P3 RF =λ×anchor mean

[0022] P4 RF =λ×(anchor mean +anchor large ) / 2

[0023] P5 RF =λ×anchor large

[0024] Among them, P1 RF 、P2 RF 、P3 RF 、P4 RF and P5 RF Represents the theoretical receptive field of each layer; λ is an adjustable hyperparameter, anchor tiny The average size and anchor of the smallest 5% signs in the dataset under 640×640 input mean Represents the average size of all signs in the dataset under 640×640 input; anchor large Represents the average size of the top 2% largest logos for 640×640 input.

[0025] In some embodiments, the calculation formula for the receptive field of each convolutional layer is:

[0026]

[0027] Among them, K n and S n Respectively represent the size and stride of the n-th layer convolution kernel; RF n-1 is the receptive field size of the previous layer.

[0028] In some embodiments, in the feature fusion network, the continuity of the feature map is maintained by combining the AGRFM module of the standard convolution and the dilated convolution to extract features of small objects.

[0029] Another aspect of the present invention provides a traffic sign detection system based on receptive field optimization and hybrid convolutional feature fusion, including:

[0030] The first module is used to obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset;

[0031] The second module is used to divide the original data set into a training set, a test set and a validation set, and process them into a uniform size;

[0032] The third module is used to build a network model, which includes three parts: feature extraction network, feature fusion network and prediction output network;

[0033] The fourth module is used to set the initial learning rate, learning rate decay method, and number of training times of the network model according to the training set, optimize the network parameters using the SGD optimizer, and train the network model;

[0034] The fifth module is used to input the image to be detected into the trained network model to detect traffic signs, and output the specific location and category information of the traffic signs in the image to be detected.

[0035] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0036] The memory is used to store programs;

[0037] The processor executes the program to implement the method described above.

[0038] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0039] Another aspect of an embodiment of the present invention further provides a computer program product, including a computer program, which implements the above-mentioned method when executed by a processor.

[0040] The embodiment of the present application at least has the following beneficial effects: the present application provides a traffic sign detection method based on receptive field optimization and mixed convolution feature fusion, the scheme obtains traffic sign pictures, and labels traffic signs in each picture to construct an original data set; the original data set is divided into a training set, a test set and a verification set, and is processed into a uniform size; a network model is constructed, the network model includes three parts of a feature extraction network, a feature fusion network and a prediction output network; according to the training set, the initial learning rate, the learning rate decay mode and the training times of the network model are set, the network parameters are optimized by using the SGD optimizer, and the network model is trained; a picture to be detected is input into the trained network model to detect traffic signs, and the specific position and category information of the traffic signs in the picture to be detected are output. The embodiment of the present application is based on optimized receptive field and mixed convolution feature fusion, efficiently improves the precision and speed of the model, and improves the accuracy of multi-scale target detection. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is an implementation environment schematic diagram provided by the embodiment of the present application;

[0042] Figure 2 is a flowchart of the overall steps provided by the embodiment of the present application;

[0043] Figure 3 is an overall network structure schematic diagram provided by the embodiment of the present application;

[0044] Figure 4 is an AGRFM module structure schematic diagram provided by the embodiment of the present application;

[0045] Figure 5 is a detection result schematic diagram provided by the embodiment of the present application;

[0046] Figure 6 is a hardware structure schematic diagram of an electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, but are only examples of devices and methods consistent with some aspects of the embodiments of the present application as described in detail in the appended claims.

[0048] It will be understood that the terms "first," "second," and the like used in the present invention may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are merely used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of," "when," or "in response to a determination."

[0049] The terms "at least one", "plurality", "each", "any", etc. used in the present invention include at least one, two or more, multiple, two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing embodiments of the present invention only and are not intended to limit the present invention.

[0051] The traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion provided by the embodiment of the present invention relates to the field of intelligent transportation technology. The traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion provided by the embodiment of the present invention can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion, etc., but is not limited to the above forms.

[0052] The present invention can be used in a wide variety of general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present invention can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0053] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 , the implementation environment includes at least one terminal 102 and a server 101. The terminal 102 and the server 101 can be connected to the network in a wireless or wired manner to complete data transmission and exchange.

[0054] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0055] In addition, server 101 can also be a node server in a blockchain network. Blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithm.

[0056] Terminal 102 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, etc. Terminal 102 may also be a vehicle-mounted terminal of the various device types described above, but is not limited thereto. Terminal 102 and server 101 may be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment of the present invention.

[0057] Based on the example Figure 1In the implementation environment shown, an embodiment of the present invention provides a traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion. The following is an example of the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion being applied to the server 101. It can be understood that the method can also be applied to the terminal 102.

[0058] Reference Figure 2 , Figure 2 The flow chart of the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion applied to the server provided in the embodiment of the present invention, the execution subject of the method can be any of the aforementioned computer devices (including servers or terminals). Figure 2 , the method may include the following steps:

[0059] Obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset;

[0060] Divide the original dataset into a training set, a test set, and a validation set, and process them into a uniform size;

[0061] Constructing a network model, which includes three parts: a feature extraction network, a feature fusion network, and a prediction output network;

[0062] According to the training set, the initial learning rate, learning rate decay method, and number of training times of the network model are set, and the network parameters are optimized using the SGD optimizer to train the network model;

[0063] The image to be detected is input into the trained network model to detect traffic signs, and the specific location and category information of the traffic signs in the image to be detected are output.

[0064] In some embodiments, constructing the network model includes the following steps:

[0065] In the feature extraction network, the features of very small signboards are extracted through the P1 layer, and the features of signboards with gradually larger sizes are extracted through the P2 layer, the P3 layer, the P4 layer, and the P5 layer respectively.

[0066] In the feature fusion network, the deep feature information of the network is fused to obtain the supplementary layer information, thereby obtaining a high-resolution feature map with rich context information;

[0067] In the prediction output network, by using the YOLO series anchor-free detection method, multi-scale target prediction is performed on a high-resolution feature map with rich contextual information to obtain the specific location and category information of the traffic sign in the input image.

[0068] In some embodiments, in the feature extraction network, the network depth of each scale in the feature extraction network is adjusted according to the size distribution of traffic signs in the training set.

[0069] In some embodiments, the expression of each layer of convolutional layer in the feature extraction network is:

[0070] P1 RF = λ × anchor tiny

[0071] P2 RF = λ × (anchor tiny + anchor mean ) / 2

[0072] P3 RF = λ × anchor mean

[0073] P4 RF = λ × (anchor mean + anchor large ) / 2

[0074] P5 RF = λ × anchor large

[0075] wherein P1 RF , P2 RF , P3 RF , P4 RF and P5 RF represent the theoretical receptive field of each layer respectively; λ is an adjustable hyperparameter, anchor tiny represents the average size of the last 5% of the smallest signs in the dataset under 640 × 640 input, anchor mean represents the average size of all signs in the dataset under 640 × 640 input; and anchor large represents the average size of the first 2% of the largest signs under 640 × 640 input.

[0076] In some embodiments, the calculation formula of the receptive field of each layer of convolutional layer is:

[0077]

[0078] wherein K n and S n represent the size and stride of the nth convolutional kernel respectively; and RF n-1 is the receptive field size of the previous layer.

[0079] In some embodiments, in the feature fusion network, the continuity of the feature map is maintained by combining the AGRFM module of the standard convolution and the dilated convolution to extract features of small objects.

[0080] The following takes a specific scenario as an example to explain the specific implementation process in detail:

[0081] The present invention proposes a real-time detection method for traffic signs based on optimized receptive field and hybrid convolution feature fusion. Specifically, the depth of each scale network in the feature extraction network is adjusted according to the size distribution of traffic signs in the data set to ensure that the receptive field of the multi-scale feature map matches the target size distribution. The information of the deep P3, P4 and P5 layers of the network is then fused to obtain the supplementary layer information. And by combining the AGRFM module of standard convolution and dilated convolution, the continuity of the feature map is maintained to further extract features of small targets. Subsequently, by injecting the supplementary layer information into the high-resolution feature map of the network, a high-resolution feature map with rich contextual information is obtained. Finally, the YOLO series Anchor-free method is used to detect multi-scale targets on it, thereby achieving a significant improvement in the accuracy and speed of the model when detecting traffic signs. The whole process includes the following steps:

[0082] S1: Data acquisition: obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset.

[0083] S2: Dataset preprocessing: divide the original dataset into training set, test set, and validation set, and process them into uniform size.

[0084] S3: Build a network model. The network model includes three parts: feature extraction, feature fusion and prediction output.

[0085] Specifically, in the feature extraction part, the embodiment of the present invention adjusts the network depth of each scale in the feature extraction network according to the size distribution of traffic signs in the dataset to ensure that the receptive field of the multi-scale feature map matches the size distribution of traffic signs in the dataset. The entire network structure is as follows Figure 3 shown.

[0086] Specifically, in the feature fusion part, the embodiment of the present invention fuses the information of the deep layers P3, P4 and P5 of the network to obtain the supplementary layer information. And by combining the AGRFM module of standard convolution and dilated convolution, the continuity of the feature map is maintained to further extract the features of small targets. Then, by injecting the supplementary layer information into the high-resolution feature map of the network, a high-resolution feature map with rich context information is obtained. The AGRFM module structure is as follows Figure 4 shown.

[0087] In the feature extraction network, layer P1 is responsible for extracting features for very small signs, while subsequent layers (P2, P3, P4, and P5) are responsible for extracting features for progressively larger signs. Given that the actual receptive field of convolution is often smaller than the theoretical receptive field, this embodiment of the present invention uses the following strategy to align the receptive field of the multi-scale feature map with the size distribution of objects in the dataset.

[0088] P1 RF =λ×anchor tiny

[0089] P2 RF =λ×(anchor tiny +anchor mean ) / 2

[0090] P3 RF =λ×anchor mean

[0091] P4 RF =λ×(anchor mean +anchor large ) / 2

[0092] P5 RF =λ×anchor large

[0093] Among them, P1 RF 、P2 RF 、P3 RF 、P4 RF and P5 RF Represents the theoretical receptive field of each layer. λ is an adjustable hyperparameter, anchor tiny 、anchor large and anchor mean Corresponding to the average size of the smallest 5% of signs in the dataset under 640×640 input, the average size of the largest 2% of signs under 640×640 input, and the average size of all signs in the dataset under 640×640 input.

[0094] The embodiment of the present invention knows that the receptive field RF of the ordinary convolution layer n The calculation formula is:

[0095]

[0096] Among them, K n and S n Respectively represent the size and stride of the convolution kernel of the nth layer. n-1 is the receptive field size of the previous layer.

[0097] By adjusting the depth of the network at each scale to the calculated theoretical receptive field size of the corresponding layer, the receptive field of the multi-scale feature map can be ensured to match the target size distribution.

[0098] In the prediction output part, the embodiment of the present invention uses the YOLO series anchor-free detection method to predict multi-scale targets on a high-resolution feature map with rich contextual information, thereby obtaining the specific location and category information of traffic signs in the input image.

[0099] S4: Train the network model. Use the training set processed in S2 and the corresponding labels as network input. Set the network model's initial learning rate, learning rate decay method, and number of training times. Use the SGD optimizer to optimize the network parameters. Save the training weight file after training is complete.

[0100] S5: Traffic sign detection results: input the input image into the trained network model to detect traffic signs, and output the specific location and category information of the traffic signs in the current input image.

[0101] By the method of the present invention, the Figure 5 The figure shows a schematic diagram of the detection results of the real-time traffic sign detection method based on the fusion of optimized receptive field and hybrid convolutional features.

[0102] In summary, the method according to the embodiment of the present invention has the following advantages:

[0103] 1. The present invention proposes a real-time traffic sign detection method based on optimized receptive field and hybrid convolution feature fusion, which simplifies the model and enhances the feature extraction capability. Compared with existing methods, it can achieve higher accuracy and speed.

[0104] 2. Based on the receptive field mismatch problem existing in current small target object detection methods, the present invention proposes a real-time traffic sign detection method based on optimized receptive field and hybrid convolutional feature fusion. The feature extraction part adjusts the network depth of each scale in the feature extraction network according to the size distribution of traffic signs in the dataset to ensure that the receptive field of the multi-scale feature map matches the size distribution of traffic signs in the dataset, thereby improving the accuracy of multi-scale target detection.

[0105] 3. Based on the problem of complex network structure in current small target object detection methods, the present invention proposes a real-time traffic sign detection method based on optimized receptive field and hybrid convolution feature fusion. The feature fusion part fuses the information of the deep P3, P4 and P5 layers of the network to obtain supplementary layer information. And by combining the AGRFM module of standard convolution and dilated convolution, the continuity of the feature map is maintained to further extract features of small targets. Subsequently, by injecting the supplementary layer information into the high-resolution feature map of the network, a high-resolution feature map with rich contextual information is obtained. The anchor-free method is used to directly perform predictions on this feature map, which simplifies the model structure, significantly improves the accuracy and speed of the model, and makes traffic sign detection in real-time applications more robust and efficient.

[0106] 4. Based on the grid effect problem existing in the current dilated convolution feature extraction, the present invention proposes a real-time traffic sign detection method based on the fusion of optimized receptive field and hybrid convolution features, and designs an AGRFM module in the feature fusion part. This module integrates a series of ordinary convolutions and a single dilated convolution with a high dilation rate. By maintaining the continuity of feature mapping, it enhances the extraction of small target features and reduces the problem of incomplete feature extraction caused by the grid effect when using dilated convolution feature extraction alone. This enables our model to maintain the integrity of the feature information extraction of the supplementary layer while effectively increasing the receptive field using dilated convolution.

[0107] Based on the problems of receptive field mismatch and complex network structure in the current traffic sign detection method, the present invention proposes a real-time detection method for traffic signs based on optimized receptive field and hybrid convolution feature fusion. By adjusting the network depth of each scale in the feature extraction network according to the size distribution of traffic signs in the data set, the receptive field of the multi-scale feature map is ensured to match the target size distribution. The information of the deep P3, P4 and P5 layers of the network is then fused to obtain the supplementary layer information. And by combining the AGRFM module of standard convolution and dilated convolution, the continuity of the feature map is maintained to further extract features of small targets. Subsequently, by injecting the supplementary layer information into the high-resolution feature map of the network, a high-resolution feature map with rich contextual information is obtained. Finally, the YOLO series Anchor-free method is used to detect multi-scale targets thereon, thereby achieving a significant improvement in the accuracy and speed of the model when detecting traffic signs.

[0108] Another aspect of the present invention provides a traffic sign detection system based on receptive field optimization and hybrid convolutional feature fusion, including:

[0109] The first module is used to obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset;

[0110] The second module is used to divide the original data set into a training set, a test set and a validation set, and process them into a uniform size;

[0111] The third module is used to build a network model, which includes three parts: feature extraction network, feature fusion network and prediction output network;

[0112] The fourth module is used to set the initial learning rate, learning rate decay method, and number of training times of the network model according to the training set, optimize the network parameters using the SGD optimizer, and train the network model;

[0113] The fifth module is used to input the image to be detected into the trained network model to detect traffic signs, and output the specific location and category information of the traffic signs in the image to be detected.

[0114] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0115] An embodiment of the present invention further provides an electronic device comprising a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program, implements the aforementioned traffic sign detection method based on receptive field optimization and hybrid convolutional feature fusion. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0116] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0117] See also Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0118] The processor 601 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided by the embodiments of the present invention.

[0119] The memory 602 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called by the processor 601 to execute the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion according to the embodiment of the present invention.

[0120] Input / output interface 603, used to implement information input and output;

[0121] Communication interface 604, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0122] Bus 605 , which transmits information between various components of the device (e.g., processor 601 , memory 602 , input / output interface 603 , and communication interface 604 );

[0123] The processor 601 , the memory 602 , the input / output interface 603 and the communication interface 604 are connected to each other in communication within the device via a bus 605 .

[0124] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion is implemented.

[0125] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0126] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The embodiments described in the embodiments of the present invention are intended to more clearly illustrate the technical solutions of the embodiments of the present invention and do not constitute a limitation on the technical solutions provided by the embodiments of the present invention. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0128] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0129] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0130] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0131] The terms "first," "second," "third," "fourth," and the like (if any) in the description of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products, or apparatus.

[0132] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0133] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0134] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0136] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in other words, the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0137] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion, characterized in that: The following steps are involved: Obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset; Divide the original dataset into a training set, a test set, and a validation set, and process them into a uniform size; Constructing a network model, which includes three parts: a feature extraction network, a feature fusion network, and a prediction output network; According to the training set, the initial learning rate, learning rate decay method, and number of training times of the network model are set, and the network parameters are optimized using the SGD optimizer to train the network model; Input the image to be detected into the trained network model to detect traffic signs, and output the specific location and category information of the traffic signs in the image to be detected; The construction of the network model includes the following steps: In the feature extraction network, the features of very small signboards are extracted through the P1 layer, and the features of signboards with gradually larger sizes are extracted through the P2 layer, the P3 layer, the P4 layer, and the P5 layer respectively. In the feature fusion network, the deep feature information of the network is fused to obtain the supplementary layer information, thereby obtaining a high-resolution feature map with rich context information; In the prediction output network, the YOLO series of anchor-free detection methods are used to predict multi-scale targets on a high-resolution feature map with rich contextual information to obtain the specific location and category information of traffic signs in the input image; In the feature fusion network, the AGRFM module combining standard convolution and dilated convolution maintains the continuity of feature mapping to extract features of small targets.

2. The traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion according to claim 1 is characterized in that: In the feature extraction network, the network depth of each scale in the feature extraction network is adjusted according to the size distribution of traffic signs in the training set.

3. The traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion according to claim 1 is characterized in that: The expressions of each convolutional layer in the feature extraction network are: in, 、 、 、 and Represent the theoretical receptive field of each layer respectively; is a tunable hyperparameter, represents the average size of the smallest 5% of signs in the dataset under 640×640 input, Represents the average size of all signboards in the dataset under 640×640 input; .

4. The traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion according to claim 3 is characterized in that: The calculation formula of the receptive field of each convolutional layer is: in, Indicates the size of the convolution kernel in the nth layer; Represents the stride of the convolution kernel in layer i; is the receptive field size of the previous layer.

5. A system for implementing the traffic sign detection method based on receptive field optimization and hybrid convolution feature fusion as described in any one of claims 1 to 4, characterized in that: include: The first module is used to obtain traffic sign images, annotate the traffic signs in each image, and construct the original dataset; The second module is used to divide the original data set into a training set, a test set and a validation set, and process them into a uniform size; The third module is used to build a network model, which includes three parts: feature extraction network, feature fusion network and prediction output network; The fourth module is used to set the initial learning rate, learning rate decay method, and number of training times of the network model according to the training set, optimize the network parameters using the SGD optimizer, and train the network model; The fifth module is used to input the image to be detected into the trained network model to detect traffic signs, and output the specific location and category information of the traffic signs in the image to be detected.

6. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images