Small traffic target detection method, device, equipment and medium for high-speed vehicles

Abstract

The present application relates to a method, device, equipment and medium for detecting small traffic targets of high-speed vehicles, which relates to the field of image processing. The method includes: inputting a traffic scene image frame into a second small traffic target detection model that has been trained to a convergence state, and in a plurality of Star Block modules of a multi-layer stacked recursion of a StarNet lightweight network, using a star operation to perform element-by-element multiplication of image features corresponding to the traffic scene image frame to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features of the traffic scene image frame; inputting the high-dimensional image features into a detection head network in a small traffic target detection model to obtain a plurality of small traffic targets in the traffic scene image frame to complete the small traffic target detection of high-speed vehicles. The present application can accurately and quickly detect small traffic targets in a high-speed traffic scene, and can achieve higher target detection performance in a resource-constrained environment such as a high-speed vehicle.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for detecting small traffic targets of high-speed vehicles, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In today's society, with the rapid development of autonomous driving technology, traffic target detection methods have received more and more attention. Traffic target detection plays a vital role in many fields such as intelligent transportation systems, autonomous driving technology, traffic monitoring and management, and safe driving assistance. Traffic target detection can monitor vehicles and pedestrians on the road in real time, helping drivers or autonomous driving systems to identify potential dangers in advance, so as to take corresponding avoidance measures and reduce the occurrence of traffic accidents.

[0003] The research methods of traffic target detection can be mainly divided into two categories: traditional methods and deep learning-based methods. Traditional traffic target detection methods mainly rely on image processing technology and machine learning algorithms. Traditional methods usually include preprocessing the input image such as image enhancement, extracting useful features from the preprocessed image, training the extracted features through machine learning algorithms, and finally using the extracted features to classify the areas in the image through a classifier. Traditional traffic target detection methods have the disadvantages of large computational complexity, complex feature design, and poor generalization ability, making it difficult to meet the requirements of real-time and accuracy.

[0004] With the rapid development of deep learning technology, traffic target detection methods based on deep learning have gradually become mainstream. These methods use deep learning models such as convolutional neural networks (CNNs) to directly learn features from original images and perform classification and positioning. Although the traffic target detection models in the prior art have been somewhat lightweight, they have not yet met the needs of small traffic target detection in high-speed driving scenarios, such as the needs of unmanned vehicles or self-driving cars in high-speed driving scenarios. In addition, the YOLO model requires preset anchor frames and NMS operations, which requires more processing time, making it difficult to perform real-time processing on resource-constrained edge devices. For the detection of small targets, the traffic target detection models in the prior art often expose high false detection rates and missed detection rates, which not only affects the accuracy of detection, but may also mislead unmanned vehicles or self-driving cars to make wrong judgments, leading to safety accidents, and even endangering the lives and property of drivers and pedestrians.

[0005] To sum up, the traditional traffic target detection methods in the existing technology have the disadvantages of large computational complexity, complex feature design, poor generalization ability, etc., which are difficult to meet the requirements of real-time and accuracy, and expose problems such as high false detection rate and missed detection rate. The applicant has made corresponding explorations in order to solve this problem. Summary of the invention

[0006] The purpose of the present application is to solve the above-mentioned problems and to provide a method for detecting small traffic targets of high-speed vehicles, a corresponding device, an electronic device and a computer-readable storage medium.

[0007] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0008] A method for detecting small traffic targets of high-speed vehicles proposed to meet one of the purposes of this application includes:

[0009] Acquire a traffic scene image frame containing a small traffic target from a high-speed vehicle;

[0010] The backbone network in the preset first small traffic target detection model is updated to a StarNet lightweight network, the strided convolution layer in the StarNet lightweight network is updated to an SPD-Conv module, and the CCFM structure of the feature fusion network in the first small traffic target detection model is updated to a Fast PANet structure to construct a second small traffic target detection model;

[0011] Inputting the traffic scene image frame into a second traffic small target detection model that has been trained to a converged state, and performing element-by-element multiplication of image features corresponding to the traffic scene image frame using a star operation in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame;

[0012] The high-dimensional image features are input into the detection head network in the small traffic target detection model to obtain multiple small traffic targets in the traffic scene image frame to complete the small traffic target detection of high-speed vehicles.

[0013] Optionally, the StarNet lightweight network includes a CBR module and multiple Star Layer modules, wherein each Star Layer module includes an SPD-Conv module and multiple StarBlock modules, and the SPD-Conv module includes a space-to-depth layer and a point convolution layer;

[0014] The Fast PANet structure includes an upsampling layer, a downsampling layer, a feature concatenation layer and a feature fusion layer, wherein the SPD-Conv module is introduced in the downsampling layer.

[0015] Optionally, in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network, a step of performing element-by-element multiplication of image features corresponding to the traffic scene image frame by using a star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine a high-dimensional image feature corresponding to the traffic scene image frame includes:

[0016] In each Star Block module, the input image features are first processed by a 7×7 first depth separable convolution module, and the processed results are sent to two 1x1 convolution layers in parallel to expand the number of channels to 4 times the original number.

[0017] The first output result of the first 1x1 convolutional layer is activated by the ReLu6 activation function, and then combined with the second output result of the second 1x1 convolutional layer by element-by-element multiplication to determine the third output result;

[0018] Afterwards, the third output result adjusts the number of channels through a 1×1 convolution layer, reducing the number of channels to one quarter to restore the original number of channels, and a 7×7 second depthwise separable convolution module is added again at the end of the Star Block module to perform a second depthwise separable convolution process;

[0019] Finally, the feature map after the second depth-wise separable convolution is regularized through a drop_path layer to reduce overfitting, and is residually connected with the original input to finally output the high-dimensional image features corresponding to the traffic scene image frame.

[0020] Optionally, in a plurality of Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, before the step of performing element-by-element multiplication of the image features corresponding to the traffic scene image frame by using a star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame, the method further comprises:

[0021] In the Star Layer module of the StarNet lightweight network, the space-to-depth layer in the SPD-Conv module divides the image features corresponding to the traffic scene image frame into multiple sub-image features from a spatial dimension, and splices the multiple sub-image features from a channel dimension to determine a first intermediate feature map;

[0022] The point convolution layer in the SPD-Conv module performs channel dimensionality reduction on the first intermediate feature map to determine a first intermediate feature map after channel dimensionality reduction;

[0023] The Star Block module in the Star Layer module performs a first depth-separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth-separable convolution and residual connection processing on the first intermediate feature map after the channel dimensionality reduction to determine a high-level feature map.

[0024] Optionally, after the step of performing a first depth-separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth-separable convolution, and a residual connection process on the first intermediate feature map after the channel dimension reduction in sequence to determine a high-level feature map, the Star Block module in the Star Layer module comprises:

[0025] In the Fast PANet structure of the traffic small target detection model, the upsampling layer uses a nearest neighbor interpolation algorithm to expand the spatial size of the high-level feature map to align the low-level feature map;

[0026] The SPD-Conv module in the downsampling layer determines image features at different levels through space-to-depth conversion and point convolution operations, and in the feature splicing layer, the image features at different levels are spliced ​​in the channel dimension to determine the spliced ​​multi-level image features;

[0027] In the feature fusion layer, a reverse bottleneck design and multiple convolution operations are used to perform feature fusion on the spliced ​​multi-level image features to determine the fused multi-scale image features;

[0028] The fused multi-scale image features are input into the detection head network in the small traffic target detection model to detect multiple small traffic targets in the traffic scene image, so as to complete the small traffic target detection of high-speed vehicles.

[0029] Optionally, the step of using a reverse bottleneck design and multiple convolution operations in the feature fusion layer to perform feature fusion on the spliced ​​feature maps to determine fused multi-scale image features includes:

[0030] Inputting the spliced ​​multi-level image features into the channel dimension reduction module in the feature fusion layer, and performing channel dimension reduction processing on the multi-level image features, wherein the feature fusion layer is constructed by connecting a channel dimension reduction module and two residual reverse bottleneck modules in series, and the channel dimension reduction module is constructed by multiple convolutional layers that do not change the size of the feature space;

[0031] The multi-level image features after channel dimension reduction processing are input into the first residual reverse bottleneck module, and the channel number increase, depthwise separable convolution, channel number recovery and residual connection processing are performed in sequence to obtain the output of the first residual reverse bottleneck module;

[0032] The output of the first residual reverse bottleneck module is used as the input of the second residual reverse bottleneck module, and the above steps are repeated to determine the fused multi-scale image features.

[0033] Optionally, the basic network architecture of the first small traffic target detection model is an original RT-DETR target detection model, and the basic network architecture of the second small traffic target detection model is an improved RT-DETR target detection model;

[0034] The small traffic targets include one or more of pedestrians, small animals, traffic facilities or non-motor vehicles; the high-speed vehicle represents an unmanned vehicle or an automatic driving vehicle traveling at high speed.

[0035] A small traffic target detection device for a high-speed vehicle provided for another purpose of the present application comprises:

[0036] An image acquisition module is configured to acquire a traffic scene image frame containing a small traffic target from a high-speed vehicle;

[0037] A detection model construction module is configured to update the backbone network in the preset first small traffic target detection model to a StarNet lightweight network, update the strided convolution layer in the StarNet lightweight network to an SPD-Conv module, and update the CCFM structure of the feature fusion network in the first small traffic target detection model to a Fast PANet structure, so as to construct a second small traffic target detection model;

[0038] A high-dimensional feature extraction module is configured to input the traffic scene image frame into a second traffic small target detection model that has been trained to a converged state, and in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network, perform element-by-element multiplication of image features corresponding to the traffic scene image frame using a star operation to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame;

[0039] The small target detection module is configured to input the high-dimensional image features into the detection head network in the traffic small target detection model to obtain multiple traffic small targets in the traffic scene image frame to complete the traffic small target detection of high-speed vehicles.

[0040] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the method for detecting small traffic targets for high-speed vehicles described in the present application.

[0041] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the method for detecting small traffic targets for high-speed vehicles in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0042] Compared with the prior art, the present application aims at the disadvantages of the traditional traffic target detection method in the prior art, such as large amount of calculation, complex feature design, poor generalization ability, etc., which are difficult to meet the requirements of real-time and accuracy, and expose high false detection rate and missed detection rate. The present application includes but is not limited to the following beneficial effects:

[0043] First, the StarNet lightweight network in the traffic small target detection model of the present application realizes the mapping of high-dimensional and nonlinear feature spaces through star operations without increasing the computational complexity. The star operations in the StarNet lightweight network can implicitly obtain high-dimensional features in low-dimensional inputs, and the backbone network in the original RT-DETR target detection model is updated to the StarNet lightweight network. While maintaining computational efficiency, it can also obtain richer and more expressive feature representations, which is particularly important for fine feature capture in the task of detecting small traffic targets in high-speed vehicles.

[0044] Secondly, the StarNet lightweight network in the traffic small target detection model of the present application has an efficient network design. The StarNet lightweight network realizes efficient feature representation through star operation without complex network design and additional computing overhead. It can perform calculations in low-dimensional space, but implicitly considers extremely high-dimensional features. The StarNet lightweight network is used as the backbone network of the traffic small target detection model of the present application, providing efficient calculation and better feature representation, and can achieve higher target detection performance in resource-constrained environments such as high-speed vehicles.

[0045] Third, the StarNet lightweight network in the traffic small target detection model of this application has multi-level implicit feature expansion. Through multi-layer star operations, StarNet can recursively increase the implicit feature dimension, approaching infinite dimension. For networks with large width and depth, this feature can significantly enhance the expressiveness of features. For the traffic small target detection model of this application, the quality of feature extraction can be significantly improved through appropriate depth and width design, thereby improving the accuracy of traffic small target detection;

[0046] Furthermore, the advantage of the StarNet lightweight network in the traffic small target detection model of the present application lies in its lightweight and efficient characteristics, making it suitable for real-time small target detection in high-speed traffic scenes. The traffic small target detection model of the present application is particularly suitable for application in environments where small targets need to be quickly processed and accurately detected, such as various traffic small target detections in traffic scene images in autonomous driving systems; through the star operation of the StarNet lightweight network, the model can significantly improve the feature dimension through recursive stacking, thereby more accurately detecting small targets in traffic scenes; the generation of high-dimensional features and subsequent detection head network processing make the StarNet lightweight network perform well in small target detection tasks in high-speed vehicles, with high real-time and accuracy;

[0047] Furthermore, the method for detecting small traffic targets for high-speed vehicles of the present application can accurately and quickly detect small traffic targets such as pedestrians, traffic facilities and non-motor vehicles in traffic scenes during high-speed driving, solving the shortcomings of traditional traffic target detection such as large amount of calculation, complex feature design and poor generalization ability. The present application can significantly improve the detection accuracy of small traffic targets in complex traffic scenes, greatly reduce the false detection rate and missed detection rate, avoid the occurrence of safety accidents, and ensure the safety of life and property of drivers, passengers and pedestrians. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0049] Figure 1 This is a flow chart of a method for detecting small traffic targets of high-speed vehicles in an embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of the StarNet lightweight network structure in the embodiment of this application;

[0051] Figure 3 Schematic diagram of the structure of the SPD-Conv module in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the computational flow between various levels of the original PANet structure in an embodiment of the present application;

[0053] Figure 5 A schematic diagram of the calculation flow between various levels of the Fast PANet structure in an embodiment of the present application;

[0054] Figure 6 A schematic diagram of the Fast PANet structure in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of a Star Block module in an embodiment of the present application;

[0056] Figure 8 This is a principle block diagram of a small traffic target detection device for high-speed vehicles in an embodiment of the present application;

[0057] Fig. 9 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0059] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0060] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0061] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playback function, or a smart TV, a set-top box and other devices.

[0062] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0063] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0064] Unless expressly specified, one or more technical features of the present application can be deployed on a server for implementation and accessed by a client through a remote call to obtain an online service interface provided by the server, or can be directly deployed and run on a client for access.

[0065] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0066] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0067] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0068] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0069] See also Figure 1 In one embodiment, the method for detecting small traffic targets of high-speed vehicles of the present application includes:

[0070] Step S10, acquiring a traffic scene image frame containing a small traffic target in a high-speed vehicle;

[0071] The automatic driving system in an unmanned vehicle or an automatic driving vehicle can respond to a small traffic target detection instruction to obtain a traffic scene image frame containing a small traffic target in a high-speed vehicle. The high-speed vehicle can be an unmanned vehicle or an automatic driving vehicle traveling at high speed, etc. The picture that needs to detect small traffic targets through the technical solution of the present application can be regarded as the traffic scene image frame of the present application. The type and source of the traffic scene image frame depend on the actual application scenario. For example, in the application scenario of an automatic driving vehicle traveling at high speed, the traffic scene image can be a static picture specified by the user, or it can be a traffic scene image frame in a traffic scene video stream submitted by a camera device in an unmanned vehicle or an automatic driving vehicle to a vehicle-mounted terminal device. The small traffic target detection model of the present application is called in the vehicle-mounted terminal device to detect the small traffic targets in the traffic scene image frame. In this way, depending on the specific application scenario, the traffic scene image frame can be determined as needed.

[0072] In some embodiments, the traffic scene image refers to an image taken in a traffic environment, and these images may contain a variety of traffic-related elements and targets. Specifically, the traffic scene image includes vehicles, pedestrians, traffic signs, traffic lights, traffic markings or road facilities, etc. The small traffic targets include one or more of pedestrians, small animals, traffic facilities or non-motor vehicles, and the non-motor vehicles include bicycles, motorcycles or electric vehicles; the high-speed vehicles represent unmanned vehicles or automatic driving vehicles traveling at high speeds, and the small animals include kittens or puppies, etc.

[0073] In some embodiments, traffic scene image frames of different scenes, weather and time in the traffic scene video stream of the camera device in the unmanned vehicle or the autonomous driving vehicle in the real driving scene can be used to construct the traffic scene image dataset of the second traffic small target detection model of the present application, and the traffic scene image dataset contains pictures of different scenes, weather and time, including high-definition and blurred scenes, and the traffic scene image dataset includes about 100,000 traffic scene pictures, and the GT frame labels have 10 categories, namely: bus (Bus), traffic light (Light), traffic sign (Sign), pedestrian (Person), bicycle (Bike), truck (Truck), motorcycle (Motor), car (Car), train (Train) and takeaway rider (Rider), with a total of about 1.84 million calibration frames. The training set, test set and validation set are divided into 7:2:1, and then the label format of the data set is converted to coco format.

[0074] Step S20, updating the backbone network in the preset first small traffic target detection model to a StarNet lightweight network, updating the strided convolution layer in the StarNet lightweight network to an SPD-Conv module, and updating the CCFM structure of the feature fusion network in the first small traffic target detection model to a Fast PANet structure, so as to construct a second small traffic target detection model;

[0075] The basic network architecture of the first small traffic target detection model is the original RT-DETR target detection model, and the basic network architecture of the second small traffic target detection model is the improved RT-DETR target detection model; since unmanned vehicles or autonomous driving vehicles need to accurately and quickly detect small traffic targets in traffic scene images while driving at high speeds, a target detection task with a fast response is required. Therefore, the present application makes improvements in the basic network architecture of the original RT-DETR target detection model to construct the second small traffic target detection model of the present application. RT-DETR (Real-Time DEtection TRansformer) is a real-time network model for target detection, which combines convolutional neural networks (CNN) and Transformer structures to achieve efficient target detection; after obtaining a traffic scene image containing small traffic targets in a high-speed vehicle, the backbone network (Backbone) in the preset original RT-DETR target detection model is updated to a StarNet lightweight network, the strided convolution layer in the StarNet lightweight network is updated to an SPD-Conv module, and the CCFM structure of the feature fusion network in the original RT-DETR target detection model is updated to a Fast PANet structure is used to determine the improved RT-DETR target detection model to construct the second traffic small target detection model. The basic network architecture of the second traffic small target detection model of the present application is the improved RT-DETR target detection model, wherein the StarNet lightweight network includes a CBR module and multiple Star Layer modules, wherein each Star Layer module includes an SPD-Conv module and multiple StarBlock modules, and the SPD-Conv module includes a space to depth layer and a point convolution layer; the FastPANet structure includes an upsampling layer, a downsampling layer, a feature splicing layer and a feature fusion layer, wherein the SPD-Conv module is introduced in the downsampling layer.

[0076] In some embodiments, the StarNet lightweight network is proposed by Microsoft Corporation, which has high-dimensional and nonlinear feature transformation. The StarNet lightweight network realizes the mapping of high-dimensional and nonlinear feature space through star operation without increasing the computational complexity. The star operation in the StarNet lightweight network can implicitly obtain high-dimensional features in low-dimensional input. The backbone network in the original RT-DETR target detection model is updated to the StarNet lightweight network, which can maintain computational efficiency while obtaining richer and more expressive feature representations, which is particularly important for fine feature capture in small traffic target detection tasks in high-speed vehicles.

[0077] The StarNet lightweight network has an efficient network design. The StarNet lightweight network achieves efficient feature representation through star operations without complex network design and additional computational overhead. It can perform calculations in low-dimensional space, but implicitly considers extremely high-dimensional features. The StarNet lightweight network is used as the backbone network of the second traffic small target detection model of the present application, providing efficient calculation and better feature representation, and can achieve higher target detection performance in resource-constrained environments such as high-speed vehicles.

[0078] The StarNet lightweight network has multi-level implicit feature expansion. By stacking multiple recursive Star Block modules in multiple layers to perform multi-layer star operations, StarNet can recursively increase the implicit feature dimension, approaching infinite dimensions. For networks with large width and depth, this feature can significantly enhance the expressiveness of features. For the second small traffic target detection model of this application, the quality of feature extraction can be significantly improved through appropriate depth and width design, thereby improving the accuracy of small traffic target detection.

[0079] In some embodiments, the star operation refers to an operation that fuses two linear transformation features through element-wise multiplication. The star operation is similar to a polynomial kernel function, and has the ability to map the input to an extremely high-dimensional nonlinear feature space. It can be calculated in a low-dimensional space while generating high-dimensional features. By stacking multiple Star Block modules recursively in multiple layers to perform multi-layer star operations, the implicit dimension exponent is recursively increased to almost infinity, thereby exponentially increasing the implicit feature dimension in the traffic scene image frame.

[0080] In some embodiments, the StarNet lightweight network structure is as follows: Figure 2As shown in the figure, the StarNet lightweight network consists of a CBR module and 4 Star Layers. The CBR module includes a 3×3 convolution (Conv) module with a step size of 2 and an output channel of 24, a batch normalization (BN) module, and a ReLu6 activation module; each StarLayer module consists of an SPD-Conv module and several Star Block modules. First, the SPD-Conv module is used to downsample and modify the number of embedded channels. The number of embedded channels can be calculated using the following formula:

[0081] ，

[0082] in, Set to 24, Indicates which Star Layer module is currently used, and then extracts features through the StarBlock module.

[0083] Furthermore, in the four Star Layer modules of the StarNet lightweight network, each Star Layer module includes an SPD-Conv module for downsampling and modifying the number of embedded channels, which replaces the general strided convolution. This is because the original strided convolution module performs convolution processing on the local area of ​​the feature, which easily leads to the loss of fine-grained information. However, the SPD-Conv module performs convolution processing on each pixel of the feature. This method helps to reduce the possible over-downsampling problem, and at the same time can more effectively retain the fine-grained feature information of the image, thereby improving the small traffic target detection capability of the small traffic target detection model of the present application; the SPD-Conv module is a convolution layer for downsampling, which combines the characteristics of spatial pyramid pooling and strided convolution (cavity convolution). The main goal of the SPD-Conv module is to improve the feature representation capability of the network during the downsampling process while retaining more spatial information.

[0084] In some embodiments, see Figure 3 , the structure of the SPD-Conv module is as follows Figure 3As shown in the figure, the SPD-Conv module replaces the traditional strided convolution to complete downsampling and modify the number of embedded channels, while improving the small target detection capability of the model; the SPD-Conv module in the Star Layer module includes a space-to-depth layer and a point convolution layer. The function of the space-to-depth (SPD) layer is to convert the spatial dimension of the image feature to the channel dimension and retain the channel information of the image feature. This process first divides the input feature from the spatial dimension into several sub-image features, and then splices these sub-image features from the channel dimension to obtain the intermediate feature. The spatial size of the intermediate feature will be reduced compared to the input feature, while the channel dimension will increase; the function of the point convolution layer is to perform channel dimension reduction on the intermediate feature.

[0085] Specifically, the SPD-Conv module processes the traffic scene image frame, including:

[0086] Assume that the image features input to the SPD-Conv module are The size is , first Divided into spatial dimensions The sub-features are as follows:

[0087] ,

[0088] ,

[0089] …

[0090] ,

[0091] The size of each sub-feature is ;

[0092] Then the above The sub-features are spliced ​​from the channel dimension to obtain the intermediate features , intermediate features The size is ; Finally, for the intermediate features use The convolution operation is performed with a convolution kernel of size 1×1 and a step size of 1, resulting in a convolution kernel of size The output features of is the spatial size of the feature map, is 2, is the channel dimension of the input feature map, is the number of embedding channels calculated above.

[0093] In some embodiments, the StarNet-S1 lightweight network includes 4 Star Layer modules, namely a first Star Layer module, a second Star Layer module, a third StarLayer module and a fourth Star Layer module, wherein the first Star Layer module includes 2 StarBlock modules, the second Star Layer module includes 2 Star Block modules, the third Star Layer module includes 8 StarBlock modules, and the fourth StarLayer module includes 3 Star Block modules.

[0094] In some embodiments, based on the original RT-DETR target detection model, the CCFM structure originally included is replaced with the FastPANet structure for the neck part. The FastPANet structure is an innovative improvement on the PANet structure. Compared with the original PANet structure, the Fast PANet structure is optimized in design, reduces the number of convolutional layers, has fewer channels, achieves stronger lightweight performance, and realizes a more efficient semantic information transmission mechanism; in the downsampling part of the strided convolution, the SPC-Conv module is used for replacement.

[0095] The Fast PANet structure specifically includes an upsampling layer, a downsampling layer, a feature concatenation layer, and a feature fusion layer, wherein the upsampling layer uses a nearest neighbor interpolation upsampling algorithm to upsample image features from a higher level in the process of image feature fusion from top to bottom, thereby expanding its spatial size so that it is aligned with the size of the image features of the lower level.

[0096] Downsampling layer uses is 2, for The SPD-Conv module, the structure of which is the same as the SPD-Conv module in the above embodiment, including a space-to-depth layer and a point convolution layer, which will not be described in detail here;

[0097] The feature concatenation layer is used to concatenate image features from different levels along the channel dimension;

[0098] The feature fusion layer adopts a reverse bottleneck design similar to that in the MobileNetV2 network, which consists of a channel dimension reduction module and two residual reverse bottleneck modules in series; first, the spliced ​​multi-level image features are input into the channel dimension reduction module, which consists of multiple convolutional layers that do not change the size of the feature space; then the features are input into the residual reverse bottleneck block, which first uses 1×1 point convolution to increase the number of feature channels; then the features are subjected to depthwise separable convolution to capture and fuse features at different scales, and then 1×1 point convolution is used again to restore the number of feature channels; finally, a residual connection is performed with the input features to obtain the output.

[0099] In order to further reduce unnecessary network parameters and improve the lightweight level, when the expansion ratio of the residual reverse bottleneck block is equal to 1, that is, the number of input channels is equal to the number of expanded intermediate layer channels, the first 1×1 point convolution is omitted.

[0100] Also, see Figure 4 and Figure 5 ,in, Figure 4 This is a schematic diagram of the computational flow between the various levels of the original PANet structure. Figure 5 This is a schematic diagram of the computational flow between each level of the Fast PANet structure of the present application. Compared with the original PANet structure, the FastPANet structure in the second traffic small target detection model of the present application also reduces the computational flow of the third and fifth levels after the backbone network and before the detection head, and enhances the level of semantic information directly entering the fourth level.

[0101] See also Figure 6 The Fast PANet structure in the traffic small target detection model of this application is as follows Figure 6 As shown in the figure, the FastPANet structure follows the information bottleneck theory, reduces the number of model parameters as much as possible, balances computational efficiency and feature expression capabilities, and achieves accurate capture of feature information while reducing the amount of computation, resulting in a more compact model architecture.

[0102] Furthermore, through the image samples included in the training set and the corresponding annotation files, the network model parameters including the size of the input image, the number of recognition categories, the initial learning rate and the number of training rounds are configured; the improved RT-DETR target detection model is trained, and the other structures of the improved RT-DETR target detection model in this application are consistent with the original RT-DETR target detection model.

[0103] Specifically, the initial learning rate is set to 0.0001 during the training process, the number of recognition categories is 10, and the cosine annealing function is used to adjust the learning rate during the training process. The training batch is set to 16, the number of training rounds is set to 500, and the iterative optimization loss function is shown in the following formula:

[0104] ,

[0105] in, Represents the loss value of iterative optimization, represents the loss value of the bounding box, Represents the loss value of the category. Represent the bounding box and category respectively, Represent the predicted bounding box and category respectively, It represents the difference between the positioning and classification prediction distributions. During the training process, the validation set is used to evaluate the performance of the model. The model is adjusted according to the validation results, and the model weight file is saved after the network model converges.

[0106] After the second small traffic target detection model of the present application is trained to a convergence state, it can be put into production for use to perform target detection on the traffic scene image frame.

[0107] Step S30, inputting the traffic scene image frame into a second traffic small target detection model that has been trained to a convergence state, and using a star operation in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network to perform element-by-element multiplication on image features corresponding to the traffic scene image frame to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame;

[0108] After the second traffic small target detection model is constructed, the traffic scene image frame is input into the second traffic small target detection model that has been trained to a convergence state, and the second traffic small target detection model that has been trained to a convergence state is used to perform target detection on the traffic scene image frame, and in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network, a star operation is used to perform element-by-element multiplication on image features corresponding to the traffic scene image frame to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame;

[0109] Specifically, multiple Star Block modules are stacked recursively through multiple layers to stack multiple layers of star operations. Multiple Star Block modules are stacked recursively through multiple layers to exponentially increase the implicit feature dimensions in the traffic scene image frame. At the same time, the original low-dimensional space is mapped to an extremely high-dimensional nonlinear feature space, and more complex features are captured in the extremely high-dimensional nonlinear feature space to increase the expression ability of image features at different levels. The StarNet lightweight network achieves the mapping of high-dimensional feature space while keeping the computational complexity low through star operations, which solves the trade-off between computational complexity and performance in traditional efficient network design.

[0110] This application requires that unmanned vehicles or autonomous vehicles traveling at high speeds accurately and in real time detect small traffic targets such as pedestrians and small animals in traffic scene images. It is necessary to find a balance between real-time performance and detection accuracy. The efficient characteristics of the StarNet lightweight network just meet this requirement.

[0111] Traditional convolutional networks have certain limitations in the high-dimensional nonlinear transformation of feature representation, while the StarNet lightweight network achieves richer feature representation through star operations. In the target detection task of high-speed vehicles, especially for small traffic targets and complex traffic scenes, the rich feature representation can significantly improve the detection effect, making the improved RT-DETR target detection model perform better in these complex traffic scenes.

[0112] In some embodiments, see Figure 7 The structure of the Star Block module is as follows Figure 7 As shown, in the Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, the step of using the star operation to perform element-by-element multiplication on the image features corresponding to the traffic scene image frame to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame includes:

[0113] Step S301: In each Star Block module, the input image features are first processed by a 7×7 first depthwise separable convolution (DWConv) module, and the processed results are sent to two 1x1 convolution layers in parallel to expand the number of channels to 4 times the original number.

[0114] Step S302: After being activated by the ReLu6 activation function, the first output result of the first 1x1 convolutional layer is combined with the second output result of the second 1x1 convolutional layer by element-by-element multiplication to determine a third output result;

[0115] Step S303: After that, the third output result adjusts the number of channels through a 1×1 convolution layer, reducing the number of channels to one quarter to restore the original number of channels, and a 7×7 second depthwise separable convolution module is added again at the end of the Star Block module to perform a second depthwise separable convolution (DWConv) process;

[0116] Step S304, finally, the feature map after the second depth-separable convolution processing is regularized through the drop_path layer to reduce overfitting, and is residually connected with the original input input, and finally the high-dimensional image features corresponding to the traffic scene image frame are output, wherein the probability of each Star Block performing drop_path is random.

[0117] Furthermore, in the Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, before the step of performing element-by-element multiplication of the image features corresponding to the traffic scene image frame by using the star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame, the method includes:

[0118] Step S3001: In the Star Layer module of the StarNet lightweight network, the space-to-depth layer in the SPD-Conv module divides the image features corresponding to the traffic scene image frame into multiple sub-image features from a spatial dimension, and splices the multiple sub-image features from a channel dimension to determine a first intermediate feature map;

[0119] Step S3002: The point convolution layer in the SPD-Conv module performs channel dimensionality reduction on the first intermediate feature map to determine a first intermediate feature map after channel dimensionality reduction;

[0120] Step S3003, the Star Block module in the Star Layer module performs the first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, second depth separable convolution and residual connection processing on the first intermediate feature map after the channel dimensionality reduction to determine a high-level feature map.

[0121] Furthermore, the Star Block module in the Star Layer module sequentially performs a first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth separable convolution and a residual connection process on the first intermediate feature map after the channel dimension reduction to determine the step of a high-level feature map, including:

[0122] Step S3004: In the Fast PANet structure of the traffic small target detection model, the upsampling layer uses a nearest neighbor interpolation algorithm to expand the spatial size of the high-level feature map to align the low-level feature map;

[0123] Step S3005: The SPD-Conv module in the downsampling layer determines image features of different levels through space-to-depth conversion and point convolution operations, and in the feature splicing layer, the image features of different levels are spliced ​​in the channel dimension to determine the spliced ​​multi-level image features;

[0124] Step S3006: using reverse bottleneck design and multiple convolution operations in the feature fusion layer to perform feature fusion on the spliced ​​multi-level image features to determine fused multi-scale image features;

[0125] Step S3007: input the fused multi-scale image features into the detection head network in the traffic small target detection model to detect multiple traffic small targets in the traffic scene image to complete the traffic small target detection of high-speed vehicles.

[0126] Furthermore, the step of using a reverse bottleneck design and multiple convolution operations in the feature fusion layer to perform feature fusion on the spliced ​​feature maps to determine the fused multi-scale image features includes:

[0127] Step S30061, inputting the spliced ​​multi-level image features into the channel dimension reduction module in the feature fusion layer, and performing channel dimension reduction processing on the multi-level image features, wherein the feature fusion layer is constructed by connecting a channel dimension reduction module and two residual reverse bottleneck modules in series, and the channel dimension reduction module is constructed by multiple convolutional layers that do not change the feature space size;

[0128] Step S30062, input the multi-level image features after the channel dimension reduction processing into the first residual reverse bottleneck module, and perform channel number increase, depthwise separable convolution, channel number recovery and residual connection processing in sequence to obtain the output of the first residual reverse bottleneck module;

[0129] Step S30063: Use the output of the first residual reverse bottleneck module as the input of the second residual reverse bottleneck module, and repeat the above steps to determine the fused multi-scale image features.

[0130] Specifically, the spliced ​​multi-level image features are input into the channel dimension reduction module in the feature fusion layer, and the multi-level image features are subjected to channel dimension reduction processing, wherein the feature fusion layer is constructed by connecting a channel dimension reduction module and two residual reverse bottleneck modules in series, and the channel dimension reduction module is constructed by multiple convolution layers that do not change the feature space size; the processed multi-level image features after channel dimension reduction processing are input into the first residual reverse bottleneck module, and the first residual reverse bottleneck module first uses a 1×1 point convolution (also called point-by-point convolution) operation to increase the number of channels of the feature map, thereby expanding the feature representation capability of the feature map; then, a depthwise separable convolution operation is performed, which performs independent spatial convolution on each channel and then performs a 1×1 convolution to achieve further extraction and fusion of features, and this operation can capture and fuse feature information at different scales; then, a 1×1 point convolution operation is applied again to restore the number of channels of the feature map to the same number as the input feature map.

[0131] Finally, the feature map after the above operation is residually connected with the input feature map (element-by-element addition) to obtain the output of the first residual reverse bottleneck module; the output of the first residual reverse bottleneck module is used as input to enter the second residual reverse bottleneck module, and the above steps are repeated: increase the number of channels, depthwise separable convolution, restore the number of channels, and residual connection processing;

[0132] After being processed by two serially connected residual reverse bottleneck modules, the final high-level feature map is obtained, which contains the information after channel dimension reduction, feature extraction, fusion and residual connection.

[0133] Through the above steps, the feature fusion layer not only effectively reduces and expands the number of channels of multi-level image features, but also captures feature information at different scales through deep separable convolutions and ensures effective information transmission through residual connections. This structure can improve the representation ability of feature maps and enhance the model's ability to process complex image data.

[0134] Step S40: input the high-dimensional image features into the detection head network in the small traffic target detection model to obtain a plurality of small traffic targets in the traffic scene image frame to complete the small traffic target detection of high-speed vehicles.

[0135] After determining the high-dimensional image features corresponding to the traffic scene image frame, the high-dimensional image features are input into the detection head network in the traffic small target detection model to obtain multiple traffic small targets in the traffic scene image frame to complete the traffic small target detection of high-speed vehicles.

[0136] Specifically, in the star operation of the StarNet lightweight network, the input feature map is mapped to an extremely high-dimensional nonlinear feature space by recursive stacking. This means that the low-dimensional features of the input image will be converted into high-dimensional features through layers of recursion, and these high-dimensional features can better capture the details and complexity in the image; the multi-layer recursive process enables the model to obtain rich implicit features, which are expressed as nonlinear feature mappings in high-dimensional space. This mapping helps the model understand and distinguish various small traffic targets in traffic scene images;

[0137] After multi-layer star operation processing, the obtained high-dimensional image features will be passed to the detection head network of the traffic small target detection model. The role of the detection head network is to extract and detect specific target information from these high-dimensional features. The detection head network focuses on detecting small traffic targets from high-dimensional features. The detection head network generally contains multiple levels of convolution and fully connected layers, through which the high-dimensional features are further processed to determine various small traffic targets in the traffic scene image. In the scene of high-speed vehicles, small traffic targets such as traffic signs, pedestrians or obstacles may be moved quickly and blocked. The detection head network can identify these small targets from the high-dimensional features and output their positions and categories.

[0138] Furthermore, the advantage of the StarNet lightweight network lies in its lightweight and efficient characteristics, making it suitable for real-time small target detection in high-speed traffic scenarios. This model is particularly suitable for applications in environments where small targets need to be processed quickly and accurately detected, such as the detection of various small traffic targets in traffic scene images in autonomous driving systems. Through the star operation of the StarNet lightweight network, the model is able to significantly improve the feature dimension through recursive stacking, thereby more accurately detecting small targets in traffic scenes; the generation of high-dimensional features and subsequent detection head network processing make the StarNet lightweight network perform well in the task of small target detection in high-speed vehicles, with high real-time and accuracy.

[0139] It can be seen from the above embodiments that, compared with the prior art, the present application aims at the disadvantages of the traditional traffic target detection method in the prior art, such as large amount of calculation, complex feature design, poor generalization ability, etc., which are difficult to meet the requirements of real-time and accuracy, and often expose high false detection rate and missed detection rate. The present application includes but is not limited to the following beneficial effects:

[0140] First, the StarNet lightweight network in the traffic small target detection model of the present application realizes the mapping of high-dimensional and nonlinear feature spaces through star operations without increasing the computational complexity. The star operations in the StarNet lightweight network can implicitly obtain high-dimensional features in low-dimensional inputs, and the backbone network in the original RT-DETR target detection model is updated to the StarNet lightweight network. While maintaining computational efficiency, it can also obtain richer and more expressive feature representations, which is particularly important for fine feature capture in the task of detecting small traffic targets in high-speed vehicles.

[0141] Secondly, the StarNet lightweight network in the traffic small target detection model of the present application has an efficient network design. The StarNet lightweight network realizes efficient feature representation through star operation without complex network design and additional computing overhead. It can perform calculations in low-dimensional space, but implicitly considers extremely high-dimensional features. The StarNet lightweight network is used as the backbone network of the traffic small target detection model of the present application, providing efficient calculation and better feature representation, and can achieve higher target detection performance in resource-constrained environments such as high-speed vehicles.

[0142] Third, the StarNet lightweight network in the traffic small target detection model of this application has multi-level implicit feature expansion. Through multi-layer star operations, StarNet can recursively increase the implicit feature dimension, approaching infinite dimension. For networks with large width and depth, this feature can significantly enhance the expressiveness of features. For the traffic small target detection model of this application, the quality of feature extraction can be significantly improved through appropriate depth and width design, thereby improving the accuracy of traffic small target detection;

[0143] Fourthly, the Fast PANet structure in the traffic small target detection model of this application follows the information bottleneck theory, reduces the number of model parameters as much as possible, balances the computational efficiency and feature expression ability, and achieves accurate capture of feature information while reducing the amount of computation, thus obtaining a more compact model architecture;

[0144] Furthermore, the advantage of the StarNet lightweight network in the traffic small target detection model of the present application lies in its lightweight and efficient characteristics, making it suitable for real-time small target detection in high-speed traffic scenes. The traffic small target detection model of the present application is particularly suitable for application in environments where small targets need to be quickly processed and accurately detected, such as various traffic small target detections in traffic scene images in autonomous driving systems; through the star operation of the StarNet lightweight network, the model can significantly improve the feature dimension through recursive stacking, thereby more accurately detecting small targets in traffic scenes; the generation of high-dimensional features and subsequent detection head network processing make the StarNet lightweight network perform well in small target detection tasks in high-speed vehicles, with high real-time and accuracy;

[0145] Furthermore, the method for detecting small traffic targets for high-speed vehicles of the present application can accurately and quickly detect small traffic targets such as pedestrians, traffic facilities and non-motor vehicles in traffic scenes during high-speed driving, solving the shortcomings of traditional traffic target detection such as large amount of calculation, complex feature design and poor generalization ability. The present application can significantly improve the detection accuracy of small traffic targets in complex traffic scenes, greatly reduce the false detection rate and missed detection rate, avoid the occurrence of safety accidents, and ensure the safety of life and property of drivers, passengers and pedestrians.

[0146] See also Figure 8, a small traffic target detection device for a high-speed vehicle provided to meet one of the purposes of the present application, includes an image acquisition module 1100, a detection model construction module 1200, a high-dimensional feature extraction module 1300 and a small target detection module 1400. Among them, the image acquisition module 1100 is configured to acquire a traffic scene image frame containing a small traffic target in a high-speed vehicle; the detection model construction module 1200 is configured to update the backbone network in the preset first small traffic target detection model to a StarNet lightweight network, update the strided convolution layer in the StarNet lightweight network to an SPD-Conv module, and update the CCFM structure of the feature fusion network in the first small traffic target detection model to a FastPANet structure to construct a second small traffic target detection model; the high-dimensional feature extraction module 1300 is configured to input the traffic scene image frame into the second small traffic target detection model that has been trained to a convergent state, and in the multiple StarNet lightweight network of the multi-layer stacked recursive layers, the high-dimensional feature extraction module 1300 is configured to extract the small traffic target detection model from the traffic scene image frame. In the Block module, the star operation is used to perform element-by-element multiplication on the image features corresponding to the traffic scene image frame to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame; the small target detection module 1400 is configured to input the high-dimensional image features into the detection head network in the traffic small target detection model to obtain multiple traffic small targets in the traffic scene image frame to complete the traffic small target detection of high-speed vehicles.

[0147] Based on any embodiment of this application, please refer to Fig. 9 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Fig. 9 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a method for detecting small traffic targets for high-speed vehicles. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the method for detecting small traffic targets for high-speed vehicles of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Fig. 9The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In this embodiment, the processor is used to execute Figure 8 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the traffic small target detection device for high-speed vehicles of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0149] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for detecting small traffic targets for high-speed vehicles described in any embodiment of the present application.

[0150] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the method for detecting small traffic targets of high-speed vehicles described in any embodiment of the present application.

[0151] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0152] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0153] To sum up, the method for detecting small traffic targets for high-speed vehicles of the present application can accurately and quickly detect small traffic targets such as pedestrians, traffic facilities and non-motor vehicles in traffic scenes during high-speed driving, solving the shortcomings of traditional traffic target detection such as large amount of calculation, complex feature design and poor generalization ability. The present application can significantly improve the detection accuracy of small traffic targets in complex traffic scenes, greatly reduce the false detection rate and missed detection rate, avoid the occurrence of safety accidents, and ensure the safety of life and property of drivers, passengers and pedestrians.

Claims

1. A method for detecting small traffic targets of high-speed vehicles, characterized in that: include: Acquire a traffic scene image frame containing a small traffic target from a high-speed vehicle; The backbone network in the preset first small traffic target detection model is updated to a StarNet lightweight network, the strided convolution layer in the StarNet lightweight network is updated to an SPD-Conv module, and the CCFM structure of the feature fusion network in the first small traffic target detection model is updated to a Fast PANet structure to construct a second small traffic target detection model, wherein the StarNet lightweight network includes a CBR module and multiple Star Layer modules, wherein each Star Layer module includes an SPD-Conv module and multiple Star Block modules, and the SPD-Conv module includes a space-to-depth layer and a point convolution layer; the Fast PANet structure includes an upsampling layer, a downsampling layer, a feature splicing layer, and a feature fusion layer, wherein the SPD-Conv module is introduced in the downsampling layer; Inputting the traffic scene image frame into a second traffic small target detection model that has been trained to a converged state, and performing element-by-element multiplication of image features corresponding to the traffic scene image frame using a star operation in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame; In the Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, before the step of performing element-by-element multiplication on the image features corresponding to the traffic scene image frame by using the star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame, the method includes: In the Star Layer module of the StarNet lightweight network, the space-to-depth layer in the SPD-Conv module divides the image features corresponding to the traffic scene image frame into multiple sub-image features from a spatial dimension, and splices the multiple sub-image features from a channel dimension to determine a first intermediate feature map; The point convolution layer in the SPD-Conv module performs channel dimensionality reduction on the first intermediate feature map to determine a first intermediate feature map after channel dimensionality reduction; The Star Block module in the Star Layer module sequentially performs a first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth separable convolution and a residual connection process on the first intermediate feature map after the channel dimension reduction to determine a high-level feature map; The Star Block module in the Star Layer module sequentially performs a first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth separable convolution and residual connection processing on the first intermediate feature map after the channel dimension reduction to determine the step of a high-level feature map, including: In the Fast PANet structure of the traffic small target detection model, the upsampling layer uses a nearest neighbor interpolation algorithm to expand the spatial size of the high-level feature map to align the low-level feature map; The SPD-Conv module in the downsampling layer determines image features at different levels through space-to-depth conversion and point convolution operations, and in the feature splicing layer, the image features at different levels are spliced ​​in the channel dimension to determine the spliced ​​multi-level image features; In the feature fusion layer, a reverse bottleneck design and multiple convolution operations are used to perform feature fusion on the spliced ​​multi-level image features to determine the fused multi-scale image features; Inputting the fused multi-scale image features into the detection head network in the traffic small target detection model to detect multiple traffic small targets in the traffic scene image, so as to complete the traffic small target detection of high-speed vehicles; The high-dimensional image features are input into the detection head network in the small traffic target detection model to obtain multiple small traffic targets in the traffic scene image frame to complete the small traffic target detection of high-speed vehicles.

2. The method for detecting small traffic targets of high-speed vehicles according to claim 1, characterized in that: In a plurality of Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, a step of performing element-by-element multiplication of image features corresponding to the traffic scene image frame by using a star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine a high-dimensional image feature corresponding to the traffic scene image frame includes: In each Star Block module, the input image features are first processed by a 7×7 first depth separable convolution module, and the processed results are sent to two 1x1 convolution layers in parallel to expand the number of channels to 4 times the original number. The first output result of the first 1x1 convolutional layer is activated by the ReLu6 activation function, and then combined with the second output result of the second 1x1 convolutional layer by element-by-element multiplication to determine the third output result; Afterwards, the third output result adjusts the number of channels through a 1×1 convolution layer, reducing the number of channels to one quarter to restore the original number of channels, and a 7×7 second depthwise separable convolution module is added again at the end of the Star Block module to perform a second depthwise separable convolution process; Finally, the feature map after the second depth-wise separable convolution is regularized through a drop_path layer to reduce overfitting, and is residually connected with the original input to finally output the high-dimensional image features corresponding to the traffic scene image frame.

3. The method for detecting small traffic targets of high-speed vehicles according to claim 1, characterized in that: The step of using reverse bottleneck design and multiple convolution operations in the feature fusion layer to fuse the spliced ​​multi-level image features to determine the fused multi-scale image features includes: Inputting the spliced ​​multi-level image features into the channel dimension reduction module in the feature fusion layer, and performing channel dimension reduction processing on the multi-level image features, wherein the feature fusion layer is constructed by connecting a channel dimension reduction module and two residual reverse bottleneck modules in series, and the channel dimension reduction module is constructed by multiple convolutional layers that do not change the size of the feature space; The multi-level image features after channel dimension reduction processing are input into the first residual reverse bottleneck module, and the channel number increase, depthwise separable convolution, channel number recovery and residual connection processing are performed in sequence to obtain the output of the first residual reverse bottleneck module; The output of the first residual reverse bottleneck module is used as the input of the second residual reverse bottleneck module, and the above steps are repeated to determine the fused multi-scale image features.

4. The method for detecting small traffic targets of high-speed vehicles according to any one of claims 1 to 3, characterized in that: The basic network architecture of the first small traffic target detection model is the original RT-DETR target detection model, and the basic network architecture of the second small traffic target detection model is the improved RT-DETR target detection model; The small traffic targets include one or more of pedestrians, small animals, traffic facilities or non-motor vehicles; the high-speed vehicle represents an unmanned vehicle or an automatic driving vehicle traveling at high speed.

5. A small traffic target detection device for high-speed vehicles, characterized in that: include: An image acquisition module is configured to acquire a traffic scene image frame containing a small traffic target from a high-speed vehicle; A detection model construction module is configured to update the backbone network in the preset first small traffic target detection model to a StarNet lightweight network, update the strided convolution layer in the StarNet lightweight network to an SPD-Conv module, and update the CCFM structure of the feature fusion network in the first small traffic target detection model to a Fast PANet structure to construct a second small traffic target detection model, wherein the StarNet lightweight network includes a CBR module and a plurality of Star Layer modules, wherein each Star Layer module includes an SPD-Conv module and a plurality of Star Block modules, and the SPD-Conv module includes a space-to-depth layer and a point convolution layer; the Fast PANet structure includes an upsampling layer, a downsampling layer, a feature concatenation layer, and a feature fusion layer, wherein the SPD-Conv module is introduced in the downsampling layer; A high-dimensional feature extraction module is configured to input the traffic scene image frame into a second traffic small target detection model that has been trained to a converged state, and in a plurality of Star Block modules of a multi-layer stacked recursion of the StarNet lightweight network, perform element-by-element multiplication of image features corresponding to the traffic scene image frame using a star operation to exponentially increase the implicit feature dimension in the traffic scene image frame, so as to determine a high-dimensional image feature corresponding to the traffic scene image frame; In the Star Block modules of the multi-layer stacked recursion of the StarNet lightweight network, before the step of performing element-by-element multiplication on the image features corresponding to the traffic scene image frame by using the star operation to exponentially increase the implicit feature dimension in the traffic scene image frame to determine the high-dimensional image features corresponding to the traffic scene image frame, the method includes: In the Star Layer module of the StarNet lightweight network, the space-to-depth layer in the SPD-Conv module divides the image features corresponding to the traffic scene image frame into multiple sub-image features from a spatial dimension, and splices the multiple sub-image features from a channel dimension to determine a first intermediate feature map; The point convolution layer in the SPD-Conv module performs channel dimensionality reduction on the first intermediate feature map to determine a first intermediate feature map after channel dimensionality reduction; The Star Block module in the Star Layer module sequentially performs a first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth separable convolution and a residual connection process on the first intermediate feature map after the channel dimension reduction to determine a high-level feature map; The Star Block module in the Star Layer module sequentially performs a first depth separable convolution, channel number expansion, nonlinear activation, element-by-element multiplication, channel number adjustment, a second depth separable convolution and residual connection processing on the first intermediate feature map after the channel dimension reduction to determine the step of a high-level feature map, including: In the Fast PANet structure of the traffic small target detection model, the upsampling layer uses a nearest neighbor interpolation algorithm to expand the spatial size of the high-level feature map to align the low-level feature map; The SPD-Conv module in the downsampling layer determines image features at different levels through space-to-depth conversion and point convolution operations, and in the feature splicing layer, the image features at different levels are spliced ​​in the channel dimension to determine the spliced ​​multi-level image features; In the feature fusion layer, a reverse bottleneck design and multiple convolution operations are used to perform feature fusion on the spliced ​​multi-level image features to determine the fused multi-scale image features; Inputting the fused multi-scale image features into the detection head network in the traffic small target detection model to detect multiple traffic small targets in the traffic scene image, so as to complete the traffic small target detection of high-speed vehicles; The small target detection module is configured to input the high-dimensional image features into the detection head network in the traffic small target detection model to obtain multiple traffic small targets in the traffic scene image frame to complete the traffic small target detection of high-speed vehicles.

6. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 4 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images