Vehicle driving target object detection method and device, vehicle-mounted terminal and vehicle

Abstract

The invention provides a vehicle driving target object detection method and device, a vehicle-mounted terminal and a vehicle. The method comprises the steps that a to-be-detected driving image is acquired; obtaining a driving feature map through the target object detection model, wherein the driving feature map comprises a low-resolution feature map and a high-resolution feature map; positioning an initial detection frame corresponding to the target object through the low-resolution feature map, refining the initial detection frame through the high-resolution feature map to obtain a target detection frame, performing non-maximum suppression on the target detection frame, and removing overlapped detection frames to obtain a target detection result. According to the method, the deep learning technology and the complex computer vision strategy are effectively utilized, the recognition process of the target object is optimized, misrecognition caused by environmental factors is reduced, meanwhile, the fast reasoning ability under the low calculation cost is achieved, and a solid safety guarantee is provided for the automatic driving automobile under the vision limitation condition.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving vehicles, and in particular to a method and device for detecting a target object while a vehicle is traveling, a vehicle-mounted terminal and a vehicle. Background Art

[0002] With the rapid development of autonomous driving technology, infrared small target detection has become a core component of the vehicle perception system. Especially in the changeable and unpredictable nighttime traffic environment, the research on infrared small target detection is particularly important.

[0003] For self-driving cars, being able to accurately identify and locate tiny obstacles on the roadside, such as animals, fallen leaves, or other small objects, is key to ensuring driving safety and avoiding potential dangers. Traditional object detection technology often faces the dual challenges of slow reasoning speed and insufficient accuracy in small object recognition. Although deep learning technology has made great technological breakthroughs in the field of object detection, there is still a significant performance bottleneck between real-time detection of small objects and maintaining high accuracy. Summary of the invention

[0004] The present application provides a vehicle moving target object detection method, device and vehicle-mounted terminal to solve the above-mentioned technical problems.

[0005] A vehicle driving target object detection method provided by an embodiment of the present application includes: acquiring a driving image to be detected; performing feature recognition on the driving image to be detected through a backbone network in a target object detection model, and extracting basic features through a depth-separable convolution layer to obtain a multi-level feature map; performing feature map fusion on the multi-scale feature map to obtain a fused feature map, and performing convolution processing on the fused feature map to obtain a driving feature map, wherein the driving feature map includes a low-resolution feature map and a high-resolution feature map; locating an initial detection frame corresponding to a target object through the low-resolution feature map, and refining the initial detection frame through the high-resolution feature map to obtain a target detection frame, wherein the target detection frame includes a confidence level and a target object; performing non-maximum suppression on the target detection frame, eliminating overlapping detection frames, and obtaining a target detection result.

[0006] In one embodiment of the present application, obtaining a driving image to be detected includes: installing an infrared camera on the target vehicle, and collecting night driving images in real time through the infrared camera; normalizing the night driving images, and enhancing the contrast of the normalized night driving images through Gaussian filtering denoising and histogram equalization technology to obtain the driving image to be detected.

[0007] In one embodiment of the present application, feature recognition is performed on the driving image to be detected through the backbone network in the target object detection model, and basic features are extracted through a deep separable convolution layer to obtain a multi-level feature map, including: spatial convolution is performed independently on each channel of the driving image to be detected, and the driving image to be detected after the spatial convolution is normalized to obtain an initial feature map, cross-channel information integration is performed on the initial feature map through a deep convolution kernel, and the initial feature map after the information integration is normalized to obtain a combined feature map; features are extracted according to multiple depth levels in the combined feature map to obtain a multi-level feature map.

[0008] In one embodiment of the present application, feature map fusion is performed on the multi-scale feature map to obtain a fused feature map, and the fused feature map is convolved to obtain a driving feature map, including: based on the multi-level feature map, through the top layer downward and the bottom layer upward path, the upsampling result of the adjacent high layer is convolved with the sum of the current layer feature map to obtain a fused feature map; the fused feature map is convolved to obtain a driving feature map.

[0009] In one embodiment of the present application, an initial detection frame corresponding to the target object is located using the low-resolution feature map, including: searching the low-resolution feature map using a query head to obtain a heat map, wherein the heat map includes multiple target object detection frames; and determining a target object detection frame that is smaller than a preset threshold as an initial detection frame, wherein the preset threshold represents the minimum anchor frame scale or minimum regression range of the driving feature map.

[0010] In one embodiment of the present application, the initial detection frame is refined by the high-resolution feature map to obtain a target detection frame, including: mapping the target detection frame in the low-resolution feature map to the high-resolution feature map, extracting the center position of the target in the initial detection frame in the high-resolution feature map, and calculating the positioning distance between the center position and the remaining targets; marking the targets corresponding to the positioning distances less than or equal to a preset distance threshold as candidate detection frames, and eliminating the targets corresponding to the positioning distances greater than the preset threshold; querying the candidate detection frame through a sparse detection head, extracting the key positions of the candidate detection frame, and obtaining the target detection frame.

[0011] In one embodiment of the present application, non-maximum suppression is performed on the target detection frame, and overlapping detection frames are removed to obtain a target detection result, including: arranging the target detection frames corresponding to the same category of targets in descending order based on the confidence of the target detection frame to obtain a plurality of target detection frame sequences; marking the target detection frame with the highest confidence in the target detection frame sequence as a selected detection frame, and adding the selected detection frame to the target detection result; calculating the overlap value between the remaining detection frames in the target detection frame sequence and the selected detection frame, and removing the target detection frames with a value greater than a preset overlap value to remove the detection frames overlapping with the same category of targets, until the target detection frame sequence is empty.

[0012] The vehicle driving target object detection device provided in the embodiment of the present application includes: an acquisition module, which is used to acquire a driving image to be detected; a recognition module, which is used to perform feature recognition on the driving image to be detected through a backbone network in a target object detection model, and extract basic features through a deep separable convolution layer to obtain a multi-level feature map; a fusion module, which is used to perform feature map fusion on the multi-scale feature map to obtain a fused feature map, and convolve the fused feature map to obtain a driving feature map, wherein the driving feature map includes a low-resolution feature map and a high-resolution feature map; a positioning module, which is used to locate an initial detection frame corresponding to a target object through the low-resolution feature map, and refine the initial detection frame through the high-resolution feature map to obtain a target detection frame, wherein the target detection frame includes a confidence level and a target object; a deduplication module, which is used to perform non-maximum suppression on the target detection frame, eliminate overlapping detection frames, and obtain a target detection result.

[0013] An embodiment of the present application provides a vehicle-mounted terminal, including a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the above-mentioned vehicle driving target detection method.

[0014] An embodiment of the present application provides a vehicle, comprising the above-mentioned vehicle driving target object detection device or the above-mentioned vehicle-mounted terminal.

[0015] The beneficial effects of the present application are as follows: obtaining a driving image to be detected; obtaining a driving feature map through a target object detection model, the driving feature map including a low-resolution feature map and a high-resolution feature map; locating an initial detection frame corresponding to the target object through the low-resolution feature map, refining the initial detection frame through the high-resolution feature map to obtain a target detection frame, performing non-maximum suppression on the target detection frame, eliminating overlapping detection frames, and obtaining a target detection result. The present invention effectively utilizes deep learning technology and complex computer vision strategies to optimize the target object recognition process, locates an initial detection frame corresponding to the target object through a low-resolution feature map, and refines the initial detection frame through a high-resolution feature map, thereby reducing misrecognition caused by environmental factors and achieving rapid reasoning capability at a relatively low computing cost, thereby providing a solid safety guarantee for autonomous driving vehicles under visually restricted conditions.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a flow chart of a vehicle driving target object detection method shown in an exemplary embodiment of the present application;

[0019] Figure 2 A schematic diagram of the integrated structure of the MobileNetv2 backbone network and the feature pyramid network shown as an exemplary embodiment of the present application;

[0020] Figure 3 This is a diagram showing an implementation framework of a cascade sparse query mechanism according to an exemplary embodiment of the present application;

[0021] Figure 4 is a block diagram of a vehicle traveling target object detection device shown in an exemplary embodiment of the present application;

[0022] Figure 5 It is a structural schematic diagram of a vehicle-mounted terminal shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0024] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0025] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0026] The embodiments of the present application respectively propose a vehicle moving object detection method, a vehicle moving object detection device, and a vehicle-mounted terminal, and these embodiments will be described in detail below.

[0027] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for detecting a vehicle moving object according to an embodiment of the present application. Figure 1 As shown, the method at least includes steps S110 to S150, which are described in detail as follows:

[0028] Step S110, obtaining a driving image to be detected.

[0029] Step S120, performing feature recognition on the driving image to be detected through the backbone network in the target object detection model, and extracting basic features through a depth-separable convolutional layer to obtain a multi-level feature map.

[0030] Step S130, performing feature map fusion on the multi-scale feature maps to obtain a fused feature map, and performing convolution processing on the fused feature map to obtain a driving feature map, which includes a low-resolution feature map and a high-resolution feature map.

[0031] Step S140, locating an initial detection frame corresponding to the target object through the low-resolution feature map, and refining the initial detection frame through the high-resolution feature map to obtain a target detection frame, where the target detection frame includes a confidence level and a target object.

[0032] Step S150, performing non-maximum suppression on the target detection frame, removing overlapping detection frames, and obtaining a target detection result.

[0033] exist Figure 1 In the technical solution shown, a driving image to be detected is obtained; a driving feature map is obtained through a target object detection model, and the driving feature map includes a low-resolution feature map and a high-resolution feature map; an initial detection frame corresponding to the target object is located through the low-resolution feature map, and the initial detection frame is refined through the high-resolution feature map to obtain a target detection frame, and non-maximum suppression is performed on the target detection frame, and overlapping detection frames are eliminated to obtain a target detection result. The present invention effectively utilizes deep learning technology and complex computer vision strategies to optimize the recognition process of the target object, locates the initial detection frame corresponding to the target object through the low-resolution feature map, and refines the initial detection frame through the high-resolution feature map, thereby reducing misrecognition caused by environmental factors, and at the same time achieves fast reasoning capability at a lower computing cost, providing a solid safety guarantee for autonomous driving vehicles under visually restricted conditions.

[0034] In one embodiment of the present application, obtaining a driving image to be detected includes: installing an infrared camera on the target vehicle, and collecting night driving images in real time through the infrared camera; normalizing the night driving images, and enhancing the contrast of the normalized night driving images through Gaussian filtering denoising and histogram equalization technology to obtain the driving image to be detected. The acquired infrared image is preprocessed, including normalization processing to meet the input requirements of the deep learning model. Gaussian filtering is applied for denoising, and the image contrast is enhanced through histogram equalization technology to improve the accuracy of subsequent feature extraction.

[0035] As an example, an infrared camera is installed on the target vehicle to collect night driving images in real time. Specifically, the infrared camera can be installed in front of the target vehicle, on the side of the target vehicle, or at the rear of the target vehicle. After collecting the night driving images, the night driving images are normalized, for example, the resolution of the night driving images is adjusted to 224x224, and the normalized night driving images are preprocessed:

[0036]

[0037] Where μ is the mean of the pixel values, σ is the sum and standard deviation, and x is the normalized night driving image. The contrast of the normalized night driving image is enhanced by Gaussian filtering denoising and histogram equalization technology to obtain the driving image to be detected. The acquired infrared image is preprocessed, including normalization processing, to meet the input requirements of the deep learning model. Gaussian filtering is applied for denoising, and the image contrast is enhanced by histogram equalization technology to improve the accuracy of subsequent feature extraction.

[0038] In one embodiment of the present application, the backbone network in the target object detection model performs feature recognition on the driving image to be detected, and extracts basic features through the deep separable convolution layer to obtain a multi-level feature map, including: performing spatial convolution independently on each channel of the driving image to be detected, and normalizing the driving image to be detected after spatial convolution to obtain an initial feature map, integrating information across channels of the initial feature map through the deep convolution kernel, and normalizing the initial feature map after information integration to obtain a combined feature map; extracting features according to multiple deep levels in the combined feature map to obtain a multi-level feature map. Through the deep separable convolution network processing based on the MobileNetv2 architecture, the network uses point-by-point convolution and deep convolution to independently process each channel of the input feature map, thereby enhancing the efficiency and effect of the model.

[0039] As an example, the target object detection model is trained based on MobileNetV2, and spatial convolution is performed independently on each channel of the driving image to be detected to obtain a multi-level feature map, including: spatial convolution is performed independently on each channel of the driving image to be detected, and the driving image to be detected after spatial convolution is normalized to obtain the initial feature map:

[0040]

[0041] in, is the driving image to be detected, is the corresponding depth convolution kernel, M i is the initial feature map, D c is the depth convolution operation, i is the number of channels. The initial feature map is integrated across channels through the depth convolution kernel, and the initial feature map after information integration is normalized to obtain a combined feature map; in the combined feature map, features are extracted according to multiple depth levels to obtain a multi-level feature map:

[0042]

[0043] Among them, F out is a multi-level feature map, M i is the initial feature map, K p is the corresponding point-by-point convolution kernel, Pc is a point-by-point convolution operation, i is the number of channels. Spatial convolution is applied independently to each channel of the input feature map to capture spatial features. Based on deep convolution, cross-channel information integration is applied to improve the semantic expressiveness of the feature map, which not only greatly reduces the computational complexity but also improves the computational accuracy. Normalization adjusts and scales the activations of the intermediate layers of the deep learning network:

[0044]

[0045] Where F is the convolution output, μ B is the mean of the batch data, is the batch data variance, γ and β are learnable parameters, and ∈ is a constant used to prevent zero division errors. The ReLU (Rectified Linear Unit) activation function is used after batch normalization to further introduce nonlinearity and enhance the network's ability to capture complex features. By applying batch normalization after each convolution operation, the feature output is standardized and the internal covariate shift is reduced. The network is processed by a deep separable convolutional network based on the MobileNetv2 architecture, which uses point-by-point convolution and depth-wise convolution to independently process each channel of the input feature map, enhancing the efficiency and effectiveness of the model.

[0046] In one embodiment of the present application, feature map fusion is performed on multi-scale feature maps to obtain a fused feature map, and the fused feature map is convolved to obtain a driving feature map, including: based on the multi-level feature map, the upsampling results of adjacent high layers are convolved with the sum of the feature maps of the current layer through the top layer downward and the bottom layer upward path to obtain a fused feature map; the fused feature map is convolved to obtain a driving feature map. The output of the target detection model is fused with multiple layers of features using a feature pyramid network to create a multi-scale feature pyramid. This structure uses high-resolution bottom-level features and low-resolution high-level features to enhance the model's ability to detect targets of different scales.

[0047] As an example, Figure 2 The schematic diagram of the integrated structure of the MobileNetv2 backbone network and the feature pyramid network shown in an exemplary embodiment of the present application, the Feature Pyramid Networks (FPN), improves the performance of the model at different scales by fusing feature information of different scales. The fused feature map is obtained through the top-down path, specifically including starting from the top-level feature map of the multi-level feature map, and improving the resolution of the high-level feature map through upsampling, for example, bilinear interpolation, so as to be fused with the low-level feature map:

[0048] P i =Conv(U(P i+1 +Fi ))

[0049] Among them, P i is the fusion feature map, P i+1 is the sampling result of the feature map of the adjacent high level, F i is the feature map of the current layer. As a specific embodiment, after obtaining the fused feature map, the fused feature map is optimized:

[0050] P′ i =Conv(P i )

[0051] Among them, P′ i is the optimized fusion feature map, P i is the fusion feature map, and Conv is the convolution operation. In the top-down process, 1x1 convolution is used to adjust the number of channels of the high-level feature map to match the low-level feature map, and then the high-level semantic information is fused with the low-level fine information by element addition or connection. Feature fusion is performed at different levels to form a feature pyramid. Each level contains rich semantic and structural information, which is suitable for object detection of different scales, improves the semantic coherence between feature layers, and optimizes the detection accuracy of small objects. The feature pyramid network is used to perform multi-layer feature fusion on the output of MobileNetv2 to create a multi-scale feature pyramid. This structure uses high-resolution underlying features and low-resolution high-level features to enhance the model's ability to detect objects of different scales.

[0052] In one embodiment of the present application, an initial detection frame corresponding to a target object is located through a low-resolution feature map, including: searching in the low-resolution feature map through a query head to obtain a heat map, the heat map including a plurality of target object detection frames; determining a target object detection frame that is smaller than a preset threshold as an initial detection frame, the preset threshold representing the minimum anchor frame scale or minimum regression range of the driving feature map.

[0053] In one embodiment of the present application, the initial detection frame is refined by a high-resolution feature map to obtain a target detection frame, including: mapping the target detection frame in the low-resolution feature map to the high-resolution feature map, extracting the center position of the target in the initial detection frame in the high-resolution feature map, and calculating the positioning distance between the center position and the remaining targets; marking the targets corresponding to the positioning distances less than or equal to a preset distance threshold as candidate detection frames, and eliminating the targets corresponding to the positioning distances greater than the preset threshold; querying the candidate detection frames through a sparse detection head, extracting the key positions of the candidate detection frames, and obtaining the target detection frame.

[0054] As an example, Figure 3The implementation framework diagram of the cascade sparse query mechanism is shown as an exemplary embodiment of the present application. First, a query head (Query Head, QH) is integrated, and the query head is used to search in the low-resolution feature map to obtain the heat map V l , V l ∈R H′×W′ , where H′ and W′ are the height and width of the heat map. As a specific example, the preset threshold is s l , the target detection box size is smaller than the preset threshold "s l " is determined as the initial detection frame. For example, the initial detection frame is a small target detection frame. The small target detection frame in the low-resolution feature map is mapped to the high-resolution feature map, and the center position of the small target in the small target detection frame in the high-resolution feature map is extracted. The positioning distance between the center position and the remaining targets is calculated. The query head calculates the distance between the center position of the small target and each position on the feature map, and sets the preset distance threshold to d. If the distance between the center position of the small target and the remaining targets is less than or equal to the preset distance threshold "d", the target is assigned a value of 1. If the distance between the center position of the small target and the remaining targets is greater than the preset distance threshold "d", the target is assigned a value of 0, thereby eliminating irrelevant elements to obtain an alternative detection frame, and the alternative detection frame is sparsely detected by the query head. The query is performed to extract the pre-selected position of the candidate detection box, and the pre-selected position of the candidate detection box is mapped to the corresponding position in the feature map of the previous layer, and the four nearest neighbors of the corresponding position are used as the key position. The value sparse vector is constructed through the key position, and the target detection is performed through the sparse vector to obtain the target detection box. Through the step-by-step query optimization from low to high layers, it is ensured that the output of each level accurately affects the input of the next level, thereby greatly improving the processing speed while maintaining high accuracy. By using the query head to predict the rough position of small targets on the low-resolution feature map, and then using this position information to guide the calculation of the detection head on the high-resolution feature map. This mechanism not only improves the detection speed, but also significantly improves the detection accuracy and efficiency of the system by focusing on the calculation of high-probability target areas.

[0055] In one embodiment of the present application, non-maximum suppression is performed on the target detection frame, and overlapping detection frames are removed to obtain a target detection result, including: arranging the target detection frames corresponding to the same category of targets in descending order based on the confidence of the target detection frame to obtain a plurality of target detection frame sequences; marking the target detection frame with the highest confidence in the target detection frame sequence as a selected detection frame, and adding the selected detection frame to the target detection result; calculating the overlap value between the remaining detection frames in the target detection frame sequence and the selected detection frame, removing the target detection frames greater than the preset overlap value, and removing the detection frames overlapping with the same category of targets until the target detection frame sequence is empty. After determining the target position and category, a non-maximum suppression (NMS) processing method is used to optimize the detection result, remove the overlapping detection frames, and retain only the detection result with the highest confidence. Non-maximum suppression can effectively reduce duplicate detection results and improve the accuracy and efficiency of target detection.

[0056] In one embodiment of the present application, the target detection results are formatted into JSON format and transmitted in real time to the auxiliary decision system of the autonomous driving vehicle for dynamically evaluating potential threats and guiding the implementation of safe driving measures. The auxiliary decision system parses the received data in real time, evaluates potential risks, and automatically takes safety measures, such as adjusting the vehicle speed or changing the driving trajectory, to ensure driving safety in real time.

[0057] Figure 4 FIG. 1 is a block diagram of a vehicle traveling target object detection device according to an exemplary embodiment of the present application. Figure 4As shown, according to an embodiment of the present application, a vehicle driving target object detection device 400 includes: an acquisition module 410, an identification module 420, a fusion module 430, a positioning module 440 and a deduplication module 450. The acquisition module is used to acquire a driving image to be detected; the identification module is used to perform feature recognition on the driving image to be detected through the backbone network in the target object detection model, and extract basic features through a deep separable convolution layer to obtain a multi-level feature map; the fusion module is used to perform feature map fusion on the multi-scale feature map to obtain a fused feature map, and perform convolution processing on the fused feature map to obtain a driving feature map, which includes a low-resolution feature map and a high-resolution feature map; the positioning module is used to locate the initial detection frame corresponding to the target object through the low-resolution feature map, and refine the initial detection frame through the high-resolution feature map to obtain a target detection frame, which includes a confidence and a target object; the deduplication module is used to perform non-maximum suppression on the target detection frame, eliminate overlapping detection frames, and obtain a target detection result. Acquire a driving image to be detected; obtain a driving feature map through a target object detection model, the driving feature map including a low-resolution feature map and a high-resolution feature map; locate an initial detection frame corresponding to the target object through the low-resolution feature map, refine the initial detection frame through the high-resolution feature map to obtain a target detection frame, perform non-maximum suppression on the target detection frame, eliminate overlapping detection frames, and obtain a target detection result. The present invention effectively utilizes deep learning technology and complex computer vision strategies to optimize the recognition process of the target object, locates the initial detection frame corresponding to the target object through the low-resolution feature map, and refines the initial detection frame through the high-resolution feature map, thereby reducing misrecognition caused by environmental factors and achieving rapid reasoning capability at a relatively low computing cost, thereby providing a solid safety guarantee for autonomous driving vehicles under visually restricted conditions.

[0058] In one embodiment of the present application, the acquisition module is used to install an infrared camera on the target vehicle, and collect night driving images in real time through the infrared camera; the night driving images are normalized, and the contrast of the normalized night driving images is enhanced through Gaussian filtering denoising and histogram equalization technology to obtain the driving image to be detected.

[0059] In one embodiment of the present application, the recognition module is used to independently perform spatial convolution on each channel of the driving image to be detected, and normalize the driving image to be detected after the spatial convolution to obtain an initial feature map, integrate the information of the initial feature map across channels through a deep convolution kernel, and normalize the initial feature map after the information integration to obtain a combined feature map; extract features according to multiple depth levels in the combined feature map to obtain a multi-level feature map.

[0060] In one embodiment of the present application, a fusion module is used to convolve the upsampling results of adjacent high layers with the sum of the feature maps of the current layer based on a multi-level feature map through a top-down path and a bottom-up path to obtain a fused feature map; the fused feature map is convolved to obtain a driving feature map.

[0061] In one embodiment of the present application, the positioning module is used to search in a low-resolution feature map through a query head to obtain a heat map, which includes multiple target detection boxes; the target detection box that is smaller than a preset threshold is determined as the initial detection box, and the preset threshold represents the minimum anchor box scale or minimum regression range of the driving feature map.

[0062] In one embodiment of the present application, the positioning module is used to map the target detection frame in the low-resolution feature map to the high-resolution feature map, extract the center position of the target in the initial detection frame in the high-resolution feature map, and calculate the positioning distance between the center position and the remaining targets; mark the targets corresponding to the positioning distance less than or equal to the preset distance threshold as alternative detection frames, and eliminate the targets corresponding to the positioning distance greater than the preset threshold; query the alternative detection frame through the sparse detection head, extract the key position of the alternative detection frame, and obtain the target detection frame.

[0063] In one embodiment of the present application, a deduplication module is used to arrange the target detection frames corresponding to the same category of targets in descending order based on the confidence of the target detection frames to obtain multiple target detection frame sequences; mark the target detection frame with the highest confidence in the target detection frame sequence as a selected detection frame, and add the selected detection frame to the target detection result; calculate the overlap value between the remaining detection frames in the target detection frame sequence and the selected detection frame, and remove the target detection frames whose overlap value is greater than a preset value to remove the detection frames overlapping with the same category of targets until the target detection frame sequence is empty.

[0064] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0065] See also Figure 5 , Figure 5 is a schematic diagram of a vehicle-mounted terminal according to an exemplary embodiment of the present application. It should be noted that: Figure 5 The vehicle-mounted terminal 500 shown is only an example and should not bring any limitation to the functions and usage area of ​​the embodiments of the present application.

[0066] like Figure 5 As shown, the vehicle terminal 500 includes a processor 501, a memory 502 and a communication bus 503; the communication bus 503 is used to connect the processor 501 and the memory 502; the processor 501 is used to execute the computer program stored in the memory 502 to implement one or more methods as in the above embodiments.

[0067] The vehicle-mounted terminal provided in the present application includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic device executes the various steps of the above method.

[0068] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0069] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0070] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for detecting a moving vehicle target, characterized in that: The method comprises: Acquire a driving image to be detected; Performing feature recognition on the driving image to be detected through a backbone network in a target object detection model, and extracting basic features through a depth-separable convolutional layer to obtain a multi-level feature map; Performing feature map fusion on the multi-scale feature map to obtain a fused feature map, and performing convolution processing on the fused feature map to obtain a driving feature map, wherein the driving feature map includes a low-resolution feature map and a high-resolution feature map; An initial detection frame corresponding to the target object is located by using the low-resolution feature map, and the initial detection frame is refined by using the high-resolution feature map to obtain a target detection frame, wherein the target detection frame includes a confidence level and a target object; Non-maximum suppression is performed on the target detection frame, overlapping detection frames are removed, and a target detection result is obtained.

2. The vehicle moving target object detection method according to claim 1, characterized in that: Obtain the driving image to be detected, including: An infrared camera is installed on the target vehicle, and night-time driving images are collected in real time by the infrared camera; The night driving image is normalized, and the contrast of the normalized night driving image is enhanced by Gaussian filtering denoising and histogram equalization technology to obtain a driving image to be detected.

3. The vehicle moving target object detection method according to claim 1, characterized in that: The backbone network in the target object detection model is used to perform feature recognition on the driving image to be detected, and the basic features are extracted through the depth-separable convolution layer to obtain a multi-level feature map, including: Performing spatial convolution independently on each channel of the to-be-detected driving image, and normalizing the to-be-detected driving image after the spatial convolution to obtain an initial feature map, integrating information across channels of the initial feature map through a deep convolution kernel, and normalizing the initial feature map after the information integration to obtain a combined feature map; In the combined feature map, features are extracted according to multiple depth levels to obtain a multi-level feature map.

4. The vehicle moving target object detection method according to claim 1, characterized in that: The multi-scale feature map is subjected to feature map fusion to obtain a fused feature map, and the fused feature map is subjected to convolution processing to obtain a driving feature map, including: Based on the multi-level feature map, convolution processing is performed on the upsampling result of the adjacent high-level layers and the sum of the feature map of the current level through the path from the top layer downward and the bottom layer upward to obtain a fused feature map; The fused feature map is convolved to obtain a driving feature map.

5. The vehicle moving target object detection method according to claim 1, characterized in that: Locating an initial detection frame corresponding to the target object using the low-resolution feature map includes: Searching the low-resolution feature map by a query head to obtain a heat map, wherein the heat map includes a plurality of target object detection boxes; An object detection frame smaller than a preset threshold is determined as an initial detection frame, where the preset threshold represents a minimum anchor frame scale or a minimum regression range of the driving feature map.

6. The vehicle moving target object detection method according to claim 5, characterized in that: The initial detection frame is refined by the high-resolution feature map to obtain a target detection frame, including: Mapping the target object detection frame in the low-resolution feature map to the high-resolution feature map, extracting the center position of the target object in the initial detection frame in the high-resolution feature map, and calculating the positioning distance between the center position and the remaining targets; Marking the target objects corresponding to the positioning distance less than or equal to the preset distance threshold as candidate detection frames, and eliminating the target objects corresponding to the positioning distance greater than the preset distance threshold; The candidate detection frame is queried through a sparse detection head, key positions of the candidate detection frame are extracted, and a target detection frame is obtained.

7. The vehicle moving target object detection method according to claim 1, characterized in that: The target detection frame is subjected to non-maximum suppression, and overlapping detection frames are removed to obtain a target detection result, including: Arrange the target detection frames corresponding to the same category of objects in descending order based on the confidence of the target detection frames to obtain multiple target detection frame sequences; Marking the target detection frame with the highest confidence in the target detection frame sequence as a selected detection frame, and adding the selected detection frame to the target detection result; The overlap values ​​of the remaining detection frames in the target detection frame sequence and the selected detection frame are calculated, and the target detection frames with an overlap value greater than a preset value are removed to remove the detection frames overlapping with the same category of objects, until the target detection frame sequence is empty.

8. A vehicle moving target object detection device, characterized in that: The device comprises: An acquisition module, used for acquiring a driving image to be detected; A recognition module, used to perform feature recognition on the driving image to be detected through the backbone network in the target object detection model, and extract basic features through a depth-separable convolutional layer to obtain a multi-level feature map; A fusion module, used for performing feature map fusion on the multi-scale feature map to obtain a fused feature map, and performing convolution processing on the fused feature map to obtain a driving feature map, wherein the driving feature map includes a low-resolution feature map and a high-resolution feature map; A positioning module, used to locate an initial detection frame corresponding to the target object through the low-resolution feature map, and refine the initial detection frame through the high-resolution feature map to obtain a target detection frame, wherein the target detection frame includes a confidence level and a target object; The deduplication module is used to perform non-maximum suppression on the target detection frame, remove overlapping detection frames, and obtain the target detection result.

9. A vehicle-mounted terminal, characterized in that: It comprises a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the vehicle moving target object detection method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that: It includes the vehicle moving target object detection device as described in claim 8 or the vehicle-mounted terminal as described in claim 9.

Images