Target detection method, device, storage medium, electronic device and vehicle

Abstract

The present disclosure relates to a target detection method, device, storage medium, electronic device and vehicle, the method comprising: obtaining a feature map and a depth estimation map corresponding to an image to be detected; determining a depth convolution kernel according to the feature map and the depth estimation map; determining a candidate region in the image to be detected through a pre-trained region candidate network model according to the feature map and the depth convolution kernel; and detecting a target object in the image to be detected according to the candidate region. That is, the present disclosure can determine a depth convolution kernel according to the feature map and the depth estimation map corresponding to the image to be detected, and process the feature map corresponding to the image to be detected according to the depth convolution kernel. Since the depth convolution kernel can perceive the depth information of the image to be detected, it overcomes the limitation of two-dimensional convolution, can accurately identify semi-occluded objects in the image to be detected, and improves the accuracy of target detection.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent connected vehicles, and in particular to a target detection method, device, storage medium, electronic equipment and vehicle. Background Art

[0002] Object detection is an image segmentation technology based on the geometric statistics and features of the object. It has a wide range of applications, for example, it can be applied to robotics or autonomous driving. At present, 2D object detection technology is gradually maturing, and some academic achievements have been put into commercial use. However, due to the lack of real-world geometric information such as depth and size, 2D object detection cannot meet the needs of practical applications. In addition, in the physical world, objects often contain geometric information such as length, width, height, and orientation angle.

[0003] In the related technology, based on 2D object detection, real information such as the size and posture of the object in three-dimensional space is provided to achieve 3D object detection. However, the accuracy of 3D object detection for occluded and truncated objects is relatively low. Therefore, how to improve the accuracy of object detection has become an urgent problem to be solved. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a target detection method, device, storage medium, electronic device and vehicle.

[0005] According to a first aspect of an embodiment of the present disclosure, a target detection method is provided, the method comprising:

[0006] Obtain the feature map and depth estimation map corresponding to the image to be detected;

[0007] Determining a depth convolution kernel according to the feature map and the depth estimation map;

[0008] According to the feature map and the deep convolution kernel, a candidate region in the image to be detected is determined by a pre-trained region candidate network model;

[0009] Detect the target object in the image to be detected according to the candidate area.

[0010] Optionally, obtaining a feature map corresponding to the image to be detected includes:

[0011] The image to be detected is input into a pre-trained feature map acquisition model to obtain the feature map output by the feature map acquisition model.

[0012] Optionally, determining a depth convolution kernel according to the feature map and the depth estimation map includes:

[0013] The feature map and the depth estimation map are input into a pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model.

[0014] Optionally, the region candidate network model includes a convolution layer, and determining the candidate region in the image to be detected through a pre-trained region candidate network model according to the feature map and the deep convolution kernel includes:

[0015] The feature map and the deep convolution kernel are input into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

[0016] Optionally, detecting the target object in the image to be detected according to the candidate area includes:

[0017] The candidate region is input into a pre-trained target detection model to obtain the type and bounding box of the target object output by the target detection model.

[0018] According to a second aspect of an embodiment of the present disclosure, there is provided a target detection device, the device comprising:

[0019] An acquisition module is configured to acquire a feature map and a depth estimation map corresponding to the image to be detected;

[0020] A convolution kernel determination module is configured to determine a depth convolution kernel according to the feature map and the depth estimation map;

[0021] A region determination module is configured to determine a candidate region in the image to be detected according to the feature map and the deep convolution kernel through a pre-trained region candidate network model;

[0022] The detection module is configured to detect the target object in the image to be detected based on the candidate area.

[0023] Optionally, the acquisition module is further configured to:

[0024] The image to be detected is input into a pre-trained feature map acquisition model to obtain the feature map output by the feature map acquisition model.

[0025] Optionally, the convolution kernel determination module is further configured to:

[0026] The feature map and the depth estimation map are input into a pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model.

[0027] Optionally, the region candidate network model includes a convolutional layer, and the region determination module is further configured to:

[0028] The feature map and the deep convolution kernel are input into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

[0029] Optionally, the detection module is further configured to:

[0030] The candidate region is input into a pre-trained target detection model to obtain the type and bounding box of the target object output by the target detection model.

[0031] Optionally, the region candidate network model is trained in the following manner:

[0032] Acquire multiple sample sets, wherein the sample sets include sample feature maps and sample depth estimation maps corresponding to the sample images;

[0033] The target neural network model is trained by using a plurality of the sample sets to obtain the region candidate network model.

[0034] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the target detection method provided in the first aspect of the present disclosure are implemented.

[0035] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0036] a memory having a computer program stored thereon;

[0037] A processor is used to execute the computer program in the memory to implement the steps of the target detection method provided in the first aspect of the present disclosure.

[0038] According to a fifth aspect of an embodiment of the present disclosure, a vehicle is provided, comprising: a vehicle body and the electronic device described in the fourth aspect.

[0039] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by obtaining the feature map and the depth estimation map corresponding to the image to be detected; determining the depth convolution kernel according to the feature map and the depth estimation map; determining the candidate region in the image to be detected through a pre-trained region candidate network model according to the feature map and the depth convolution kernel; detecting the target object in the image to be detected according to the candidate region. In other words, the present disclosure can determine the depth convolution kernel according to the feature map and the depth estimation map corresponding to the image to be detected, and process the feature map corresponding to the image to be detected according to the depth convolution kernel. Since the depth convolution kernel can perceive the depth information of the image to be detected, it overcomes the limitations of two-dimensional convolution, can accurately identify the semi-occluded objects in the image to be detected, and improves the accuracy of target detection.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0042] Figure 1 is a flow chart of a target detection method according to an exemplary embodiment of the present disclosure;

[0043] Figure 2 is a schematic diagram of target detection according to an exemplary embodiment of the present disclosure;

[0044] Figure 3 is a schematic diagram of a frame according to an exemplary embodiment of the present disclosure;

[0045] Figure 4 is a flow chart of another target detection method according to an exemplary embodiment of the present disclosure;

[0046] Figure 5 is a block diagram of a target detection device according to an exemplary embodiment of the present disclosure;

[0047] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0049] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0050] Before introducing the specific implementation methods of the present disclosure in detail, the application scenarios of the present disclosure are first described. At present, 3D target detection based on monocular images includes direct target detection methods and target detection methods with additional information. The direct target detection method is to regress the three-dimensional box through geometric constraints, and does not explicitly express the depth information missing from the monocular image, so it relies heavily on prior knowledge and the accuracy of 2D target detection. Based on this, the direct target detection method has a relatively large error and low precision. The target detection method with additional information is to convert the depth estimation map predicted by the two-dimensional image into pseudo-lidar data or BEV (Bird Eye View). Compared with the direct target detection method, the detection performance of the target detection method with additional information is improved. However, the inventors of the present disclosure found that the target detection method with additional information is limited by the gap between the predicted depth and the real-world depth, and the accuracy of target detection is still relatively low. Moreover, this method cannot effectively utilize the high-level semantic information extracted from the RGB image, resulting in misdetection of targets such as traffic light signs and lane lines.

[0051] In order to overcome the technical problems existing in the above-mentioned related technologies, the present disclosure provides a target detection method, device, storage medium, electronic device and vehicle, which can determine the depth convolution kernel according to the feature map and depth estimation map corresponding to the image to be detected, and process the feature map corresponding to the image to be detected according to the depth convolution kernel. Since the depth convolution kernel can perceive the depth information of the image to be detected, it overcomes the limitations of two-dimensional convolution, can accurately identify semi-occluded objects in the image to be detected, and improves the accuracy of target detection.

[0052] The present disclosure is described below in conjunction with specific embodiments.

[0053] Figure 1 is a flow chart of a target detection method according to an exemplary embodiment of the present disclosure. Figure 1 As shown, the method may include:

[0054] S101, obtaining a feature map and a depth estimation map corresponding to the image to be detected.

[0055] Among them, the image to be detected can be an RGB image. Taking the autonomous driving scenario as an example, the image to be detected can be an RGB image taken by the vehicle during driving; the feature map can be an image that can reflect the characteristics of the object to be detected in the image to be detected. The object to be detected can be determined according to the application scenario. For example, if the application scenario is autonomous driving, the object to be detected can be a car, a pedestrian, a traffic light, etc.

[0056] In this step, after acquiring the image to be detected, the image to be detected can be input into a pre-trained feature map acquisition model to obtain the feature map output by the feature map acquisition model. Among them, the feature map acquisition model can be a model obtained by training the first neural network model based on the ImageNet classification data set through the model training method of the prior art, which will not be repeated here. The first neural network model can include modules other than the last full convolution layer and pooling layer in the ResNet-50 network model. The first neural network model disclosed in the present invention can also embed an attention mechanism on the basis of the ResNet-50 network model. Based on the attention mechanism, the feature map acquired by the feature map acquisition model can be connected to the context information of the neighboring pixels of the target object in the image to be detected.

[0057] It should be noted that the above method of obtaining the feature map corresponding to the image to be detected is only one implementation method, and the feature map can also be obtained by other methods of related technologies, and the present disclosure does not limit this.

[0058] After acquiring the image to be detected, the depth estimation map corresponding to the image to be detected can be acquired by the DORN algorithm. In addition, the present disclosure does not limit the order of acquiring the feature map and the depth estimation map.

[0059] S102: Determine a depth convolution kernel according to the feature map and the depth estimation map.

[0060] In this step, after obtaining the feature map and depth estimation map corresponding to the image to be detected, the feature map and the depth estimation map can be input into the pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model. Among them, the convolution kernel generation model can be a model obtained by training the second neural network model through the model training method of the prior art, which will not be repeated here. The second neural network model can include the first three modules (conv1, conv2_x, conv3_x, conv4_x) in the ResNet-50 network model.

[0061] S103: According to the feature map and the deep convolution kernel, a candidate region in the image to be detected is determined by a pre-trained region candidate network model.

[0062] Among them, the region candidate network model may include a convolutional layer.

[0063] In this step, after obtaining the deep convolution kernel, the feature map and the deep convolution kernel can be input into the region candidate network model, so that the convolution layer of the region candidate network model convolves the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

[0064] For example, Figure 2 is a schematic diagram of target detection according to an exemplary embodiment of the present disclosure, such as Figure 2 As shown, after the image to be detected is input into the feature map acquisition model, the feature map corresponding to the image to be detected is obtained, and after the feature map and the depth estimation map are input into the convolution kernel generation model, the target convolution kernel is obtained, and the feature map and the target convolution kernel are input into the region candidate network model to obtain the candidate region in the image to be detected.

[0065] Among them, the region candidate network model can be trained in the following way: obtain multiple sample sets, the sample sets include sample feature maps and sample depth estimation maps corresponding to the sample images; train the target neural network model through multiple sample sets to obtain the region candidate network model.

[0066] For example, after obtaining multiple sample sets, the model training step can be executed repeatedly until the trained target neural network model meets the preset stop iteration condition according to the target candidate area, and the trained target neural network model is used as the candidate network model of the area, wherein the multiple sample sets can be KITTI and NuScenes data sets, and the preset stop iteration condition can be a stop iteration condition commonly used in related technologies, which is not limited in the present disclosure. The model training step may include:

[0067] S1. Input the sample feature map and the sample depth estimation map into the target neural network model to obtain the target candidate area output by the target neural network model.

[0068] S2. When it is determined that the target neural network model does not meet the preset stop iteration condition according to the target candidate area, a target loss value is determined according to the target candidate area, and the parameters of the target neural network model are updated according to the target loss value to obtain a trained target neural network model, and the trained target neural network model is used as a new target neural network model. For example, the parameters of the target neural network model can be updated by an SGD (Stochastic Gradient Descent) algorithm.

[0069] S104: Detect the target object in the image to be detected according to the candidate area.

[0070] In this step, after determining the candidate region in the image to be detected, a target region with a high probability of belonging to the foreground can be screened out from the candidate region, and the type of the object in the target region and the frame corresponding to the object can be determined. Figure 3 is a schematic diagram of a frame according to an exemplary embodiment of the present disclosure, such as Figure 3 As shown, the type of the object is a car, and the white line frame is the border corresponding to the object, which is a visualized three-dimensional frame.

[0071] By adopting the above method, the depth convolution kernel can be determined according to the feature map and depth estimation map corresponding to the image to be detected, and the feature map corresponding to the image to be detected can be processed according to the depth convolution kernel. Since the depth convolution kernel can perceive the depth information of the image to be detected, it overcomes the limitations of two-dimensional convolution, can accurately identify semi-occluded objects in the image to be detected, and improves the accuracy of target detection.

[0072] Figure 4 is a flowchart of another target detection method according to an exemplary embodiment of the present disclosure, such as Figure 4 As shown, the method may include:

[0073] S401. Input the image to be detected into a pre-trained feature map acquisition model to obtain a feature map output by the feature map acquisition model.

[0074] Among them, the image to be detected can be an RGB image. Taking the autonomous driving scenario as an example, the image to be detected can be an RGB image taken by the vehicle during driving; the feature map can be an image that can reflect the characteristics of the object to be detected in the image to be detected. The object to be detected can be determined according to the application scenario. For example, if the application scenario is autonomous driving, the object to be detected can be a car, a pedestrian, a traffic light, etc.

[0075] S402: Obtain a depth estimation map corresponding to the image to be detected.

[0076] S403: Input the feature map and the depth estimation map into a pre-trained convolution kernel generation model to obtain a depth convolution kernel output by the convolution kernel generation model.

[0077] S404: Input the feature map and the deep convolution kernel into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

[0078] Among them, the region candidate network model may include a convolutional layer.

[0079] S405: Input the candidate region into a pre-trained target detection model to obtain the type and bounding box of the target object output by the target detection model.

[0080] Among them, the target detection model can be a 2D-3D single-stage detection based on YOLO (You Only Look Once), and the model is trained by the model training method of the relevant technology, which will not be repeated here.

[0081] By adopting the above method, the deep convolution kernel can be determined according to the feature map and depth estimation map corresponding to the image to be detected, and the feature map corresponding to the image to be detected can be processed according to the deep convolution kernel. Since the deep convolution kernel can perceive the depth information of the image to be detected, the limitation of two-dimensional convolution can be overcome, and the semi-occluded object in the image to be detected can be accurately identified, thereby improving the accuracy of target detection. Furthermore, the feature map obtained by the present disclosure can be linked to the context information of the neighboring pixels of the target object in the image to be detected, so that the deep convolution kernel obtained according to the feature map can effectively extract the features of the local context information of the semi-occluded object, and the recognition of the semi-occluded object is more accurate, thereby further improving the accuracy of target detection. In addition, the target detection method disclosed in the present disclosure simplifies the network structure, so that the computational overhead in the detection process is relatively small, thereby saving computing resources.

[0082] Figure 5 is a block diagram of a target detection device according to an exemplary embodiment of the present disclosure. Figure 5 As shown, the device may include:

[0083] The acquisition module 501 is configured to acquire a feature map and a depth estimation map corresponding to the image to be detected;

[0084] A convolution kernel determination module 502 is configured to determine a depth convolution kernel according to the feature map and the depth estimation map;

[0085] The region determination module 503 is configured to determine the candidate region in the image to be detected according to the feature map and the deep convolution kernel through a pre-trained region candidate network model;

[0086] The detection module 504 is configured to detect the target object in the image to be detected according to the candidate area.

[0087] Optionally, the acquisition module 501 is further configured to:

[0088] The image to be detected is input into a pre-trained feature map acquisition model to obtain the feature map output by the feature map acquisition model.

[0089] Optionally, the convolution kernel determination module 502 is further configured to:

[0090] The feature map and the depth estimation map are input into a pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model.

[0091] Optionally, the region candidate network model includes a convolutional layer, and the region determination module 503 is further configured to:

[0092] The feature map and the deep convolution kernel are input into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

[0093] Optionally, the detection module 504 is further configured to:

[0094] The candidate region is input into a pre-trained target detection model to obtain the type and bounding box of the target object output by the target detection model.

[0095] Optionally, the region candidate network model is trained in the following way:

[0096] Acquire multiple sample sets, the sample sets including sample feature maps and sample depth estimation maps corresponding to the sample images;

[0097] The target neural network model is trained through multiple sample sets to obtain a candidate network model for the region.

[0098] Through the above-mentioned device, the depth convolution kernel can be determined according to the feature map and depth estimation map corresponding to the image to be detected, and the feature map corresponding to the image to be detected can be processed according to the depth convolution kernel. Since the depth convolution kernel can perceive the depth information of the image to be detected, it overcomes the limitations of two-dimensional convolution, can accurately identify semi-occluded objects in the image to be detected, and improves the accuracy of target detection.

[0099] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0100] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the target detection method provided by the present disclosure are implemented.

[0101] Figure 6 1 is a block diagram of an electronic device 600 according to an exemplary embodiment of the present disclosure. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0102] Reference Figure 6 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0103] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned target detection method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0104] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0105] The power component 606 provides power to the various components of the electronic device 600. The power component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0106] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0107] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0108] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0109] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or a component of the electronic device 600, the presence or absence of contact between the user and the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0110] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0111] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned target detection method.

[0112] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of an electronic device 600 to perform the above-mentioned target detection method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0113] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above target detection method when executed by the programmable device.

[0114] According to an embodiment of the present disclosure, a vehicle is also provided, including: a vehicle body and the above-mentioned electronic device 600.

[0115] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0116] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A target detection method, characterized in that: The method comprises: Obtain the feature map and depth estimation map corresponding to the image to be detected; Determining a depth convolution kernel according to the feature map and the depth estimation map; According to the feature map and the deep convolution kernel, a candidate region in the image to be detected is determined by a pre-trained region candidate network model; Detecting a target object in the image to be detected according to the candidate area; Determining the depth convolution kernel according to the feature map and the depth estimation map includes: Inputting the feature map and the depth estimation map into a pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model; The region candidate network model includes a convolution layer, and determining the candidate region in the image to be detected through a pre-trained region candidate network model according to the feature map and the deep convolution kernel includes: The feature map and the deep convolution kernel are input into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

2. The method according to claim 1, characterized in that The obtaining of a feature map corresponding to the image to be detected comprises: The image to be detected is input into a pre-trained feature map acquisition model to obtain the feature map output by the feature map acquisition model.

3. The method according to claim 1 or 2, characterized in that: The detecting the target object in the image to be detected according to the candidate area comprises: The candidate region is input into a pre-trained target detection model to obtain the type and bounding box of the target object output by the target detection model.

4. A target detection device, characterized in that: The device comprises: An acquisition module is configured to acquire a feature map and a depth estimation map corresponding to the image to be detected; A convolution kernel determination module is configured to determine a depth convolution kernel according to the feature map and the depth estimation map; A region determination module is configured to determine a candidate region in the image to be detected according to the feature map and the deep convolution kernel through a pre-trained region candidate network model, wherein the region candidate network model includes a convolution layer; A detection module, configured to detect a target object in the image to be detected according to the candidate area; The convolution kernel determination module is further configured to input the feature map and the depth estimation map into a pre-trained convolution kernel generation model to obtain the depth convolution kernel output by the convolution kernel generation model; The region determination module is also configured to input the feature map and the deep convolution kernel into the region candidate network model, so that the convolution layer of the region candidate network model performs convolution processing on the feature map according to the deep convolution kernel to obtain the candidate region in the image to be detected.

5. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 3 are implemented.

6. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 3.

7. A vehicle, characterized in that: The device comprises a vehicle body and the electronic device as claimed in claim 6.

Images