Target detection model construction method, target detection method, device and computing equipment

Abstract

The present disclosure provides a target detection model construction method, a target detection method, a device and a computing device to solve the problem of poor small object detection performance of single-stage target detectors in existing solutions. The target detection model construction method comprises: constructing a feature extraction network, the feature extraction network being configured to perform feature extraction on an input image to obtain a plurality of feature maps, the plurality of feature maps comprising a first feature map and a second feature map; and constructing a target detection network, the target detection network comprising a plurality of network layers corresponding to the plurality of feature maps, the plurality of network layers comprising a first network layer and a second network layer; the first network layer being configured to perform a query operation on the first feature map and transmit a query result obtained to the second network layer, the query result comprising a query point of a specific target in the first feature map; and the second network layer being configured to determine a mapping region of the query point in the second feature map and perform a detection operation in the mapping region to obtain a detection result.

Description

Technical Field

[0001] This disclosure relates to the field of target detection, and in particular to a method for constructing a target detection model, a target detection method, an apparatus, and a computing device. Background Technology

[0002] In recent years, object detectors based on deep neural networks have achieved great success. Common object detectors can be divided into single-stage detectors and two-stage detectors. Among them, single-stage detectors have advantages such as simple structure and fast inference speed. For example, the anchor-based RetinaNet object detector pre-defines candidate boxes of specific sizes and shapes in the image, and then uses a neural network to classify and regress these candidate boxes to perform object detection. However, existing single-stage object detectors do not perform well in detecting small objects. To improve the performance of small object detection, high-resolution input images and features are usually used, but this will bring huge computational costs and seriously slow down the inference speed. Therefore, there is a need to provide a more efficient and faster object detection method. Summary of the Invention

[0003] The embodiments of this disclosure provide a method for constructing a target detection model, a target detection method, an apparatus, and a computing device to improve the detection efficiency of a single-stage target detector for small objects.

[0004] To achieve the above objectives, the embodiments of this disclosure adopt the following technical solutions:

[0005] A first aspect of this disclosure provides a method for constructing a feature extraction network to extract features from an input image, resulting in multi-layer feature maps of different sizes, each multi-layer feature map including a first feature map and a second feature map. A target detection network is also provided, comprising multiple network layers corresponding to the multi-layer feature maps, each network layer including a first network layer and a second network layer. The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map, transmitting the obtained query result to the second network layer. The query result includes a query point of a specific target in the first feature map. The second network layer corresponds to the second feature map and is used to determine the mapping region of the query point in the second feature map, performing a detection operation within the mapping region to obtain a detection result.

[0006] A second aspect of this disclosure provides a target detection method, comprising: inputting an image to be detected into a target detection model, the target detection model including a feature extraction network and a target detection network; using the feature extraction network to extract features from the image to be detected, obtaining multi-layer feature maps of different sizes, the multi-layer feature maps including a first feature map and a second feature map; and using the target detection network to output detection results on the multi-layer feature maps, the target detection network including multiple network layers corresponding to the multi-layer feature maps, the multiple network layers including a first network layer and a second network layer; wherein, the first network layer corresponds to the first feature map, is used to perform a query operation on the first feature map, and transmits the obtained query result to the second network layer, the query result including a query point of a specific target in the first feature map; the second network layer corresponds to the second feature map, is used to determine the mapping region of the query point in the second feature map, and performs a detection operation within the mapping region to obtain a detection result.

[0007] A third aspect of this disclosure provides an apparatus for constructing an object detection model, comprising: a first construction unit for constructing a feature extraction network, the feature extraction network being used to extract features from an input image to obtain multi-layer feature maps of different sizes, the multi-layer feature maps including a first feature map and a second feature map; a second construction unit for constructing an object detection network, the object detection network including multiple network layers corresponding to the multi-layer feature maps, the multiple network layers including a first network layer and a second network layer; the first network layer corresponding to the first feature map being used to perform a query operation on the first feature map and transmit the obtained query result to the second network layer, the query result including a query point of a specific object in the first feature map; and the second network layer corresponding to the second feature map being used to determine the mapping region of the query point in the second feature map and perform a detection operation within the mapping region to obtain a detection result.

[0008] A fourth aspect of this disclosure provides a target detection apparatus, comprising: an input unit for inputting an image to be detected into a target detection model, the target detection model including a feature extraction network and a target detection network; a feature extraction unit for using the feature extraction network to extract features from the image to be detected, obtaining multi-layer feature maps of different sizes, the multi-layer feature maps including a first feature map and a second feature map; and a target detection unit for using the target detection network to output detection results on the multi-layer feature maps, the target detection network including multiple network layers corresponding to the multi-layer feature maps, the multiple network layers including a first network layer and a second network layer; wherein the first network layer corresponds to the first feature map, is used to perform a query operation on the first feature map, and transmits the obtained query result to the second network layer, the query result including a query point of a specific target in the first feature map; the second network layer corresponds to the second feature map, is used to determine the mapping region of the query point in the second feature map, and perform a detection operation within the mapping region to obtain a detection result.

[0009] A fifth aspect of this disclosure provides a computing device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor; wherein, when the processor runs the computer program, it executes the method for constructing a target detection model and / or the target detection method as described above.

[0010] A sixth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method for constructing a target detection model and / or the target detection method as described above.

[0011] A seventh aspect of this disclosure provides a vehicle including the computing device described above.

[0012] According to the technical solution of this disclosure, a novel single-stage object detector based on a query mechanism is proposed, which achieves rapid detection of small objects on high-resolution features. The basic idea is to first find the approximate location of the small object on the low-resolution feature map, i.e., the query point of the specific target, so as to perform mapping and detection on the high-resolution feature map based on the query point. This disclosure also includes a query network added to the detection head, which is used to determine whether there is an object smaller than the size threshold of the layer at a certain location. Simultaneously, this disclosure can also be combined with sparse convolution to improve inference speed. Based on these query points, a sparse feature tensor is constructed using high-resolution features, and the result is calculated using sparse convolution. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A structural diagram of a vehicle 100 provided in an embodiment of this disclosure;

[0015] Figure 2 A flowchart of a method 200 for constructing a target detection model provided in this embodiment of the disclosure;

[0016] Figure 3 A schematic diagram of a target detection model provided in an embodiment of this disclosure;

[0017] Figure 4 A schematic diagram of another target detection model provided in an embodiment of this disclosure;

[0018] Figure 5 A flowchart of another target detection method 500 provided in this disclosure embodiment;

[0019] Figure 6 A schematic diagram illustrating the detection effect of a target detection method provided in an embodiment of this disclosure;

[0020] Figure 7 A structural diagram of a target detection model construction apparatus 700 provided in an embodiment of this disclosure;

[0021] Figure 8 A structural diagram of another target detection model device 800 provided in this embodiment of the present disclosure;

[0022] Figure 9 This is a structural diagram of a computing device 900 provided in an embodiment of the present disclosure. Detailed Implementation

[0023] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Figure 1This is a schematic diagram of a vehicle 100 in which the various technologies disclosed herein can be implemented. Vehicle 100 can be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawnmower, excavator, snowmobile, aircraft, recreational vehicle, amusement park vehicle, farm equipment, construction equipment, tram, golf cart, train, trolleybus, or other vehicle. Vehicle 100 can operate fully or partially in an autonomous driving mode. In autonomous driving mode, vehicle 100 can control itself; for example, vehicle 100 can determine the current state of the vehicle and the current state of the environment in which the vehicle is located, determine the predicted behavior of at least one other vehicle in the environment, determine the trust level corresponding to the probability that the at least one other vehicle will perform the predicted behavior, and control vehicle 100 itself based on the determined information. In autonomous driving mode, vehicle 100 can operate without human interaction.

[0026] Vehicle 100 may include various vehicle systems, such as drive system 142, sensor system 144, control system 146, user interface system 148, control computer system 150, and communication system 152. Vehicle 100 may include more or fewer systems, and each system may include multiple units. Furthermore, each system and unit of vehicle 100 may be interconnected. For example, control computer system 150 is capable of data communication with one or more of vehicle systems 142-148 and 152. Thus, one or more of the described functions of vehicle 100 may be divided into additional functional components or physical components, or combined into a smaller number of functional components or physical components. In a further example, additional functional components or physical components may be increased to, for example... Figure 1 In the example shown.

[0027] The drive system 142 may include a plurality of operable components (or units) that provide kinetic energy to the vehicle 100. In one embodiment, the drive system 142 may include an engine or electric motor, wheels, a transmission, electronic systems, and a power source (or power source). The engine or electric motor may be any combination of internal combustion engines, electric motors, steam engines, fuel cell engines, propane engines, or other forms of engines or electric motors. In some embodiments, the engine may convert a power source into mechanical energy. In some embodiments, the drive system 142 may include multiple engines or electric motors. For example, a hybrid vehicle may include a gasoline engine and an electric motor, or other configurations may be included.

[0028] The wheels of vehicle 100 can be standard wheels. The wheels of vehicle 100 can be of various forms, including single-wheel, two-wheel, three-wheel, or four-wheeled, such as the four wheels on a car or truck. Other numbers of wheels are also possible, such as six or more wheels. One or more wheels of vehicle 100 can be operated to rotate in a different direction than the other wheels. A wheel can be at least one wheel fixedly connected to a transmission. The wheel can include a combination of metal and rubber, or other materials. The transmission can include units operable to transmit mechanical power from the engine to the wheels. For this purpose, the transmission can include a gearbox, clutch, differential gears, and driveshaft. The transmission can also include other units. The driveshaft can include one or more axles that match the wheels. The electronic system can include units for transmitting or controlling electronic signals of vehicle 100. These electronic signals can be used to activate multiple lights, multiple servo mechanisms, multiple electric motors, and other electronic drives or controls in vehicle 100. The power source can be an energy source that provides power to the engine or electric motor, either wholly or partially. That is, the engine or electric motor is capable of converting the power source into mechanical energy. For example, the power source may include gasoline, petroleum, petroleum-based fuels, propane, other compressed gaseous fuels, ethanol, fuel cells, solar panels, batteries, and other electrical energy sources. The power source may optionally include any combination of a fuel tank, battery, capacitor, or flywheel. The power source may also provide energy to other systems of vehicle 100.

[0029] Sensor system 144 may include multiple sensors for sensing information about the environment and conditions of vehicle 100. For example, sensor system 144 may include an inertial measurement unit (IMU), a global positioning system (GPS) transceiver, a radar (RADAR) unit, a laser rangefinder / LIDAR unit (or other distance measurement device), acoustic sensors, and cameras or image capture devices. Sensor system 144 may include multiple sensors for monitoring vehicle 100 (e.g., oxygen (O2) monitor, fuel gauge sensor, engine oil pressure sensor, etc.). Other sensors may also be configured. One or more sensors included in sensor system 144 may be driven individually or collectively to update the position, orientation, or both of the sensors.

[0030] The IMU may include a combination of sensors (e.g., accelerometers and gyroscopes) for sensing changes in the position and orientation of vehicle 100 based on inertial acceleration. The GPS transceiver may be any sensor used to estimate the geographic location of vehicle 100. For this purpose, the GPS transceiver may include a receiver / transmitter to provide position information of vehicle 100 relative to the Earth. It should be noted that GPS is an example of a Global Navigation Satellite System; therefore, in some embodiments, the GPS transceiver may be replaced with a BeiDou Navigation Satellite System transceiver or a Galileo Navigation Satellite System transceiver. The radar unit may use radio signals to sense objects in the environment in which vehicle 100 is located. In some embodiments, in addition to sensing objects, the radar unit may also be used to sense the speed and direction of travel of objects approaching vehicle 100. The laser rangefinder or LIDAR unit (or other distance measurement device) may be any sensor that uses lasers to sense objects in the environment in which vehicle 100 is located. In one embodiment, the laser rangefinder / LIDAR unit may include a laser source, a laser scanner, and a detector. The laser rangefinder / LIDAR unit is used to operate in continuous (e.g., using heterodyne detection) or discontinuous detection modes. The camera may include means for capturing multiple images of the environment in which the vehicle 100 is located. The camera may be a still image camera or a video camera.

[0031] The control system 146 is used to control the operation of the vehicle 100 and its components (or units). Accordingly, the control system 146 may include various units, such as a steering unit, a power control unit, a braking unit, and a navigation unit.

[0032] The steering unit may be a combination of mechanisms for adjusting the forward direction of vehicle 100. A power control unit (e.g., a throttle) may be used to control the engine speed, thereby controlling the speed of vehicle 100. The braking unit may include a combination of mechanisms for decelerating vehicle 100. The braking unit may utilize friction to decelerate the vehicle in a standard manner. In other embodiments, the braking unit may convert the kinetic energy of the wheels into electrical current. The braking unit may also take other forms. The navigation unit may be any system that determines a driving path or route for vehicle 100. The navigation unit may also dynamically update the driving path as vehicle 100 travels. The control system 146 may also additionally or optionally include other components (or units) not shown or described.

[0033] User interface system 148 can be used to allow vehicle 100 to interact with external sensors, other vehicles, other computer systems, and / or the user of vehicle 100. For example, user interface system 148 may include standard visual display devices (e.g., plasma displays, liquid crystal displays (LCDs), touchscreen displays, head-mounted displays, or other similar displays), speakers or other audio output devices, microphones or other audio input devices. For example, user interface system 148 may also include navigation interfaces and interfaces for controlling the internal environment of vehicle 100 (e.g., temperature, fan, etc.).

[0034] Communication system 152 can provide vehicle 100 with a means of communicating with one or more devices or other vehicles in the vicinity. In an exemplary embodiment, communication system 152 can communicate with one or more devices directly or through a communication network. Communication system 152 can be, for example, a wireless communication system. For example, the communication system can use 3G cellular communication (e.g., CDMA, EVDO, GSM / GPRS) or 4G cellular communication (e.g., WiMAX or LTE), and can also use 5G cellular communication. Optionally, the communication system can communicate with a wireless local area network (WLAN) (e.g., using...). In some embodiments, the communication system 152 can communicate directly with one or more devices or other vehicles in the vicinity, for example, using infrared light. Or ZigBee. Other wireless protocols, such as various vehicular communication systems, are also within the scope of this application. For example, the communication system may include one or more Dedicated Short Range Communication (DSRC) devices, V2V devices, or V2X devices that conduct public or private data communication with vehicles and / or roadside stations.

[0035] The control computer system 150 can control some or all of the functions of the vehicle 100. The autonomous driving control unit in the control computer system 150 can be used to identify, assess, and avoid or traverse potential obstacles in the environment in which the vehicle 100 is located. Typically, the autonomous driving control unit can be used to control the vehicle 100 without a driver or to assist a driver in controlling the vehicle. In some embodiments, the autonomous driving control unit is used to combine data from a GPS transceiver, radar data, LiDAR data, camera data, and data from other vehicle systems to determine the driving path or trajectory of the vehicle 100. The autonomous driving control unit can be activated to enable the vehicle 100 to be driven in autonomous driving mode.

[0036] The control computer system 150 may include at least one processor (which may include at least one microprocessor), which executes processing instructions (i.e., machine-executable instructions) stored in a non-volatile computer-readable medium (e.g., a data storage device or memory). The memory stores at least one machine-executable instruction, and the processor executes this instruction to implement functions including a map engine, a positioning module, a perception module, a navigation or path module, and an automatic control module. The map engine and positioning module provide map and positioning information. The perception module perceives objects in the vehicle's environment based on information acquired by the sensor system and map information provided by the map engine. The navigation or path module plans a driving path for the vehicle based on the processing results of the map engine, positioning module, and perception module. The automatic control module parses and converts the decision information input from modules such as the navigation or path module into control commands for the vehicle control system, and sends these commands to corresponding components in the vehicle control system via an in-vehicle network (e.g., an in-vehicle electronic network system implemented via CAN bus, local area network, multimedia orientation system transmission, etc.) to achieve automatic vehicle control; the automatic control module can also obtain information about various components in the vehicle via the in-vehicle network.

[0037] The control computer system 150 may also be multiple computing devices that distribute and control components or systems of the vehicle 100. In some embodiments, the memory may contain processing instructions (e.g., program logic) that are executed by a processor to implement various functions of the vehicle 100. In one embodiment, the control computer system 150 is capable of data communication with systems 142, 144, 146, 148, and / or 152. Interfaces within the control computer system facilitate data communication between the control computer system 150 and systems 142, 144, 146, 148, and 152.

[0038] The memory may also include other instructions, including instructions for sending data, instructions for receiving data, instructions for interaction, or instructions for controlling the drive system 140, sensor system 144, control system 146, or user interface system 148.

[0039] In addition to storing processing instructions, the memory can store various types of information or data, such as image processing parameters, road maps, and route information. This information can be used by the vehicle 100 and the control computer system 150 while the vehicle 100 is operating in automatic, semi-automatic, and / or manual mode.

[0040] Although the autonomous driving control unit is shown as separate from the processor and memory, it should be understood that in some embodiments, some or all of the functions of the autonomous driving control unit may be implemented using program code instructions residing in one or more memories (or data storage devices) and executed by one or more processors, and in some cases, the autonomous driving control unit may be implemented using the same processor and / or memory (or data storage device). In some embodiments, the autonomous driving control unit may be implemented at least in part using various special-purpose circuit logics, various processors, various field-programmable gate arrays (“FPGAs”), various application-specific integrated circuits (“ASICs”), various real-time controllers, and hardware.

[0041] The control computer system 150 can control the functions of the vehicle 100 based on inputs received from various vehicle systems (e.g., drive system 142, sensor system 144, and control system 146) or from the user interface system 148. For example, the control computer system 150 can use inputs from the control system 146 to control the steering unit to avoid obstacles detected by the sensor system 144. In one embodiment, the control computer system 150 can be used to control multiple aspects of the vehicle 100 and its systems.

[0042] Although Figure 1 The diagram shows various components (or units) integrated into vehicle 100, one or more of which may be mounted on or separately associated with vehicle 100. For example, a control computer system may exist partially or entirely independent of vehicle 100. Thus, vehicle 100 can exist as separate or integrated device units. The device units constituting vehicle 105 can communicate with each other via wired or wireless communication. In some embodiments, additional components or units may be added to or removed from various systems (e.g., ...). Figure 1 (LiDAR or radar shown).

[0043] As mentioned earlier, existing single-stage object detectors perform poorly in detecting small objects. Although the introduction of feature pyramids has significantly improved performance for small objects, the results are still unsatisfactory: the highest resolution feature of a typical feature pyramid is 1 / 8 of the input resolution, which is still insufficient for very small objects. While introducing higher resolution features can further alleviate this problem, it will bring a huge computational burden. For example, in RetinaNet, if features with a resolution of 1 / 4 of the input resolution are introduced, the computational burden on the detection head will increase by 300%, which will severely slow down the inference speed. Therefore, the embodiments of this disclosure aim to propose a more efficient object detection scheme.

[0044] like Figure 2 As shown in the figure, an embodiment of this disclosure provides a method 200 for constructing a target detection model, comprising:

[0045] Step S201: Construct a feature extraction network. This feature extraction network is used to extract features from the input image to obtain multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map and a second feature map.

[0046] Step S202: Construct an object detection network, which includes multiple network layers corresponding to multi-layer feature maps, and the multiple network layers include a first network layer and a second network layer.

[0047] The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map, transmitting the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The second network layer corresponds to the second feature map and is used to determine the mapping region of the query point in the second feature map, and perform a detection operation within the mapping region to obtain the detection result.

[0048] To enable those skilled in the art to better understand this disclosure, the embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings, examples, etc.

[0049] In one embodiment of this disclosure, the target detection model is as follows: Figure 3 and Figure 4 As shown, it includes a feature extraction network and an object detection network. The feature extraction network includes a backbone network and a feature pyramid network.

[0050] The backbone network is used to extract features from the input image, obtaining initial multi-layer feature maps. That is, the backbone network obtains initial multi-layer feature maps by downsampling the input image. The feature maps obtained by the downsampling operation have corresponding size scaling relationships, such as... Figure 3 The feature map of layer P2 in the backbone network is 1 / 4 the resolution of the input image, and the feature map of layer P3 is 1 / 2 the resolution of the feature map of layer P2.

[0051] The feature pyramid network is used to upsample and fuse features from the initial feature map, resulting in an improved multi-layer feature map. For example... Figure 3 The feature maps in the feature pyramid network are obtained by fusing feature maps from the backbone network, and the resulting improved multi-layer feature maps still have corresponding size scaling relationships. Here, a first feature map is defined, and multiple additional feature maps are obtained by downsampling from the first feature map. Upsampling is then performed starting from the first feature map, and the additional feature maps are fused with feature maps of the same level in the backbone network to obtain multiple high-resolution feature maps in sequence. Figure 3 In this model, layers P7 and P8 of the feature pyramid network are additional feature maps used for traditional object detection, while layers P1-P6 are query feature maps used for query-based object detection. The object is a target object in the image data, which can include static and dynamic objects such as pedestrians, vehicles, animals, obstacles, traffic lights, and road signs. Object detection involves using algorithms to locate the target object in the raw sensor data, typically represented by a rectangle or cuboid in 2D or 3D space.

[0052] It should be noted that this disclosure can perform object detection on the initial multi-layer feature map obtained from the backbone network, or on the improved multi-layer feature map obtained from the feature pyramid network. This disclosure does not restrict the source of the feature map for detection.

[0053] Each feature map layer has a corresponding size threshold, such as the minimum target size that each layer can detect, or the range of target sizes that each layer can detect. For feature maps with anchors, the size threshold includes the minimum and / or maximum anchor size of that layer. In this way, different feature maps output targets of different sizes. By merging these output targets of different sizes, the target detection result of the input image can be obtained.

[0054] Object detection networks consist of multiple network layers, each corresponding to a feature map. The layer corresponding to the additional feature map is called the additional network layer, and the layer corresponding to the query feature map is called the query layer. For example... Figure 3 The object detection network in the model consists of two additional network layers and four query layers.

[0055] The additional network layer is a traditional network layer, also known as a non-query layer, used to perform detection operations on the corresponding additional feature maps to obtain detection results. The detection network of the additional network layer includes a classification network and / or a regression network. The classification network and the regression network are used to perform target classification detection and regression detection on the corresponding additional feature maps, respectively, to obtain classification results and regression results. The classification result is, for example, the classification category, and the regression result is, for example, the regression bounding box.

[0056] The query layer is a network layer based on query operations. Each network layer in the query layer can implement one or more of the following functions: detect query points of a specific target in the feature map of this layer and transmit the detected query points to lower-level network layers; receive query points transmitted from higher-level network layers and map the query points to the mapped regions in the feature map of this layer. Here, the layer containing the low-resolution feature map is a higher-level layer, and the layer containing the high-resolution feature map is a lower-level layer, such as... Figure 3In this model, layer P2 is at a lower level than layer P3. Therefore, the resolution of the multi-level feature maps disclosed herein increases progressively from higher to lower levels, and there is a corresponding proportional relationship between the upper and lower feature maps.

[0057] According to one embodiment, the query layer includes a first network layer corresponding to a first feature map, a second network layer corresponding to a second feature map, a third network layer corresponding to a third feature map, ..., an i-th network layer corresponding to an i-th network layer, ..., an n-th network layer corresponding to an n-th feature map.

[0058] The first network layer is the starting layer of the query layer, used to perform query operations on the first feature map to obtain query results. The query results of the query layer include query points of a specific target in the corresponding layer's feature map. For example, the query results of the first network layer include query points of a specific target in the first feature map. Optionally, the query results include a query result map, where the query result map of a certain network layer has the same size as the feature map corresponding to that network layer, and the query result map includes the probability of a specific target existing at each location point in the corresponding feature map. Correspondingly, the query point is a location point in the feature map with a probability value greater than or equal to a preset threshold. This preset threshold can be set as needed, such as 0.5, and this disclosure does not limit it in this way.

[0059] The query operation can be performed by a query network, which is also a type of detection network (i.e., a detection head). A specific target is a target whose size is less than or equal to the size threshold of the corresponding layer's feature map. Taking the first feature map as an example, assuming the minimum size threshold of the first feature map is 'a', then the specific target is a target whose size is less than or equal to 'a'. Of course, a specific target can also be a target whose size is less than or equal to a predetermined multiple 'b' (b > 1) of the size threshold of the feature map at that layer; in this case, the specific target of the first feature map is a target whose size is less than or equal to a * b. This disclosure sets such an extended range for the size of specific targets in each layer's feature map, which is beneficial for generating a wider range of query points and improving the detection accuracy of the lower network layers.

[0060] The query point can be a location point contained within the feature target, a location point within the detection box of the specific target, or a point within a preset range of the center point of the specific target. This invention does not impose specific limitations on the range of query points for a specific target. The query point can be represented by coordinates, that is, the coordinate position of the query point in the first feature map.

[0061] The first network layer transmits the query result to the second network layer. The second network layer extracts the query point from the transmitted result and maps it to multiple mapping points in the first feature map, obtaining a mapped region. Generally, the second feature map is obtained by upsampling and feature fusion of the first feature map. The size of the second feature map is m times that of the first feature map, where m > 1. Therefore, the second network layer maps the query point to multiple points in the second feature map according to the magnification factor m between the first and second feature maps, obtaining a mapped region. If the second feature map is twice the size of the first feature map, the second network layer maps each received query point to a four-neighbor point in the second feature map according to the coordinate mapping relationship between the images; if the second feature map is three times the size of the first feature map, the second network layer maps each received query point to an eight-neighbor point in the second feature map, and so on. Here, the mapping operation of the query point by the second network layer can be performed by the mapping module. A query operation is then performed on the mapped region in the second feature map to obtain the corresponding query result.

[0062] The obtained query results are transmitted to the third network layer, so that the third network layer maps the query points in the query results to the corresponding mapping regions in the third feature map, and performs detection operations within the mapping regions to obtain the detection results.

[0063] Next, the second network layer performs a detection operation on the determined mapped region. This detection operation is also performed by a detection network, such as a classification network and / or a regression network, to perform classification detection and regression detection respectively. Generally, if a network layer receives query results from a higher-level network layer, that network layer usually has a mapping module and a detection network. The mapping module is used to map the query points within the query results, and the detection module is used to detect the mapped regions. Similarly, if a network layer performs a query operation, it will pass the query results to lower levels.

[0064] Here, the first network layer mainly includes a query network for performing query operations, and the second network layer mainly includes a detection network for performing detection operations. Optionally, the first network layer may also include a detection network for performing detection operations on the first feature map to obtain detection results. The second network layer may also include a query network for performing query operations on the corresponding second feature map and transmitting the query results to the third network layer. The query result output by each network layer is the query point of a specific target in the feature map of that layer. In this case, the third network layer must have a mapping module and a detection network to map the query points received from the second network layer into query regions and perform detection operations within these mapped regions.

[0065] Similarly, in addition to mapping and detection networks, the third network layer can also have a query network to perform query operations on the corresponding third feature map and transmit the query result to the fourth network layer. The fourth network layer then maps the query result into a mapped region for target detection, and so on, until the bottom layer of the query layer is reached. The bottom layer of the query layer only includes mapping and detection networks. This bottom layer only needs to obtain the mapped region and output the detection result, and no longer needs a query network.

[0066] It should be noted that this disclosure considers network layers that receive the same set of query points as the same set of network layers. That is, the second, third, and i-th network layers in the target detection network can all be multiple. Figure 4 (This example uses two features; there could actually be more). Feature maps at the same level as the second network layer are called second feature maps, those at the same level as the third network layer are called third feature maps, and those at the same level as the i-th network layer are called i-th feature maps. Each second network layer receives query results from the first network layer and maps the query results to the corresponding mapping region in the second feature map; each third network layer receives query results from the second network layer and maps the query results to the corresponding mapping region in the third feature map; and so on.

[0067] Specifically, after the first network layer performs a query operation on the first feature map, it obtains a first set of query points and passes these first set of query points to one or more second network layers. That is, the one or more second network layers receive the same set of query points and map the first query result to a mapped region in the corresponding layer's feature map, so as to perform a detection operation within that mapped region.

[0068] Similarly, after a second network layer performs a query operation on the second feature map of the same level, it obtains a second set of query points, which are then passed to one or more third network layers. At this time, the one or more third network layers respectively map the second set of query points to the corresponding mapping regions in the feature map of the same level, so as to perform detection operations within the mapping regions.

[0069] Here, multiple second network layers can perform query operations to obtain corresponding query results. Then, the scaling relationship between the feature maps of different layers is used to combine the multiple query results to obtain the final merged result. For example, based on the query point coordinates of each second network layer and the mapping relationship between the coordinates, the combined query point of the multiple second network layers is obtained and transmitted to the third network layer.

[0070] Similarly, after the i-th network layer performs a query operation on the i-th feature map of the same level, it obtains the i-th set of query points and passes them to one or more (i+1)-th network layers. At this time, the one or more (i+1)-th network layers respectively map the i-th set of query points to the corresponding mapping regions in the feature map of the level, so as to perform detection operations within the mapping regions.

[0071] Preferably, each query layer has both a query network and a detection network, and sequentially transmits the query results to the next lower-level network layer. At this point, the top layer of the query layer, i.e., the first network layer, outputs the query result of the first feature map and transmits it to the second network layer for mapping and detection operations. Simultaneously, the second network layer performs a query operation on the second feature map and transmits the query result to the third network layer for mapping and detection operations. That is, the intermediate layers of the query layer obtain query results from higher-level network layers for mapping and detection operations, and also perform query operations on the feature map of their own layer, transmitting the query results to lower-level network layers. This process continues until the bottom layer of the network layer is reached, where only mapping and detection operations are performed, and no further query operations are performed. In other words, for the top and intermediate layers of the query layer, the detection network includes a classification network, a regression network, and a query network, used to output classification results, regression results, and query results; while for the bottom layer of the query layer, the detection network only contains a classification network and a query network, used to output classification results and regression results.

[0072] According to one embodiment of this disclosure, in order to improve the overall detection efficiency of the object detection network, this disclosure employs sparse convolution to perform detection operations on the feature map. Specifically, each network layer performing the detection operation (such as the second network layer) extracts image features of the mapped region to construct a sparse tensor, and uses sparse convolution to perform detection and / or query operations within the mapped region.

[0073] For convolutional neural networks, the input is typically a set of four-dimensional tensors, corresponding to the four dimensions of batch, channel, height, and width. The convolutional kernel computes the result at every position in the height and width plane. However, sometimes only the computation result at a specific position in this plane is needed, making computation across the entire plane inefficient. To address this, sparse convolution was proposed to reduce computational cost and accelerate inference. In addition to the usual four-dimensional tensor input, additional positions for which computation is needed are specified, allowing the convolutional kernel to compute only at those designated positions.

[0074] When sparse convolution is applied in the query layer, for the highest layer (i.e., the first network layer), since it has no input from the query points in the upper layers, ordinary convolution is used to calculate the result. For the features of subsequent layers, the input query points are first mapped to the multi-neighbor locations of that layer, then features are extracted from all locations to construct sparse features, and finally the result for the queried location is calculated.

[0075] Alternatively, the sparse convolution can be replaced with a cropping operation, that is, cropping the features of the queried mapping region and calculating the convolution result on the cropped features.

[0076] As can be seen from the foregoing, the object detection network includes multiple network layers capable of outputting detection results, such as additional network layers, second network layers, third network layers, etc. Based on this, method 200 may further include the step of constructing an output network, which is used to merge and output the detection results obtained after the detection operations of each network layer. There are various ways to merge the outputs, such as using a weighted nonmaximum suppression method to merge multiple bounding boxes; this disclosure does not limit this approach.

[0077] In addition, such as Figure 3 As shown, based on the detection type, the object detection network comprises three branches: a classification network, a regression network, and a query network, which are used to perform classification, regression, and query operations, respectively. To improve the overall efficiency of model training, the classification, regression, and query networks of this disclosure share parameters across multiple network layers. That is, all classification networks have the same parameter types and values, all regression networks have the same parameter types and values, and all query networks have the same parameter types and values, enabling synchronous training of the same network branch.

[0078] Furthermore, this disclosure also includes a step of training the object detection model: generating a training sample set, and training the constructed object detection model based on the training sample set to obtain the trained object detection model. The training sample set includes multiple training images and annotations for each training image, the annotations including at least one of classification annotations and location annotations. At this point, the classification network branch can be iteratively updated based on the classification annotations of each training image and the predicted classification result, and the regression network branch can be iteratively updated based on the regression annotations of each training image and the predicted regression result.

[0079] In addition, the annotation of each training image can also include the query point annotations of each layer of feature maps generated from that training image. In this case, a loss function can be calculated based on the query point annotations of each layer of feature maps and the predicted query points of that layer. The constructed object detection network is then trained based on the loss function, thereby iteratively updating the query network branches. When annotating query points of a training image, for any layer of feature maps: if the distance from a location point to the center of a specific object in that layer of feature maps is less than a preset threshold, that location point is labeled as a query point; or if the overlap between the anchor box corresponding to a location point and the specific object in that layer of feature maps is greater than a preset threshold, that location point is labeled as a query point. This overlap can be the intersection-union ratio, which is the ratio of the intersection region to the union region.

[0080] Optionally, this disclosure employs focused loss to train the classification network branch and the query network branch, and utilizes smooth L1 loss to train the regression network branch.

[0081] Once the object detection model is trained, it can be used for object detection. Figure 5 A target detection method 500 according to an embodiment of the present disclosure is shown, such as... Figure 5 As shown, the target detection method 500 includes:

[0082] Step S501: Input the image to be detected into the trained object detection model. This object detection model includes a feature extraction network and an object detection network. The specific network structure is as follows: Figure 3 and Figure 4 As shown, further details will not be elaborated here.

[0083] Step S502: Use a feature extraction network to extract features from the image to be detected, and obtain multi-layer feature maps with different sizes. The multi-layer feature maps include a first feature map and a second feature map.

[0084] Step S503: The target detection network outputs the detection results of the multi-layer feature map. This target detection network includes multiple network layers corresponding to the multi-layer feature map, including a first network layer and a second network layer. The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map, transmitting the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The second network layer corresponds to the second feature map and is used to determine the mapping region of the query point in the second feature map, performing a detection operation within the mapping region to obtain the detection result.

[0085] like Figure 3 and Figure 4As shown, the object detection network also includes additional network layers and third, fourth, ..., nth network layers. Each network layer has a detection network used to obtain the corresponding detection result. Therefore, in step S530, the object detection network outputs the detection result of the multi-layer feature map, including: merging the detection results of each network layer to obtain the detection result of the input image.

[0086] Figure 6 The illustration shows the effect of target detection using an object detection model according to an embodiment of the present disclosure. The low-resolution feature map is mainly used to detect large objects, and a query point is transmitted to the high-resolution feature map. After the high-resolution feature map determines the mapping region corresponding to the query point, small objects within that mapping region are detected. Using the object detection model of the present disclosure for target detection has advantages such as high robustness, fast inference speed, and high efficiency.

[0087] Figure 7 An apparatus 700 for constructing a target detection model according to an embodiment of the present disclosure is shown, such as Figure 7 As shown, the device 700 includes:

[0088] The first building unit 701 is used to build a feature extraction network, which is used to extract features from the input image to obtain multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map and a second feature map.

[0089] The second construction unit 702 is used to construct an object detection network, which includes multiple network layers corresponding to multi-layer feature maps. These multiple network layers include a first network layer and a second network layer. The first network layer corresponds to a first feature map and is used to perform a query operation on the first feature map, transmitting the obtained query result to the second network layer. The query result includes the query point of a specific object in the first feature map. The second network layer corresponds to a second feature map and is used to determine the mapping region of the query point in the second feature map, performing a detection operation within the mapping region to obtain a detection result.

[0090] Optionally, the object detection model construction apparatus 700 may further include a third construction unit and a model training unit (neither shown in the figure). The third construction unit is used to construct an output network, which merges and outputs the detection results obtained after the detection operations of each network layer. The model training unit is used to generate a training sample set and train the constructed object detection model based on this training sample set to obtain the trained object detection model. The training sample set includes multiple training images and annotations for each training image, including at least one of classification annotations and location annotations.

[0091] In addition, the annotation of each training image also includes the query point annotations of each layer of feature maps generated from that training image. At this point, the model training unit calculates the loss function based on the query point annotations of each layer of feature maps and the predicted query points of that layer, and trains the constructed object detection network based on this loss function. Generally, for any layer of feature maps; when the distance from a certain location point to the center of a specific object in that layer of feature maps is less than a preset threshold, that location point is labeled as a query point; or, when the intersection-union ratio (IoU) of the anchor box corresponding to a certain location point and the specific object in that layer of feature maps is greater than a preset threshold, that location point is labeled as a query point.

[0092] Figure 8 A target detection apparatus 800 according to an embodiment of the present disclosure is shown, such as Figure 8 As shown, the device 800 includes:

[0093] The input unit 801 is used to input the image to be detected into the object detection model, which includes a feature extraction network and an object detection network.

[0094] The feature extraction unit 802 is used to extract features from the image to be detected using a feature extraction network to obtain multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map and a second feature map.

[0095] The target detection unit 803 is used to output detection results on a multi-layer feature map using a target detection network. This target detection network includes multiple network layers corresponding to the multi-layer feature map, including a first network layer and a second network layer. The first network layer corresponds to a first feature map and is used to perform a query operation on the first feature map, transmitting the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The second network layer corresponds to a second feature map and is used to determine the mapping region of the query point in the second feature map, performing a detection operation within the mapping region to obtain the detection result.

[0096] Optionally, the target detection unit 803 is used to merge and output the detection results obtained after the detection operations of each network layer (including non-query layers and query layers with detection networks).

[0097] It should be noted that the specific implementation methods of the target detection model construction device 700 and the target detection device 800 provided in the embodiments of this disclosure have been based on... Figures 1-6 The details have been disclosed in the description and will not be repeated here.

[0098] In addition, embodiments of this disclosure also provide a computer-readable storage medium including a program or instructions that, when executed on a computer, implement the object state estimation method as described above.

[0099] In addition, this disclosure also provides an embodiment such as Figure 9 The computing device 900 shown includes a memory 901 and one or more processors 902 communicatively connected to the memory. The memory 901 stores instructions executable by the one or more processors 902, which, when executed, cause the one or more processors 902 to implement the object state estimation method described above. The computing device 900 may further include a communication interface 903 that can implement one or more communication protocols (LTE, Wi-Fi, etc.).

[0100] According to the technical solution of this disclosure, the detection performance of small objects is improved by introducing higher resolution features, and a query header is added to the detection head, realizing a query-based and sparse convolution-based object detection method, thus solving the problem of slow inference speed after introducing high-resolution features. By deploying the object detection method of this disclosure on autonomous vehicles, the performance of small object detection can be made both fast and good.

[0101] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] This disclosure uses specific embodiments to illustrate the principles and implementation methods of this disclosure. The description of the above embodiments is only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.

Claims

1. A method for constructing an object detection model, comprising: A feature extraction network is constructed to extract features from an input image, resulting in multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map, a second feature map, and an additional feature map. The second feature map is obtained by upsampling and feature fusion of the first feature map, and the additional feature map is obtained by downsampling the first feature map. Construct an object detection network, the object detection network including multiple network layers corresponding to multi-layer feature maps, the multiple network layers including a first network layer, a second network layer and an additional network layer; as well as An output network is constructed, which merges the detection results obtained from the detection operations of each network layer to obtain the detection result of the input image; wherein, The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map and transmit the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The specific target is a target whose size is less than or equal to the size threshold that the corresponding layer feature map can detect. The query point is a point in the corresponding feature map whose probability value of the specific target exists is greater than or equal to a preset threshold. The second network layer corresponds to the second feature map and is used to map the query point to multiple points in the second feature map according to the magnification factor between the first feature map and the second feature map, to obtain a mapping region, and to perform a detection operation within the mapping region to obtain a detection result; The additional network layer corresponds to the additional feature map and is used to perform detection operations on the additional feature map to obtain detection results.

2. The method according to claim 1, wherein, The first network layer is also used to perform detection operations on the first feature map to obtain detection results.

3. The method according to claim 1, wherein, The target detection network has multiple second network layers. Each second network layer receives the query result from the first network layer and maps the query result to a corresponding mapping region in the second feature map.

4. The method according to claim 1, wherein, The multi-layer feature map further includes a third feature map, and the target detection network further includes a third network layer corresponding to the third feature map. The second network layer is further used for: Perform a query operation on the mapped region in the second feature map to obtain the corresponding query results; The obtained query results are transmitted to the third network layer, so that the third network layer maps the query points in the query results to the corresponding mapping regions in the third feature map, and performs detection operations within the mapping regions to obtain the detection results.

5. The method according to claim 1, wherein, The second network layer is also used for: Extract image features from the mapped region to construct a sparse tensor; Sparse convolution is used to perform detection and / or query operations within the mapped region.

6. The method according to claim 1, wherein, The feature extraction network includes: The backbone network is used to extract features from the input image, obtaining an initial multi-layer feature map; and A feature pyramid network is used to upsample and fuse the initial feature map to obtain an improved multi-layer feature map.

7. The method according to claim 1, wherein, The size of the second feature map is m times that of the first feature map, and m > 1.

8. The method according to claim 1, wherein, The detection operation includes at least one of regression operation and classification operation; The detection results include at least one of regression results and classification results.

9. The method according to claim 8, wherein, The target detection network includes a classification network, a regression network, and a query network; The classification, regression, and query operations are executed by the classification network, regression network, and query network, respectively. The classification network, regression network, and query network share parameters across multiple network layers.

10. The method according to claim 1, wherein, The query results include a query result image; The query result image includes the probability that the specific target exists at each location point in the corresponding feature image; The query point is a location with a probability value greater than or equal to a preset threshold.

11. The method according to claim 1, wherein, Each feature map layer has a corresponding size threshold, and the specific target of each feature map layer is the target whose size is less than or equal to the size threshold of that feature map layer.

12. The method according to claim 1, further comprising: Generate a training sample set, which includes multiple training images and annotations for each training image, wherein the annotations include at least one of classification annotations and location annotations; The constructed target detection model is trained based on the training sample set to obtain the trained target detection model.

13. The method according to claim 12, wherein, The annotation of each training image also includes the query point annotations of each layer of feature maps generated from that training image. The object detection model is trained based on the training sample set, including: Based on the query point annotations of each feature map layer and the predicted query points of that feature map layer, a loss function is calculated, and the constructed object detection network is trained based on the loss function.

14. The method according to claim 13, wherein, For any layer of feature map; When the distance from a certain location point to the center of a specific target in the feature map of that layer is less than a preset threshold, that location point is marked as a query point; or When the intersection-union ratio (IUU) of the anchor box corresponding to a certain location point with a specific target in the feature map of that layer is greater than a preset threshold, that location point is marked as a query point.

15. A target detection method, comprising: The image to be detected is input into the target detection model, which includes a feature extraction network and a target detection network; The feature extraction network is used to extract features from the image to be detected, resulting in multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map, a second feature map, and an additional feature map. The second feature map is obtained by upsampling and feature fusion of the first feature map, and the additional feature map is obtained by downsampling the first feature map. as well as The target detection network outputs the detection results of the multi-layer feature map. The target detection network includes multiple network layers corresponding to the multi-layer feature map, and these multiple network layers include a first network layer, a second network layer, and an additional network layer. The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map and transmit the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The specific target is a target whose size is less than or equal to the size threshold that the corresponding layer feature map can detect. The query point is a point in the corresponding feature map whose probability value of the specific target exists is greater than or equal to a preset threshold. The second network layer corresponds to the second feature map and is used to map the query point to multiple points in the second feature map according to the magnification factor between the first feature map and the second feature map, to obtain a mapping region, and to perform a detection operation within the mapping region to obtain a detection result; The additional network layer corresponds to the additional feature map and is used to perform detection operations on the additional feature map to obtain detection results; The detection results of the multi-layer feature map output by the target detection network include: The detection results of each network layer are merged using the target detection network to obtain the detection results of the input image.

16. An apparatus for constructing a target detection model, comprising: The first construction unit is used to construct a feature extraction network, which is used to extract features from the input image to obtain multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map, a second feature map, and an additional feature map. The second feature map is obtained by upsampling and feature fusion of the first feature map, and the additional feature map is obtained by downsampling the first feature map. The second construction unit is used to construct an object detection network, which includes multiple network layers corresponding to multi-layer feature maps, and the multiple network layers include a first network layer, a second network layer, and an additional network layer. as well as The third construction unit is used to construct an output network, which is used to merge the detection results obtained after the detection operation of each network layer to obtain the detection result of the input image. The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map and transmit the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The specific target is a target whose size is less than or equal to the size threshold that the corresponding layer feature map can detect. The query point is a point in the corresponding feature map whose probability value of the specific target exists is greater than or equal to a preset threshold. The second network layer corresponds to the second feature map and is used to map the query point to multiple points in the second feature map according to the magnification factor between the first feature map and the second feature map, to obtain a mapping region, and to perform a detection operation within the mapping region to obtain a detection result; The additional network layer corresponds to the additional feature map and is used to perform detection operations on the additional feature map to obtain detection results.

17. A target detection device, comprising: An input unit is used to input the image to be detected into a target detection model, the target detection model including a feature extraction network and a target detection network; The feature extraction unit is used to extract features from the image to be detected using the feature extraction network to obtain multi-layer feature maps of different sizes. The multi-layer feature maps include a first feature map, a second feature map, and a third feature map. The second feature map is obtained by upsampling and feature fusion of the first feature map, and the additional feature map is obtained by downsampling the first feature map. as well as The target detection unit is used to output the detection result of the multi-layer feature map using the target detection network. The target detection network includes multiple network layers corresponding to the multi-layer feature map, and the multiple network layers include a first network layer, a second network layer, and an additional network layer; wherein, The first network layer corresponds to the first feature map and is used to perform a query operation on the first feature map and transmit the obtained query result to the second network layer. The query result includes the query point of a specific target in the first feature map. The specific target is a target whose size is less than or equal to the size threshold that the corresponding layer feature map can detect. The query point is a point in the corresponding feature map whose probability value of the specific target exists is greater than or equal to a preset threshold. The second network layer corresponds to the second feature map and is used to map the query point to a mapping region in the second feature map according to the magnification factor between the first feature map and the second feature map, and to perform a detection operation in the mapping region to obtain a detection result; The additional network layer corresponds to the additional feature map and is used to perform detection operations on the additional feature map to obtain detection results; The target detection unit is further used to merge the detection results obtained after the detection operations of each network layer to obtain the detection result of the input image.

18. A computing device, comprising a memory and one or more processors communicatively connected to the memory; The memory stores instructions that can be executed by the one or more processors to cause the one or more processors to implement the method as described in any one of claims 1 to 15.

19. A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method of any one of claims 1-15.

20. A vehicle comprising the computing device of claim 18.

Images