Computing device for performing object detection and computing method for object detection

Abstract

The application provides a computing device and a computing method for target detection, wherein the computing device comprises a plurality of processing cores and a storage core, and the processing cores and the storage core are configured to implement operator fusion, that is, while the processing cores screen the confidence of the detection frame corresponding to each class by using an NMS operator, a Topk operator is used to perform a sorting operation, so that the final operation result can be directly obtained and stored in the on-chip storage core, without data transfer on-chip and off-chip in the calculation process, thereby compressing the calculation amount and reducing the IO overhead without affecting the calculation correctness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to a computing device for performing target detection and a target detection computing method. BACKGROUND

[0002] Target detection refers to determining the category of a target object and the position of the target object in a to-be-detected image.

[0003] At present, a common target detection computing method is as follows: feature extraction is performed on a to-be-detected image to obtain multiple categories, each category corresponds to multiple detection boxes, a Non-Maximum Suppression (NMS) algorithm is used to screen the multiple detection boxes corresponding to each category of the to-be-detected image, so that multiple detection boxes with the maximum score corresponding to each category can be obtained, and then the multiple detection boxes with the maximum score corresponding to each category are mixed together, and a Topk target detection box is selected from the mixed detection boxes, which is the final detection box used to determine the category of a target object contained in the to-be-detected image and the position of the target object.

[0004] In a target detection network based on a deep learning algorithm, the combination of an NMS operator and a Topk operator is very common, for example, which can be applied in an SSD network and a YOLO series target detection network. However, the above target detection computing method has a large amount of calculation and a large IO overhead, and therefore, how to compress the calculation amount and reduce the IO overhead without affecting the calculation correctness becomes a problem to be solved. SUMMARY

[0005] The present application provides a computing device for performing target detection and a target detection computing method, which are used to compress the calculation amount and reduce the calculation overhead without affecting the calculation correctness.

[0006] In a first aspect, an embodiment of the present application provides a computing device for performing target detection, the computing device comprising: a plurality of processing cores and a storage core; wherein,

[0007] The processing core is configured to:

[0008] obtain a to-be-detected image, and obtain multiple categories when performing target detection on the to-be-detected image; wherein each category corresponds to multiple detection boxes, and the detection boxes corresponding to different categories are different; the detection box has corresponding position information and confidence;

[0009] The confidence of the detection frame corresponding to each class is screened by using a non-maximum suppression algorithm to obtain a detection frame with the maximum confidence of each class in the current screening; the confidence of the detection frame of the class obtained in the current screening is compared with the confidence of the detection frame stored in the storage core, and the content stored in the storage core after the current screening is determined according to the comparison result;

[0010] The storage core is configured to:

[0011] The confidence and position information of the detection frame of the class are stored after each screening.

[0012] In a second aspect, an embodiment of the present application provides a computing method for target detection, applied to a computing device for performing target detection, the computing device comprising a plurality of processing cores and a storage core; the computing method comprising:

[0013] The processing core obtains an image to be detected, and obtains a plurality of classes when performing target detection on the image to be detected; wherein each class corresponds to a plurality of detection frames, and the detection frames corresponding to different classes are different; the detection frame has corresponding position information and confidence;

[0014] The processing core screens the confidence of the detection frame corresponding to each class by using a non-maximum suppression algorithm to obtain a detection frame with the maximum confidence of each class in the current screening; the confidence of the detection frame of the class obtained in the current screening is compared with the confidence of the detection frame stored in the storage core, and the content stored in the storage core after the current screening is determined according to the comparison result;

[0015] After each screening, the storage core stores the confidence and position information of the detection frame of the class.

[0016] The application provides a computing device for performing target detection and a computing method for target detection. The computing device comprises a plurality of processing cores and a storage core. The processing core is configured to: acquire an image to be detected, and obtain a plurality of detection boxes corresponding to each class and having corresponding position information and confidence when performing target detection on the image to be detected. The confidence of the detection box corresponding to each class is filtered by using a non-maximum suppression algorithm to obtain a detection box with the maximum confidence corresponding to each class after current filtering. The confidence of the detection box of the class obtained after the current filtering is compared with the confidence of the detection box stored in the storage core, and the content stored in the storage core after the current filtering is determined according to the comparison result. The storage core is configured to store the confidence and position information of the detection box of the class after each filtering. That is, the application realizes operator fusion by configuring the processing core and the storage core, that is, performs a sorting operation while filtering the confidence of the detection box corresponding to each class, so that the target detection box can be directly determined, the confidence and position information of the target detection box are stored in the on-chip storage core, and data transfer on-chip and off-chip in the calculation process is not required, so that the calculation amount is compressed and the IO overhead is reduced without affecting the calculation accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0018] Figure 1 A structural schematic diagram of a board card 10 provided for an embodiment of the application is shown in the figure.

[0019] Figure 2 A structural diagram of a combined processing device in a chip 101 provided for an embodiment of the application is shown in the figure.

[0020] Figure 3 An internal structural schematic diagram of a multi-core computing device 201 provided for an embodiment of the application is shown in the figure.

[0021] Figure 4 An execution flow schematic diagram of a computing device for target detection provided for an embodiment of the application is shown in the figure.

[0022] Figure 5 A flowchart of a computing method for target detection provided for an embodiment of the application is shown in the figure.

[0023] The above figures have shown the specific embodiments of the application, and more detailed descriptions will be given in the following. The figures and the text are not intended to limit the scope of the concept of the application by any means, but to illustrate the concept of the application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is made with reference to the accompanying drawings in which like reference numerals designate identical or similar elements in the several figures. The implementations described in the following exemplary embodiments are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure.

[0025] Exemplary hardware environment

[0026] Figure 1 A schematic diagram of a board card 10 is provided for the embodiments of the present disclosure. As shown in the figure, the board card 10 includes a chip 101, which is a system on chip (SoC) or a system on chip, integrating one or more combined processing devices, which is an artificial intelligence operation unit, to support various deep learning and machine learning algorithms, to meet the intelligent processing needs of complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in cloud intelligent fields. One of the notable features of cloud intelligent applications is the large amount of input data, which has high requirements for the storage and computing capabilities of the platform. The board card 10 of the embodiments is suitable for cloud intelligent applications, with a large off-chip storage, on-chip storage and powerful computing capability. Figure 1

[0027] The chip 101 is connected to an external device 103 through an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card or a wifi interface, etc. The data to be processed can be transmitted from the external device 103 to the chip 101 through the external interface device 102. The computing results of the chip 101 can be transmitted back to the external device 103 through the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe interface, etc.

[0028] The board card 10 further includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected and transmits data with the control device 106 and the chip 101 through a bus. The control device 106 in the board card 10 is configured to regulate the state of the chip 101. For this purpose, in one application scenario, the control device 106 can include a micro controller unit (MCU).

[0029] Figure 2 A structural diagram of the combined processing device in the chip 101 is provided for the embodiments of the present disclosure. As shown in the figure, Figure 2 ​As shown in FIG. 1, the combined processing device 20 comprises a computing device 201, an interface device 202, a processing device 203 and a storage device 204.

[0030] The computing device 201 is configured to perform user-specified operations, mainly implemented as a single-core or multi-core intelligent processor to perform deep learning or machine learning calculations, which can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0031] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it to the storage device on the computing device 201. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them to the control buffer on the computing device 201. Alternatively or optionally, the interface device 202 can also read data from the storage device of the computing device 201 and transmit it to the processing device 203.

[0032] The processing device 203 serves as a general-purpose processing device to perform basic controls including but not limited to data transfer, starting and / or stopping of the computing device 201, etc. Depending on the implementation, the processing device 203 can be one or more types of processors including but not limited to a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) or other general-purpose and / or special-purpose processors such as a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., and the number thereof can be determined according to actual needs. As mentioned before, the computing device 201 of the present disclosure can be considered as having a single-core or homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0033] The storage device 204 is used to store data to be processed, which can be a DRAM, a DDR memory, usually with a size of 16G or more, for saving data of the computing device 201 and / or the processing device 203.

[0034] Figure 3 The computing device 201 provided by the embodiments of the present application is a schematic diagram of the internal structure of a multi-core. The multi-core computing device 300 adopts a hierarchical structure design. The multi-core computing device 300 is a system on a chip, which includes at least one computing cluster, and each computing cluster includes a plurality of processor cores. In other words, the multi-core computing device 300 is composed of a system on a chip-computing cluster-processor core hierarchy.

[0035] As shown in the system on a chip hierarchy, Figure 3 the multi-core computing device 300 includes an external storage controller 31, a peripheral communication module 32, an on-chip interconnection module 33, a synchronization module 34, and a plurality of computing clusters 35.

[0036] The external storage controller 31 can be multiple, and two are exemplarily shown in the figure. The external storage controller 31 is used to access an external storage device, such as the storage device 204 in Figure 2 , in response to an access request issued by a processing core, so as to read data from off-chip or write data. The peripheral communication module 32 is used to receive a control signal from the processing device 203 through the interface device 202 to start the computing device 201 to perform a task. The on-chip interconnection module 33 connects the external storage controller 31, the peripheral communication module 32, and the plurality of computing clusters 35 to transmit data and control signals between the modules. The synchronization module 34 is a global synchronization barrier controller (GBC) used to coordinate the working progress of each computing cluster and ensure the synchronization of information. The plurality of computing clusters 35 are the computing cores of the multi-core computing device 300. Four are exemplarily shown in the figure. With the development of hardware, the multi-core computing device 300 of the present application can also include eight, sixteen, sixty-four, or even more computing clusters 35. The computing cluster 35 is used to efficiently execute a deep learning algorithm. The plurality of computing clusters 35 can form a mesh structure that can be communicated in a ring, that is, the plurality of computing clusters can have a mesh interconnection circuit.

[0037] As shown in the computing cluster hierarchy, Figure 3 the upper right, each computing cluster 35 includes a processing unit 302 and a memory core 304. The processing unit 302 performs various computing tasks. In some implementations, the processing unit can be a multi-core architecture, for example, including a plurality of processing cores (IPU core) 311-1~311-n to complete tasks such as large-scale vector computation. The present application does not limit the number of processing cores 311.

[0038] The internal architecture of the processing core 311 is as shown in Figure 3The computing modules 324-1 ~ 324-m are on-chip basic tensor computing units, including but not limited to vector operation units, tensor operation units performing matrix multiplication, operation units directly performing convolution operations, or convolution computing units integrating img2col (image to row and column) and gemm (general matrix multiplication).

[0039] The computing modules 324-1 ~ 324-m are on-chip basic tensor computing units, including but not limited to vector operation units, tensor operation units performing matrix multiplication, operation units directly performing convolution operations, or convolution computing units integrating img2col (image to row and column) and gemm (general matrix multiplication).

[0040] The local storage module 323 can serve as a cache level (for example, a first-level cache L1 cache) in the computing cluster 35, which can include a neuron storage unit (Neuron RAM, NRAM for short) and a weight storage unit (Weight RAM, WRAM for short). The NRAM is used to store input neurons, output neurons, and intermediate results after calculation; the WRAM is used to store the convolution kernel of the deep learning network, that is, the weight. It should be particularly pointed out that the local storage module 323 can also include various communication modules to exchange data with external storage units. For example, the local storage module 323 can include a communication module 321 to communicate with the shared storage module 315 in the storage core 304. The communication module 321 can be, for example, a move direct memory access module (Move Direct Memory Access, MVDMA for short). The local storage module 323 can also include a communication module 322 to exchange data with off-chip memory, such as the DRAM 308. The communication module 322 can be, for example, an input / output direct memory access module (Input / Output Direct Memory Access, IODMA for short). The IODMA 322 controls the access of the NRAM / WRAM in the local storage module 323 to the DRAM 308; the MVDMA 321 is used to control the access of the NRAM / WRAM in the local storage module 323 to the shared storage module 315.

[0041] Continuing Figure 3 The upper right view, the storage core 304 is mainly used for storage and communication, that is, to store shared data or intermediate results between processing cores 311, and to perform communication between the computing cluster 35 and the DRAM 308, communication between the computing clusters 35, communication between the processing cores 311, and the like. In other embodiments, the storage core 304 has the ability of scalar operation to perform scalar operation to achieve operation tasks in data communication.

[0042] The memory core 304 includes a large shared memory module (SRAM) 315, a broadcast bus 314, a cluster direct memory access module (CDMA) 318, a global direct memory access module (GDMA) 316, and a communication time computing module 317. The SRAM 315 functions as a high-performance data relay station. Data reused among the processing cores 311 in the same computing cluster 35 does not need to be obtained by the processing cores 311 from the DRAM 308 individually, but is relayed among the processing cores 311 through the SRAM 315. The memory core 304 only needs to quickly distribute the reused data from the SRAM 315 to the multiple processing cores 311, so as to improve the efficiency of inter-core communication and greatly reduce on-chip and off-chip input / output access.

[0043] The broadcast bus 314, the CDMA 318, and the GDMA 316 are respectively used to perform communication among the processing cores 311, communication among the computing clusters 35, and data transmission between the computing cluster 35 and the DRAM 308. The following will be described respectively.

[0044] The broadcast bus 314 is used to complete high-speed communication among the processing cores 311 in the computing cluster 35. The broadcast bus 314 of the embodiment supports inter-core communication modes including unicast, multicast, and broadcast. Unicast refers to point-to-point (for example, single processing core to single processing core) data transmission. Multicast is a communication mode in which a piece of data is transmitted from the SRAM 315 to specific processing cores 311. Broadcast is a communication mode in which a piece of data is transmitted from the SRAM 315 to all processing cores 311, which is a special case of multicast.

[0045] Inside each computing cluster 35, each processing core 311 can initiate a broadcast to simultaneously broadcast data into the local memory module 323 (for example, NRAM or WRAM) of each core. Broadcasting to the NRAM and the WRAM belongs to two data channels and can be performed concurrently. However, at a certain time node, each processing core can only initiate a broadcast once, that is, the broadcast of the WRAM and the NRAM cannot be initiated at the same core at the same time.

[0046] The CDMA 318 is used to control the access of the SRAM 315 between different computing clusters 35 in the same computing device 301. The GDMA 316 cooperates with the external storage controller 31 to control the access of the SRAM 315 of the computing cluster 35 to the DRAM 308, or read data from the DRAM 308 to the SRAM 315. As described above, the communication between the DRAM 308 and the NRAM / WRAM in the local storage module 323 can be realized through two channels. The first channel is to directly contact the DRAM 308 and the local storage module 323 through the IODMA 322; the second channel is to first transmit data between the DRAM 308 and the SRAM 315 through the GDMA 316, and then transmit data between the SRAM 315 and the local storage module 323 through the MVDMA 321. Although the second channel needs more elements to participate and the data flow is longer, in fact, in some embodiments, the bandwidth of the second channel is much larger than that of the first channel, so that the communication between the DRAM 308 and the local storage module 323 through the second channel can be more efficient. The embodiments of the present application can select the data transmission channel according to the hardware conditions.

[0047] Exemplary operational scheme

[0048] At present, the common target detection calculation method is executed in the computing device 301 as follows: the processing core 311 uses the NMS operator to screen the confidence of each class corresponding detection frame, and moves the screening result to the off-chip DRAM 308 for storage, and when the processing core 311 executes the Topk operator operation, the screening result stored in the DRAM 308 is moved back to the on-chip storage core 304. Due to the limited bandwidth resources between off-chip and on-chip, such data movement will cause large IO overhead, and even cause IO bottleneck.

[0049] Therefore, how to compress the calculation amount and reduce the IO overhead without affecting the calculation correctness has become a problem to be solved.

[0050] Exemplary optimization scheme

[0051] The computing device for executing target detection and the target detection calculation method provided by the present application aim to solve the above technical problems of the prior art.

[0052] In the computing device for performing target detection and the computing method for target detection provided in the application, the processing core 311 and the storage core 304 are configured to implement operator fusion, that is, while the processing core 311 adopts an NMS operator to screen the confidence of the detection frame corresponding to each class, a Topk operator is adopted to perform a sorting operation, so that the final operation result can be directly obtained and stored in the on-chip storage core 304, without data transfer on-chip and off-chip in the calculation process, thereby compressing the calculation amount and reducing the IO overhead without affecting the calculation accuracy.

[0053] The technical solutions of the application and how the technical solutions of the application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the application will be described below with reference to the drawings.

[0054] Embodiment one

[0055] The embodiment one of the application provides a computing device for performing target detection, which comprises a plurality of processing cores 311 and a storage core 304 as shown in the figure. Figure 3

[0056] The processing core 311 is configured to: acquire a to-be-detected image, and obtain a plurality of classes when performing target detection on the to-be-detected image. Each class corresponds to a plurality of detection frames, and the detection frames corresponding to different classes are different. The detection frame has corresponding position information and confidence.

[0057] The confidence of the detection frame corresponding to each class is screened by using a non-maximum suppression algorithm to obtain the detection frame with the maximum confidence corresponding to each class in the current screening. The confidence of the detection frame of the class obtained in the current screening is compared with the confidence of the detection frame stored in the storage core 304, and the content stored in the storage core 304 after the current screening is determined according to the comparison result.

[0058] The storage core 304 is configured to: store the confidence and position information of the detection frame of the class after each screening.

[0059] Specifically, the position information corresponding to the detection frame can include the horizontal and vertical coordinate values of the four vertices of the detection frame, and the confidence corresponding to the detection frame can be any value greater than or equal to 0, which is not limited in the embodiment.

[0060] On the basis of the above-mentioned embodiment one, the processing core 311 is further configured to:

[0061] After the detection frame with the maximum confidence corresponding to each class in the current screening is obtained, it is judged whether the number of the confidence and position information of the detection frame stored in the storage core 304 reaches a preset number.​

[0062] If it is determined that the number of confidence and position information of the detection frame stored in the storage core 304 does not reach the preset number, it is determined to store the confidence and position information of the detection frame of the class obtained in the current screening into the storage core 304.

[0063] If it is determined that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, a step of comparing the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame stored in the storage core 304 and determining the content stored in the storage core 304 after the current screening according to the comparison result is performed.

[0064] In one example, when target detection is performed on the to-be-detected image, 5 classes are obtained, and the processing core 311 can perform screening on the confidence of the detection frame corresponding to the 5 classes respectively by using the non-maximum suppression algorithm. Specifically, the processing core 311 can first perform first screening on the confidence of the detection frame corresponding to the first class by using the non-maximum suppression algorithm, perform first screening on the confidence of the detection frame corresponding to the second class by using the non-maximum suppression algorithm, and so on until first screening on the confidence of the detection frame corresponding to the 5 classes is completed.

[0065] Correspondingly, after first screening on the confidence of the detection frame corresponding to each class is performed, the processing core 311 can determine whether the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number. Specifically, the preset number can be the number of target detection frames finally wanted to be obtained.

[0066] If it is determined that the number of confidence and position information of the detection frame stored in the storage core 304 does not reach the preset number, it indicates that sufficient detection frames have not been obtained at present, and it can be determined to store the confidence and position information of the detection frame of the class obtained in the current screening into the storage core 304.

[0067] For example, if the processing core 311 performs first screening on the confidence of the detection frame corresponding to the first class and determines that the number of confidence and position information of the detection frame stored in the storage core 304 does not reach the preset number, it can be determined to store the confidence and position information of the detection frame of the first class obtained in the current screening into the storage core 304.

[0068] If it is determined that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, it indicates that sufficient detection frames have been obtained at present, but the detection frames obtained at present can not be the final target detection frame, and a step of comparing the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame stored in the storage core 304 and determining the content stored in the storage core 304 after the current screening according to the comparison result can be performed.

[0069] In the above example, if the processing core 311 performs the first screening on the confidence of the detection frame corresponding to the fifth category, and determines that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, the step of comparing the confidence of the detection frame of the fifth category obtained in the current screening with the confidence of the detection frame stored in the storage core 304, and determining the content stored in the storage core 304 after the current screening according to the comparison result can be performed.

[0070] In the above example, if the processing core 311 performs the first screening on the confidence of the detection frame corresponding to the fifth category, and determines that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, the step of comparing the confidence of the detection frame of the fifth category obtained in the current screening with the confidence of the detection frame stored in the storage core 304, and determining the content stored in the storage core 304 after the current screening according to the comparison result can be performed.

[0071] In the above example, if the processing core 311 performs the first screening on the confidence of the detection frame corresponding to the fifth category, and determines that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, the step of comparing the confidence of the detection frame of the fifth category obtained in the current screening with the confidence of the detection frame stored in the storage core 304, and determining the content stored in the storage core 304 after the current screening according to the comparison result can be performed.

[0072] In the above example, if the processing core 311 performs the first screening on the confidence of the detection frame corresponding to the fifth category, and determines that the number of confidence and position information of the detection frame stored in the storage core 304 reaches the preset number, the step of comparing the confidence of the detection frame of the fifth category obtained in the current screening with the confidence of the detection frame stored in the storage core 304, and determining the content stored in the storage core 304 after the current screening according to the comparison result can be performed.

[0073] On the basis of the above embodiment one, in another possible implementation, after obtaining the comparison result, the processing core 311 is further specifically configured to: if the comparison result is that the confidence of the detection frame of the class obtained in the current screening is greater than the confidence of the detection frame with the minimum confidence stored in the storage core 304, it is determined that the confidence and position information of the detection frame of the class obtained in the current screening are stored in the storage core 304.

[0074] Correspondingly, the storage core 304 is specifically configured to: store the confidence and position information of the detection frame of the class obtained in the current screening, and delete the confidence and position information of the detection frame with the minimum confidence currently stored.

[0075] Taking the above example, if the confidence of the detection frame of the fifth class obtained after the first screening of the confidence of the detection frame of the fifth class is greater than the confidence of the detection frame with the minimum confidence stored in the storage core 304, it can be determined that the confidence and position information of the detection frame of the fifth class obtained in the first screening are stored in the storage core 304. Correspondingly, the storage core 304 can store the confidence and position information of the detection frame of the fifth class obtained in the first screening, and delete the confidence and position information of the detection frame with the minimum confidence currently stored.

[0076] Specifically, the storage core 304 can delete the confidence and position information of the detection frame located at the tail end of the linked list, and store the confidence and position information of the detection frame of the fifth class obtained in the first screening in the corresponding position of the linked list, so as to realize the updating of the linked list.

[0077] On the basis of the above embodiment one, in one example, the number of target detection frames finally wanted to be obtained, that is, the preset number is K, the number of confidence and position information of the detection frame currently stored in the storage core 304 is Kcur, and is initialized to 0, the minimum value of the confidence of the detection frame currently stored in the storage core 304 is smin, and is initialized to negative infinity. After each time the confidence of the detection frame of a class is screened by using the non-maximum suppression (NMS) algorithm, the maximum value of the confidence obtained is s.

[0078] Correspondingly, Figure 4 The execution flow diagram of the target detection calculation device provided in the embodiments of the present application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the execution method of the target detection calculation device provided in the embodiments of the present application includes: the processing core 311 screens the confidence of the detection frame of each class corresponding to the to-be-detected image by using the NMS algorithm. After each screening ends, the detection frame with the confidence s can be obtained.

[0079] Next, the processing core 311 can determine the number Kcur of the confidence and position information of the detection frame currently stored in the storage core 304. In one case, Kcur is equal to 0, indicating that the storage core 304 does not store the confidence and position information of the detection frame, and thus the confidence and position information of the detection frame obtained in this screening can be stored in the storage core 304, and smin is updated to s, and Kcur is updated to 1. In another case, Kcur is greater than 0 and less than K, indicating that the storage core 304 stores the confidence and position information of the detection frame, but does not reach the preset number, and thus the confidence and position information of the detection frame obtained in this screening can be stored in the storage core 304, and Kcur is increased by 1, and s is compared with the minimum confidence smin of the detection frame currently stored in the storage core 304. If s is less than smin, smin is updated to s, and if s is greater than or equal to smin, smin remains the current value.

[0080] In yet another case, Kcur is equal to K, indicating that the storage core 304 stores the confidence and position information of the detection frame up to the preset number, and thus s is compared with the minimum confidence smin of the detection frame currently stored in the storage core 304. If s is less than or equal to smin, the confidence and position information of the detection frame obtained in this screening is not stored in the storage core 304, and the screening of the confidence of the detection frame corresponding to the class is ended. Correspondingly, if s is greater than smin, the confidence and position information of the detection frame with the minimum confidence currently stored in the storage core 304 is deleted, and the confidence and position information of the detection frame obtained in this screening is stored in the storage core 304.

[0081] When the processing core 311 ends the screening of the confidence of the detection frame corresponding to all classes of the image to be detected, the position information of the detection frame stored in the storage core 304 at this time is the position information of the target detection frame.

[0082] Embodiment Two

[0083] Figure 5 The flowchart of the target detection calculation method provided by the embodiments of the present application is shown in FIG. 1, and the target detection calculation method provided by the embodiments of the present application is applied to a computing device for performing target detection. The computing device includes a plurality of processing cores and a storage core. Correspondingly, the computing method includes the following steps: Figure 5

[0084] Step 101, the processing core obtains an image to be detected, and obtains a plurality of classes when performing target detection on the image to be detected. Each class corresponds to a plurality of detection frames, and the detection frames corresponding to different classes are different. The detection frame has corresponding position information and confidence.

[0085] ​In step 102, the processing core screens the confidence of the detection frame corresponding to each class by using a non-maximum suppression algorithm to obtain the detection frame with the maximum confidence of each class in the current screening; and compares the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame stored in the storage core, and determines the content stored in the storage core after the current screening according to the comparison result.

[0086] In step 103, the storage core stores the confidence and position information of the detection frame of the class after each screening.

[0087] In the above embodiment two, the processing core obtains the detection frame with the maximum confidence of each class in the current screening, and further includes: the processing core judging whether the number of the confidence and position information of the detection frame stored in the storage core reaches a preset number.

[0088] In an optional implementation, if the processing core judges that the number of the confidence and position information of the detection frame stored in the storage core does not reach the preset number, the processing core determines to store the confidence and position information of the detection frame of the class obtained in the current screening in the storage core.

[0089] In another optional implementation, if the processing core judges that the number of the confidence and position information of the detection frame stored in the storage core reaches the preset number, the processing core executes the step of comparing the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame stored in the storage core, and determining the content stored in the storage core after the current screening according to the comparison result.

[0090] In the above embodiment two, the processing core compares the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame stored in the storage core, and specifically includes: the processing core compares the confidence of the detection frame of the class obtained in the current screening with the confidence of the detection frame with the minimum confidence stored in the storage core to obtain a comparison result.

[0091] In the above embodiment two, the processing core determines the content stored in the storage core after the current screening according to the comparison result, and includes: if the comparison result is that the confidence of the detection frame of the class obtained in the current screening is less than or equal to the confidence of the detection frame with the minimum confidence stored in the storage core, the processing core determines not to store the confidence and position information of the detection frame of the class obtained in the current screening in the storage core, and ends the screening of the confidence of all detection frames corresponding to the class.

[0092] On the basis of the embodiment two, the processing core determines the content stored in the storage core after the current screening according to the comparison result, and the method further includes: if the comparison result is that the confidence of the detection frame of the class obtained in the current screening is greater than the confidence of the detection frame with the minimum confidence stored in the storage core, the processing core determines to store the confidence and position information of the detection frame of the class obtained in the current screening in the storage core.

[0093] Correspondingly, the storage core stores the confidence and position information of the detection frame of the class, including: the storage core stores the confidence and position information of the detection frame of the class obtained in the current screening, and deletes the confidence and position information of the detection frame with the minimum confidence currently stored.

[0094] It should be noted that the technical solutions and effects of the target detection calculation method provided in this embodiment can refer to the related content of the foregoing execution target detection calculation device embodiments, which will not be repeated here.

[0095] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0096] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place, or can be distributed to multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purpose of the embodiment.

[0097] In addition, each functional module in each embodiment of the present application can be integrated in one processing module, or each module can exist physically, or two or more modules can be integrated in one module. The above integrated module can be realized in the form of hardware or in the form of hardware plus software functional module.

[0098] Program code for carrying out the methods of the present application can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be embodied in whole or in part within a machine-readable medium. The machine-readable medium can be a storage medium, a memory device, or a memory on a processor, or a combination thereof.

[0099] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of a processor, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0100] Further, while operations are depicted in a particular order, this should not be understood as requiring such an order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, while several specific implementation details are contained in the above discussion, these should not be construed as limitations on the scope of the present application, but rather as exemplification of one or more embodiments thereof. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in subcombination, or as separate embodiments, absent the features of the single embodiment. Embodiments described herein can be implemented in hardware, software, firmware, or any combination thereof.

[0101] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the claims and their equivalents. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the application indicated by the following claims.

[0102] It is to be understood that the application is not limited to the precise construction already described above and shown in the drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application should only be limited by the claims appended hereto.

Claims

1. A computing device for performing target detection, characterized in that, The computing device includes: multiple processing cores and storage cores; wherein... The processing core is configured to: When acquiring an image to be detected and performing target detection on the image, multiple classes are obtained; each class corresponds to multiple detection boxes, and different classes correspond to different detection boxes; each detection box has corresponding location information and confidence level; For each class, the confidence scores of the detection boxes are filtered using a non-maximum suppression algorithm to obtain the detection boxes with the highest confidence scores for each class in the current filtering. The confidence scores of the detection boxes for the class obtained in the current filtering are compared with the confidence scores of the detection boxes with the lowest confidence scores stored in the storage core. Based on the comparison result, the content to be stored in the storage core after the current filtering is determined. Specifically, if the comparison result shows that the confidence scores of the detection boxes for the class obtained in the current filtering are less than or equal to the confidence scores of the detection boxes with the lowest confidence scores stored in the storage core, then it is determined that the confidence scores and position information of the detection boxes for the class obtained in the current filtering will not be stored in the storage core, and the filtering of the confidence scores of all detection boxes corresponding to the class ends. If the comparison result shows that the confidence scores of the detection boxes for the class obtained in the current filtering are greater than the confidence scores of the detection boxes with the lowest confidence scores stored in the storage core, then it is determined that the confidence scores and position information of the detection boxes for the class obtained in the current filtering will be stored in the storage core. The storage core is configured for: After each filtering, the confidence level and location information of the detection box for that class are stored; The storage core is an on-chip storage unit that is tightly integrated with the processing core within the same computing device.

2. The computing device according to claim 1, characterized in that, The processing core is also configured to: After obtaining the detection box with the maximum confidence value for each class during the current filtering, it is determined whether the number of confidence and location information of the detection boxes stored in the storage core has reached a preset number. If it is determined that the number of confidence and location information of the detection boxes stored in the storage core has not reached the preset number, then it is determined that the confidence and location information of the detection boxes of the class obtained in the current filtering will be stored in the storage core. If it is determined that the number of confidence scores and location information of the detection boxes stored in the storage core reaches a preset number, then the step of comparing the confidence scores of the detection boxes of the class obtained in the current filtering with the confidence scores of the detection boxes stored in the storage core is executed, and the content to be stored in the storage core after the current filtering is determined based on the comparison result.

3. The computing device according to claim 2, characterized in that, When the number of confidence and location information of the detection frames stored in the storage core reaches a preset number, the processing core is specifically configured to: The confidence scores of the detection boxes of the class obtained in the current filtering are compared with the confidence scores of the detection boxes with the lowest confidence scores stored in the storage core to obtain the comparison results.

4. The computing device according to claim 3, characterized in that, The storage core is specifically configured for: Store the confidence and location information of the detection boxes of the class obtained in the current filtering, and delete the confidence and location information of the detection box with the lowest confidence currently stored.

5. A computational method for target detection, applied to a computing device performing target detection, characterized in that, The computing device includes multiple processing cores and storage cores; the computing method includes: The processing core acquires the image to be detected and performs target detection on the image to be detected, obtaining multiple classes; each class corresponds to multiple detection boxes, and different classes correspond to different detection boxes; each detection box has corresponding location information and confidence level; The processing core uses a non-maximum suppression algorithm to filter the detection boxes corresponding to each class based on their confidence scores, and obtains the detection boxes with the highest confidence scores for each class at the time of the current filtering. The confidence scores of the detection boxes for each class obtained at the time of the current filtering are compared with the confidence scores of the detection boxes with the lowest confidence scores stored in the storage core. Based on the comparison results, the content to be stored in the storage core after the current filtering is determined. After each filtering, the storage core stores the confidence and location information of the detection box of the class; the storage core is an on-chip storage unit, which is tightly integrated with the processing core in the same computing device; The processing core determines the content to be stored in the storage core after the current filtering based on the comparison results, including: If the comparison result is that the confidence score of the detection box of the class obtained in the current filtering is less than or equal to the confidence score of the detection box with the lowest confidence score stored in the storage core, the processing core determines not to store the confidence score and position information of the detection box of the class obtained in the current filtering into the storage core, and ends the filtering of the confidence scores of all detection boxes corresponding to the class. If the comparison result shows that the confidence score of the detection box of the class obtained in the current filtering is greater than the confidence score of the detection box with the lowest confidence score stored in the storage core, then the processing core determines to store the confidence score and position information of the detection box of the class obtained in the current filtering into the storage core.

6. The calculation method according to claim 5, characterized in that, After the processing kernel obtains the detection box with the maximum confidence value for each class during the current filtering, it also includes: The processing core determines whether the number of confidence and location information of the detection frame stored in the storage core reaches a preset number; If the processing core determines that the number of confidence and location information of the detection boxes stored in the storage core has not reached the preset number, then it determines to store the confidence and location information of the detection boxes of the class obtained in the current filtering into the storage core. If the processing core determines that the number of confidence and location information of the detection boxes stored in the storage core reaches a preset number, then it performs the step of comparing the confidence of the detection boxes of the class obtained in the current filtering with the confidence of the detection boxes stored in the storage core, and determining the content to be stored in the storage core after the current filtering based on the comparison result.

7. The calculation method according to claim 6, characterized in that, The processing core compares the confidence scores of the detection boxes for the class obtained during the current filtering with the confidence scores of the detection boxes stored in the storage core, specifically including: The processing kernel compares the confidence score of the detection box of the class obtained in the current screening with the confidence score of the detection box with the lowest confidence score stored in the storage kernel to obtain a comparison result.

8. The calculation method according to claim 7, characterized in that, The storage core stores the confidence and location information of the detection boxes of the class, including: The storage core stores the confidence and location information of the detection boxes of the class obtained during the current filtering, and deletes the confidence and location information of the detection box with the lowest confidence currently stored.

Images