Object detection method and apparatus

Abstract

The present application relates to the technical field of artificial intelligence, and discloses an object detection method and apparatus. A specific embodiment of the method comprises: performing feature extraction on an input image to obtain a feature matrix, and flattening the feature matrix in a planar dimension to obtain a query content vector set; performing first score evaluation on the query content vector set, and extracting a key query content vector set from the query content vector set on the basis of a first score evaluation result; performing second score evaluation on a pre-generated query vector set, and extracting a target query vector set from the query vector set on the basis of a second score evaluation result; and executing attention mechanism calculation on the basis of the key query content vector set and the target query vector set to obtain an object detection result. According to the embodiment, the amount of attention mechanism calculation is remarkably reduced and the calculation speed is increased, without decreasing the feature resolution and reducing effective features, thereby achieving the effect of performing object detection in real time at a vehicle end.

Description

A method and device for target detection

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311747933.2, filed on December 19, 2023, entitled “A method and device for target detection.” The contents disclosed in the above-mentioned Chinese patent application are hereby cited in their entirety as part or all of this application. Technical Field

[0003] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method and device for target detection. Background Art

[0004] Purely visual 3D object detection methods use input image information, extract features from a model, and ultimately output 3D detection results for the target objects in the scene. In recent years, with the development of attention-based sequence models like the Transformer model, an increasing number of methods have applied attention mechanisms to 3D object detection. While attention mechanisms can effectively extract global features, their computational complexity is often significantly higher than that of conventional convolutional neural networks (CNNs), making it difficult to achieve fast computational speeds in scenarios such as autonomous driving, which require real-time on-board computation.

[0005] In the process of implementing the present disclosure, the inventors discovered that the prior art has at least the following problems:

[0006] Existing target detection methods based on attention mechanisms have high computational complexity and slow computational speed, making it difficult to achieve real-time target detection. Summary of the Invention

[0007] In view of this, the embodiments of the present disclosure provide a method and device for target detection, which can evaluate the quality score of the QKV vector of the attention mechanism, and then filter out the QKV vectors that carry the most effective information. Based on the dynamic screening strategy of the number of QKVs, the computational complexity of the attention mechanism is significantly reduced without reducing the feature resolution or the effective features, and the computing speed is improved, thereby achieving the effect of real-time target detection on the vehicle side.

[0008] To achieve the above objectives, according to one aspect of an embodiment of the present disclosure, a method for target detection is provided, comprising:

[0009] Extract features from the input image to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group;

[0010] performing a first score evaluation on the query content vector group, and extracting a key query content vector group from the query content vector group according to the first score evaluation result;

[0011] performing a second score evaluation on the pre-generated query vector group, and extracting a target query vector group from the query vector group according to the second score evaluation result;

[0012] An attention mechanism is performed based on the key query content vector group and the target query vector group to obtain a target detection result.

[0013] According to one or more embodiments of the present disclosure, a first score evaluation is performed on the query content vector group, and a key query content vector group is extracted from the query content vector group based on the first score evaluation result, including: obtaining a query content vector evaluation graph by introducing an additional model branch, and using the query content vector evaluation graph to perform a first score evaluation on the query content vector group; sorting the query content vectors in the query content vector group from high to low according to the first score evaluation result; and selecting the first specified number of query content vectors from the sorted query content vector group to obtain the key query content vector group.

[0014] According to one or more embodiments of the present disclosure, the method further includes: using a classification loss function supervision signal based on binary cross entropy to supervise the query content vector evaluation graph.

[0015] According to one or more embodiments of the present disclosure, a second score evaluation is performed on a pre-generated query vector group, and a target query vector group is extracted from the query vector group based on the second score evaluation result, including: obtaining a query vector evaluation vector by introducing an additional model branch, and using the query vector evaluation vector to perform a second score evaluation on the pre-generated query vector group; sorting the query vectors in the query vector group from high to low according to the second score evaluation result; and selecting the first second specified number of query vectors from the sorted query vector group to obtain the target query vector group.

[0016] According to one or more embodiments of the present disclosure, the method further includes: supervising the query vector evaluation vector using a loss function based on mean absolute error.

[0017] According to one or more embodiments of the present disclosure, the additional model branch is implemented based on a convolutional neural network.

[0018] According to another aspect of an embodiment of the present disclosure, there is provided an apparatus for target detection, comprising:

[0019] An image feature extraction module is used to extract features from an input image to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group;

[0020] a first vector extraction module, configured to perform a first score evaluation on the query content vector group, and extract a key query content vector group from the query content vector group according to the first score evaluation result;

[0021] a second vector extraction module, configured to perform a second score evaluation on the pre-generated query vector group, and extract a target query vector group from the query vector group according to the second score evaluation result;

[0022] The attention mechanism calculation module is used to perform attention mechanism calculation based on the key query content vector group and the target query vector group to obtain a target detection result.

[0023] According to one or more embodiments of the present disclosure, the first vector extraction module is further used to: obtain a query content vector evaluation graph by introducing an additional model branch, and use the query content vector evaluation graph to perform a first score evaluation on the query content vector group; sort the query content vectors in the query content vector group from high to low according to the first score evaluation result; and select the first specified number of query content vectors from the sorted query content vector group to obtain the key query content vector group.

[0024] According to one or more embodiments of the present disclosure, the second vector extraction module is further configured to: obtain a query vector evaluation vector by introducing an additional model branch, and use the query vector evaluation vector to perform a second score evaluation on a pre-generated query vector group; sort the query vectors in the query vector group from high to low according to the second score evaluation result; and select the first second specified number of query vectors from the sorted query vector group to obtain the target query vector group.

[0025] According to another aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the target detection method provided by an embodiment of the present disclosure.

[0026] According to another aspect of the embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the target detection method provided by the embodiments of the present disclosure is implemented.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present disclosure and do not constitute an improper limitation of the present disclosure.

[0029] Figure 1 is a schematic diagram of the implementation principle of the existing attention mechanism for target detection;

[0030] FIG2 is a schematic diagram of the implementation principle of target detection based on the attention mechanism according to an embodiment of the present disclosure;

[0031] FIG3 is a schematic diagram of the main steps of a method for target detection according to an embodiment of the present disclosure;

[0032] FIG4 is a schematic diagram of main modules of an apparatus for target detection according to an embodiment of the present disclosure;

[0033] FIG5 is a diagram of an exemplary system architecture in which embodiments of the present disclosure may be applied;

[0034] FIG6 is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0036] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed in this application all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures will be taken with respect to user personal information to prevent unauthorized access to user personal information data and to maintain the security of user personal information, network security, and national security.

[0037] The computational complexity of the attention mechanism is proportional to the lengths of its three weight matrices: the query vector (Q), the key vector (K), and the value vector (V). Therefore, to accelerate attention-based models, the length of the QKV matrix needs to be reduced, thereby exponentially increasing computational speed. However, reducing the length of the QKV matrix (i.e., reducing the vector dimension) often means reduced feature resolution and loss of feature information, which is likely to result in reduced model performance.

[0038] In order to solve this problem, the present invention proposes a strategy for dynamically screening the attention mechanism QKV, which significantly reduces the length of QKV without reducing the feature resolution or the number of effective features, reduces the amount of calculation, and achieves a significant improvement in the model calculation speed without losing accuracy (or even improving accuracy), so that the vehicle side can perform real-time target detection.

[0039] Figure 1 is a schematic diagram of the existing implementation principle of target detection based on the attention mechanism. As shown in Figure 1, in the prior art, the input image is first extracted with features through a conventional convolutional neural network (CNN) to obtain a feature matrix F. The height of F is H, the width is W, and the number of channels is C. F is flattened in the plane dimension to obtain H*W feature vectors of dimension C, which serve as K and V of the attention mechanism, and the length of KV is H*W. At the same time, the length of the pre-generated query vector group Q is N. Q and KV are input together into the decoder of the sequence model Transformer based on the attention mechanism, and the calculation of the attention mechanism is performed to obtain the output of the decoder. The decoder output is passed through the subsequent neural network to obtain the final target detection result. In this process, the length H*W of KV and the length N of Q significantly affect the computational complexity of the attention mechanism, and often it is impossible to achieve a faster computing speed.

[0040] In order to accelerate the calculation of the attention mechanism, the embodiment of the present disclosure proposes a strategy for dynamic screening of KVQ. By introducing additional model branches, KV and Q are evaluated separately to achieve dynamic screening of KV and Q, so as to screen out the QKV vector that carries the most effective information, significantly reducing the computational complexity of the attention mechanism, improving the calculation speed, and realizing real-time target detection on the vehicle side.

[0041] FIG2 is a schematic diagram of the implementation principle of target detection based on the attention mechanism in an embodiment of the present disclosure. As shown in FIG2, in an embodiment of the present disclosure, the input image first adopts the same method as the prior art to obtain an original vector group KV of length H*W. Specifically, the input image is extracted with a conventional convolutional neural network (CNN) to obtain a feature matrix F. The height of F is H, the width is W, and the number of channels is C. F is flattened in the plane dimension to obtain H*W feature vectors of dimension C, which serve as K and V of the attention mechanism, and the length of KV is H*W.

[0042] At the same time, the target detection model of the present disclosure introduces an additional branch to obtain a matrix KV evaluation graph (Key & Value Score) for scoring KV. The number of elements in the KV evaluation graph is the same as the length of the original KV vector group, both of which are H*W, and the value of each element represents the score of the effective information carried by the K / V at the corresponding position. The higher the score, the richer the information and the more useful it is. Among all the original KV vector groups, a considerable number of positions correspond to invalid (or inefficient) areas in the input image, such as the sky. These areas have extremely low benefits for the final target detection, but they will bring a huge amount of computation. The lower the score of a position in the KV evaluation graph, the more likely it is to be the above-mentioned invalid (or inefficient) area. In order to allow only a few efficient areas to participate in the calculation, the KV evaluation graph is subjected to TOP K1 screening, that is, the areas with high K1 scores are screened out (for example, if K1=100, the top 100 positions with the highest scores are screened out). Specifically, in the model of the present disclosure, the value of K1 is, for example, H*W / 4, which reduces the number of KV vector groups to 1 / 4 of the original. After obtaining the index position of the TOP K1 score, only the elements corresponding to the TOP K1 index position of the original KV vector group are retained, resulting in a filtered K'V' vector group with a length of K1. During this screening process, the feature resolution is not reduced. Instead, by evaluating the KV quality, only the "foreground" areas with key features are retained, and the background areas are eliminated. Therefore, the amount of information loss is extremely small and will not cause a decrease in model accuracy. In fact, this method will make the model pay more attention to valuable key foreground areas (such as roads) and ignore background areas that may cause interference (such as the sky). In the actual training process, it will improve model accuracy while significantly reducing the number of KVs and increasing calculation speed.

[0043] Similar to the screening of the KV vector group, the target detection model disclosed in this disclosure introduces an additional branch to obtain a Q evaluation vector (Query Score) of length N that scores and evaluates Q. The number of elements in this vector is the same as the length of the original Q query vector group, and the score at each position is an assessment of the quality of each Q vector. Similarly, the Q evaluation vector is screened by the TOP K2 index (K2 is specifically set to 0.5N in this embodiment, that is, the number of Q vector groups is reduced by half), and only the index positions of the TOP K2 of the original Q vector group are retained, thus obtaining the dynamically screened query vector group Q' of length K2.

[0044] Finally, the K'V' vector group of length K1 and the Q' vector of length K2 enter the Transformer decoder for attention mechanism calculation, and ultimately output the object detection result. Because K1 is only 1 / 4 of H*W and K2 is only 1 / 2 of N, the attention mechanism's calculation speed is significantly increased without compromising model accuracy, achieving real-time on-board object detection.

[0045] In addition, in order to ensure the prediction accuracy of the target detection model, the KV evaluation map and the Q evaluation vector need to be able to objectively, impartially and accurately give the quality score of each index position QKV. For this purpose, it is necessary to introduce an additional loss function to supervise the two. First, for the KV evaluation map, a classification loss function supervision signal based on Focal loss (Focal loss is a loss function based on binary cross entropy) is adopted. For the production of the true value supervision signal, the present disclosure projects the geometric center point of the target object into the image and disposes 1 at the corresponding position in the true value map, and assigns a floating point score true value that decays to 0 within a certain Gaussian radius. In this way, the true value KV evaluation map has significant physical meaning, that is, the closer the pixel is to the center point of the target object, the higher the quality score is. Using this signal to supervise the predicted KV evaluation map can ensure that it can objectively and impartially give the quality score of each KV. For the Q evaluation vector, since the query vector in target detection is usually bound to the three-dimensional position coordinates (x, y, z) of the object, it can be supervised based on the conventional L1-Loss (mean absolute error) loss function. The closer the query vector is to the true target object, the higher the score.

[0046] Figure 3 is a schematic diagram of the main steps of the target detection method according to an embodiment of the present disclosure. As shown in Figure 3, the target detection method according to the embodiment of the present disclosure mainly includes the following steps S301 to S304.

[0047] Step S301: Feature extraction is performed on the input image to obtain a feature matrix, and the feature matrix is ​​flattened in a two-dimensional dimension to obtain a query content vector group. In one embodiment of the present disclosure, feature extraction is performed on the input image using a conventional convolutional neural network (CNN) to obtain a feature matrix F. F has a height of H, a width of W, and a number of channels C. F is flattened in a two-dimensional dimension to obtain H*W feature vectors of dimension C, which serve as the query vector K and content vector V for the attention mechanism. The query content vector group KV composed of K and V has a length of H*W.

[0048] Step S302: performing a first score evaluation on the query content vector group, and extracting a key query content vector group from the query content vector group according to the first score evaluation result.

[0049] According to one embodiment of the present disclosure, a first score evaluation is performed on the query content vector group, and a key query content vector group is extracted from the query content vector group based on the first score evaluation result. Specifically, it may include: obtaining a query content vector evaluation graph by introducing an additional model branch, and using the query content vector evaluation graph to perform a first score evaluation on the query content vector group; sorting the query content vectors in the query content vector group from high to low according to the first score evaluation result; and selecting the first specified number of query content vectors from the sorted query content vector group to obtain the key query content vector group. In an embodiment of the present disclosure, the additional model branch is implemented based on a convolutional neural network. The query content vector evaluation graph is used to perform a first score evaluation on the query content vector group, for example, by performing a convolution operation on the query content vector group through a convolutional neural network to perform the first score evaluation. In a specific implementation, when sorting the query content vectors in the query content vector group from high to low scores based on the first score evaluation result, for example, the sorting is performed based on the index position of each query content vector in the query content vector group. Subsequently, when selecting the first specified number of query content vectors that are ranked high, the first specified number of index positions that are ranked high can also be selected, and the corresponding query content vectors can be obtained based on the selected index positions. The first specified number can be flexibly set according to the needs of the business scenario.

[0050] By selecting key query vectors from the query vector group based on their scores, we can retain only the "foreground" regions of the input image that contain key features, while eliminating background regions. This minimizes information loss and prevents degradation of model accuracy. In fact, this approach allows the model to focus more on valuable key foreground regions (such as roads) and ignore potentially distracting background regions (such as the sky). This actually improves model accuracy during training, significantly reducing the number of key vectors in the query vector group and increasing computation speed.

[0051] Step S303: performing a second score evaluation on the pre-generated query vector group, and extracting a target query vector group from the query vector group according to the second score evaluation result;

[0052] According to another embodiment of the present disclosure, a second score evaluation is performed on a pre-generated query vector group, and a target query vector group is extracted from the query vector group based on the second score evaluation result. Specifically, it may include: obtaining a query vector evaluation vector by introducing an additional model branch, and using the query vector evaluation vector to perform a second score evaluation on the pre-generated query vector group; sorting the query vectors in the query vector group from high to low according to the score according to the second score evaluation result; and selecting the first second specified number of query vectors from the sorted query vector group to obtain the target query vector group. In an embodiment of the present disclosure, the additional model branch is implemented based on a convolutional neural network. The query vector evaluation vector is used to perform a second score evaluation on the query vector group, for example, by performing a convolution operation on the query vector group through a convolutional neural network to perform the second score evaluation. In a specific implementation, when sorting the query vectors in the query vector group from high to low scores based on the second score evaluation result, for example, the sorting is performed based on the index position of each query vector in the query vector group. Subsequently, when selecting the second specified number of query vectors that are ranked higher, the second specified number of index positions that are ranked higher can also be selected, and the corresponding query vectors can be obtained based on the selected index positions. The second specified number can be flexibly set according to the needs of the business scenario.

[0053] By selecting target query vectors from a set of query vectors based on their scores, only query vectors Q with high query information scores are retained, resulting in minimal information loss and no degradation in model accuracy. In fact, this approach allows the model to focus more on valuable query information, which can actually improve model accuracy during training. It also significantly reduces the number of query vectors in the set Q, increasing computational speed.

[0054] Step S304: Execute an attention mechanism calculation based on the key query content vector group and the target query vector group to obtain an object detection result. In the embodiment of the present disclosure, since the length of the KVQ input to the model is reduced, the calculation speed of the attention mechanism is significantly improved without reducing the model accuracy, thereby achieving real-time vehicle-side object detection.

[0055] According to one embodiment of the present disclosure, a classification loss function based on binary cross entropy is used as a supervision signal to supervise the query content vector evaluation graph, and a loss function based on mean absolute error is used to supervise the query vector evaluation vector. To ensure the model's prediction accuracy, so that the KV evaluation graph and Q evaluation vector can objectively, impartially, and accurately provide the QKV quality score for each index position, an additional loss function is introduced in this embodiment to supervise the KV evaluation graph and Q evaluation vector.

[0056] The KV evaluation map uses a supervision signal based on the Focal Loss classification loss function. To generate the ground-truth supervision signal, the geometric center of the target object (for loss function calculation, real-world objects are manually annotated; the target object is the ground-truth object derived from these annotations) is projected into the image and a 1 is assigned to the corresponding location in the ground-truth map. A floating-point score decaying to 0 is assigned within a certain Gaussian radius. This gives the ground-truth KV evaluation map significant physical meaning: pixels closer to the target object center receive higher quality scores. Using this signal to supervise the predicted KV evaluation map ensures that it objectively and impartially provides quality scores for each KV vector. For the Q evaluation vector, since the query vector in object detection is typically bound to the three-dimensional position coordinates (x, y, z), supervision can be performed based on conventional L1-loss, with query vectors closer to the ground-truth target object receiving higher scores.

[0057] By designing and introducing additional model branches, we obtain the KV evaluation graph and Q evaluation vector of the attention mechanism, accurately describing the quality score of the QKV vector. By screening it, we obtain the QKV vector with the highest score and the most effective information. This significantly reduces the computational complexity of the attention mechanism, improves the calculation speed, and enhances the prediction accuracy of the model, enabling real-time target detection on the vehicle side.

[0058] Figure 4 is a schematic diagram of the main modules of an object detection apparatus according to an embodiment of the present disclosure. As shown in Figure 4, the object detection apparatus 400 according to the embodiment of the present disclosure mainly includes an image feature extraction module 401, a first vector extraction module 402, a second vector extraction module 403, and an attention mechanism calculation module 404.

[0059] An image feature extraction module 401 is configured to extract features from an input image to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group;

[0060] a first vector extraction module 402 configured to perform a first score evaluation on the query content vector group and extract a key query content vector group from the query content vector group based on the first score evaluation result;

[0061] A second vector extraction module 403 is configured to perform a second score evaluation on the pre-generated query vector group, and extract a target query vector group from the query vector group according to the second score evaluation result;

[0062] The attention mechanism calculation module 404 is used to perform attention mechanism calculation based on the key query content vector group and the target query vector group to obtain a target detection result.

[0063] According to one embodiment of the present disclosure, the first vector extraction module 402 can also be used to: obtain a query content vector evaluation graph by introducing an additional model branch, and use the query content vector evaluation graph to perform a first score evaluation on the query content vector group; sort the query content vectors in the query content vector group from high to low according to the first score evaluation result; and select the first specified number of query content vectors from the sorted query content vector group to obtain the key query content vector group.

[0064] According to another embodiment of the present disclosure, the target detection device 400 may further include an evaluation vector supervision module (not shown in the figure), which is used to supervise the query content vector evaluation graph using a classification loss function supervision signal based on binary cross entropy.

[0065] According to another embodiment of the present disclosure, the second vector extraction module 403 may also be used to: obtain a query vector evaluation vector by introducing an additional model branch, and use the query vector evaluation vector to perform a second score evaluation on a pre-generated query vector group; sort the query vectors in the query vector group from high to low according to the second score evaluation result; and select the first second specified number of query vectors from the sorted query vector group to obtain the target query vector group.

[0066] According to another embodiment of the present disclosure, the evaluation vector supervision module (not shown in the figure) may also be configured to supervise the query vector evaluation vector using a loss function based on mean absolute error.

[0067] According to yet another embodiment of the present disclosure, the additional model branch is implemented based on a convolutional neural network.

[0068] According to the technical solution of the embodiment of the present disclosure, a feature matrix is ​​obtained by extracting features from the input image, and the feature matrix is ​​flattened in the plane dimension to obtain a query content vector group; a first score evaluation is performed on the query content vector group, and a key query content vector group is extracted from the query content vector group according to the first score evaluation result; a second score evaluation is performed on the pre-generated query vector group, and a target query vector group is extracted from the query vector group according to the second score evaluation result; an attention mechanism is calculated based on the key query content vector group and the target query vector group to obtain a technical solution for target detection results, by performing a quality score evaluation on the QKV vector of the attention mechanism, and then filtering out some QKV vectors carrying the most effective information, based on a dynamic screening strategy for the number of QKVs, the calculation amount of the attention mechanism is significantly reduced without reducing the feature resolution and the effective features, and the calculation speed is improved, thereby achieving the effect of real-time target detection on the vehicle side.

[0069] FIG5 shows an exemplary system architecture 500 to which the object detection method or the object detection apparatus according to the embodiments of the present disclosure may be applied.

[0070] As shown in Figure 5, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0071] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as vehicle navigation applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0072] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0073] Server 505 can be a server that provides various services, such as a backend management server (for example only) that supports websites browsed by users using terminal devices 501, 502, and 503. The backend management server can extract features from the input image for the received target detection request and other data to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group; perform a first score evaluation on the query content vector group, and extract a key query content vector group from the query content vector group based on the first score evaluation result; perform a second score evaluation on the pre-generated query vector group, and extract a target query vector group from the query vector group based on the second score evaluation result; perform attention mechanism calculation based on the key query content vector group and the target query vector group to obtain target detection results and other processing, and feed back the processing results (for example, target detection results - for example only) to the terminal device.

[0074] It should be noted that the target detection method provided in the embodiments of the present disclosure is generally executed by the server 505 , and accordingly, the target detection device is generally set in the server 505 .

[0075] It should be understood that the number of terminal devices, networks and servers in Figure 5 is merely illustrative and any number of terminal devices, networks and servers may be provided as required.

[0076] 6, which shows a schematic diagram of a computer system 600 suitable for implementing a terminal device or server according to an embodiment of the present disclosure. The terminal device or server shown in FIG6 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0077] As shown in FIG6 , a computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the system 600 are also stored in the RAM 603. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0078] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, and the like; an output section 607 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or a modem. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 610 as needed, so that computer programs read therefrom can be installed into the storage section 608 as needed.

[0079] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present disclosure are performed.

[0080] It should be noted that the computer-readable medium described in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0081] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0082] The units or modules involved in the embodiments described in the present disclosure may be implemented in software or in hardware. The units or modules described may also be provided in a processor. For example, they may be described as: a processor including an image feature extraction module, a first vector extraction module, a second vector extraction module, and an attention mechanism calculation module. In some cases, the names of these units or modules do not constitute a limitation on the units or modules themselves. For example, the attention mechanism calculation module may also be described as "a module for performing attention mechanism calculations based on the key query content vector group and the target query vector group to obtain target detection results."

[0083] As another aspect, the present disclosure further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: extracting features from an input image to obtain a feature matrix, and flattening the feature matrix in a plane dimension to obtain a query content vector group; performing a first score evaluation on the query content vector group, and extracting a key query content vector group from the query content vector group based on the first score evaluation result; performing a second score evaluation on a pre-generated query vector group, and extracting a target query vector group from the query vector group based on the second score evaluation result; performing an attention mechanism calculation based on the key query content vector group and the target query vector group to obtain a target detection result.

[0084] According to the technical solution of the embodiment of the present disclosure, a feature matrix is ​​obtained by extracting features from the input image, and the feature matrix is ​​flattened in the plane dimension to obtain a query content vector group; a first score evaluation is performed on the query content vector group, and a key query content vector group is extracted from the query content vector group according to the first score evaluation result; a second score evaluation is performed on the pre-generated query vector group, and a target query vector group is extracted from the query vector group according to the second score evaluation result; an attention mechanism is calculated based on the key query content vector group and the target query vector group to obtain a technical solution for target detection results, by performing a quality score evaluation on the QKV vector of the attention mechanism, and then filtering out some QKV vectors carrying the most effective information, based on a dynamic screening strategy for the number of QKVs, the calculation amount of the attention mechanism is significantly reduced without reducing the feature resolution and the effective features, and the calculation speed is improved, thereby achieving the effect of real-time target detection on the vehicle side.

[0085] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for target detection, comprising: Extract features from the input image to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group; performing a first score evaluation on the query content vector group, and extracting a key query content vector group from the query content vector group according to the first score evaluation result; performing a second score evaluation on the pre-generated query vector group, and extracting a target query vector group from the query vector group according to the second score evaluation result; An attention mechanism is performed based on the key query content vector group and the target query vector group to obtain a target detection result.

2. The method according to claim 1, wherein Performing a first score evaluation on the query content vector group, and extracting a key query content vector group from the query content vector group according to the first score evaluation result, includes: Obtaining a query content vector evaluation graph by introducing an additional model branch, and performing a first score evaluation on the query content vector group using the query content vector evaluation graph; sorting the query content vectors in the query content vector group according to the first score evaluation result from high to low scores; The first specified number of query content vectors are selected from the sorted query content vector group to obtain the key query content vector group.

3. The method according to claim 2, further comprising: A classification loss function supervision signal based on binary cross entropy is used to supervise the query content vector evaluation graph.

4. The method according to claim 1, wherein Performing a second score evaluation on the pre-generated query vector group, and extracting a target query vector group from the query vector group according to the second score evaluation result, includes: Obtaining a query vector evaluation vector by introducing an additional model branch, and performing a second score evaluation on the pre-generated query vector group using the query vector evaluation vector; sorting the query vectors in the query vector group from high to low scores according to the second score evaluation result; The first second specified number of query vectors are selected from the sorted query vector group to obtain the target query vector group.

5. The method according to claim 4, further comprising: The query vector evaluation vector is supervised using a loss function based on mean absolute error.

6. The method according to claim 2 or 4, wherein: The additional model branch is implemented based on a convolutional neural network.

7. A target detection device, comprising: An image feature extraction module is used to extract features from an input image to obtain a feature matrix, and flatten the feature matrix in a plane dimension to obtain a query content vector group; a first vector extraction module, configured to perform a first score evaluation on the query content vector group, and extract a key query content vector group from the query content vector group according to the first score evaluation result; a second vector extraction module, configured to perform a second score evaluation on the pre-generated query vector group, and extract a target query vector group from the query vector group according to the second score evaluation result; The attention mechanism calculation module is used to perform attention mechanism calculation based on the key query content vector group and the target query vector group to obtain a target detection result.

8. The device according to claim 7, wherein The first vector extraction module is further configured to: Obtaining a query content vector evaluation graph by introducing an additional model branch, and performing a first score evaluation on the query content vector group using the query content vector evaluation graph; sorting the query content vectors in the query content vector group according to the first score evaluation result from high to low scores; The first specified number of query content vectors are selected from the sorted query content vector group to obtain the key query content vector group.

9. The device according to claim 7, wherein The second vector extraction module is further configured to: Obtaining a query vector evaluation vector by introducing an additional model branch, and performing a second score evaluation on the pre-generated query vector group using the query vector evaluation vector; sorting the query vectors in the query vector group from high to low scores according to the second score evaluation result; The first second specified number of query vectors are selected from the sorted query vector group to obtain the target query vector group.

10. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

11. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images