Target detection method and device, electronic equipment, medium and program product

Abstract

The present disclosure provides a target detection method and device, electronic equipment, medium and program product, which can be applied to the field of computer and financial technology. The target detection method comprises: acquiring a target image to be detected; performing feature extraction on the target image to obtain a preprocessed feature map; based on a Transformer module, splitting the preprocessed feature map along a first direction to obtain a first feature map, and splitting the first feature map along a second direction to obtain a second feature map; performing feature fusion on the first feature map and the second feature map to obtain a fused feature map; and performing detection on the fused feature map to obtain a detection result for a target object.

Description

Technical Field

[0001] This disclosure relates to the fields of computers and finance, and more specifically to a target detection method, apparatus, device, medium, and program product. Background Technology

[0002] Image object detection algorithms are an important research direction in deep learning. Before deep learning, based on the labeled features, the target in the image or video frame is found, and candidate boxes are generated through selective search, followed by classification and regression.

[0003] Currently, object detection methods can extract most of the features through convolutional networks and then perform object detection, but the detection accuracy is not high. Summary of the Invention

[0004] In view of the above problems, this disclosure provides a target detection method, apparatus, device, medium, and program product to improve the accuracy of target detection, and to at least partially solve the above technical problems.

[0005] According to a first aspect of the present disclosure, a target detection method is provided, comprising: acquiring a target image to be detected; extracting features from the target image to obtain a preprocessed feature map; splitting the preprocessed feature map along a first direction based on a Transformer module to obtain a first feature map, and splitting the first feature map along a second direction to obtain a second feature map; fusing the first feature map and the second feature map to obtain a fused feature map; and detecting the fused feature map to obtain a detection result for the target object.

[0006] According to an embodiment of the present disclosure, splitting the preprocessed feature map along a first direction to obtain a first feature map includes: splitting the preprocessed feature map along the width direction of the preprocessed feature map to obtain a plurality of first sub-feature maps with a unit width, wherein the size of the first sub-feature map is the dimension of the preprocessed feature map multiplied by the length of the preprocessed feature map.

[0007] The first feature map is obtained by fusing the first sub-feature map with multiple units of width with the preprocessed feature map.

[0008] According to embodiments of this disclosure, fusing multiple first sub-feature maps of unit width with a preprocessed feature map to obtain a first feature map includes: obtaining a query matrix, a key matrix, and a value matrix of the first sub-feature map using a Transformer module; multiplying the first sub-feature map with the query matrix, key matrix, and value matrix of the first sub-feature map respectively to obtain a first query vector, a first key vector, and a first value vector; normalizing the first query vector, the first key vector, and the first value vector to obtain a second sub-feature map; concatenating multiple second sub-feature maps in the width direction to obtain a third sub-feature map; and obtaining the first feature map based on the preprocessed feature map and the third sub-feature map.

[0009] According to an embodiment of this disclosure, normalizing the first query vector, the first key vector, and the first value vector to obtain a second sub-feature map includes: transposing the first key vector to obtain a first transpose result; normalizing the first transpose result to obtain a first normalization result; and calculating the first transpose result, the first normalization result, the first query vector, and the first value vector using an activation function to obtain a second sub-feature map.

[0010] According to an embodiment of this disclosure, obtaining a first feature map based on a preprocessed feature map and a third sub-feature map includes: adding the preprocessed feature map and the third sub-feature map to obtain the first feature map.

[0011] According to an embodiment of this disclosure, splitting the first feature map along a second direction to obtain a second feature map includes: splitting the first feature map along the length direction of the first feature map to obtain multiple fourth sub-feature maps of unit length, wherein the size of the fourth sub-feature map is the dimension of the preprocessed feature map multiplied by the width of the preprocessed feature map; and fusing the multiple fourth sub-feature maps of unit length to obtain the second feature map.

[0012] According to an embodiment of this disclosure, fusing multiple fourth sub-feature maps of unit length to obtain a second feature map includes: obtaining the query matrix, key matrix, and value matrix of the fourth sub-feature map according to the Transformer module;

[0013] The fourth sub-feature map is multiplied by its query matrix, key matrix, and value matrix to obtain the second query vector, second key vector, and second value vector, respectively. The second query vector, second key vector, and second value vector are normalized to obtain the fifth sub-feature map. Multiple fifth sub-feature maps are concatenated along their length to obtain the second feature map.

[0014] According to an embodiment of this disclosure, fusing the first feature map and the second feature map to obtain a fused feature map includes: adding the first feature map and the second feature map to obtain a third feature map; and inputting the third feature map into the Transformer module based on the Transformer module to obtain the fused feature map.

[0015] According to embodiments of this disclosure, acquiring a target image to be detected includes: acquiring multiple types of target images and target region information in the target images; wherein at least one type of target image contains a target to be detected.

[0016] According to embodiments of this disclosure, feature extraction of the target image to obtain a preprocessed feature map includes: inputting the target image into a lightweight convolutional network MobileNet for feature extraction to obtain a preprocessed feature map.

[0017] A second aspect of this disclosure provides a target detection apparatus, comprising: an acquisition module for acquiring a target image to be detected; a Transformer module for splitting a preprocessed feature map from a first direction to obtain a first feature map, and splitting the first feature map from a second direction to obtain a second feature map; a fusion module for fusing the first feature map and the second feature map to obtain a fused feature map; and a classification detection module for detecting the fused feature map to obtain a detection result for a target object.

[0018] A third aspect of this disclosure provides an electronic device, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the target detection method described above.

[0019] A fourth aspect of this disclosure also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the target detection method described above.

[0020] A fifth aspect of this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described target detection method. Attached Figure Description

[0021] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0022] Figure 1 The illustration shows an application scenario of the target detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure;

[0023] Figure 2 A flowchart illustrating a target detection method according to an embodiment of the present disclosure is shown schematically.

[0024] Figure 3 This schematically illustrates a flowchart of splitting a preprocessed feature map along a first direction in a target detection method according to an embodiment of the present disclosure;

[0025] Figure 4 This schematically illustrates the principle of splitting a preprocessed feature map along a first direction in a target detection method according to an embodiment of the present disclosure;

[0026] Figure 5 The flowchart illustrating the fusion of a first sub-feature map and a preprocessed feature map in a target detection method according to an embodiment of the present disclosure is shown in the schematic diagram.

[0027] Figure 6 This schematically illustrates a flowchart of the normalization process for the first query vector, the first key vector, and the first value vector in the target detection method according to an embodiment of the present disclosure;

[0028] Figure 7 This schematically illustrates a flowchart of splitting a preprocessed feature map along a second direction in a target detection method according to an embodiment of the present disclosure;

[0029] Figure 8 This schematically illustrates the principle of splitting a preprocessed feature map along a second direction in a target detection method according to an embodiment of the present disclosure;

[0030] Figure 9 A flowchart illustrating the fourth sub-feature map fusion in a target detection method according to an embodiment of the present disclosure is shown schematically.

[0031] Figure 10 The flowchart illustrating the fusion of a first feature map and a second feature map in a target detection method according to an embodiment of the present disclosure is shown in the schematic diagram.

[0032] Figure 11 A schematic block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown; and

[0033] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a target detection method according to an embodiment of the present disclosure. Detailed Implementation

[0034] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0035] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0036] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0037] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0038] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0039] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0040] It should be noted that the methods and apparatus specified in this disclosure for target detection methods, devices, equipment, media, and program products can be used in the financial field for applications of face recognition, and can also be used in any field other than the financial field for applications of face recognition. This disclosure does not limit the application field of the target detection methods and apparatus.

[0041] Embodiments of this disclosure provide a target detection method, comprising: acquiring a target image to be detected; extracting features from the target image to obtain a preprocessed feature map; splitting the preprocessed feature map along a first direction to obtain a first feature map based on a Transformer module, and splitting the first feature map along a second direction to obtain a second feature map; fusing the first feature map and the second feature map to obtain a fused feature map; and performing detection on the fused feature map to obtain a detection result for the target object.

[0042] Figure 1 The illustration shows an application scenario of the target detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure.

[0043] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0044] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0045] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0046] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0047] It should be noted that the target detection method provided in this embodiment can generally be executed by server 105. Correspondingly, the target detection device provided in this embodiment can generally be located in server 105. The target detection method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the target detection device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0048] It should be understood that Figure 1 The number of terminal devices, networks, and servers / server clusters shown is merely illustrative. Any number of terminal devices, networks, and servers / server clusters can be included depending on actual needs.

[0049] The following will be based on Figure 1 The described scene, through Figures 2-8 The target detection method according to the disclosed embodiments is described in detail. It should be noted that the sequence numbers of each operation in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0050] Figure 2 A flowchart illustrating a target detection method according to an embodiment of the present disclosure is shown schematically.

[0051] like Figure 2 As shown, the target detection method in this embodiment includes operations S210 to S250.

[0052] In operation S210, the target image to be detected is acquired.

[0053] In some embodiments, acquiring the target image to be detected includes: acquiring multiple types of target images and target region information within the target images. Wherein, at least one type of target image contains the target to be detected.

[0054] For example, a target image can be understood as image data containing a target object collected in a target scene. The image data can be a screenshot from a video file or a photograph taken. Target scenes can include: remote authorization-based business processing scenarios, remote authorization-based credit card issuance scenarios, or remote authorization-based transaction payment scenarios, etc. In the above target scenes, users can handle specific business online through remote authorization without going to the site. Target objects can include the user's identification documents (e.g., the user's ID card, passport, etc.) or the user's business documents (e.g., the user's business application form, etc.). It should be noted that, depending on the specific circumstances and the target scene, the above target objects can also include other categories; this disclosure does not specifically limit the target objects.

[0055] Multiple types of target images can be image data acquired from different target scenes. The image data may or may not contain target objects. Multiple types of target images can improve the accuracy of target detection results.

[0056] In some embodiments, the target image can be obtained from a known database, and the target region information in the target image in the known database is also known; the target image can also be temporarily acquired image data, and the target region information in the temporarily acquired image data can be the target region corresponding to the target object determined based on a deep learning neural network model. The deep learning neural network model can be, but is not limited to, a CNN model of a convolutional neural network, and the target region information can be, but is not limited to, the position, size, and other data of the target object in the target image.

[0057] The accuracy of the method can be verified by comparing the target region information of the pre-acquired target image with the detection results of the method, confirming the improvement in accuracy of the method compared with the existing technology.

[0058] In operation S220, feature extraction is performed on the target image to obtain a preprocessed feature map.

[0059] The process of extracting features from the target image to obtain a preprocessed feature map includes: inputting the target image into the lightweight convolutional network MobileNet for feature extraction to obtain a preprocessed feature map.

[0060] For example, the MobileNet network mentioned above uses the CNN network structure of MobileNetV3, which not only reduces the number of parameters and computational cost but also improves the accuracy of object detection. It should be noted that this application does not impose specific limitations on the CNN network; an RNN network can also be used.

[0061] In operation S230, based on the Transformer module, the preprocessed feature map is split along the first direction to obtain the first feature map, and the first feature map is split along the second direction to obtain the second feature map.

[0062] The preprocessed feature map is a feature vector, and the features include, but are not limited to, color features, texture features, shape features, spatial relationship features, etc. The size of the preprocessed feature map is the product of its width, length, and dimension (channels). Let the preprocessed feature map be denoted as M1, the width as w, the length as h, and the dimension as c, then the size of M1 is equal to c × h × w. The first direction includes the width and length of the preprocessed feature map, and the second direction includes the width and length of the preprocessed feature map. It should be noted that the first direction and the second direction are two different directions, that is, if the first direction is the width, the second direction is the length, and if the first direction is the length, the second direction is the width.

[0063] The Transformer module includes an encoder layer and a decoder layer. In this embodiment, the preprocessed feature map is split in the encoder.

[0064] In operation S240, the first feature map and the second feature map are fused to obtain a fused feature map;

[0065] The S250 is used to detect the fused feature map, and the detection results for the target object are obtained.

[0066] Those skilled in the art will understand that the object detection method according to embodiments of this disclosure employs a model combining a CNN network and a Transformer, which increases the receptive field of features in the target image while preserving the original attributes of the underlying features, thereby making the prediction of the target region in object detection more accurate. In the Transformer, the model structure is split and computed from the first and second directions respectively, reducing the model's memory consumption, accelerating model training, and improving the accuracy of model prediction.

[0067] Figure 3 The flowchart illustrating the splitting of a preprocessed feature map along a first direction in a target detection method according to an embodiment of the present disclosure is shown.

[0068] like Figure 3 As shown, the preprocessed feature map is split along the first direction in this embodiment to obtain the first feature map, including operations S2310 to S2320.

[0069] In operation S2310, the preprocessed feature map is split along the width direction to obtain multiple first sub-feature maps with a width of one unit. The size of the first sub-feature map is the dimension of the preprocessed feature map multiplied by the length of the preprocessed feature map.

[0070] In operation S2320, the first sub-feature map with multiple unit widths is fused with the preprocessed feature map to obtain the first feature map.

[0071] The width direction of the preprocessed feature map is the same as the row direction of the preprocessed feature map. The first sub-feature map after splitting is denoted as Xn, and the number of Xn is w. The w Xn are denoted as X1, X2, X3, ..., and the size of Xn is c×h.

[0072] Figure 4 This schematically illustrates the principle of splitting a preprocessed feature map along a first direction in a target detection method according to an embodiment of the present disclosure; Figure 5 The flowchart illustrating the fusion of a first sub-feature map and a preprocessed feature map in a target detection method according to an embodiment of the present disclosure is shown.

[0073] like Figure 4 and 5 As shown, the preprocessed feature map is split along the first direction in this embodiment to obtain the first feature map, including operations S2321 to S2325.

[0074] In operation S2321, the query matrix, key matrix, and value matrix of the first sub-feature map are obtained according to the Transformer module.

[0075] Specifically, the query matrix, key matrix, and value matrix of each of the w first sub-feature maps are obtained, and the query matrix of each first sub-feature map is denoted as W. Q The key matrix is ​​denoted as W. K The value matrix is ​​denoted as W. V The size of the query matrix, key matrix, and value matrix is ​​h×w.

[0076] In operation S2322, the first sub-feature map is multiplied by the query matrix, key matrix and value matrix of the first sub-feature map respectively to obtain the first query vector, the first key vector and the first value vector respectively.

[0077] Each first sub-feature map is multiplied by its corresponding query matrix, key matrix, and value matrix, and a first sub-feature map corresponds to a set of query matrices, key matrices, and value matrices.

[0078] For example, taking a first sub-feature map as an example, let's denote the first sub-feature map as X1, then:

[0079] First query vector = X1 × WQ The size of the first query vector is c×w;

[0080] First key vector (key vector) = X1 × W K The size of the first key vector is c×w;

[0081] First value vector = X1 × W V The size of the first value vector is c×w.

[0082] In operation S2323, the first query vector, the first key vector, and the first value vector are normalized to obtain the second sub-feature map.

[0083] In this process, the first query vector, the first key vector, and the first value vector corresponding to each first sub-feature map are normalized to obtain w second sub-feature maps. The second sub-feature map is denoted as An, and the w second sub-feature maps are denoted as A1, A2, A3, ... in sequence.

[0084] In operation S2324, multiple second sub-feature maps are concatenated in the width direction to obtain a third sub-feature map.

[0085] Among them, the second sub-feature maps A1, A2, A3... (w) are sequentially spliced ​​along the width direction to obtain the third sub-feature map, which is denoted as M2. The size of the third sub-feature map is c×h×w.

[0086] In operation S2325, the first feature map is obtained based on the preprocessed feature map and the third sub-feature map.

[0087] According to an embodiment of this disclosure, obtaining a first feature map based on a preprocessed feature map and a third sub-feature map includes: adding the preprocessed feature map and the third sub-feature map to obtain the first feature map. The first feature map is denoted as M3, i.e., M3 = M1 + M2.

[0088] Figure 6 The flowchart illustrating the normalization process of the first query vector, the first key vector, and the first value vector in the target detection method according to an embodiment of the present disclosure is shown.

[0089] like Figure 4 and 6 As shown, the normalization process of the first query vector, the first key vector, and the first value vector in this embodiment to obtain the second sub-feature map includes: operations S2323a to S2323c.

[0090] In operation S2323a, the first key vector is transposed to obtain the first transpose result.

[0091] In operation S2323b, the first transpose result is normalized to obtain the first normalized result.

[0092] In operation S2323c, the first transpose result, the first normalization result, the first query vector, and the first value vector are calculated using an activation function to obtain the second sub-feature map.

[0093] Specifically, taking a second sub-feature map as an example, the second sub-feature map corresponds one-to-one with the first sub-feature map X1. The relationship is constructed based on the first query vector, the first key vector, and the first value vector:

[0094]

[0095] Where Z1 is the second sub-feature image, Q1 is the first query vector, and K1 is the second sub-feature image. T This is the result of transposing the first key vector. For Q1×K1 T The result is normalized and calculated. V1 is the first value vector, and Q1, K1 and V1 are the first query vector, first key vector and first value vector corresponding to the first sub-feature map X1, respectively.

[0096] It should be noted that A1, A2, A3… are respectively related to Z1 1 Z1 2 Z1 3 …one-to-one correspondence.

[0097] Figure 7 This schematically illustrates a flowchart of splitting a preprocessed feature map along a second direction in a target detection method according to an embodiment of the present disclosure; Figure 8 The diagram illustrates the principle of splitting a preprocessed feature map along a second direction in a target detection method according to an embodiment of the present disclosure.

[0098] like Figure 7 and 8 As shown, the method of splitting the first feature map along the second direction to obtain the second feature map in this embodiment includes operations S2330 to S2340.

[0099] In operation S2330, the first feature map is split along the length direction of the first feature map to obtain multiple fourth sub-feature maps of unit length, wherein the size of the fourth sub-feature map is the dimension of the preprocessed feature map multiplied by the width of the preprocessed feature map.

[0100] The length direction of the preprocessed feature map is the same as the column direction of the preprocessed feature map. The fourth sub-feature map after splitting is denoted as Yn, and the number of Yn is h. The h Yn are denoted as Y1, Y2, Y3, ..., and the size of Yn is c×w.

[0101] In operation S2340, multiple fourth sub-feature maps of unit length are fused to obtain the second feature map.

[0102] Figure 9 A flowchart illustrating the fourth sub-feature map fusion in a target detection method according to an embodiment of the present disclosure is shown schematically.

[0103] like Figure 8 and 9 As shown, the second feature map obtained by fusing multiple fourth sub-feature maps of unit length in this embodiment includes operations S2341 to S2344.

[0104] In operation S2341, the query matrix, key matrix, and value matrix of the fourth sub-feature map are obtained according to the Transformer module.

[0105] Specifically, the query matrix, key matrix, and value matrix of each of the h fourth sub-feature maps are obtained, and the query matrix of each fourth sub-feature map is denoted as W. Q The key matrix is ​​denoted as W. K The value matrix is ​​denoted as W. V The size of the query matrix, key matrix, and value matrix is ​​h×w.

[0106] In operation S2342, the fourth sub-feature map is multiplied by the query matrix, key matrix and value matrix of the fourth sub-feature map respectively to obtain the second query vector, the second key vector and the second value vector respectively.

[0107] Each fourth sub-feature map is multiplied by its corresponding query matrix, key matrix, and value matrix, and a fourth sub-feature map corresponds to a set of query matrices, key matrices, and value matrices.

[0108] For example, taking a fourth sub-feature map as an example, let's denote the fourth sub-feature map as Y1, then:

[0109] Second query vector = Y1 × W Q The size of the first query vector is c×w;

[0110] Second key vector (key vector) = Y1 × W K The size of the first bond vector is c×h;

[0111] Second value vector = Y1 × W V The size of the first value vector is c×h.

[0112] In operation S2343, the second query vector, the second key vector, and the second value vector are normalized to obtain the fifth sub-feature map.

[0113] Taking the fourth sub-feature map as an example, the fourth sub-feature corresponds one-to-one with the fifth sub-feature Y1. The relationship is constructed based on the second query vector, the second key vector, and the second value vector:

[0114]

[0115] Where Z2 is the fifth sub-feature image, Q2 is the second query vector, and K2 is the second query vector. T This is the result of transposing the second key vector. For Q2×K2 T The result is normalized and calculated. V2 is the second value vector. Q2, K2 and V2 are the second query vector, second key vector and second value vector corresponding to the fourth sub-feature map Y1, respectively.

[0116] Normalize the second query vector, second key vector and second value vector corresponding to each fourth sub-feature map to obtain h fifth sub-feature maps. The fifth sub-feature map is denoted as Bn, and the h fifth sub-feature maps are denoted as B1, B2, B3 and so on.

[0117] It should be noted that B1, B2, B3… are respectively related to Z2 1 Z2 2 Z2 3 …one-to-one correspondence.

[0118] In operation S2344, multiple fifth sub-feature maps are concatenated along the length direction to obtain the second feature map.

[0119] Specifically, the fifth sub-feature maps B1, B2, B3... (h) are sequentially spliced ​​along the width direction to obtain the second feature map. The second feature map is denoted as M4, and its size is c×h×w.

[0120] Figure 10 The flowchart illustrating the fusion of a first feature map and a second feature map in a target detection method according to an embodiment of the present disclosure is shown.

[0121] like Figure 10 As shown, the feature fusion of the first feature map and the second feature map in this embodiment to obtain the fused feature map includes operations S2510 to S2520.

[0122] In operation S2510, the first feature map and the second feature map are added together to obtain the third feature map.

[0123] Specifically, if the third feature map is denoted as M5, then M3 + M4 = M5.

[0124] When operating the S2520, based on the Transformer module, the third feature map is input into the Transformer module to obtain the fused feature map.

[0125] Specifically, the third feature map is input into the Transformer module, and the third feature map is processed by the encoder and decoder layers in the Transformer module in sequence to obtain the fused feature map.

[0126] Understandably, performing a first-direction split (row split) on the preprocessed feature map reduces the computational cost of this method, thereby reducing the model's memory consumption and accelerating model training. At the same time, performing a second-direction split (column split) on the feature map after the first-direction (row split) split enables interaction between features, enhances the correlation between features, and improves the accuracy of model detection.

[0127] It should be noted that in this embodiment, there is no specific limitation on whether the preprocessed feature map is split in the first direction (row split) or the second direction (column split). It can be that the preprocessed feature map is first split in the row and then the feature map obtained by the row split is further split in the column, or the preprocessed feature map is first split in the column and then the feature map obtained by the column split is further split in the row. It can be set according to the actual design.

[0128] Based on the above-described target detection method, this disclosure also provides a target detection device. The following will be combined with... Figure 11 The device is described in detail.

[0129] Figure 11 A schematic block diagram of a target detection apparatus according to an embodiment of the present disclosure is shown.

[0130] like Figure 11 As shown, the target detection device 400 in this embodiment includes an acquisition module 410, a Transformer module 420, a fusion module 430, and a classification detection module 440.

[0131] The acquisition module 410 is used to acquire the target image to be detected. In one embodiment, the acquisition module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0132] Transformer module 420 is used to split the preprocessed feature map along a first direction to obtain a first feature map, and to split the first feature map along a second direction to obtain a second feature map. In one embodiment, Transformer module 420 can be used to perform the operation S220 described above, which will not be repeated here.

[0133] The fusion module 430 is used to fuse the first feature map and the second feature map to obtain a fused feature map. In one embodiment, the fusion module 430 can be used to perform the operation S230 described above, which will not be repeated here.

[0134] The classification and detection module 440 is used to detect the fused feature map to obtain the detection result for the target object. In one embodiment, the classification and detection module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0135] According to embodiments of this disclosure, any plurality of modules among the acquisition module 410, Transformer module 420, fusion module 430, and classification detection module 440 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 410, Transformer module 420, fusion module 430, and classification detection module 440 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 410, Transformer module 420, fusion module 430 and classification detection module 440 may be implemented at least partially as a computer program module that can perform corresponding functions when the computer program module is run.

[0136] Figure 12 A block diagram schematically illustrates an electronic device suitable for implementing a target detection method according to an embodiment of the present disclosure.

[0137] like Figure 12As shown, an electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0138] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0139] According to embodiments of this disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0140] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0141] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0142] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the item recommendation method provided in the embodiments of this disclosure.

[0143] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0144] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0145] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0146] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0149] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A target detection method, comprising: Acquire the target image to be detected; Feature extraction is performed on the target image to obtain a preprocessed feature map; Based on the Transformer module, the preprocessed feature map is split along a first direction to obtain a first feature map, and then split along a second direction to obtain a second feature map. This includes: splitting the preprocessed feature map along its width to obtain multiple first sub-feature maps of unit width, wherein the size of each first sub-feature map is the dimension of the preprocessed feature map multiplied by its length; fusing the multiple first sub-feature maps of unit width with the preprocessed feature map to obtain the first feature map; splitting the first feature map along its length to obtain multiple fourth sub-feature maps of unit length, wherein the size of each fourth sub-feature map is the dimension of the preprocessed feature map multiplied by its width; and fusing the multiple fourth sub-feature maps of unit length to obtain the second feature map. The first feature map and the second feature map are fused to obtain a fused feature map; and The fused feature map is then used for detection to obtain detection results for the target object; The process of fusing the multiple unit-width first sub-feature maps with the preprocessed feature map to obtain the first feature map includes: obtaining the query matrix, key matrix, and value matrix of the first sub-feature map according to the Transformer module; multiplying the first sub-feature map with the query matrix, key matrix, and value matrix of the first sub-feature map respectively to obtain a first query vector, a first key vector, and a first value vector; normalizing the first query vector, the first key vector, and the first value vector to obtain a second sub-feature map; concatenating multiple second sub-feature maps in the width direction to obtain a third sub-feature map; and obtaining the first feature map based on the preprocessed feature map and the third sub-feature map includes: adding the preprocessed feature map and the third sub-feature map to obtain the first feature map.

2. The method according to claim 1, wherein, The normalization process of the first query vector, the first key vector, and the first value vector to obtain the second sub-feature map includes: The first key vector is transposed to obtain the first transpose result. The first transpose result is normalized to obtain the first normalized result; The first transpose result, the first normalization result, the first query vector, and the first value vector are used to calculate the second sub-feature map through an activation function.

3. The method according to claim 1, wherein, The step of fusing the multiple fourth sub-feature maps of unit length to obtain the second feature map includes: Based on the Transformer module, obtain the query matrix, key matrix, and value matrix of the fourth sub-feature map; The fourth sub-feature map is multiplied by the query matrix, key matrix and value matrix of the fourth sub-feature map respectively to obtain the second query vector, the second key vector and the second value vector respectively; The second query vector, the second key vector, and the second value vector are normalized to obtain the fifth sub-feature map; The second feature map is obtained by splicing multiple fifth sub-feature maps along the length direction.

4. The method according to claim 3, wherein, The step of fusing the first feature map and the second feature map to obtain a fused feature map includes: The first feature map and the second feature map are added together to obtain the third feature map; Based on the Transformer module, the third feature map is input into the Transformer module to obtain the fused feature map.

5. The method according to claim 1, wherein, The acquisition of the target image to be detected includes: Acquire various types of target images and target region information within those target images; Among the various types of target images, at least one type contains a detected target.

6. The method according to claim 1, wherein, The step of extracting features from the target image to obtain a preprocessed feature map includes: The target image is input into the lightweight convolutional network MobileNet for feature extraction to obtain the preprocessed feature map.

7. A target detection device, comprising: The acquisition module is used to acquire the target image to be detected, extract features from the target image, and obtain a preprocessed feature map. The Transformer module is configured to split the preprocessed feature map along a first direction to obtain a first feature map, split the first feature map along a second direction to obtain a second feature map, and further split the preprocessed feature map along its width to obtain multiple first sub-feature maps of unit width, wherein the size of the first sub-feature map is the dimension of the preprocessed feature map multiplied by its length; merge the multiple first sub-feature maps of unit width with the preprocessed feature map to obtain the first feature map; split the first feature map along its length to obtain multiple fourth sub-feature maps of unit length, wherein the size of the fourth sub-feature map is the dimension of the preprocessed feature map multiplied by its width; and merge the multiple fourth sub-feature maps of unit length to obtain the second feature map. A fusion module is used to fuse the first feature map and the second feature map to obtain a fused feature map; and The classification and detection module is used to detect the fused feature map and obtain the detection result for the target object; The Transformer module is further configured to: obtain the query matrix, key matrix, and value matrix of the first sub-feature map; multiply the first sub-feature map by the query matrix, key matrix, and value matrix of the first sub-feature map respectively to obtain a first query vector, a first key vector, and a first value vector respectively; normalize the first query vector, the first key vector, and the first value vector to obtain a second sub-feature map; concatenate multiple second sub-feature maps in the width direction to obtain a third sub-feature map; and obtain the first feature map based on the preprocessed feature map and the third sub-feature map, including: adding the preprocessed feature map and the third sub-feature map to obtain the first feature map.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images