Attention-based target detection method and system

Abstract

The application discloses an attention-based target detection method and system, and relates to the field of pedestrian identity recognition. In the method, the following steps are included: acquiring a to-be-detected video; processing a to-be-detected video frame contained in the to-be-detected video by a first feature network to generate a feature representation of the to-be-detected video, wherein the first feature network comprises a simplified ResNet bottom network and a spatial pyramid top network, the simplified ResNet bottom network is used for extracting a multi-dimensional video frame feature of the to-be-detected video frame, and the spatial pyramid top network is used for performing fusion processing on the multi-dimensional video frame feature of the to-be-detected video frame to generate the feature representation of the to-be-detected video frame; and generating a target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames. By adopting the technical scheme provided in the application, the structure of the first feature network is simplified, and the number of parameters of the first feature network is reduced, so that the detection speed of the pedestrian detection method is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pedestrian identity recognition, and in particular to a target detection method and system based on attention. BACKGROUND

[0002] The pedestrian detection method based on HOG feature extraction and SVM classifier is a very classic detection method. For an image or video to be detected, the method outputs the descriptor of the pedestrian in the image or video to be detected through the steps of grayscale, image gradient calculation, Cell segmentation and Block, and then classifies and recognizes the pedestrian based on the descriptor of the pedestrian.

[0003] Starting from the common pedestrian detection method, it is not difficult to analyze that for the problem of pedestrian detection, the usual solution is to extract the features of the pedestrian based on the image or video to be detected, and then describe the identity of the pedestrian based on these features. However, for a large amount of data contained in the image or video to be detected, the existing pedestrian detection method has the problem of low detection rate. SUMMARY

[0004] In order to improve the detection rate of the existing pedestrian detection method, the present application provides a target detection method and system based on attention.

[0005] In a first aspect, the present application provides a target detection method based on attention, which comprises the following steps:

[0006] Obtaining a video to be detected;

[0007] Processing the video frames to be detected contained in the video to be detected by a first feature network to generate the feature representation of the video frames to be detected, the first feature network comprising a simplified ResNet bottom network and a spatial pyramid top network, the simplified ResNet bottom network being used to extract the multi-dimensional video frame features of the video frames to be detected, and the spatial pyramid top network being used to perform fusion processing on the multi-dimensional video frame features of the video frames to be detected to generate the feature representation of the video frames to be detected;

[0008] Generating a target detection result in the video to be detected according to the feature representations of all the video frames to be detected.

[0009] By adopting the above technical solution, the first feature network used to generate the feature representation of the video frames to be detected takes the simplified ResNet as the bottom network to extract the multi-dimensional video frame features of the video frames to be detected, without using the average pooling layer and the fully connected layer of the traditional ResNet, but adding the spatial pyramid top network at the top of the simplified ResNet, thereby reducing the number of parameters of the first feature network and improving the overall calculation speed of the first feature network, so as to improve the overall detection speed of the pedestrian detection method.

[0010] Optionally, in the processing of the to-be-detected video frames contained in the to-be-detected video by the first feature network to generate the feature representation of the to-be-detected video, further comprising:

[0011] learning the attention weights of the feature output channels of the multi-dimensional video frame features by the spatial pyramid top network, the output feature channels including a plurality of;

[0012] fusing the multi-dimensional video frame features of the to-be-detected video frame based on the attention weights of each output feature channel to generate the feature representation of the to-be-detected video frame.

[0013] By adopting the above technical solution, after extracting the multi-dimensional video frame features of the to-be-detected video frame by simplifying the ResNet bottom network, the attention mechanism is introduced in the spatial pyramid top network, the discrimination and diversity of the features are improved, and the accuracy of target detection is improved. At the same time, using the attention mechanism can reduce redundant feature information, and further reduce the computational amount of the network.

[0014] Optionally, in the generating of the target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames, specifically comprising:

[0015] inputting the feature representations of the to-be-detected video frames into a second feature network according to the time axis of the to-be-detected video;

[0016] performing convolution operation on the feature representations of the to-be-detected video frames along the time axis of the to-be-detected video by the second feature network to generate the output result of the second feature network;

[0017] generating the target detection result in the to-be-detected video according to the output result of the second feature network.

[0018] By adopting the above technical solution, the temporal relationship of the feature representations of the to-be-detected video frames is captured by the second feature network, so that the motion and deformation of the target in the to-be-detected video are better captured, and the accuracy and speed of target detection are improved.

[0019] Optionally, after generating the target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames, further comprising:

[0020] According to the target detection result of the to-be-detected video, obtaining the target region image in the to-be-detected video frame contained in the to-be-detected video;

[0021] processing the target region image by an image segmentation network to segment a plurality of region of interest images from the target region image;

[0022] respectively inputting each of the region of interest images into a third feature network to generate a DNA identifier of each of the region of interest images;

[0023] generating an identity code of the target region image according to the DNA identifier of each of the regions of interest.

[0024] By adopting the technical solutions, after completing the target detection on the to-be-detected video, the target region image is analyzed and processed to generate an identity code of the target region image, so as to determine the identity information of the pedestrian contained in the target region image and uniquely represent the specific pedestrian, thereby realizing effective utilization of the data of the to-be-detected video.

[0025] Optionally, in the processing of the target region image by the image segmentation network and the segmentation of the several region of interest images from the target region image, the method specifically comprises:

[0026] The image segmentation network comprises a parameter generator and a grid generator, the target region image is processed by the parameter generator to extract the region features of the target region image, and the two-dimensional affine transformation parameters of the target region image are outputted;

[0027] The region features of the target region image and the two-dimensional affine transformation parameters are transmitted to the grid generator, and the sampling grid applied to the target region image is generated by the grid generator;

[0028] The target region image is subjected to affine transformation by the sampling grid, and the several region of interest images are segmented from the target region image.

[0029] By adopting the technical solutions, the target region image is subjected to affine transformation, and the target region image is segmented into the plurality of region of interest images, each of which corresponds to a part of the pedestrian contained in the target region image, thereby providing a data basis for subsequent processing.

[0030] Optionally, in the respectively inputting each of the region of interest images into a third feature network to generate a DNA identifier of each of the region of interest images, the method specifically comprises:

[0031] The first DNA feature vector of the region of interest image is acquired by the third feature network, and the first DNA feature vector is generated according to the quantifiable attribute of the region of interest image;

[0032] The second DNA feature vector of the region of interest image is acquired by the third feature network, and the second DNA feature vector is generated according to the output result of the feature extraction layer of the third feature network;

[0033] The first DNA feature vector is spliced with a second DNA feature vector, and an empty feature vector is spliced on the basis of splicing of the first DNA feature vector and the second DNA feature vector, to generate a DNA identifier of the region of interest image.

[0034] By using the technical solutions, the region of interest image is described based on quantifiable attribute features and abstract features of the region of interest image. The quantifiable attribute features are key information of the pedestrian attribute DNA expression, and the abstract features are keys of further subdivided features, which can distinguish specific pedestrian identities. Through splicing of the multiple features, the generated DNA identifier can more accurately describe specific regions of the pedestrian.

[0035] Optionally, in the first DNA feature vector of the region of interest image obtained by the third feature network, the first DNA feature vector specifically includes:

[0036] The quantifiable attributes in the region of interest image are classified and recognized by the third feature network.

[0037] The quantifiable attributes in the region of interest image are one-hot encoded based on the classification result, to generate the first DNA feature vector.

[0038] By using the technical solutions, the classification result of the quantifiable attributes in the region of interest image is encoded, and the classification result expressing actual meaning is converted into a feature vector form by one-hot encoding.

[0039] In a second aspect of the present application, a target detection system based on attention is provided, and the system includes the following modules:

[0040] A to-be-detected video acquisition module is configured to acquire a to-be-detected video.

[0041] A to-be-detected video frame processing module is configured to process to-be-detected video frames included in the to-be-detected video by a first feature network, to generate a feature representation of the to-be-detected video. The first feature network includes a simplified ResNet bottom network and a spatial pyramid top network. The simplified ResNet bottom network is configured to extract multi-dimensional video frame features of the to-be-detected video frames. The spatial pyramid top network is configured to perform fusion processing on the multi-dimensional video frame features of the to-be-detected video frames, to generate a feature representation of the to-be-detected video frames.

[0042] A target detection result generation module is configured to generate a target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames.

[0043] In a third aspect of the present application, an electronic device is provided.

[0044] In a fourth aspect of the present application, a computer-readable storage medium is provided;

[0045] In summary, the one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0046] 1. When generating the feature representation of the to-be-detected video frame through the first feature network, the average pooling layer and the fully connected layer of the traditional ResNet are not used, and a spatial pyramid top network is added at the top of the simplified ResNet, thereby reducing the number of parameters of the first feature network and improving the overall calculation speed of the first feature network, so as to improve the overall detection speed of the pedestrian detection method

[0047] 2. The time sequence relationship of the feature representation of the to-be-detected video frame in time is captured through the second feature network, so as to better capture the motion and deformation of the target in the to-be-detected video, thereby improving the accuracy and speed of target detection.

[0048] 3. After completing the target detection of the to-be-detected video, the target detection result is obtained, and the DNA identification of the pedestrian contained in the target region is generated based on the obtained target region image, so as to effectively utilize the data of the to-be-detected video and identify the identity of the pedestrian in the to-be-detected video. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is a flowchart of a target detection method based on attention provided by the embodiments of the present application.

[0050] Figure 2 is a feasible structure diagram of a spatial pyramid top network of a first feature network in a target detection method based on attention provided by the embodiments of the present application;

[0051] Figure 3 is a structure diagram of an Attentive fuse Block in a spatial pyramid top network in a target detection method based on attention provided by the embodiments of the present application;

[0052] Figure 4 is a structure diagram of a target detection system based on attention disclosed by the embodiments of the present application.

[0053] Figure 5 is a structure diagram of an electronic device disclosed by the embodiments of the present application.

[0054] Explanation of reference numerals in the attached figures: 401, Module for acquiring video to be detected; 402, Module for processing video frames to be detected; 403, Module for generating target detection results; 500, Electronic device; 501, Processor; 502, Communication bus; 503, User interface; 504, Network interface; 505, Memory. Detailed Implementation

[0055] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0056] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0057] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0058] Reference Figure 1 This application provides an attention-based object detection method, which specifically includes the following steps:

[0059] S1: Obtain the video to be tested;

[0060] S2: The first feature network processes the video frames to be detected in the video to be detected, generating feature representations of the video frames to be detected;

[0061] Specifically, for a to-be-detected video with a time length of T, a to-be-detected video frame at the t-th moment of the to-be-detected video can be represented as I(t), I(t) e R^(cols x rows x 3), I(t) is a three-dimensional array, each dimension is the number of image columns, the number of image rows, and the number of channels of each pixel point in the image, the to-be-detected video frame at the t-th moment of the to-be-detected video is input into the first feature network, and the feature representation of the to-be-detected video frame at the t-th moment is extracted, the first feature network specifically includes a simplified ResNet bottom network and a spatial pyramid top network, and the first feature network takes the simplified ResNet bottom network as the backbone, and adds the spatial pyramid top network at the top of the simplified ResNet bottom network.

[0062] The simplified ResNet bottom network is the backbone network of the first feature network, specifically, the simplified ResNet bottom network is formed by removing the average pooling layer and the fully connected layer of the traditional ResNet on the basis of the traditional ResNet, and the basis of the simplified ResNet bottom network can be ResNet-50. For the input I(t), the simplified ResNet bottom network will extract the multi-dimensional video frame features of I(t), and the extracted multi-dimensional video frame features are transmitted to the spatial pyramid top network.

[0063] The spatial pyramid top network is essentially a pooling layer that can increase the receptive field while keeping the resolution of the feature map unchanged, and can extract multi-scale information, the spatial pyramid top network processes the received multi-dimensional video frame features, captures and fuses the multi-dimensional video frame features from different perspectives using attention weights, and outputs the feature representation of I(t). It should be noted that due to the structure of the spatial pyramid top network, the output feature representation of I(t) is a fixed-size feature vector.

[0064] Referring to Figure 2 With Figure 3 In an available embodiment of the present application, the spatial pyramid top network is composed of four parallel Attentive fuse Blocks, the Attentive fuse Block performs convolution operation on the input feature map to obtain the corresponding feature vector, then performs weighted average on the feature vector according to certain rules to obtain the fused feature vector, and finally generates the fused feature map through deconvolution operation.

[0065] In the Attentive fuse Block, an attention mechanism is introduced. When performing weighted average on the feature vectors to obtain the fused feature vectors, the Attentive fuse Block performs dot product operation on each feature vector before fusion and an attention vector to obtain an attention weight, and then normalizes the weights to obtain the final weight coefficients. Then, the weight coefficients are applied to the feature vectors to obtain the weighted feature vectors. Then, the weighted feature maps are obtained through the deconvolution operation. Finally, all the weighted feature maps are concatenated to obtain the final feature map.

[0066] S3: processing the feature representations of all the video frames to be detected by the second feature network to generate the target detection result in the video to be detected;

[0067] Specifically, after the feature representations of the video frames to be detected included in the video to be detected are extracted by the first feature network, the feature representations are input into the second feature network according to the time axis of the video to be detected, so as to generate the target detection result in the video to be detected.

[0068] In an available embodiment of the present application, the second feature network can be ConvLSTM. The second feature network performs convolution operation on the feature representations of the video frames to be detected along the time axis of the video to be detected, so as to encode the time feature into the network. In the second feature network, the input data at each time is a two-dimensional tensor, in which one dimension represents spatial information (such as the width and height of the image), and the other dimension represents the number of feature channels (such as the RGB three channels of the image). The second feature network realizes the extraction of spatial information by performing convolution operation on the two-dimensional tensor. Meanwhile, the second feature network also maintains a state vector at each time to store the internal state of the network at the current time. The state vector can pass the state information of the previous time to the current time, so as to realize the transmission of time information.

[0069] The output result of the second feature network can be a multi-dimensional tensor, such as (batch_size, num_boxes, num_classes+4). batch_size represents the number of video frames input remotely, num_boxes represents the number of target boxes in the current video frame to be detected, num_classes represents the classification category of the target box, and 4 represents the coordinate information of the target box. The coordinate information of the target box can be represented as (x, y, w, h), and the center coordinates (x, y) and the width and height (w, h) of the target box. Based on the output result of the second feature network, multiple target boxes can be generated in the video to be detected, each target box corresponding to a pedestrian appearing in the video to be detected, and the target detection of the video to be detected is completed.

[0070] In another possible implementation of the present application, the second feature network can also be an FC-LSTM, which is a variant of the traditional LSTM model, and has a peep mechanism and a full connection structure added to consider the spatial correlation of the input sequence, so as to improve the expression ability and prediction accuracy of the model.

[0071] Specifically, the FC-LSTM converts the input time sequence vector into a matrix to represent the connection relationship in space, considers the correlation between different positions, and thus better captures the features of the input sequence. The full connection structure can also enable the model to better process large-scale data sets and improve the expression ability of the model. In addition, the model also has a peep mechanism, and the input of each gate is determined by three parts, namely the input of the current layer, the output of the previous layer, and the information on the memory line. The peep mechanism can enable the model to better learn the relationship between the input sequences, and improve the accuracy and generalization ability of the model.

[0072] S4: According to the target detection result of the to-be-detected video, a target region image in a to-be-detected video frame contained in the to-be-detected video is obtained;

[0073] Specifically, after the to-be-detected video is processed by the first feature network and the second feature network, multiple target boxes exist in the to-be-detected video, and each target box frames the pedestrian appearing in the to-be-detected video. The image in each target box region of the to-be-detected video is intercepted to obtain the target region image in the to-be-detected video frame contained in the to-be-detected video.

[0074] S5: The target region image is processed by an image segmentation network to segment a plurality of region of interest images from the target region image;

[0075] Specifically, for the obtained target region image, the image segmentation network is used to process the target region image to segment a plurality of region of interest images from the target region image, and each region of interest image is a part of the pedestrian in the target region image. In one possible implementation of the present application, the target region image can be segmented into five region of interest images, i.e., a head image, an upper body image, a lower body image, a foot image, and an external object image.

[0076] The image segmentation network can be Faster-RCNN, which divides the target region image into multiple region of interest images based on the image features of the target region image. Faster-RCNN generates multiple candidate boxes in the target region image through steps such as feature extraction, candidate box generation, candidate box selection, and boundary regression and classification, and each candidate box has a range and a part of the pedestrian in the target region image.

[0077] The image segmentation network comprises a parameter generator and a grid generator. The parameter generator extracts region features of an input target region image, outputs region features of the target region image, and generates two-dimensional affine transformation parameters applied to the target region image. The output of the parameter generator is transmitted to the grid generator. The grid generator generates a sampling grid applied to the target region image based on the two-dimensional affine transformation parameters. The sampling grid performs affine transformation on the target region image, so that the generated candidate frame is transformed to better adapt to the target, thereby accurately segmenting the target region image into multiple region-of-interest images.

[0078] In another embodiment of the present application, a similarity-based video feature aggregation method is introduced when the image segmentation network processes the target region image. Specifically, the method takes a certain candidate region output by the RPN network in the Faster-RCNN as the center, then iteratively searches for the candidate region with the highest correlation in the adjacent frame, and forms a correlation enhancement table for the current frame. The correlation enhancement table contains candidate regions in adjacent frames with high similarity to the candidate region in the current frame. The features of these candidate regions can be aggregated through similarity screening, thereby improving the utilization of network calculation. The similarity screening process can be designed according to specific requirements, for example, correlation coefficient, cosine similarity, etc. can be used to measure the similarity between candidate regions. Through this method, similarity screening can be used in the Faster-RCNN network to aggregate features in adjacent frames, thereby improving the accuracy of target detection and reducing the computational cost.

[0079] S6: processing each region-of-interest image through a third feature network to generate a DNA identifier of each region-of-interest image;

[0080] Specifically, the third feature network can be a convolutional network. After inputting the region-of-interest image into the third feature network, the third feature network will classify and identify the region-of-interest image, identify the category of quantifiable attributes contained in the region-of-interest image, such as clothing color, item category, whether to carry an item, etc. The classification results of quantifiable attributes are one-hot encoded to convert the classification results into encoded form and obtain the first DNA feature vector of the region-of-interest. The second DNA feature vector of the region-of-interest image is generated according to the output of the feature extraction layer in the third feature network.

[0081] The first DNA feature vector of the region-of-interest image is spliced with the second DNA feature vector, and a fixed number of empty feature vectors are spliced on the basis of the splicing of the first DNA feature vector and the second DNA feature vector, to generate a DNA identifier of the region-of-interest image, and the DNA identifier of the region-of-interest image is used to describe the region-of-interest image.

[0082] S7: generating an identity code of the target region image according to the DNA identifiers of the region-of-interest images;

[0083] Specifically, the target region image contains a plurality of region-of-interest images, each region-of-interest image has a corresponding DNA identifier, and the DNA identifiers of the region-of-interest images contained in the target region image can uniquely represent a target region image. The combination of the DNA identifiers of the plurality of region-of-interest images contained in the target region image is the identity code of the target region image.

[0084] The identity code of the target region image can uniquely represent a pedestrian appearing in the to-be-detected video, and different pedestrians have different identity codes, thereby completing the identity recognition of the pedestrian in the to-be-detected video.

[0085] An embodiment of the target detection method based on attention provided in the present application has the following implementation principle: a first feature network is used to extract a feature representation of a to-be-detected video frame of a to-be-detected video, the first feature network takes ResNet as a backbone network, removes the average pooling layer and the fully connected layer of the traditional ResNet, and adds an attention-based spatial pyramid network at the top of the ResNet; a second feature network is used to extract a spatio-temporal feature of the to-be-detected video based on the feature representation of the to-be-detected video frame of the to-be-detected video, so as to generate a target detection result in the to-be-detected video and complete pedestrian detection of the to-be-detected video; a target region image is obtained based on the target detection result, an image segmentation network is used to segment a plurality of region-of-interest images from the target region image, each region-of-interest image corresponds to a part of a pedestrian in the target region image; a third feature network is used to generate a DNA identifier of each region-of-interest image, and the pedestrian in the target region image is uniquely represented based on the DNA identifiers of the region-of-interest images, thereby completing the identity recognition of the pedestrian.

[0086] Reference Figure 4 The present application also provides a target detection system based on attention, which specifically includes the following modules:

[0087] The to-be-detected video acquisition module 401 is configured to acquire a to-be-detected video.

[0088] The to-be-detected video frame processing module 402 is configured to process the to-be-detected video frames contained in the to-be-detected video by using a first feature network to generate feature representations of the to-be-detected video, the first feature network comprising a simplified ResNet bottom network and a spatial pyramid top network, the simplified ResNet bottom network being configured to extract multi-dimensional video frame features of the to-be-detected video frames, and the spatial pyramid top network being configured to perform fusion processing on the multi-dimensional video frame features of the to-be-detected video frames to generate the feature representations of the to-be-detected video frames;

[0089] The target detection result generation module 403 is configured to generate a target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames.

[0090] It should be noted that, in implementing the functions thereof, the apparatuses provided in the above embodiments are only used as examples for the division of the functional modules, and in actual applications, the above functions can be completed by different functional modules according to requirements, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above-described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be described here.

[0091] The present application also discloses an electronic device 500. Referring to Figure 5 , Figure 5 is a structural schematic diagram of an electronic device 500 disclosed by an embodiment of the present application. The electronic device 500 can include at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0092] The communication bus 502 is configured to realize the connection and communication between the components.

[0093] The user interface 503 can include a display screen (Display) and a camera (Camera), and the optional user interface 503 can further include a standard wired interface and a wireless interface.

[0094] The network interface 504 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0095] The processor 501 can include one or more processing cores. The processor 501 connects various parts within the server through various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 505, and calling data stored in the memory 505. Alternatively, the processor 501 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 501 can integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 501, but can be realized by a separate chip.

[0096] The memory 505 can include a random access memory (RAM) and a read-only memory (ROM). Alternatively, the memory 505 includes a non-transitory computer-readable storage medium. The memory 505 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 505 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 505 can alternatively be at least one storage device located away from the aforementioned processor 501. Referring to Figure 5 The memory 505 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an application program of the attention-based target detection method.

[0097] In Figure 5In the electronic device 500 shown, the user interface 503 is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor 501 can be used to invoke an application program of a target detection method based on attention stored in the memory 505, and when executed by one or more processors 501, make the electronic device 500 execute the method described in one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all expressed as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0098] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0099] In several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, and the division of units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some service interface, device or unit, and can be electrical or other forms.

[0100] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0101] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0102] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium 505. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium 505, and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium 505 includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0103] The above is only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure.

[0104] The present application is intended to cover any variations, uses, or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field of the present disclosure not described in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. An attention-based object detection method, characterized in that, The method comprises the following steps: acquiring a video to be detected; processing a video frame to be detected contained in the video to be detected by a first feature network to generate a feature representation of the video frame to be detected, the first feature network comprising a simplified ResNet bottom network and a spatial pyramid top network, the simplified ResNet bottom network being used to extract a multi-dimensional video frame feature of the video frame to be detected, and the spatial pyramid top network being used to perform fusion processing on the multi-dimensional video frame feature of the video frame to be detected to generate the feature representation of the video frame to be detected, wherein the spatial pyramid top network is composed of parallel Attentive fuse Blocks, an attention mechanism is introduced in the Attentive fuse Block, and feature maps obtained through parallel processing are concatenated together to obtain a final feature map; generating a target detection result in the video to be detected according to the feature representations of all the video frames to be detected; after generating the target detection result in the video to be detected according to the feature representations of all the video frames to be detected, the method further comprises the following steps: acquiring a target region image in the video frame to be detected contained in the video to be detected according to the target detection result of the video to be detected; processing the target region image through an image segmentation network to segment a plurality of region of interest images from the target region image, each region of interest image being a part of a pedestrian in the target region image, and the target region image being segmented into five region of interest images, i.e., a head image, an upper body image, a lower body image, a foot image and an external object image; wherein when the image segmentation network processes the target region image, the following steps are included: a plurality of candidate regions are output from the image segmentation network, the candidate regions are region of interest, one candidate region is selected as a center candidate region from the plurality of candidate regions, a region having a similarity value greater than a preset threshold value with the center candidate region is searched in a neighboring frame of the current frame, and is aggregated to obtain a related enhancement table corresponding to the current frame, the image segmentation network is Faster-RCNN, and is used to divide a plurality of region of interest images in the target region image; respectively inputting each region of interest image into a third feature network to generate a DNA identifier of each region of interest image; generating an identity code of the target region image according to the DNA identifiers of each region of interest; in the step of respectively inputting each region of interest image into the third feature network to generate a DNA identifier of each region of interest image, the following steps are included: acquiring a first DNA feature vector of the region of interest image through the third feature network, the first DNA feature vector being generated according to a quantifiable attribute of the region of interest image; acquiring a second DNA feature vector of the region of interest image through the third feature network, the second DNA feature vector being generated according to an output result of a feature extraction layer of the third feature network; Splice the first DNA feature vector with a second DNA feature vector, and splice an empty feature vector on the basis of the splicing of the first DNA feature vector and the second DNA feature vector, to generate a DNA identifier of the region of interest image, the DNA identifier uniquely representing a target region image, the target region image containing a combination of DNA identifiers of a plurality of region of interest images.

2. The attention-based object detection method of claim 1, wherein, In the process of processing the to-be-detected video frame contained in the to-be-detected video by the first feature network to generate the feature representation of the to-be-detected video frame, further comprising: Learning the attention weight of the feature output channel of the multi-dimensional video frame feature through the spatial pyramid top network, the output feature channel including a plurality of; Fusing the multi-dimensional video frame feature of the to-be-detected video frame based on the attention weight of each output feature channel to generate the feature representation of the to-be-detected video frame.

3. The attention-based object detection method of claim 1, wherein, In the process of generating a target detection result in the to-be-detected video according to the feature representations of all the to-be-detected video frames, specifically comprising: Inputting the feature representation of the to-be-detected video frame into the second feature network according to the time axis of the to-be-detected video; Performing convolution operation on the feature representation of the to-be-detected video frame along the time axis of the to-be-detected video through the second feature network to generate the output result of the second feature network; Generating the target detection result in the to-be-detected video according to the output result of the second feature network.

4. The attention-based object detection method of claim 1, wherein, In the process of processing the target region image through the image segmentation network to segment a plurality of region of interest images from the target region image, specifically comprising: The image segmentation network includes a parameter generator and a grid generator, the parameter generator is used to process the target region image, extract the region feature of the target region image, and output the two-dimensional affine transformation parameter of the target region image; The region feature of the target region image and the two-dimensional affine transformation parameter are transmitted to the grid generator, and the grid generator is used to generate a sampling grid applied to the target region image; Performing affine transformation on the target region image through the sampling grid to segment a plurality of region of interest images from the target region image.

5. The attention-based object detection method of claim 1, wherein, In the process of acquiring the first DNA feature vector of the region of interest image through the third feature network, specifically comprising: Classifying and identifying the quantifiable attribute in the region of interest image through the third feature network; Based on the classification result, the quantifiable attribute in the region of interest image is one-hot encoded to generate the first DNA feature vector.

6. An attention-based object detection system, comprising: The system comprises: A to-be-detected video acquisition module (401) is used to acquire a to-be-detected video; The video frame to be detected processing module (402) is used for processing the video frame to be detected contained in the video to be detected by a first feature network to generate a feature representation of the video to be detected, the first feature network comprises a simplified ResNet bottom network and a spatial pyramid top network, the simplified ResNet bottom network is used for extracting a multi-dimensional video frame feature of the video frame to be detected, the spatial pyramid top network is used for performing fusion processing on the multi-dimensional video frame feature of the video frame to be detected to generate a feature representation of the video frame to be detected, wherein the spatial pyramid top network is composed of parallel Attentive fuse Block, an attention mechanism is introduced in the Attentive fuse Block, and the feature maps obtained by parallel processing are cascaded together to obtain a final feature map; The target detection result generation module (403) is used for generating a target detection result in the video to be detected according to the feature representations of all the video frames to be detected. The target detection result generation module (403) is further configured to acquire a target region image in the to-be-detected video frame included in the to-be-detected video according to a target detection result of the to-be-detected video; and process the target region image through an image segmentation network to segment a plurality of region of interest images from the target region image, each of the region of interest images being a part of a pedestrian in the target region image, and the target region image being segmented into five region of interest images, i.e., a head image, an upper body image, a lower body image, a foot image, and an external object image. When processing the target region image through the image segmentation network, the following operations are included: a plurality of candidate regions are output from the image segmentation network for a current frame, the candidate regions being region of interest, a candidate region is selected from the plurality of candidate regions as a central candidate region, a region in a neighboring frame of the current frame having a similarity value greater than a preset threshold value with the central candidate region is searched and aggregated to obtain a relevant enhancement table corresponding to the current frame, the image segmentation network being Faster-RCNN, and being configured to divide a plurality of region of interest images in the target region image. Each of the region of interest images is input into a third feature network to generate a DNA identifier of each of the region of interest images. An identity code of the target region image is generated according to the DNA identifiers of the region of interest images. In the process of inputting each of the region of interest images into the third feature network to generate the DNA identifier of each of the region of interest images, the following operations are included: a first DNA feature vector of the region of interest image is acquired through the third feature network, the first DNA feature vector being generated according to a quantifiable attribute of the region of interest image; a second DNA feature vector of the region of interest image is acquired through the third feature network, the second DNA feature vector being generated according to an output result of a feature extraction layer of the third feature network; the first DNA feature vector and the second DNA feature vector are spliced, and an empty feature vector is spliced on the basis of the splicing of the first DNA feature vector and the second DNA feature vector to generate the DNA identifier of the region of interest image, the DNA identifier uniquely representing one target region image, and the target region image including a combination of DNA identifiers of a plurality of region of interest images.

7. An electronic device, comprising: The electronic device (500) includes a processor (501), a memory (505), a user interface (503), and a network interface (504). The memory (505) is configured to store instructions. The user interface (503) and the network interface (504) are configured to communicate with other devices. The processor (501) is configured to execute the instructions stored in the memory (505) to cause the electronic device (500) to perform the method of any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method steps of any one of claims 1-5.

Images