Target object detection method and device, storage medium and computer program product

Abstract

The invention provides a target object detection method and device, a storage medium and a computer program product. The method comprises the steps of obtaining scene semantic feature information and target feature information of a to-be-detected picture; based on the scene semantic feature information and the target feature information, determining relation feature information between scene information and target object information in the to-be-detected picture; and detecting the target object in the to-be-detected picture based on the relation feature information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a target object detection method, device, storage medium and computer program product. Background Art

[0002] In video surveillance detection tasks, most methods collect on-site pictures and complete the analysis through detection models. In actual monitoring, complex scenes, changing lighting, target occlusion, etc. often occur. When using local features of the target area for independent detection, a large number of target omissions and false detections will occur when the target features are blurred. For example, in a river scene, a boat may be mistakenly detected as a car, and a vehicle on a road may be missed after being obscured, resulting in a low accuracy rate in target object detection. Summary of the invention

[0003] In view of this, the present application hopes to provide a target object detection method, device, storage medium and computer program product, which can improve the accuracy of target object detection.

[0004] The technical solution of this application is implemented as follows:

[0005] In a first aspect, the present application provides a target object detection method, the method comprising:

[0006] Obtain scene semantic feature information and target feature information of the image to be detected;

[0007] Based on the scene semantic feature information and the target feature information, determining the relationship feature information between the scene information and the target object information in the image to be detected;

[0008] Based on the relationship feature information, the target object in the image to be detected is detected.

[0009] In a second aspect, the present application provides a target object detection device, the target object detection device comprising:

[0010] An acquisition unit, used to acquire scene semantic feature information and target feature information of the image to be detected;

[0011] A determination unit, used to determine the relationship feature information between the scene information and the target object information in the image to be detected based on the scene semantic feature information and the target feature information;

[0012] The detection unit is used to detect the target object in the image to be detected based on the relationship feature information.

[0013] In a third aspect, the present application provides a target object detection device, which includes: a processor and a memory; when the processor executes an operating program stored in the memory, the above-mentioned target object detection method is implemented.

[0014] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, which implements the above-mentioned target object detection method when executed by a processor.

[0015] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned target object detection method when executed by a processor.

[0016] The present application provides a target object detection method, device, storage medium and computer program product, the method comprising: obtaining scene semantic feature information and target feature information of a picture to be detected; determining the relationship feature information between the scene information and the target object information in the picture to be detected based on the scene semantic feature information and the target feature information; and detecting the target object in the picture to be detected based on the relationship feature information. Using the above implementation scheme, in the process of detecting the target object, not only the target feature information of the picture to be detected is obtained, but also the scene semantic feature information of the picture to be detected is obtained, and the relationship feature information between the scene information and the target object information of the picture to be detected is constructed by combining the scene semantic feature information and the target feature information, because the relationship feature information integrates the scene information where the target object is located, and combines the relevance of the scene information and the target object information, so that when the relationship feature information is used to detect the target object in the picture to be detected, the fused relationship feature information can reflect the relationship between the scene information and the target object information, thereby reducing the probability of the target object being misdetected and improving the accuracy of target object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a target object detection method provided in an embodiment of the present application;

[0018] Figure 2 A schematic diagram of a graph model of a structure provided in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of the overall process of a target object detection method provided in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of a process for obtaining scene semantic feature information using an RGA-S network provided in an embodiment of the present application;

[0021] Figure 5 A schematic diagram of the structure of a target object detection device provided in an embodiment of the present application Figure 1 ;

[0022] Figure 6 A schematic diagram of the structure of a target object detection device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0023] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the technical solution of the present application is further elaborated in detail below in combination with the drawings and specific embodiments of the specification. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0024] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art to which the present application belongs. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0025] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first / second / third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Video surveillance uses cameras to capture real-time images or video streams for recording and analysis, and uses visual perception to enhance security measures, improve monitoring efficiency, and facilitate decision-making. As a key component of modern social security management, video surveillance technology has evolved from a single video record to an advanced system with integrated intelligent analysis capabilities. With the acceleration of urbanization and the improvement of public safety awareness, the demand for video surveillance in modern society is growing, and it is widely used in urban governance, traffic management, industrial monitoring, environmental monitoring, home security and other fields. At present, video surveillance mainly uses computer vision technology for intelligent analysis, and realizes the detection capabilities of abnormal behavior of personnel, vehicle violations, fire, etc. through methods such as target detection, recognition, and tracking, which improves the efficiency of urban management and reduces safety risks. However, related video surveillance detection methods still face challenges in dealing with complex scenes, lighting changes, target occlusion, etc., and a large number of targets will be missed and misdetected.

[0027] Commonly used computer vision processing methods for video surveillance detection include target detection and instance segmentation. Target detection is to identify the category and position of an object in an image by extracting object features. In related technologies, target detection frameworks based on deep learning are widely used, including: Faster RCNN (Region-based Convolutional Neural Network), Single Shot MultiBox Detector (SSD), YOLO (You Only Look Once) series, etc. Instance segmentation is to obtain the shape, contour, position and category information of the target through target features. Common algorithms include Mask Region-based Convolutional Neural Network (R-CNN, YOLACT (You Only Look At CoefficienT), SOLO (Segmenting Objects by Locations), BlendMask, etc. Related video surveillance detection methods mainly collect and annotate on-site pictures for specific monitoring and analysis needs, and obtain detection models for corresponding tasks through network training. In various scene tasks, it is necessary to train their own models to complete video surveillance analysis.

[0028] In order to improve the detection performance, the attention mechanism is used to obtain important features of the target when extracting target features, so that useful information is enhanced and useless information is suppressed. Common methods include Squeeze-and-Excitation Networks (SENet), Convolutional Block Attention Module (CBAM), Dual Attention Network (DANet), etc. However, these attention mechanisms only focus on the local information of the target area and lack the acquisition of the entire image information.

[0029] Based on this, in the video surveillance detection task, most methods collect on-site pictures and use detection models to complete the analysis. In actual monitoring, there are often complex scenes, changing lighting, and occlusion of target objects. Some related technical methods mainly rely on the local features of the target area to perform independent detection, ignoring the scene information where the target is located, as well as the relationship between the scene and the target. When the target features are blurred, a large number of target misses and false detections will occur. For example, in the river scene, a boat will be mistakenly detected as a vehicle, and on the road, a vehicle will be missed after being blocked. In addition, some related detection methods often require different detection tasks to train different detection models for deployment and analysis. For example, smoke detection, vehicle detection, etc., all require retraining of different network models. With the increase of video surveillance detection tasks, the number of detection models will also increase, requiring a large amount of computing resources, and also causing great pressure on model deployment and storage.

[0030] To solve the above problems, a method for detecting a target object by using scene information as a guide in the process of video surveillance is proposed in the embodiment of the present application. The image scene semantic information features are obtained by adding a global spatial relationship attention (Spatial Relation-Aware Global Attention, RGA-S) network to the detection method, and the extracted target features are fused using graph convolution to construct the relationship between the scene semantic feature information and the target feature information of the target object, and the relationship feature information obtained after fusion is used for subsequent detection, recognition and other processing tasks. The relationship between the scene semantic feature information and the target feature information of the target object can greatly reduce the probability of missed detection and false detection, and improve the detection accuracy. In addition, the method proposed in the embodiment of the present application integrates the information of the scene as a guide in the detection, so that a single network structure can learn the information of various scenes, and can be applied to a variety of detection tasks in a variety of scenes. Different scenes can use the same network for detection, and can be used for target detection, tracking, instance segmentation and other video surveillance detection tasks, reducing the model deployment amount and saving computing resources.

[0031] The specific implementation of the embodiment of this application is as follows:

[0032] The present application embodiment provides a method for detecting a target object. Figure 1 As shown, the method may include:

[0033] S101, obtaining scene semantic feature information and target feature information of a picture to be detected.

[0034] In the embodiment of the present application, the image to be detected can be a image taken by using an acquisition device, or it can be a image captured from a video stream acquired by the acquisition device. Specifically, it can be selected according to actual conditions, and no specific limitation is made in the embodiment of the present application.

[0035] In the embodiment of the present application, the scene semantic feature information can be represented by the symbol F s express.

[0036] In the embodiment of the present application, the target feature information can be represented by the symbol F0.

[0037] In the embodiment of the present application, the scene semantic feature information and target feature information of the image to be detected can be obtained by the following methods:

[0038] Based on the pre-trained feature extraction network, feature information of the image to be detected is extracted to obtain feature data of the image to be detected; based on the feature data and the global perception network, the scene semantic feature information is determined; based on the feature data and the target feature extraction network, the target feature information is determined.

[0039] In an embodiment of the present application, the pre-trained feature extraction network may be a pre-trained backbone network (represented by N), wherein the backbone network may be a deep convolutional neural network composed of convolutional layers, and the output of the backbone network is a feature map with certain high-level semantic information.

[0040] In the embodiment of the present application, the feature data may be a feature map (which may be represented by F) having certain high-level semantic information.

[0041] In an embodiment of the present application, the global perception network can be a global spatial relational attention network (ie, RGA-S).

[0042] In the embodiment of the present application, the target feature extraction network can adopt a network for extracting feature information of a target object in related technologies, and can be selected according to actual conditions, and the embodiment of the present application does not make any specific limitation.

[0043] In an embodiment of the present application, feature extraction is performed on the image to be detected based on a pre-trained feature extraction network. The image to be detected in video surveillance can be input into a trained general backbone network N to extract feature information and obtain a feature map F of the image.

[0044] In the embodiment of the present application, the obtained feature map F is input into RGA-S to obtain the scene semantic feature information F of the image to be detected in the entire image space. s .

[0045] In an embodiment of the present application, while the feature map F is input into RGA-S, the feature map F can also be input into a general target feature extraction network, which is generally composed of a convolutional layer and outputs target feature information F0, wherein the target feature information F0 is usually the local feature information of the image to be detected.

[0046] S102: Determine relationship feature information between scene information and target object information in the image to be detected based on the scene semantic feature information and the target feature information.

[0047] In the embodiment of the present application, the relationship feature information can be used to G express.

[0048] In the embodiment of the present application, the scene semantic feature information F s The target feature information F0 is input into the graph convolutional network (GCN) to obtain the scene information of the image to be detected and the relationship feature information F between the target object G .

[0049] S103: Detect the target object in the to-be-detected picture based on the relationship feature information.

[0050] In the present embodiment, the F G The input is then used in the subsequent detection network to implement classification and detection, and the detection result of the image to be detected is obtained.

[0051] In the embodiment of the present application, using F G It can perform visual detection tasks such as target detection, tracking, instance segmentation, etc. to obtain the final detection result. And this feature contains the global scene semantic feature information and the target feature information of the original target object. Using the scene information as auxiliary information for subsequent detection tasks can improve the accuracy of target object detection.

[0052] It can be understood that a target object detection method provided by an embodiment of the present application, in the process of detecting the target object, not only obtains the target feature information of the picture to be detected, but also obtains the scene semantic feature information of the picture to be detected, and constructs the relationship feature information between the scene information and the target object information of the picture to be detected by combining the scene semantic feature information and the target feature information. Because the relationship feature information integrates the scene information where the target object is located, combined with the correlation between the scene information and the target object information, when using the relationship feature information to detect the target object in the picture to be detected, the fused relationship feature information can reflect the relationship between the scene information and the target object information, thereby reducing the probability of the target object being misdetected and improving the accuracy of target object detection.

[0053] In an embodiment of the present application, based on feature data and a global perception network, determining scene semantic feature information can be specifically implemented in the following manner:

[0054] The feature data is input into the global perception network to obtain spatial relationship feature information; based on the feature data and the spatial relationship feature information, the spatial relationship attention information is determined; based on the spatial relationship attention information, the attention weight value of each spatial position in the feature data is determined; based on the feature data and the attention weight value, the scene semantic feature information is determined.

[0055] In the embodiment of the present application, the spatial relationship feature information can be represented as a spatial relationship vector r i .

[0056] In the embodiment of the present application, the above-mentioned inputting of feature data into the global perception network to obtain spatial relationship feature information can be specifically implemented in the following manner:

[0057] Based on the global perception network, the feature node data of each spatial position in the feature data is determined; based on the feature node data of each spatial position, the node graph information is determined; the target feature node information is selected from the node graph information, and based on the target feature node information, the spatial relationship information is determined; based on the spatial relationship information, the spatial relationship feature information corresponding to the target feature node information is determined.

[0058] In the embodiment of the present application, the spatial relationship information can be represented as a spatial relationship matrix.

[0059] In the embodiment of the present application, a feature map F is input, where: The C-dimensional feature vector of each spatial position of the feature map F is obtained by raster scanning, and the obtained C-dimensional feature vector is used as a feature node data.

[0060] In the embodiment of the present application, the characteristic node data of all spatial positions are combined to form node graph information, which can be expressed as T s , where the node graph information T s The number of feature node data in N = W × H, and each feature node data is expressed as Among them, i=1, 2,…,N.

[0061] In the embodiment of the present application, the node graph information T is formed s In , select the target feature node information to be input, input the target feature node information into two 1×1 convolutional layers, and so on, until the node graph T s All feature node information in is input into two 1×1 convolutional layers respectively to obtain two vectors of feature node information. The two vectors are dot-product processed to obtain the spatial relationship matrix The spatial relationship matrix represents the relationship between each vector. For example, the relationship between the i-th node and the j-th node can be expressed as r ij , where r ij The calculation method is as follows:

[0062]

[0063] Among them, θ S and It is two 1×1 convolution operations with shared parameters.

[0064] After the calculation of the above formula (1), a batch normalization layer (Batch Normalization, BN) and a rectified linear unit (Rectified Linear Unit, ReLU) are added for further processing, which will be explained in detail in the following process.

[0065] It should be noted that if the relationship between the jth node and the ith node can be expressed as the following formula (2):

[0066]

[0067] In the embodiment of the present application, for the i-th feature node information, the i-th feature node information and all related feature node information are paired and spliced ​​in a certain order to obtain the spatial relationship vector of the i-th feature node information (also referred to as spatial relationship feature information), and the spatial relationship feature information can be expressed as r i =[R S (i, :),

[0068] In the embodiment of the present application, the spatial relationship attention information is determined based on the feature data and the spatial relationship feature information, which can be implemented in the following manner:

[0069] The feature data is pooled to obtain pooled feature data; and the spatial relationship attention information is determined based on the pooled feature data and the spatial relationship feature information.

[0070] In an embodiment of the present application, all feature information in the feature map F is globally averaged pooled to obtain pooled feature data, wherein the obtained pooled feature data is an operation of performing pooling processing on the entire feature map F.

[0071] In the embodiment of the present application, the feature data after pooling processing is spliced ​​with the spatial relationship vector to obtain the spatial relationship attention information, wherein the spatial relationship attention information can be used to obtain the spatial relationship attention information. s express.

[0072] In the present embodiment, E s It has both global structural information and local original feature information. s It can be expressed by the following formula (3):

[0073] E s =pool C (ψ s (x i ),δ s (r j )) (3)

[0074] in, k represents the average pooling dimensionality reduction ratio, ψ S and δ S It is an operation on the original feature information and the spatial relationship feature information, which consists of two 1×1 convolutions, followed by a batch normalization layer BN and a rectified linear unit ReLU, pool C (·) represents the global average pooling operation on the channel dimension, which reduces the original feature channel dimension by 1.

[0075] In the embodiment of the present application, based on the spatial relationship attention information, the attention weight value of each spatial position in the feature data is determined, which can be specifically implemented in the following manner:

[0076] The spatial relationship attention information is regularized and nonlinearly processed to obtain the attention weight value.

[0077] In the embodiment of the present application, the spatial relationship attention information E is obtained s Get the attention weight value of each position, the attention weight value can be used to obtain the attention weight value of each position. i Indicates that, according to the spatial relationship attention information E s Get a i The process is as follows:

[0078] The spatial relationship attention information is processed by batch normalization BN 1×1 convolution operation and nonlinear processing. The nonlinear processing can be processed by nonlinear unit RELU to obtain the spatial relationship attention weight value a i , a i It can be expressed by the following formula (4):

[0079] a i =sigmoid(W2RELU(W1E s )) (4)

[0080] Among them, W1 and W2 are 1×1 convolution operations with batch normalization layer BN added. W1 reduces the channel dimension by a ratio of l, and W2 converts the channel dimension to 1.

[0081] In the embodiment of the present application, after obtaining the attention weight value a of the spatial position i Afterwards, the attention weight value a i Multiply it with the original feature map F to obtain the feature data weighted by spatial relationship attention, which is the obtained scene semantic feature information F S , where F S It can be expressed as F S =F×a i .

[0082] In an embodiment of the present application, based on the scene semantic feature information and the target feature information, determining the relationship feature information between the scene information and the target object information in the image to be detected can be achieved in the following manner:

[0083] Based on the scene semantic feature information and the target feature information, a data structure is constructed; based on the data structure and the pre-trained neural network model, the relationship feature information is determined.

[0084] In the embodiment of the present application, the data structure may be a graph model.

[0085] In an embodiment of the present application, the pre-trained neural network model may be GCN.

[0086] In the embodiment of the present application, the obtained scene semantic feature information F s and target feature information F0, construct graph model, scene semantic feature information F s , the target feature information F0 is defined as the node v in the graph model. Among them, the target feature information F0 is composed of each target feature f i Composition, scene semantic feature information F s And each target feature f i The relationship between , and each target feature f i The relationship between is defined as the edge e of the graph model. The graph model is constructed as follows Figure 2 As shown, the graph model is represented by G = (v, e), where v = {F s , f1, f2, …, f n}, n is the number of target feature information.

[0087] In the embodiment of the present application, the edge e of the graph model can be obtained using a multi-layer fully connected neural network MLP, which can be expressed as e ij =MLP(|v i -v j |), where v i 、vj Represented as two nodes in the graph model, the features of node i and node j are subtracted and taken to have the property of measuring the difference between features. Then they are input into a multi-layer fully connected neural network to learn the value of the edge between the two specific nodes. In this way, the edges of the graph model G can be adaptively learned.

[0088] In the embodiment of the present application, the graph model obtained above is input into GCN to extract the relationship feature information F between the aggregated scene information and the target object information, the target objects, and the scene information and the target feature information. G Among them, F G It can be expressed as F G =GCN(G), which can be used in subsequent detection networks to achieve classification and detection, etc.

[0089] Based on the above embodiments, the present application also provides an overall flow diagram of a target object detection method, which uses scene information as a guide for detecting target objects in video surveillance. The specific process is as follows: Figure 3 As shown, the detailed implementation process of the flowchart is as follows:

[0090] First, the image I to be detected obtained from video surveillance is input into the backbone network N for feature extraction to obtain a feature map F with certain high-level semantic information, where the backbone network N is composed of convolutional layers.

[0091] Then, the feature map F is input into the RGA-S network to extract the global spatial relationship feature information of the image to be detected I, and the scene semantic feature information F is further obtained based on the spatial relationship feature information S , and at the same time, the feature map F is input into the general target feature extraction network to obtain the target feature F0.

[0092] Finally, the scene semantic feature information F S And the target feature information F0 are input into GCN, and the scene information is added to the target feature information. In this way, the relationship between the scene semantic feature information and the target feature information is constructed. G , then, F G The final detection result is obtained through subsequent classification, regression and other detection processes.

[0093] Compared with the related art, the technical solution in the embodiment of the present application has the following technical advantages:

[0094] (1) The target object detection method can be used for a variety of detection tasks such as target detection, tracking, instance segmentation, etc., and is suitable for detection in a variety of scenarios. It avoids training multiple models for different application scenarios, reduces model training time and deployment, saves storage and computing resources, and realizes multi-task, multi-scenario intelligent analysis and monitoring of video surveillance.

[0095] (2) The detection tasks in the related technologies only consider local target feature information and ignore global scene information, which will result in a large number of missed detections and false detections in actual video surveillance detection. However, the embodiments of the present application add global spatial relationship attention and graph convolution structure to the general detection network, and obtain spatially global scene information by compactly representing the pairwise relationship of each feature node information in the spatial dimension. The graph convolution network is used to represent the scene information and the target as a graph structure, and the global spatial relationship attention and graph convolution network are used to obtain the scene information and the contextual relationship with the target object. The scene information and the relationship information between the scene information and the target feature information can be added to the original feature information as a guide for detection, which can reduce missed detections and false detections, effectively improve the detection accuracy, and improve the video surveillance performance.

[0096] Based on the above embodiment, the present application also provides a method for processing the feature graph F using the RGA-S network to obtain F s The specific implementation process diagram is as follows: Figure 4 As shown. Figure 4 In the above example, the input feature graph F is processed to obtain the node graph information T s ; The node graph information T s All feature node information in is input into two 1×1 convolutional layers to obtain the vectors of two feature nodes. The two vectors are dot-producted to obtain the spatial relationship matrix R S ; For the i-th feature node information, stack it and all the paired relationships in the feature nodes in a certain order to obtain the spatial relationship feature information r i ; Perform global average pooling on the feature map F and combine it with the obtained spatial relationship feature information r i Splice and get E s ; for E s Perform 1×1 convolution operation and RELU processing of the regularization layer BN to obtain the attention weight value a i ; Finally, using the feature map F and a i Multiply to get F S .

[0097] It should be noted that Figure 4 The detailed process of each step in the above-mentioned embodiment has been described. For details, please refer to the specific implementation process in the above-mentioned embodiment, which will not be repeated here.

[0098] Based on the above embodiment, in another embodiment of the present application, a target object detection device 1 is provided. Figure 5 As shown, the target object detection device 1 includes:

[0099] The acquisition unit 10 is used to acquire scene semantic feature information and target feature information of the image to be detected.

[0100] The determination unit 11 is used to determine the relationship feature information between the scene information and the target object information in the image to be detected based on the scene semantic feature information and the target feature information.

[0101] The detection unit 12 is used to detect the target object in the image to be detected based on the relationship feature information.

[0102] In one embodiment, the target object detection device 1 may further include: an extraction unit.

[0103] The extraction unit is used to extract feature information of the image to be detected based on a pre-trained feature extraction network to obtain feature data of the image to be detected.

[0104] The determination unit 11 is further used to determine scene semantic feature information based on the feature data and the global perception network.

[0105] The determination unit 11 is further used to determine target feature information based on the feature data and the target feature extraction network.

[0106] In one embodiment, the extraction unit is further used to input the feature data into the global perception network to obtain spatial relationship feature information.

[0107] The determination unit 11 is also used to determine the spatial relationship attention information based on the feature data and the spatial relationship feature information.

[0108] The determination unit 11 is also used to determine the attention weight value of each spatial position in the feature data based on the spatial relationship attention information.

[0109] The determination unit 11 is further used to determine scene semantic feature information based on the feature data and the attention weight value.

[0110] In one embodiment, the determination unit 11 is further configured to determine feature node data of each spatial position in the feature data based on a global perception network.

[0111] The determination unit 11 is further used to determine the node graph information based on the characteristic node data of each spatial position.

[0112] The determination unit 11 is further configured to select target feature node information from the node graph information, and determine spatial relationship information based on the target feature node information.

[0113] The determination unit 11 is further configured to determine the spatial relationship feature information corresponding to the target feature node information based on the spatial relationship information.

[0114] In one embodiment, the target object detection device 1 may further include: a processing unit.

[0115] The processing unit is used to perform regularization and nonlinear processing on the spatial relationship attention information to obtain the attention weight value.

[0116] In one embodiment, the processing unit is further used to perform pooling processing on the feature data to obtain feature data after pooling processing.

[0117] The determination unit 11 is further used to determine the spatial relationship attention information based on the feature data after pooling processing and the spatial relationship feature information.

[0118] In one embodiment, the determining unit 11 is further configured to construct a data structure based on the scene semantic feature information and the target feature information.

[0119] The determination unit 11 is further used to determine the relationship feature information based on the data structure and the pre-trained neural network model.

[0120] The embodiment of the present application provides a target object detection device, which obtains scene semantic feature information and target feature information of a picture to be detected; determines the relationship feature information between the scene information and the target object information in the picture to be detected based on the scene semantic feature information and the target feature information; and detects the target object in the picture to be detected based on the relationship feature information. It can be seen that the target object detection device proposed in the embodiment of the present application, in the process of detecting the target object, not only obtains the target feature information of the picture to be detected, but also obtains the scene semantic feature information of the picture to be detected, and constructs the relationship feature information between the scene information and the target object information of the picture to be detected by combining the scene semantic feature information and the target feature information. Because the relationship feature information integrates the scene information where the target object is located, combined with the correlation between the scene information and the target object information, when the target object in the picture to be detected is detected using the relationship feature information, the fused relationship feature information can reflect the relationship between the scene information and the target object information, thereby reducing the probability of the target object being misdetected and improving the accuracy of target object detection.

[0121] Figure 6 The schematic diagram of the structure of a target object detection device 1 provided in an embodiment of the present application is shown in FIG. 1 . In practical applications, based on the same disclosure concept of the above embodiments, Figure 6 As shown, the target object detection device 1 of the embodiment of the present application includes a processor 13, a memory 14 and a communication bus 15.

[0122] In the specific implementation process, the acquisition unit 10, the determination unit 11, the detection unit 12, the extraction unit, and the processing unit can be implemented by a processor 13 located on the target object detection device 1, and the processor 13 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing image processing device (DSPD), a programmable logic image processing device (PLD), a field programmable gate array (FPGA), a CPU, a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the embodiments of the present application are not specifically limited.

[0123] In the embodiment of the present application, the communication bus 15 is used to realize the connection communication between the processor 13 and the memory 14; when the processor 13 executes the running program stored in the memory 14, the following target object detection method is realized:

[0124] Obtain scene semantic feature information and target feature information of the image to be detected; determine the relationship feature information between the scene information and the target object information in the image to be detected based on the scene semantic feature information and the target feature information; detect the target object in the image to be detected based on the relationship feature information.

[0125] In one embodiment, the processor 13 is further used to extract feature information of the image to be detected based on a pre-trained feature extraction network to obtain feature data of the image to be detected; determine scene semantic feature information based on the feature data and the global perception network; and determine target feature information based on the feature data and the target feature extraction network.

[0126] In one embodiment, the processor 13 is also used to input feature data into a global perception network to obtain spatial relationship feature information; determine spatial relationship attention information based on the feature data and the spatial relationship feature information; determine the attention weight value of each spatial position in the feature data based on the spatial relationship attention information; and determine scene semantic feature information based on the feature data and the attention weight value.

[0127] In one embodiment, the processor 13 is further used to determine feature node data of each spatial position in the feature data based on a global perception network; determine node graph information based on the feature node data of each spatial position; select target feature node information from the node graph information, and determine spatial relationship information based on the target feature node information; and determine spatial relationship feature information corresponding to the target feature node information based on the spatial relationship information.

[0128] In one embodiment, the processor 13 is further used to perform regularization and nonlinear processing on the spatial relationship attention information to obtain an attention weight value.

[0129] In one embodiment, the processor 13 is further used to perform pooling processing on the feature data to obtain the feature data after pooling processing; and determine the spatial relationship attention information based on the feature data after pooling processing and the spatial relationship feature information.

[0130] In one embodiment, the processor 13 is further used to construct a data structure based on the scene semantic feature information and the target feature information; and determine the relationship feature information based on the data structure and the pre-trained neural network model.

[0131] Based on the above embodiments, an embodiment of the present application provides a storage medium on which a computer program is stored. The above computer-readable storage medium stores one or more programs. The above one or more programs can be executed by one or more processors and applied to a target object detection device. The computer program implements the target object detection method as described above.

[0132] Based on the above embodiments, an embodiment of the present application provides a computer program product, including a computer program. The computer program can be executed by one or more processors and applied to a target object detection device. The computer program implements the target object detection method as described above.

[0133] It should be noted that in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for an image display device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the method described in each embodiment of the embodiment of the present application.

[0135] The above is only a specific implementation of the embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target object detection method, characterized in that: The method comprises: Obtain scene semantic feature information and target feature information of the image to be detected; Based on the scene semantic feature information and the target feature information, determining the relationship feature information between the scene information and the target object information in the image to be detected; Based on the relationship feature information, the target object in the to-be-detected picture is detected.

2. The method according to claim 1, characterized in that The step of obtaining scene semantic feature information and target feature information of the image to be detected includes: Extracting feature information of the image to be detected based on a pre-trained feature extraction network to obtain feature data of the image to be detected; Determining the scene semantic feature information based on the feature data and the global perception network; Based on the feature data and a target feature extraction network, the target feature information is determined.

3. The method according to claim 2, characterized in that The determining the scene semantic feature information based on the feature data and the global perception network includes: Inputting the feature data into the global perception network to obtain spatial relationship feature information; Determining spatial relationship attention information based on the feature data and the spatial relationship feature information; Based on the spatial relationship attention information, determining an attention weight value for each spatial position in the feature data; Based on the feature data and the attention weight value, the scene semantic feature information is determined.

4. The method according to claim 3, characterized in that The step of inputting the feature data into the global perception network to obtain spatial relationship feature information includes: Based on the global perception network, determining feature node data of each spatial position in the feature data; Determining node graph information based on the characteristic node data of each spatial position; Selecting target feature node information from the node graph information, and determining spatial relationship information based on the target feature node information; Based on the spatial relationship information, the spatial relationship feature information corresponding to the target feature node information is determined.

5. The method according to claim 3, characterized in that: The determining, based on the spatial relationship attention information, an attention weight value for each spatial position in the feature data comprises: Regularization and nonlinear processing are performed on the spatial relationship attention information to obtain the attention weight value.

6. The method according to claim 3, characterized in that The determining of spatial relationship attention information based on the feature data and the spatial relationship feature information includes: Performing pooling processing on the feature data to obtain feature data after pooling processing; Based on the pooled feature data and the spatial relationship feature information, spatial relationship attention information is determined.

7. The method according to claim 1, characterized in that The determining, based on the scene semantic feature information and the target feature information, the relationship feature information between the scene information and the target object information in the to-be-detected picture includes: Constructing a data structure based on the scene semantic feature information and the target feature information; Based on the data structure and the pre-trained neural network model, the relationship feature information is determined.

8. A target object detection device, characterized in that: The target object detection device comprises: An acquisition unit, used to acquire scene semantic feature information and target feature information of the image to be detected; A determination unit, configured to determine, based on the scene semantic feature information and the target feature information, relationship feature information between the scene information and the target object information in the image to be detected; A detection unit is used to detect the target object in the image to be detected based on the relationship feature information.

9. A target object detection device, characterized in that: The target object detection device comprises: a processor and a memory; when the processor executes the running program stored in the memory, the method according to any one of claims 1 to 7 is implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.

Images