Object detection method and device

Abstract

The embodiment of the present invention provides a method and device for detecting an object, and relates to the technical field of target detection. The method includes: using a preset algorithm to detect a first target frame image in a video image to obtain the position and state information of the first object; extracting the features of the first object according to the position and state information to obtain the first feature information; using a preset algorithm to detect a second target frame image in the video image to obtain the second feature information of the second object; determining the repeated detection objects in the first object and the second object based on the first feature information and the second feature information; wherein the spatial attention module included in the preset algorithm is used to obtain the global features of the object, and determine the position and state of the object according to the global features. Through the present invention, the preset algorithm includes a spatial attention module, which can obtain the global features of the detection object, and the determined position and state of the detection object are more accurate, which can improve the detection accuracy.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of target detection, and in particular, to a method and device for detecting an object. Background Art

[0002] Object detection is an important task in the field of computer vision, which aims to determine the specified objects and their specific locations in images or videos. With the rapid development of neural networks and deep learning, the performance and accuracy of object detection have been continuously improved, and it has been widely used in many fields. For example, robot vision, autonomous driving, human-computer interaction, content-based image retrieval, intelligent video surveillance and augmented reality, scenic area garbage, etc.

[0003] Taking garbage in scenic spots as an example, the main goal of garbage detection is to determine whether there is garbage in the image or video by analyzing the features in the image or video, so as to timely discover and deal with the garbage in the scenic spot and maintain the environment of the scenic spot. In the existing technology, when performing garbage detection, the accuracy of garbage detection is low, resulting in a high missed detection rate. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for detecting an object, so as to at least solve the problem that the accuracy of garbage detection in scenic spots is low, resulting in a high missed detection rate.

[0005] According to an embodiment of the present invention, a method for detecting an object is provided, comprising:

[0006] Using a preset algorithm to detect a first target frame image in the video image, and obtaining first position information and first state information of a first object in the first target frame image;

[0007] Extracting a feature of the first object according to the first position information and the first state information to obtain first feature information of the first object;

[0008] Using the preset algorithm to detect a second target frame image in the video image, to obtain second position information and second state information of a second object in the second target frame image, wherein the second target frame image is an image acquired after the first target frame image;

[0009] Extracting a feature of the second object according to the second position information and the second state information to obtain second feature information of the second object;

[0010] Performing feature matching on the first object and the second object based on the first feature information and the second feature information, and determining a repeated detection object in the first object and the second object according to the matching result;

[0011] Among them, the preset algorithm includes a spatial attention module, which is used to obtain the global features of the object and determine the location information and state information of the object based on the global features.

[0012] According to one embodiment of the present invention, there is provided a device for detecting an object, comprising:

[0013] A first detection module, configured to detect a first target frame image in a video image using a preset algorithm to obtain first position information and first state information of a first object in the first target frame image;

[0014] A first extraction module, configured to extract a feature of the first object according to the first position information and the first state information to obtain first feature information of the first object;

[0015] A second detection module is used to detect a second target frame image in the video image using the preset algorithm to obtain second position information and second state information of a second object in the second target frame image, where the second target frame image is an image collected after the first target frame image;

[0016] A second extraction module, configured to extract a feature of the second object according to the second position information and the second state information to obtain second feature information of the second object;

[0017] a determination module, configured to perform feature matching on the first object and the second object based on the first feature information and the second feature information, and determine a repeated detection object between the first object and the second object according to the matching result;

[0018] Among them, the preset algorithm includes a spatial attention module, which is used to obtain the global features of the object and determine the location information and state information of the object based on the global features.

[0019] According to another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps in any embodiment of the above-mentioned object detection method when running.

[0020] According to another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any embodiment of the above-mentioned object detection method.

[0021] Through the present invention, based on the preset algorithm adopted including a spatial attention module, the spatial attention module can obtain the global features of the detection object. Therefore, the position and state of the detection object are determined more accurately, thereby improving the accuracy of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a hardware structure diagram of a mobile terminal of an object detection method according to an embodiment of the present invention;

[0023] Figure 2 is a flow chart of a method for detecting an object according to an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a network structure of a specific embodiment of the present invention;

[0025] Figure 4 is a structural diagram of a spatial attention (CA) module of an embodiment of the present invention;

[0026] Figure 5 is a structural diagram of a channel attention (ECA) module of an embodiment of the present invention;

[0027] Figure 6 It is a structural diagram of an object detection device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0030] The method embodiments provided in the embodiments of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of an object detection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more (only one is shown in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a method for detecting an object in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0032] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0033] This embodiment provides a method for detecting an object, wherein: Figure 2 is a flow chart of a method for detecting an object according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0034] Step S201: Use a preset algorithm to detect a first target frame image in a video image to obtain first position information and first state information of a first object in the first target frame image.

[0035] Among them, the preset algorithm includes a spatial attention module, which is used to obtain the global features of the object and determine the location information and state information of the object based on the global features.

[0036] In this step, video images can be collected through monitoring, drones, mobile terminals, etc., and any frame of the image can be obtained from the video image, that is, the first target frame image. In order to obtain better image effects, a frame of image with better shooting quality can be obtained from multiple frames of images taken continuously as the first target frame image. The number of frames of the video can also be linearly sampled to obtain a reasonable number of frame-by-frame pictures, or it can be the first frame in the picture sequence after linear sampling. After obtaining the first target frame image, the first target frame image is detected to obtain the position information and state information of the first object in the image, that is, the first position information and the second state information. The first position information can be understood as the position of the first object in the image, for example, the coordinates of the feature points of the first object in the image. The first state information may include information such as the length, width, and shape of the first object.

[0037] The first object may be an object to be detected, such as a vehicle, a target person, garbage, etc. The first object may be a set of all detection objects in the first target frame image, and the first object may include one or more objects. In order to facilitate understanding of this embodiment, the present invention takes the detection of garbage in a scenic area as an example, and the first object is garbage. When the first target frame image containing garbage is captured, the image is detected to obtain the location information and status information of the garbage in the image.

[0038] In order to improve the accuracy of object detection, a spatial attention module is added to the above-mentioned preset algorithm adopted by the present invention, and the above-mentioned preset algorithm is, for example, the Yolov5 algorithm. It should be noted that other algorithms that can implement the scheme of the present invention are also within the protection scope of the present invention. For ease of understanding, the present invention only takes the improved Yolov5 algorithm as an example for explanation.

[0039] like Figure 3 As shown in FIG. 1 , a spatial attention (CA) module is added to the backbone neural network of Yolov5 to transfer deep features to shallow features to enhance the extraction of global features of the object. Based on the acquired global features, more accurate state information and position information of the first object can be obtained.

[0040] like Figure 4 As shown in the figure, four CA modules are added to the backbone neural network of Yolov5. The CA module performs average pooling along the row and column directions to obtain the local features of each row and column, thereby obtaining the global features of the first object, and then concatenates the features of each row and column along the channel direction, embedding the position information into the channel attention, thereby achieving the effect of simultaneously obtaining channel attention and spatial attention.

[0041] At the same time, the output is activated through a two-dimensional convolution, a batch normalization, and a nonlinear function. The specific calculation process is as follows:

[0042]

[0043]

[0044] f = δ(BN(Conv2d 1×1 (z h ,z w ))) (3)

[0045] Where x is the input feature map, x∈R C×H×W ;z h is the output after average pooling along the height direction, z h ∈R C×H×1 ;z w is the output after average pooling along the width direction, z w ∈R C×1×W The two outputs are then concatenated along the channel direction, and the output is activated through a two-dimensional convolution, a batch normalization, and a nonlinear function, resulting in the expression shown in (3).

[0046] In the above formula, (,) represents the connection along the channel direction; Conv2d 1×1 represents a two-dimensional convolution with a convolution kernel size of 1×1; BN represents a batch normalization operation; δ is a nonlinear activation function; f is an intermediate output feature map, f∈R C / r×1×(H+W) r is the same as in the SE block and is used to control the channel reduction rate. Then f is split into two independent tensors f h and f w , where f h ∈R C ×H×1 , f w ∈R C×1×W , perform two-dimensional convolution and Sigmoid function activation on these two tensors to output two attention g h and g w ,g h ∈R C×H×1 ,g w ∈R C×1×W Finally, these two attentions are multiplied with the original input x to obtain the final feature map output y, y∈R C×H×W , whose expression is:

[0047] f h ,f w =Split(f) (4)

[0048] g h =σ(Conv2d 1×1 (f h )) (5)

[0049] g w = σ(Conv2d 1×1 (f w )) (6)

[0050] y=g h ·g w ·x (7)

[0051] Where Split is the split tensor; σ is the Sigmoid function.

[0052] Since convolution operations can often only focus on the relationship between similar pixels, on this basis, adding a spatial attention CA module after each layer of convolution can expand the receptive field, enhance the object's extraction of global features, and achieve feature fusion, which can further improve the accuracy of detection.

[0053] Step S202: Extract features of the first object according to the first position information and the first state information to obtain first feature information of the first object.

[0054] In this step, the features of the first object can be extracted based on the first position information and the first state information, thereby obtaining the first feature information of the first object. After determining the position and state, the specific detail features of the first object can be further identified, which can improve the recognition accuracy.

[0055] Optionally, the preset algorithm also includes a channel attention module, which is used to fuse the features of the object and determine the location information and state information of the object based on the fused features.

[0056] In order to further improve the accuracy of object detection, the present invention also adds a channel attention (ECA) module to the improved Yolov5 algorithm. Figure 5 As shown in Figure 1, the ECA module is an improvement of the SENet module. ECA proposes to replace the fully connected network of SENet by using a one-dimensional convolution of size k, where the size of the convolution kernel k represents the coverage of local cross-channel interactions, and also proposes an adaptive method to adjust the size of k. Experiments show that ECA is more efficient in extracting channel information, and has fewer parameters and computations.

[0057] The specific calculation process of the ECA module is:

[0058] s=σ(C1D k (GAP(x))) (8)

[0059] y=s·x (9)

[0060]

[0061] Where x is the input feature map, x∈R C×H×W ; s is the channel attention output, s∈R C×1×1 ; y is the final output, y∈R C×H×W ; GAP stands for global average pooling; C1D k represents a one-dimensional convolution with a convolution length of k; σ is the Sigmoid function. Formula (10) is the mapping expression between k and C. Where γ and b are hyperparameters, we set γ and b to 2 and 1 respectively; |t| odd represents the odd integer closest to t.

[0062] Among them, the convolution length k determines the number of local domain channels captured by one-dimensional convolution. The larger k is, the larger the receptive field is. For different convolutional neural network structures, the mapping relationship between k and C can be obtained by formula (10), thereby achieving an adaptive convolution effect.

[0063] The Neck module in yolov5 performs feature fusion on feature maps of different sizes extracted by the backbone network to perform multi-scale prediction.

[0064] Optionally, extracting the feature of the first object according to the first position information and the first state information to obtain first feature information of the first object includes:

[0065] Separating the image of the first object from the first target frame image according to the first position information and the first state information;

[0066] The first feature information of the first object is obtained by performing feature extraction based on the separated image of the first object.

[0067] In this embodiment, object detection is performed by adding a yolov5 module with an attention mechanism to obtain first position information and first state information of the first object in the first target frame image. For example, in the captured image, it is determined that the garbage is located in the middle of the image, and the coordinate information of the feature points of the garbage in the image is obtained to determine the length and width of the garbage.

[0068] The first object is separated from the first target frame image using the position information and the state information, and the image of the separated first object is further subjected to feature recognition to obtain the first feature information. Feature matching based on the first feature information can improve the accuracy of feature matching, and can improve the accuracy of detection for situations such as garbage occlusion and coverage. For example, if a piece of garbage is detected in the middle area of ​​the image, the garbage is separated from the original image based on the position and state, and further identified to obtain the features of the garbage, so as to perform feature matching with the garbage detected subsequently based on the features of the garbage.

[0069] When the garbage contains multiple different types of garbage piled up and covered, further identification based on the separated images can obtain the quantity and category of the garbage. When detecting garbage that is obscured or tiny, the accuracy of garbage identification can be improved, thereby improving the accuracy of garbage duplicate detection object detection in the subsequent matching process of garbage duplicate detection.

[0070] Step S203: Use the preset algorithm to detect the second target frame image in the video image to obtain second position information and second state information of the second object in the second target frame image, where the second target frame image is an image captured after the first target frame image.

[0071] The second target frame image may be an image captured after the first target frame image, and the number of frames of the video may be linearly sampled to obtain a reasonable number of frame-by-frame images, or the first frame image may be linearly sampled, and a frame image after the first target frame image may be obtained, i.e., the second target frame image, and the object in the second target frame image may be detected to obtain the second position information and the second state information. The second object may also be a set of detection objects, and the specific detection process of the second target frame image may refer to the detection of the first target frame image, which will not be described in detail here.

[0072] Step S204: extracting the features of the second object according to the second position information and the second state information to obtain second feature information of the second object.

[0073] The feature extraction process in this step can refer to the feature extraction of the first object, and will not be repeated here.

[0074] Step S205: perform feature matching on the first object and the second object based on the first feature information and the second feature information, and determine a repeated detection object in the first object and the second object according to the matching result.

[0075] In this step, the first feature information and the second feature information may be feature matched, and the feature information and location information of the first object may be matched with the feature information and location information of the second object. When the first object contains multiple objects, or the second object contains multiple objects, or both the first object and the second object contain multiple objects, the object with matching features in the two objects is determined as a duplicate detection object.

[0076] Optionally, after detecting the first target frame image in the video image using a preset algorithm to obtain the first position information and the first state information of the first object in the first target frame image, the method further includes:

[0077] Predicting the position and state of the first object in an image acquired after the first target frame image according to the first position information and the first state information to obtain predicted position information and predicted state information;

[0078] The performing feature matching on the first object and the second object based on the first feature information and the second feature information, and determining a repeated detection object in the first object and the second object according to the matching result, includes:

[0079] Acquire a first attribute feature of the first object, where the first attribute feature includes the predicted position information, the predicted state information, and the first feature information;

[0080] Acquire a second attribute feature of the second object, where the second attribute feature includes the second position information, the second state information, and the second feature information;

[0081] The first attribute feature and the second attribute feature are matched, and a repeated detection object in the first object and the second object is determined according to the matching result.

[0082] Since the camera may be in a moving state during the shooting process, based on the first state information and first position information of the first object in the acquired first target frame image, the state and position of the first object in the subsequent frame, that is, the image acquired after the first target frame image, can be predicted, that is, the predicted state information and predicted position information. For example, in the acquired first target frame image, the garbage is slightly blocked on one side of the tree, and the garbage and the tree are both located in the middle area of ​​the first target frame image; based on the first target frame image, it can be predicted that in the next acquired frame image, the garbage is blocked by the tree more, and the garbage and the tree are both located in the edge area of ​​the image, and the length of the garbage is reduced. Since the position and state of the object in the image change, predicting the position and state of the object in the image can avoid repeated detection of the same object.

[0083] Among them, the attribute feature may include any one or more of the feature information of the object, the position information of the object, and the state information of the object. In order to make the matching result more accurate and improve the detection rate of duplicate objects, the second feature information of the second object may be matched with the first feature information of the first object, and the position information of the second object may be matched with the predicted position information of the first object, and the state information of the second object may be matched with the predicted state information of the first object. Through the above-mentioned multiple matching, the matching accuracy can be improved. When the attribute feature includes the above-mentioned feature information, position information and state information at the same time, cascade matching can also be performed according to the attribute features of the object. For example, the priority of setting the feature information of the object is the highest, and the priority of setting the state and position information of the object is lower. When matching, the feature information can be matched first, and when the feature information matches, no other feature matching is required. When the matching degree of the feature information of the object is slightly low and is within the preset range, but the matching degree of the position and state information of the object is very high, the attribute feature can also be considered to match. When the feature matching degree is low, and the matching degree of the state information and the position information is also low, it is considered not to match.

[0084] In order to ensure the accuracy and efficiency of the detection algorithm, the present invention selects yolov5s as the reference network to add the attention mechanism, so as to conduct comparative experiments and select a suitable network model as the detection model. The Neck module in yolov5 uses the structure of FPN and PAN. For the prediction of each scale, it connects the feature map of the next layer with the feature map of this layer through upsampling and convolution to obtain the final prediction result. Finally, each layer has its meaning, which is respectively the prediction score, the position of the target center point, the width, height and offset of the prediction frame, and the probability of each target type. Each layer is interconnected, and the accuracy of its final result is related to each layer. Therefore, an ECA module is added after each feature fusion operation to improve the prediction between the hierarchical relationship, thereby improving the accuracy of detection.

[0085] In the case where the first object and the second object include objects with matching attribute features, the objects will not be counted repeatedly when counting the number of objects, thereby avoiding repeated inspection of the same object.

[0086] In order to achieve the above effect, the present invention adopts the deepsort algorithm. The deepsort algorithm is mainly divided into two parts: Kalman filter and Hungarian matching. The Kalman filter predicts the next frame video result according to the current detection result, and uses the Hungarian algorithm to match the detection result of the next moment with the prediction result of the current moment. If there is a matching object, it means that the same object exists in the current frame and the next frame. The same object is identified and counted, thereby solving the problem of repeated inspection of adjacent frames. At the same time, the Kalman filter will save the object frame detection results of the specified number of times, and if it appears again under the specified number of frames, it will be re-identified, thereby avoiding repeated detection of the same object.

[0087] The Kalman filter is composed of two modules: the Prediction module and the Update module. The Prediction module predicts the detection result of the next frame based on the detection result of the current frame, and predicts the covariance matrix of the next frame based on the covariance matrix of the current frame. The specific calculation process is as follows:

[0088]

[0089] P - k = AP k-1 A T + Q (12)

[0090] A and B are state change matrices, represents the detection result of the k-1th frame,

[0091] represents the prediction of the detection result of the Kalman filter for the kth frame, P k-1 It represents the covariance matrix of the k-1th frame, P - k It represents the prediction of the covariance matrix of the kth frame by the Kalman filter, and Q represents the prediction error, which is usually generated by normal distribution.

[0092] The covariance matrix describes the change of the state matrix over time, and represents the covariance between the eight state quantities (x, y, g, h, x', y', g', h'). Among them, (x, y) is the center coordinate of the target bounding box, g is the aspect ratio of the bounding box, h is the height of the bounding box, and (x', y', g', h') represents the rate of change of the corresponding parameters.

[0093] The Update module is and P k The iterative update of these two results. The specific calculation process is as follows:

[0094]

[0095]

[0096] P k = (I - K k C) P - k (15)

[0097] Where K k is the Kalman gain, C is the state change matrix, Represents the detection result of the kth frame, y k represents the detection value of the detector in the kth frame, P k represents the covariance matrix of the k-th frame.

[0098] The Hungarian matching algorithm is then used to match the prediction results of the Kalman filter with the detection values ​​of the detector. It builds a cost matrix and uses relevant matching rules for matching. The specific matching process is as follows:

[0099] 1) For each row of the matrix, subtract its minimum element;

[0100] 2) For each column of the matrix, subtract its minimum element;

[0101] 3) Cover the zeros in the matrix with the least number of horizontal or vertical lines;

[0102] 4) If the number of lines = N, the optimal allocation is found and the algorithm ends, otherwise it goes to step 5);

[0103] 5) Find the smallest element that is not covered by any line, subtract this element from each row that is not covered by the line, add this element to each column that is covered by the line, and return to step 3);

[0104] In the deepsort algorithm, the appearance metric, motion state metric, and position metric are used to measure the similarity of the detected objects, the prediction performance of the Kalman filter, and the correlation of the spatial position of the detected objects. Here, we use cosine distance, Mahalanobis distance, and IOU loss to calculate them.

[0105] After obtaining the prediction information, the Mahalanobis distance is used to perform similarity motion matching between the target detection position and the predicted target state. The calculation formula is shown in (16).

[0106]

[0107] In the formula, y i represents the predicted target position of the i-th trajectory, d j Detect the bounding box for the jth object, S iis the covariance matrix of the current frame predicted by the Kalman filter. After obtaining the Mahalanobis distance between the two, use X 2 Distribution, by setting a threshold to determine whether the j-th target detection bounding box is similar to the i-th track.

[0108]

[0109] Where, t (1) For X 2 Corresponding to the 95% confidence threshold, here we take 9.4877; if d (1) (i,j) is less than the threshold, Take 1 to indicate successful association, otherwise take 0 to indicate failed association. In order to prevent the identity conversion phenomenon caused by the lost target re-entering the field of view, the target feature vector within a certain frame number is retained and the set R k express.

[0110]

[0111] Where, L k The number of target features to be retained is set to 100. For the measurement of the appearance features of the detection results and trajectories, deepsort uses the minimum cosine distance to measure the feature distance between the two.

[0112] For the calculation of the minimum cosine distance, the calculation formulas are shown in (19), (20), (21) and (22).

[0113]

[0114] Where, d (2) (i, j) is the minimum cosine distance between the N successfully associated trajectories of the i-th tracker and the j-th detected target feature vector. represents the normalized minimum cosine distance between the jth detected target and the i-th trajectory feature vector. Similarly, set the threshold t (2) , according to d (2) (i,j) determines whether the two are successfully associated.

[0115]

[0116] After obtaining the matching correlation between motion and appearance information through a series of calculations, the two mechanisms are integrated to construct the correlation measurement function as shown in (21).

[0117] C i,j =λd (1) (i,j)+(1-λ)d (2) (i,j) (21)

[0118] Where λ is the proportional coefficient of different association metrics, C i,j The smaller the value, the more similar the ith tracking trajectory is to the jth detected target. The motion features and appearance features are jointly considered to determine whether the ith tracking trajectory is successfully associated with the jth detected target.

[0119]

[0120] Finally, the optimal tracking trajectory can be obtained using the Hungarian matching algorithm based on the association measurement results.

[0121] The IOU loss function is actually the intersection-over-union ratio, as shown in formula (23). It is mainly used here to evaluate the effect between the predicted value and the true value.

[0122]

[0123] Where B is the predicted bounding box, B gt is the true bounding box. From the formula, we can understand that when the predicted bounding box has no intersection with the true bounding box, the calculated loss value is 0, which means that the IOU loss function does not work, and the gradient contributed during the neural network training process is 0. Therefore, the convergence speed of training the neural network using the IOU loss function will be very slow.

[0124] At the same time, Hungarian matching is divided into two matchings. The first matching is to perform cascade matching on the determined trajectory frame that has been matched m times continuously with the detection frame detected by the detector. The loss functions used are Mahalanobis distance and cosine distance. The second matching is to match the determined trajectory frame and uncertain predicted trajectory frame that are not matched in the cascade matching with the unmatched detection frame. Finally, the unmatched uncertain trajectory frame and the determined frame that have not been matched for a specified number of times n are discarded, and the unmatched detection frame becomes an uncertain trajectory frame.

[0125] In order to improve the accuracy of the control matching, gate units are added to the Hungarian matching algorithm. These gate units control the accuracy of the matching through threshold control. The appearance metric used in the cascade matching of the Hungarian algorithm requires us to separate the targets detected by the detector and perform appearance metric on the targets detected in the next frame one by one. In the appearance metric, it is necessary to extract the features of the target and use the cosine loss to calculate the similarity of the appearance. Therefore, the quality of the extracted target greatly affects the tracking accuracy.

[0126] Optionally, the duplicate detection object is garbage; after determining the duplicate detection object in the first object and the second object according to the matching result, the method further includes:

[0127] identifying the type of waste;

[0128] Get the number of each type of garbage.

[0129] When the above method in this embodiment is applied to the garbage detection scene in a scenic area, after the garbage is detected by the above method, the garbage can be further identified to determine the type of garbage, for example, hazardous garbage, recyclable garbage, kitchen waste, other garbage, etc., and the quantity of each type of garbage can be obtained. For example, there are 2 recyclable garbage, 1 kitchen waste, and 3 other garbage. In order to determine the size of the garbage truck, the length and width of the garbage can be further obtained, and the volume of the garbage can be estimated according to the size of the garbage in the image.

[0130] Based on the type, quantity, and volume of the garbage, the size and type of garbage truck used to remove the garbage can be selected to remove the garbage and improve the efficiency of garbage removal.

[0131] The present invention adds a CA module and an ECA module to the yolov5s model and combines it with the deep sort algorithm. This can increase the detection accuracy of the algorithm while ensuring real-time performance, while also avoiding repeated detection of the same object in different frames, thereby improving the overall detection rate.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0133] In the present embodiment, a detection device for an object is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0134] Figure 6 is a structural diagram of an object detection device according to an embodiment of the present invention. Figure 6 As shown, the device comprises:

[0135] A first detection module 601 is used to detect a first target frame image in a video image using a preset algorithm to obtain first position information and first state information of a first object in the first target frame image;

[0136] A first extraction module 602, configured to extract a feature of the first object according to the first position information and the first state information to obtain first feature information of the first object;

[0137] A second detection module 603 is used to detect a second target frame image in the video image using the preset algorithm to obtain second position information and second state information of a second object in the second target frame image, where the second target frame image is an image collected after the first target frame image;

[0138] A second extraction module 604, configured to extract a feature of the second object according to the second position information and the second state information to obtain second feature information of the second object;

[0139] A determination module 605 is used to perform feature matching on the first object and the second object based on the first feature information and the second feature information, and determine a repeated detection object in the first object and the second object according to the matching result;

[0140] Among them, the preset algorithm includes a spatial attention module, which is used to obtain the global features of the object and determine the location information and state information of the object based on the global features.

[0141] Optionally, the preset algorithm also includes a channel attention module, which is used to fuse the features of the object and determine the location information and state information of the object based on the fused features.

[0142] Optionally, the device further includes a prediction module, configured to predict the position and state of the first object in an image acquired after the first target frame image according to the first position information and the first state information to obtain predicted position information and predicted state information;

[0143] The determination module comprises:

[0144] A first acquisition submodule, configured to acquire a first attribute feature of the first object, wherein the first attribute feature includes the predicted position information, the predicted state information, and the first feature information;

[0145] A second acquisition submodule, configured to acquire a second attribute feature of the second object, where the second attribute feature includes the second position information, the second state information, and the second feature information;

[0146] The determination submodule is used to match the first attribute feature with the second attribute feature, and determine the repeated detection object in the first object and the second object according to the matching result.

[0147] Optionally, the prediction module includes:

[0148] a separation submodule, configured to separate the image of the first object from the first target frame image according to the first position information and the first state information;

[0149] The extraction submodule is used to perform feature extraction based on the separated image of the first object to obtain the first feature information of the first object.

[0150] Optionally, the repeated detection object is garbage; and the device further includes:

[0151] An identification module, used for identifying the type of the garbage;

[0152] The acquisition module is used to obtain the number of each type of garbage.

[0153] The present invention adds a CA module and an ECA module to the yolov5s model and combines it with the deep sort algorithm. This can increase the detection accuracy of the algorithm while ensuring real-time performance, while also avoiding repeated detection of the same object in different frames, thereby improving the overall detection rate.

[0154] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0155] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0156] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0157] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0158] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0159] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0160] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting an object, characterized in that: include: Using a preset algorithm to detect a first target frame image in the video image, and obtaining first position information and first state information of a first object in the first target frame image; Extracting a feature of the first object according to the first position information and the first state information to obtain first feature information of the first object; Using the preset algorithm to detect a second target frame image in the video image, to obtain second position information and second state information of a second object in the second target frame image, wherein the second target frame image is an image acquired after the first target frame image; Extracting a feature of the second object according to the second position information and the second state information to obtain second feature information of the second object; Based on the first feature information and the second feature information, perform feature matching on the first object and the second object using a Hungarian matching algorithm including a gate unit, and determine a repeated detection object in the first object and the second object according to the matching result, wherein the gate unit is used to perform threshold control on the feature matching process of the Hungarian matching algorithm; Among them, the preset algorithm includes a CA spatial attention module and an ECA channel attention module. The ECA channel attention module is used to fuse the features of the object and determine the location information and status information of the object based on the fused features; the repeated detection object is garbage.

2. The method according to claim 1, characterized in that After the first target frame image in the video image is detected by using a preset algorithm to obtain the first position information and the first state information of the first object in the first target frame image, the method further includes: Predicting the position and state of the first object in an image acquired after the first target frame image according to the first position information and the first state information to obtain predicted position information and predicted state information; The performing feature matching on the first object and the second object based on the first feature information and the second feature information, and determining a repeated detection object in the first object and the second object according to the matching result, includes: Acquire a first attribute feature of the first object, where the first attribute feature includes the predicted position information, the predicted state information, and the first feature information; Acquire a second attribute feature of the second object, where the second attribute feature includes the second position information, the second state information, and the second feature information; The first attribute feature and the second attribute feature are matched, and a repeated detection object in the first object and the second object is determined according to the matching result.

3. The method according to claim 1, characterized in that The extracting the feature of the first object according to the first position information and the first state information to obtain first feature information of the first object includes: Separating the image of the first object from the first target frame image according to the first position information and the first state information; The first feature information of the first object is obtained by performing feature extraction based on the separated image of the first object.

4. The method according to any one of claims 1 to 3, characterized in that After determining the duplicate detected objects in the first object and the second object according to the matching result, the method further includes: identifying the type of waste; Get the number of each type of garbage.

5. A device for detecting an object, characterized in that: include: A first detection module, configured to detect a first target frame image in a video image using a preset algorithm to obtain first position information and first state information of a first object in the first target frame image; A first extraction module, configured to extract a feature of the first object according to the first position information and the first state information to obtain first feature information of the first object; A second detection module is used to detect a second target frame image in the video image using the preset algorithm to obtain second position information and second state information of a second object in the second target frame image, where the second target frame image is an image collected after the first target frame image; A second extraction module, configured to extract a feature of the second object according to the second position information and the second state information to obtain second feature information of the second object; a determination module, configured to perform feature matching on the first object and the second object using a Hungarian matching algorithm including a gate unit based on the first feature information and the second feature information, and determine a repeated detection object in the first object and the second object according to the matching result, wherein the gate unit is configured to perform threshold control on the feature matching process of the Hungarian matching algorithm; Among them, the preset algorithm includes a CA spatial attention module and an ECA channel attention module. The ECA channel attention module is used to fuse the features of the object and determine the location information and status information of the object based on the fused features; the repeated detection object is garbage.

6. The device according to claim 5, characterized in that The device further includes a prediction module, configured to predict the position and state of the first object in an image acquired after the first target frame image according to the first position information and the first state information to obtain predicted position information and predicted state information; The determination module comprises: A first acquisition submodule, configured to acquire a first attribute feature of the first object, wherein the first attribute feature includes the predicted position information, the predicted state information, and the first feature information; A second acquisition submodule, configured to acquire a second attribute feature of the second object, where the second attribute feature includes the second position information, the second state information, and the second feature information; The determination submodule is used to match the first attribute feature with the second attribute feature, and determine the repeated detection object in the first object and the second object according to the matching result.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 4 when executed.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 4.

Images