A target detection method, device, terminal and computer readable storage medium

Abstract

The application provides a target detection method, device, terminal and computer readable storage medium, the target detection method comprises: obtaining a to-be-detected video frame of a preset area; performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; performing feature extraction on the candidate target to obtain feature information of the candidate target; and determining whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object. The application performs target detection on the to-be-detected video frame to obtain a candidate target, compares the feature information of the candidate target with the first sample set containing the target object, determines whether the candidate target is the target object, avoids the phenomenon of false detection of the target object, and thus improves the detection accuracy and the generalization performance of the detection method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a target detection method and device, a terminal and a computer readable storage medium. BACKGROUND

[0002] In security monitoring, target detection technology is applied in various intelligent technologies as a basic module. Due to the complexity and variability of real scenes, background false detection and target missing detection may occur in intelligent devices. At present, the main optimization methods mainly include two categories: the first category is model training data enhancement, which is performed for different scenes, model fine-tuning, mainly increasing the background false detection data and missing detection data of the scene, and improving the generalization ability of the model. This method needs to collect and label a large amount of fine-tuning data and process the data for different scenes, which is tedious and has high maintenance cost; the second category is to solve the background false detection by pre-establishing a false detection target whitelist, comparing the similarity of the detection target features and the target features in the false detection whitelist, and setting a fixed threshold to determine whether it is false detection. This method requires strong prior knowledge to pre-set false detection targets that may appear in the scene, and still needs to expand the background whitelist for different scenes. After setting the fixed threshold, the scene generalization ability is not strong, and there may be a problem of only being applicable to certain specific scenes. SUMMARY

[0003] The technical problem solved by the present application is to provide a target detection method, device, terminal and computer readable storage medium, which solves the problem of low accuracy of target object detection in the prior art.

[0004] To solve the above technical problems, the first technical solution adopted by the present application is to provide a target detection method, which comprises: acquiring a to-be-detected video frame of a preset region; performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; performing feature extraction on the candidate target to obtain feature information of the candidate target; and determining whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object.

[0005] Among them, based on the first similarity between the feature information of the candidate target and the first sample set containing the target object, whether the candidate target is a target object is determined, comprising: in response to the first similarity between the feature information of the candidate target and the feature information of the target object contained in the first sample set being greater than a preset value, determining that the candidate target is a target object.

[0006] The first sample set includes at least two sub-image sets of different categories; each sub-image set includes a plurality of images containing the same part of the target object; and in response to the first similarity between the feature information of the candidate target and the feature information of the target object contained in the first sample set being greater than a preset value, the candidate target is determined to be the target object, including: based on the respective similarities between the feature vector of the candidate target and the feature information corresponding to the images in each sub-image set, the respective second similarities between each sub-image set and the candidate target are calculated; and according to the respective second similarities of each sub-image set, the first similarity between the first sample set and the target object is determined.

[0007] The method further includes: before obtaining the to-be-detected video frame of the preset region, constructing a second sample set, and the second sample set includes a plurality of background images not containing the target object.

[0008] The method further includes: calculating the third similarity between the feature information of the candidate target and the second sample set; and in response to the first similarity being greater than the third similarity, the candidate target is determined to be the target object.

[0009] The method further includes: in response to the first similarity being greater than the third similarity, the confidence of the target object is further weighted and reset.

[0010] The method further includes: in response to the first similarity being less than the third similarity, the candidate target is deleted, and the candidate target is attributed to the second sample set as a non-target object.

[0011] The method further includes: before extracting the feature information of the candidate target, determining whether the confidence of the candidate target exceeds a preset confidence; and if the confidence of the candidate target exceeds the preset confidence, the feature information of the candidate target is obtained through feature detection of the candidate target.

[0012] The method further includes: before obtaining the to-be-detected video frame of the preset region, obtaining a background video frame of the preset region, the collection time of the background video frame being earlier than the collection time of the to-be-detected video frame; performing target detection on the background video frame to obtain initial targets corresponding to the background video frame; extracting region images containing each initial target, and attributing the region images to the second sample set.

[0013] The background video frame of the preset area is acquired, and then the background video frame is subjected to data enhancement processing to obtain a plurality of enhanced images; the background video frame is subjected to target detection to obtain an initial target corresponding to the background video frame, including: the plurality of enhanced images are subjected to target detection respectively to obtain an initial target corresponding to the enhanced image.

[0014] The target detection network model is trained by: acquiring a first training data set, the first training data set including a plurality of first sample images containing targets and a plurality of first sample images not containing targets; the first sample image is labeled with a corresponding real target frame and a real category, and the real target frame of the first sample image not containing the target is a null value; each first sample image is input into the target detection network model to obtain a corresponding predicted target frame and a predicted category of each first sample image; and the target detection network model is iteratively trained based on an error value between the corresponding real category and the predicted category of the same first sample image, and an error value between the corresponding real target frame and the predicted target frame.

[0015] The feature extraction network model is trained by: acquiring a second training data set, the second training data set including a plurality of second sample images containing targets and a plurality of second sample images not containing targets; the second sample image is labeled with a real feature vector; each second sample image is input into the feature extraction network model to obtain a corresponding predicted feature vector of each second sample image; and the feature extraction network model is iteratively trained based on an error value between the corresponding real feature vector and the predicted feature vector of the same second sample image.

[0016] To solve the above technical problems, a second technical solution adopted by the present application is to provide a target detection device, which comprises: an acquisition module configured to acquire a to-be-detected video frame of a preset area; a detection module configured to perform target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; a feature extraction module configured to perform feature extraction on the candidate target to obtain feature information of the candidate target; and an analysis module configured to determine whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object.

[0017] To solve the above technical problems, a third technical solution adopted by the present application is to provide a terminal, which comprises a memory, a processor and a computer program stored in the memory and running on the processor, and the processor is used to execute the program data to realize the steps in the above target detection method.

[0018] To solve the above technical problems, a fourth technical solution adopted by the present application is to provide a computer readable storage medium, which stores a computer program, and the computer program is executed by the processor to realize the steps in the above target detection method.

[0019] The beneficial effects of the present application are: different from the prior art, the provided target detection method, device, terminal and computer readable storage medium, the target detection method comprises: obtaining a to-be-detected video frame of a preset area; performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; performing feature extraction on the candidate target to obtain feature information of the candidate target; determining whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object. The present application performs target detection on the to-be-detected video frame to obtain a candidate target, compares the feature information of the candidate target with the first sample set containing the target object to determine whether the candidate target is the target object, avoids the phenomenon of false detection of the target object, and further improves the detection accuracy and the generalization performance of the detection method. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a flowchart of the target detection method provided by the present application;

[0022] Figure 2 is a flowchart of an embodiment of the target detection method provided by the present application;

[0023] Figure 3 is a flowchart of a specific embodiment of the target detection method provided by the present application;

[0024] Figure 4 is Figure 2 is a flowchart of a specific embodiment of step S201 in the target detection method provided by the present application;

[0025] Figure 5 is Figure 2A flowchart of a specific embodiment of step S202 in the provided target detection method is provided.

[0026] Figure 6 A flowchart of a specific embodiment of step S203 in the provided target detection method is provided. Figure 2 A flowchart of a specific embodiment of step S202 in the provided target detection method is provided.

[0027] Figure 7 A schematic block diagram of the target detection device provided by the present application is provided.

[0028] Figure 8 A schematic block diagram of an embodiment of the terminal provided by the present application is provided.

[0029] Figure 9 A schematic block diagram of an embodiment of the computer-readable storage medium provided by the present application is provided. DETAILED DESCRIPTION

[0030] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0031] In the following description, specific details are set forth in order to provide a thorough understanding of the present application. However, persons having ordinary skill in the art will appreciate that embodiments of the present application can be practiced without many of the details.

[0032] The term "and / or" herein merely describes an association relationship of associated objects, and can represent three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects. In addition, "multiple" herein represents two or more than two.

[0033] In order for those skilled in the art to better understand the technical solutions of the present application, a target detection method provided by the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] Please refer to Figure 1 , Figure 1 A flowchart of the target detection method provided by the present application is provided. In the present embodiment, a target detection method is provided, which includes the following steps.

[0035] S11: Obtain a to-be-detected video frame of a preset area.

[0036] Specifically, an image acquisition device is installed, and a video frame in the preset area can be obtained in real time by the image acquisition device as a to-be-detected video frame. Alternatively, a monitoring video can be obtained by the image acquisition device, and any frame in the monitoring video frame can be taken as a to-be-detected video frame. The to-be-detected video frame includes a foreground image and a background image, and the foreground image is specifically a target object. For example, the target object can be a pedestrian, an animal, etc.

[0037] S12: performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame.

[0038] Specifically, the target detection network model is used to perform target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame.

[0039] S13: performing feature extraction on the candidate target to obtain feature information of the candidate target.

[0040] Specifically, it is determined whether the confidence of the candidate target exceeds a preset confidence; if the confidence of the candidate target exceeds the preset confidence, feature detection is performed on the candidate target to obtain the feature information of the candidate target.

[0041] In an embodiment, the feature extraction network model is used to perform feature detection on the candidate target to obtain the feature information of the candidate target.

[0042] S14: determining whether the candidate target is the target object based on a first similarity between the feature information of the candidate target and the first sample set containing the target object.

[0043] Specifically, in response to the first similarity between the feature information of the candidate target and the feature information of the target object contained in the first sample set being greater than a preset value, it is determined that the candidate target is the target object. The confidence of the target object is weighted and reset.

[0044] In an embodiment, the first sample set includes at least two sub-image sets of different categories; each sub-image set includes a plurality of images containing the same part of the target object; based on the respective similarities between the feature vector of the candidate target and the feature information corresponding to the images contained in each sub-image set, a weighted sum of the respective second similarities of each sub-image set contained in the first sample set is calculated and an average value is obtained, and the obtained average value is taken as the first similarity between the first sample set and the target object.

[0045] In an embodiment, a second sample set is constructed, and the second sample set includes a plurality of background images not containing the target object. The feature information of the candidate target and the second sample set are calculated to obtain a corresponding third similarity; in response to the first similarity being greater than the third similarity, it is determined that the candidate target is the target object. The confidence of the target object is weighted and reset.

[0046] In an embodiment, in response to the first similarity being less than the third similarity, the candidate target is deleted, and the candidate target is attributed to the second sample set as a non-target object.

[0047] The embodiment provides a target detection method, which comprises the following steps.

[0048] Referring to Figure 2 and Figure 3 , Figure 2 is a flowchart of an embodiment of the target detection method provided by the present application. Figure 3 is a flowchart of a specific embodiment of the target detection method provided by the present application.

[0049] The embodiment provides a target detection method, which comprises the following steps.

[0050] S201: training a target detection network model.

[0051] Specifically, the training method of the target detection network model comprises the following steps.

[0052] Referring to Figure 4 , Figure 4 is Figure 2 a flowchart of a specific embodiment of step S201 in the target detection method provided by the present application.

[0053] S2011: obtaining a first training data set.

[0054] Specifically, the first training data set comprises a plurality of first sample images containing at least one target and a plurality of first sample images not containing a target; the first sample images are labeled with corresponding real target frames and real categories, and the real target frame of the first sample image not containing a target is a null value.

[0055] Each first sample image contains at most one or more targets. Specifically, the first training data set comprises a head-shoulder image of a pedestrian, an upper body image of a pedestrian or a full-body image of a pedestrian. In order to improve the detection accuracy of the target detection network model, the first training data set further comprises a plurality of images not containing a target. The image containing a non-target also belongs to the image not containing a target. The image containing a non-target can be an image containing interference targets such as columnar objects, kitchen sundries and clothes.

[0056] When the target is contained in the first sample image, the first sample image is labeled with a real target frame corresponding to each target and a real category of the target. The real category can be head and shoulder, half body, and full body. The real position of the target can be determined based on the real target frame of the target.

[0057] When the target is not contained in the first sample image, the real category labeled by the first sample image is non-target; the first sample image is not labeled with a real target frame, or the real target frame of the first sample image is set to 0.

[0058] S2012: input each first sample image into a target detection network model to obtain a predicted target frame and a predicted category corresponding to each first sample image respectively.

[0059] Specifically, the first sample image is input into the target detection network model, and the target detection network model performs target detection on each first sample image to obtain a predicted target frame and a predicted category of each target contained in each first sample image. The target detection network model is a multi-classification detection network, that is, the target detection network model detects multiple categories of a preset target in the first sample image. That is, when the first sample image contains a pedestrian, the pedestrian head and shoulder, the pedestrian upper body, and the pedestrian full body in the first sample image are detected.

[0060] The target detection network model can be a Faster RCNN, YOLOV3, CenterNet, or other target detection network, and the specific implementation is not limited.

[0061] S2013: iteratively train the target detection network model based on an error value between a real category and a predicted category corresponding to the same first sample image, and an error value between a real target frame and a predicted target frame corresponding to the same first sample image.

[0062] Specifically, the error value between the real target frame and the predicted target frame corresponding to the same target in the same first sample image and the error value between the real category and the predicted category corresponding to the same target are calculated based on a cross-entropy loss Cross-entropy Loss.

[0063] The target detection network model is iteratively trained based on the error value between the real target frame and the predicted target frame corresponding to the same target in the same first sample image and the error value between the real category and the predicted category corresponding to the same target.

[0064] In an optional embodiment, the result of the target detection network model is back propagated, the weight of the target detection network model is corrected based on the error value between the real target frame and the predicted target frame corresponding to the same target in the same first sample image and the error value between the real category and the predicted category corresponding to the same target, and the training of the target detection network model is realized.

[0065] The first sample image is input into the target detection network model, and the target detection network model detects the target in the first sample image. When the error value between the predicted target frame and the real target frame corresponding to the same target and the error value between the corresponding predicted category and the real category are both less than a preset threshold, the preset threshold can be set by itself, for example, 1%, 5%, etc., the training of the target detection network model is stopped.

[0066] Through the training of the target detection network model, the detection accuracy of the network can be increased.

[0067] S202: training a feature extraction network model.

[0068] Specifically, the training method of the feature extraction network model specifically includes the following steps.

[0069] Please refer to Figure 5 , Figure 5 is Figure 2 a flowchart of a specific embodiment of step S202 in the target detection method provided by the application.

[0070] S2021: obtaining a second training data set.

[0071] Specifically, the second training data set includes a plurality of second sample images containing a target and a plurality of second sample images not containing a target; the second sample images are labeled with real feature vectors.

[0072] In another specific embodiment, the second sample image is a local image of each target in the first sample image. That is, the second sample image is an image containing a target, and the second sample image is a sub-image of the first sample image. Wherein, images of the same category are attributed to the same sub-data set. In order to improve the accuracy of feature extraction of the feature extraction network model, the second training set also includes second sample images not containing a target.

[0073] S2022: inputting each second sample image into the feature extraction network model to obtain a corresponding predicted feature vector of each second sample image.

[0074] Specifically, each second sample image is input into the feature extraction network model, and the feature extraction network model extracts features from the second sample image to obtain a predicted feature vector corresponding to the second sample image. Wherein, the feature extraction network can be VGG, ResNet, MobileNet, etc.

[0075] S2023: iteratively training the feature extraction network model based on the error value between the real feature vector and the predicted feature vector corresponding to the same second sample image.

[0076] Specifically, the error value between the corresponding real feature vector and the predicted feature vector in the same second sample image is calculated based on cross-entropy loss.

[0077] The feature extraction network model is iteratively trained based on the error value between the real feature vector and the predicted feature vector corresponding to the same second sample image.

[0078] In another embodiment, the similarity between feature vectors of the same category is large, and the similarity between feature vectors of different categories is small. The feature extraction network model can also be optimized based on the similarity between feature vectors. The similarity between the predicted feature vectors corresponding to each image included in the same sub-data set is calculated, and the images corresponding to the predicted feature vectors can be further verified based on the similarity whether they belong to the same category of sub-data set.

[0079] In a specific embodiment, the cosine distance between the predicted feature vector A and the predicted feature vector B corresponding to each image included in the same sub-data set is calculated based on the following formula 1.

[0080]

[0081] In formula 1, cos(ɑ) represents the cosine distance; A and B represent the feature information corresponding to different images respectively.

[0082] In response to the cosine distance between the predicted feature vector A and the predicted feature vector B being less than the preset distance, it is determined that the second sample image corresponding to the predicted feature vector A and the second sample image corresponding to the predicted feature vector B belong to the same category of sub-data set. In response to the cosine distance between the predicted feature vector A and the predicted feature vector B being greater than the preset distance, it is determined that the second sample image corresponding to the predicted feature vector A and the second sample image corresponding to the predicted feature vector B belong to different categories of sub-data set. The feature extraction network model can also be iteratively trained based on the difference between the categories corresponding to the second sample images.

[0083] In an optional embodiment, the result of the feature extraction network model is back-propagated, the weight of the feature extraction network model is corrected according to the error value between the real feature vector and the predicted feature vector corresponding to the same second sample image, and the training of the feature extraction network model is realized.

[0084] The second sample image is input into the feature extraction network model, and the feature extraction network model extracts features from the second sample image. When the error value between the real feature vector and the predicted feature vector corresponding to the second sample image is less than a preset threshold, which can be set by itself, for example, 1%, 5%, etc., the training of the feature extraction network model is stopped.

[0085] S203: Construct a second sample set.

[0086] Specifically, the second sample set includes a plurality of background images not containing the target object. The second sample set contains a universal feature with a high false detection frequency obtained empirically and commonly existing in different scenes, and the image thereof is registered as the second sample set. If the second sample set is not registered, the number of features of the image not containing the target is set to 0.

[0087] In an embodiment, the second sample image not containing the target is attributed to the second sample set. The second sample image containing the non-target is also attributed to the second sample set.

[0088] Please refer to Figure 6 , Figure 6 is Figure 2 A flowchart of a specific embodiment of step S203 in the target detection method provided.

[0089] S2031: Obtain a background video frame of a preset region.

[0090] Specifically, an image acquisition device is installed, and the background video frame of the preset region is acquired by the image acquisition device. The background video frame is an image not containing the preset target in the preset region. That is, the background video frame is a background image not containing a foreground image.

[0091] S2032: Perform data enhancement processing on the background video frame to obtain a plurality of enhanced images.

[0092] Specifically, in order to enrich the second sample set and avoid mistaking the target in the background video frame as the target object, the background video frame can be subjected to data enhancement processing. For example, the obtained background video frame is respectively rotated by 90°, 180° and 270° to obtain a plurality of enhanced images corresponding to the background video frame. The background video frame image can also be stretched and compressed to obtain corresponding enhanced images.

[0093] S2033: Perform target detection on the enhanced images using a target detection network model to obtain initial targets corresponding to the background video frame.

[0094] Specifically, the target detection network model trained in step S201 is used to perform target detection on the background video frame and the enhanced images respectively to obtain initial targets contained in the background video frame. The initial target is a non-target object.

[0095] S2034: Extract a region image containing each initial target and attribute the region image to the second sample set.

[0096] Specifically, in order to avoid identifying the initial target in the background video frame as the target object when identifying the target object, the region image of the initial target identified in the background video frame is attributed to the second sample set, thereby improving the detection accuracy of the target object.

[0097] S204: Obtain a to-be-detected video frame of a preset region.

[0098] Specifically, the image acquisition device continues to acquire images as the to-be-detected video frame. The acquisition time of the to-be-detected video frame is later than the acquisition time of the background video frame. The to-be-detected video frame contains at least one target object. The target object can be a pedestrian.

[0099] S205: Perform target detection on the to-be-detected video frame by using a target detection network model to obtain candidate targets contained in the to-be-detected video frame.

[0100] Specifically, the target detection network model trained in step S201 is used to perform target detection on the to-be-detected video frame to obtain information of each candidate target contained in the to-be-detected video frame. The information of each candidate target includes a target bounding box and a target confidence.

[0101] In an embodiment, in order to reduce the workload, the target confidence of each candidate target detected is compared with a preset confidence, and the candidate target with a target confidence greater than the preset confidence is retained, and the candidate target with a target confidence less than the preset confidence is deleted. The preset confidence can be set to 0.35, and if the time consumption is allowed, the preset confidence can be set to a smaller value.

[0102] S206: Perform feature extraction on the candidate target by using a feature extraction network model to obtain feature information of the candidate target.

[0103] Specifically, the region image containing each candidate target is extracted, and the region image of the retained candidate target is subjected to feature extraction by using the feature extraction network trained in step S202 to obtain feature information of each candidate target. Specifically, the feature information is a feature map of the candidate target.

[0104] S207: Calculate a first similarity between the feature information of the candidate target and a first sample set containing the target object.

[0105] Specifically, the images containing the target object are clustered to obtain the first sample set. The first sample set contains multiple sub-image sets of different categories. Each sub-image set includes multiple images of the same category of the target object. For example, the first sample set includes three sub-image sets corresponding to the head-shoulder category, the half-body category, and the full-body category, respectively.

[0106] The second similarity corresponding to each candidate target and each sub-image set in the first sample set is calculated.

[0107] In a specific embodiment, the similarity corresponding to the feature information of the candidate target and the feature information of each image in the sub-image set is calculated, and the similarities corresponding to each image are added to obtain the second similarity corresponding to the candidate target and the sub-image set.

[0108] Specifically, the cosine distance corresponding to the feature information of the candidate target and the feature vector corresponding to each image in the sub-image set is calculated based on formula 1 using a metric learning method, and then the similarity between the candidate target and the feature vector of the image is determined.

[0109] All sub-image sets are traversed, and the second similarity corresponding to each candidate target and each sub-image set is calculated.

[0110] In an embodiment, based on multi-attribute decision, the second similarity corresponding to each sub-image set in the first sample set is weighted and averaged to obtain the first similarity corresponding to the candidate target and the first sample set.

[0111] S208: Calculate the third similarity between the feature information of the candidate target and the second sample set not containing the target object.

[0112] Specifically, the similarity between the feature information of the candidate target and the feature information corresponding to each image in the second sample set is calculated, and the similarities corresponding to each image are added and averaged to obtain the third similarity corresponding to the candidate target and the second sample set.

[0113] In a specific embodiment, the cosine distance corresponding to the feature information of the candidate target and the feature vector corresponding to each image in the second sample set is calculated based on formula 1 using a metric learning method, and then the similarity between the candidate target and the feature vector of the image is determined.

[0114] S209: Determine whether the first similarity is greater than the third similarity.

[0115] Specifically, in order to determine whether the candidate target is the target object, the first similarity corresponding to the candidate target and the first sample set is compared with the third similarity corresponding to the candidate target and the second sample set.

[0116] In a specific embodiment, the judgment can be made based on the voting formula shown in formula 2.

[0117]

[0118] In formula 2, M represents the output voting result; P represents the first sample set; N represents the second sample set; k represents the weight value of each category, k = 1 / (x+y); x represents the number of sub-image sets contained in the first sample set; and y represents the number of sub-image sets contained in the second sample set.

[0119] In another specific embodiment, when the feature information of the image contained in the second sample set is 0, the first similarity between the first sample set and the candidate target is compared with a preset value. Specifically, as shown in formula 3, the preset value can be 0.2.

[0120]

[0121] In formula 3, K represents the output voting result; P represents the first sample set; k represents the weight value of each category; and x represents the number of sub-image sets contained in the first sample set.

[0122] If the first similarity is greater than the third similarity or the preset value, the process directly jumps to step S210; if the first similarity is less than the third similarity or the preset value, the process directly jumps to step S212.

[0123] S210: determining that the candidate target is the target object.

[0124] Specifically, if the first similarity is greater than the third similarity or the preset value, it is determined that the candidate target in the to-be-detected video frame is the target object.

[0125] S211: weighting and resetting the confidence of the target object.

[0126] Specifically, in order to avoid missing detection of the target object in the to-be-detected video frame, the confidence of the target object in the to-be-detected video frame is weighted and reset. For example, the confidence of the target object is reset to 1.5.

[0127] S212: deleting the candidate target.

[0128] Specifically, if the first similarity is less than the third similarity or the preset value, it is determined that the candidate target in the to-be-detected video frame is not the target object, and the candidate target is deleted.

[0129] S213: and the candidate target is attributed to the second sample set as a non-target object.

[0130] Specifically, in order to enrich the second sample set, the candidate target can be attributed to the second sample set. Further, the second sample set is adaptively expanded, and the filtering of the misdetected target object by the target detection network model is enhanced.

[0131] The embodiment provides a target detection method, which comprises the following steps: acquiring a to-be-detected video frame of a preset region; performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; performing feature extraction on the candidate target to obtain feature information of the candidate target; and determining whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object. The to-be-detected video frame is subjected to target detection to obtain a candidate target, and the feature information of the candidate target is compared with the first sample set containing the target object to determine whether the candidate target is the target object, so that the phenomenon of false detection of the target object is avoided, and the detection accuracy and the generalization performance of the detection method are improved.

[0132] Referring to Figure 7 , Figure 7 is a schematic block diagram of a target detection device provided by the application. The embodiment provides a target detection device 60, which comprises an acquisition module 61, a detection module 62, a feature extraction module 63 and an analysis module 64.

[0133] The acquisition module 61 is configured to acquire a to-be-detected video frame of a preset region.

[0134] The detection module 62 is configured to perform target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame.

[0135] Specifically, the detection module 62 is configured to determine whether the confidence of the candidate target exceeds a preset confidence.

[0136] The feature extraction module 63 is configured to perform feature extraction on the candidate target to obtain feature information of the candidate target.

[0137] Specifically, if the confidence of the candidate target exceeds the preset confidence, the feature extraction module 63 is configured to perform feature detection on the candidate target to obtain the feature information of the candidate target.

[0138] The analysis module 64 is configured to determine whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object.

[0139] In an embodiment, the analysis module 64 is configured to determine that the candidate target is the target object in response to the first similarity between the feature information of the candidate target and feature information of a target object contained in the first sample set being greater than a preset value.

[0140] In an embodiment, the first sample set includes at least two sub-image sets of different categories; each sub-image set includes a plurality of images containing the same part of the target object. The analysis module 64 is configured to calculate a second similarity between each sub-image set and the candidate target based on a respective similarity between the feature vector of the candidate target and feature information corresponding to the images contained in each sub-image set; and determine the first similarity between the first sample set and the target object based on a weighted sum of the respective second similarities of each sub-image set.

[0141] In an embodiment, the acquisition module 61 is configured to construct a second sample set including a plurality of background images not containing the target object.

[0142] The acquisition module 61 is further configured to acquire a background video frame of a preset region, the acquisition time of the background video frame being earlier than the acquisition time of the to-be-detected video frame; and the acquisition module 61 is further configured to perform data enhancement processing on the background video frame to obtain a plurality of enhanced images.

[0143] The detection module 62 is configured to perform target detection on the background video frame to obtain an initial target corresponding to the background video frame. The detection module 62 is configured to perform target detection on each of the plurality of enhanced images to obtain an initial target corresponding to each enhanced image.

[0144] The feature extraction module 63 is configured to extract a region image containing each initial target and attribute the region image to the second sample set.

[0145] In an embodiment, the analysis module 64 is configured to calculate a third similarity between the feature information of the candidate target and the second sample set; and in response to the first similarity being greater than the third similarity, determine that the candidate target is the target object. The analysis module 64 is further configured to perform weighted resetting on the confidence of the target object.

[0146] In an embodiment, the analysis module 64 is configured to, in response to the first similarity being less than the third similarity, delete the candidate target and attribute the candidate target to the second sample set as a non-target object.

[0147] The detection module 62 is configured to perform target detection on the to-be-detected video frame using a target detection network model to obtain a candidate target contained in the to-be-detected video frame.

[0148] The feature extraction module 63 is configured to perform feature detection on the candidate target based on a feature extraction network model to obtain feature information of the candidate target.

[0149] The target detection device provided in the embodiment performs target detection on a video frame to be detected to obtain a candidate target, and determines whether the candidate target is a target object by comparing feature information of the candidate target with a first sample set containing the target object, so that the phenomenon of false detection of the target object is avoided, and the detection accuracy and the generalization performance of the detection method are improved.

[0150] Please refer to Figure 8 , Figure 8 is a schematic block diagram of an embodiment of the terminal provided by the present application. The terminal 80 comprises a memory 81 and a processor 82 coupled with each other. The processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps of any of the target detection method embodiments described above. In a specific implementation scenario, the terminal 80 can include but is not limited to a microcomputer, a server, and in addition, the terminal 80 can also include a notebook computer, a tablet computer and other mobile devices, which are not limited here.

[0151] Specifically, the processor 82 is configured to control itself and the memory 81 to implement the steps of any of the target detection method embodiments described above. The processor 82 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 82 can be an integrated circuit chip with signal processing capability. The processor 82 can also be a general-purpose processor, a DSP (Digital Signal Processor, digital signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field-Programmable Gate Array, field programmable gate array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 82 can be implemented by an integrated circuit chip together.

[0152] Please refer to Figure 9 , Figure 9 is a schematic block diagram of an embodiment of the computer readable storage medium provided by the present application. The computer readable storage medium 90 stores program instructions 901 capable of being executed by a processor, and the program instructions 901 are used to implement the steps of any of the target detection method embodiments described above.

[0153] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0154] The above description of the various embodiments tends to emphasize differences between the various embodiments, and the same or similar elements can be referred to each other for brevity, and will not be repeated here.

[0155] In several embodiments provided in the present application, it should be understood that the disclosed method and device can be implemented in other ways. For example, the above-described device implementation is only schematic; for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a unit or component can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed elements can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0156] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0157] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0158] If the technical solution of the present application involves personal information, the product applying the technical solution of the present application has been explicitly informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solution of the present application involves sensitive personal information, the product applying the technical solution of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent sign is set to inform that it has entered the personal information collection range and will collect personal information. If the individual voluntarily enters the collection range, it is considered to agree to collect personal information. Or, on the device for processing personal information, through the pop-up information or by asking the individual to upload his / her personal information, the individual's authorization is obtained under the condition of using obvious signs / information to inform the individual of the personal information processing rules. The personal information processing rules can include personal information processor, personal information processing purpose, processing method and personal information type, etc.

[0159] The above is only an embodiment of the present application, and does not limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation using the content of the present application specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A target detection method characterized by, The target detection method comprises: obtaining a to-be-detected video frame of a preset region; performing target detection on the to-be-detected video frame to obtain a candidate target contained in the to-be-detected video frame; performing feature extraction on the candidate target to obtain feature information of the candidate target; determining whether the candidate target is a target object based on a first similarity between the feature information of the candidate target and a first sample set containing the target object; the first sample set comprises at least two sub-image sets of different categories; each of the sub-image sets comprises a plurality of images containing the same part of the target object; the determination of whether the candidate target is the target object based on the first similarity between the feature information of the candidate target and the first sample set containing the target object comprises: calculating a second similarity between each of the sub-image sets and the candidate target based on a respective similarity between a feature vector of the candidate target and feature information corresponding to the images contained in each of the sub-image sets; determining the first similarity between the first sample set and the target object based on an average value of a weighted sum of the respective second similarities of each of the sub-image sets; the obtaining of the to-be-detected video frame of the preset region further comprises: constructing a second sample set, the second sample set comprising a plurality of background images not containing the target object; the determination of whether the candidate target is the target object based on the first similarity between the feature information of the candidate target and the first sample set containing the target object further comprises: calculating a third similarity between the feature information of the candidate target and the second sample set; in response to the first similarity being greater than the third similarity, determining that the candidate target is the target object; in response to the first similarity being less than the third similarity, deleting the candidate target and attributing the candidate target to the second sample set as a non-target object.

2. The target detection method of claim 1, wherein: in response to the first similarity being greater than the third similarity, the determination that the candidate target is the target object further comprises: weighting and resetting a confidence level of the target object.

3. The target detection method of claim 1, wherein: the feature extraction on the candidate target to obtain the feature information of the candidate target further comprises: determining whether a confidence level of the candidate target exceeds a preset confidence level; the feature extraction on the candidate target to obtain the feature information of the candidate target comprises: if the confidence level of the candidate target exceeds the preset confidence level, performing feature detection on the candidate target to obtain the feature information of the candidate target.

4. The target detection method of claim 1, wherein: the obtaining of the to-be-detected video frame of the preset region further comprises: obtaining a background video frame of the preset region, the collection time of the background video frame being earlier than the collection time of the to-be-detected video frame. perform target detection on the background video frame to obtain initial targets corresponding to the background video frame; extract a region image containing each of the initial targets and attribute the region image to a second sample set.

5. The target detection method of claim 4, wherein, after the background video frame of the preset region is obtained, the method further comprises: performing data enhancement processing on the background video frame to obtain a plurality of enhanced images; performing target detection on the plurality of enhanced images to obtain the initial targets corresponding to the enhanced images.

6. The target detection method of claim 1, wherein, the performing target detection on the to-be-detected video frame to obtain candidate targets contained in the to-be-detected video frame comprises: performing target detection on the to-be-detected video frame using a target detection network model to obtain the candidate targets contained in the to-be-detected video frame; wherein, the training method of the target detection network model comprises: obtaining a first training data set, the first training data set comprising a plurality of first sample images containing targets and a plurality of first sample images not containing the targets; the first sample images are labeled with corresponding real target boxes and real categories, and the real target box of the first sample image not containing the target is a null value; inputting each of the first sample images into the target detection network model to obtain a corresponding predicted target box and a predicted category for each of the first sample images; iteratively training the target detection network model based on error values between the real category and the predicted category corresponding to the same first sample image, and error values between the real target box and the predicted target box.

7. The target detection method of claim 1, wherein, the performing feature extraction on the candidate targets to obtain feature information of the candidate targets comprises: performing feature detection on the candidate targets based on a feature extraction network model to obtain feature information of the candidate targets; wherein, the training method of the feature extraction network model comprises: obtaining a second training data set, the second training data set comprising a plurality of second sample images containing targets and a plurality of second sample images not containing the targets; the second sample images are labeled with real feature vectors; inputting each of the second sample images into the feature extraction network model to obtain a corresponding predicted feature vector for each of the second sample images; iteratively training the feature extraction network model based on error values between the real feature vector and the predicted feature vector corresponding to the same second sample image. The target detection device comprises: an acquisition module configured to acquire a to-be-detected video frame of a preset region; a detection module configured to perform target detection on the to-be-detected video frame to obtain candidate targets contained in the to-be-detected video frame; a feature extraction module configured to perform feature extraction on the candidate targets to obtain feature information of the candidate targets; 8. A target detection apparatus characterized by comprising: ​ ​ ​ ​ The analysis module is configured to determine whether the candidate target is the target object based on a first similarity between feature information of the candidate target and a first sample set containing the target object. The first sample set includes at least two sub-image sets of different categories, and each sub-image set includes a plurality of images containing the same part of the target object. The analysis module is further configured to calculate a second similarity between each sub-image set and the candidate target based on a respective similarity between a feature vector of the candidate target and feature information corresponding to the images contained in each sub-image set, and determine the first similarity between the first sample set and the target object based on an average value of a weighted sum of the respective second similarities of each sub-image set. The acquisition module is further configured to construct a second sample set, and the second sample set includes a plurality of background images not containing the target object. The analysis module is further configured to calculate a third similarity between the feature information of the candidate target and the second sample set, determine that the candidate target is the target object in response to the first similarity being greater than the third similarity, and delete the candidate target and attribute the candidate target as a non-target object to the second sample set in response to the first similarity being less than the third similarity.

9. A terminal, characterized by comprising: The terminal includes a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor is configured to execute program data to implement the steps of the target detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the target detection method according to any one of claims 1 to 7.

Images