Target object detection method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a target object detection method and device, electronic equipment and a storage medium, which can be applied to the fields of artificial intelligence, target object tracking, target object detection, image processing and the like, and the method comprises: detecting a target object in an image according to a target template set to obtain a detection result of the target object in the image. By using the embodiments of the present application, the accuracy of target tracking and detection is improved.

Description

Technical Field

[0001] This application relates to fields such as artificial intelligence, target object tracking, target object detection, and image processing, and in particular to target object detection methods, devices, electronic devices, and storage media. Background Technology

[0002] Visual object tracking (VOT) is an important research technique in computer vision. It is commonly used to track moving objects such as people, animals, airplanes, and cars.

[0003] During target object tracking, the object to be tracked is marked in the first frame of the video (also known as the initial frame). The target tracking algorithm needs to continuously track this object in subsequent frames of the video, providing its positional information. The core idea of ​​target tracking technology is to extract template information based on the target marked in the first frame, calculate the matching degree between different candidate positions and the template in the search region of subsequent video frames, and select the position with the highest matching degree to determine the target position. Depending on the scene, common challenges in target tracking technology include, but are not limited to, the following: scale transformation, background interference, exceeding the field of view, illumination changes, in-plane and out-of-plane rotation, deformation, low resolution, fast motion, motion blur, occlusion, etc. These issues cause discrepancies between the tracked and detected target objects, thus affecting the accuracy of the target tracking and detection algorithms.

[0004] Therefore, improving the accuracy of target tracking and detection has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a target object detection method, apparatus, electronic device, and storage medium, which can improve the accuracy of target tracking and detection and has high applicability.

[0006] On one hand, embodiments of this application provide a target object detection method, the method comprising: Based on the target template set, the target objects in the image are detected, and the detection results of the target objects in the above image are obtained.

[0007] On one hand, embodiments of this application provide a target object detection device for tracking target objects in a video. For any frame of the video, the device includes: The target object processing module is used to detect target objects in the image based on the target template set, and obtain the detection results of the target objects in the image.

[0008] On one hand, embodiments of this application provide an electronic device, including a processor and a memory, which are interconnected; The aforementioned memory is used to store computer programs; The processor is configured to execute the method provided by any optional implementation of the target object detection method when the computer program is invoked.

[0009] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program that is executed by a processor to implement the method provided in any possible implementation of the target object detection method described above.

[0010] On one hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in any possible implementation of the target object detection method described above.

[0011] The beneficial effects of the solution provided in this application are as follows: The target object detection method, apparatus, electronic device, and storage medium provided in this application detect target objects by using a target template set to detect target objects in an image, thereby obtaining the detection result of the target objects in the image. This method of using a target template set, since the target template set contains one or more target templates, makes the information contained in the target template set richer, avoiding the use of only the first frame image for target object detection. In other words, the target template set records multiple feature information that the target object may appear, which can more comprehensively describe the information of the target object, thereby improving the accuracy of target object detection. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the network structure of a target tracking method provided in an embodiment of this application; Figure 2This is a schematic diagram of the network structure of another target tracking method provided in the embodiments of this application; Figure 3 This is a flowchart illustrating a method for tracking a target object provided in an embodiment of this application; Figure 4 This is a schematic diagram illustrating the principle of memory matching provided in an embodiment of this application; Figure 5 This is a schematic diagram of the network structure of a lightweight target object detection method provided in an embodiment of this application; Figure 6 This is a schematic diagram of the network structure of another lightweight target object detection method provided in the embodiments of this application; Figure 7 This is a schematic diagram of a memory update process provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a target object detection device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0016] When tracking a target object, the feature information of the target object will change to varying degrees due to changes in lighting, occlusion, deformation, etc., which in turn affects the accuracy of target tracking.

[0017] To address the aforementioned problems, this application provides a target object detection method, aiming to solve at least one of the problems mentioned above. This method can be executed by any electronic device, including a terminal device or a server. The method includes: Step S101: Detect the target objects in the image based on the target template set to obtain the detection results of the target objects in the image.

[0018] Optionally, the above method can be used for tracking and detecting target objects in videos, or for detecting target objects in any frame of an image, without any limitations.

[0019] This application does not limit the source of the video. For example, it can be a video taken by a visible light camera, such as surveillance footage, live sports event video, etc.

[0020] The target object detection method in this application embodiment can be applied to visual object tracking, where visual object tracking (VOT) technology is an important research task in computer vision and has many practical applications in our daily lives. For example, it can be used to find target objects (such as criminal suspects, lost children, elderly people, animals, etc.) in surveillance videos, to locate target athletes in live sports broadcasts, to determine the trajectory of vehicles in surveillance videos, and to track small aircraft in flight. Furthermore, on smartphones, the target object detection method in this application embodiment can be used to assist camera focusing, video effects generation, and moving object analysis. The target object detection method shown in this application can be applied to specific scenarios according to actual needs, and no limitations are imposed here.

[0021] When applied to video tracking and detection scenarios, the aforementioned target template set includes the initial target template of the aforementioned target object, as well as the target template corresponding to at least one frame of the preceding image in the aforementioned video. The target template corresponding to any preceding image is the image region where the target object is located in that image.

[0022] The target object mentioned above can be understood as the object to be tracked or detected, such as a person, animal, or vehicle (such as a car or airplane) in motion. The target object to be tracked or detected can be selected according to the actual application requirements, and there are no restrictions here. Optionally, the object to be tracked in the video (i.e., the target object mentioned above) can be determined by user annotation and regarded as a real value, or it can be determined by an image of a known target object. For example, the user can annotate the video frame in which the target image first appears in the video, and the annotated area can be used as the initial target template of the target object. The initial target template can also be a known image of the target object.

[0023] The aforementioned target template set can be understood as a kind of target memory, and this target memory is stored. This target memory includes the image region corresponding to the detection result of the target object in each detected frame of the video (i.e., the region where the target object is located in each frame). Of course, in practical applications, the target template set can also store the feature information of the image region. That is to say, the target template set can be the image region where the target object is located in each frame (i.e., the image region where the target object is located in each frame is used as the target template), or it can be the image feature information of the image region, with the image feature information representing the image region (i.e., the features of the image region where the target object appears in each frame are used as the target template).

[0024] Specifically, the target template set may include an initial target template, which can be determined based on the first frame image where the target object first appears in the video. For example, the target object to be tracked can be marked in the first frame image to obtain the annotation information of the target object to be tracked, and then the initial target template can be obtained based on the annotation information. Alternatively, the initial target template can also be obtained based on an image of a known target object. Similarly, the target image region corresponding to the marking information in the first frame image can be used as the initial target template, or the feature information of the target image region can be used as the initial target template.

[0025] For any frame of the image to be detected, the target template set may also include target templates corresponding to other detected frames of images besides the initial target template, that is, the target template corresponding to at least one frame of the preceding images of the image. For example, when the image is the second frame, the preceding image is the first frame, and the target template is the initial target template. When the image is the third frame, the preceding image includes at least one of the first and second frames, and the target template can be the initial target template, as well as the image region (i.e., the target region) where the detection result of the preceding image (i.e., the second frame) of the third frame of the target object is located, and so on.

[0026] Optionally, for any frame image, all target templates corresponding to all preceding images can be added to the target template set, or target templates that meet certain conditions among all target templates corresponding to preceding images can be added to the set. The specific conditions can be configured according to actual needs. For example, for any preceding image, whether to add the target template corresponding to the preceding image to the set can be determined based on the matching degree between the target template corresponding to the preceding image and each target template already stored in the target template set. Optionally, if all the above matching degrees are less than the set value, the target template corresponding to the preceding image can be added to the target template set. Using this scheme can avoid the appearance of target templates with excessively high similarity in the target template set, which can effectively reduce the amount of computation for subsequent data.

[0027] It is understandable that for different frame images, the set of target templates used to detect the target object in the frame image may be different, and the number of templates in the set of target templates used in each frame image may be the same or different.

[0028] When tracking and detecting target objects in a video, for any frame of an image, the target object can be detected by the above target template set to obtain the detection result of the target object. The detection result can be the position information of the target object in the image and the confidence level of the position information. The confidence level is used to characterize the credibility and reliability of the detection result, and the value range is [0,1].

[0029] In the embodiments of this application, when detecting target objects in an image, the detection results are obtained by detecting target objects in the image through a target template set. This method, which employs a memory mechanism, contains not only the initial target template but also the target template corresponding to the image region where the target object is located in the preceding image. This makes the information contained in the target template set richer. In other words, due to the diversity of the target template set, the information that can be relied upon in the detection of target objects is richer, thereby improving the accuracy of the detection results.

[0030] In one optional embodiment, detecting target objects in an image based on a set of target templates includes: Based on the target template set and interference template set mentioned above, the target objects in the above images are detected.

[0031] The set of interference templates includes each interference template of the target object, and each interference template is the image region where the interference object of the target object is located in at least one frame of the preceding image in the video.

[0032] Optionally, the aforementioned interfering object can also be called an interfering element, that is, an object that may interfere with the detection of the target object. The interfering object can be an object, texture (non-object), or shape in a frame image of the video that is (visually) similar to the tracked target object. In other words, it can also be understood as an object that may cause interference.

[0033] The aforementioned set of interference templates can be understood as a kind of interference memory, and this interference memory is stored. This interference memory includes image regions other than the image region corresponding to the detection result of the target object in each detected frame of the video. In other words, the interference memory includes the image region where the interference object corresponding to the target object is located in each detected frame. Of course, in practical applications, the interference template set can also store feature information of the image region where the interference object is located. The interference template set includes at least one interference template, and each interference template can be the image region where the interference object is located, or it can be feature information of the image region where the interference object is located; no limitation is made here.

[0034] Specifically, the interference template set includes various interference templates for the target object, where each interference template represents the image region where the interference object of the target object is located in the preceding image of the image. For example, when the image is the third frame, the preceding images include the first frame and the second frame. For the first frame, if the target object is determined by labeling the first frame (i.e., the initial target template is the labeled image region in the first frame), optionally, the first frame may not be used for target object and interference object detection. Instead, target object tracking and detection can begin from the second frame. When detecting the target object in the second frame, multiple detection results will be obtained. Part or all of the image regions corresponding to the other detection results (excluding the image region corresponding to the final determined detection result) can be identified as the image region where the interference object of the target object is located. In other words, when detecting the third frame, the interference template set includes the interference template corresponding to the second frame. Alternatively, interference object detection can be performed on the first frame. Correspondingly, when detecting the third frame, the interference template set includes both the interference template corresponding to the second frame and the interference template corresponding to the first frame. As an alternative approach, the interference template corresponding to the first frame image can also be determined by annotation.

[0035] Optionally, when detecting target objects in an image, both the target template set and the interference template set can be considered simultaneously. This means that both target memory and interference memory are considered at the same time, making the detection result of the target object closer to the target memory and farther away from the interference memory.

[0036] In this embodiment, during the movement of a target object in a video, due to factors such as lighting changes, rotation, rapid movement, motion blur, occlusion, deformation, and background interference, the object's features will change to varying degrees compared to the information annotated in the first frame image. By memorizing these changes during movement using the aforementioned target template set (i.e., target memory), a more complete description of the target object can be achieved. Furthermore, the interference template set (i.e., interference memory) can memorize the image features of potential interfering objects, allowing the detection of the target object to avoid their influence. This method of combining target memory and interference memory for target object detection comprehensively considers the target object's own factors and avoids the influence of interfering objects, greatly improving the accuracy of target object tracking.

[0037] In one optional embodiment, the target object in the image is detected based on the target template set to obtain the detection result of the target object in the image, including: Based on the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Determine the target region from one or more of the above target object candidate regions; Based on the aforementioned target region, the detection results of the aforementioned target object corresponding to the image are obtained.

[0038] Optionally, the aforementioned one or more target object candidate regions can also be referred to as candidate detection boxes, anchor boxes, or anchor points, representing the various possible positions of the target object in the image. Anchor boxes, in object detection, represent fixed reference boxes, which are a pre-defined set of fixed reference boxes at different scales and positions, covering almost all positions and scales.

[0039] When detecting target objects in an image, one or more candidate regions for target objects can be obtained from the target template set (i.e., target memory). The specific process is as follows: When detecting target objects in an image, multiple search regions are searched to obtain multiple search bounding boxes. Each search bounding box can then be matched one-to-one with each target template in the image's corresponding target template set, or matched with a fused template resulting from the fusion of all target templates in the image's corresponding target template set. This yields one or more candidate regions for the target object with high matching scores. Finally, the final target region is selected from these candidate regions, and the detection result of the target object corresponding to the image is obtained based on this target region.

[0040] It is understood that the solutions provided in the various optional embodiments of this application can be implemented using pre-trained neural network models. As an optional example, Figure 1 This is a schematic diagram of the network structure of a neural network model for implementing a target tracking method provided in an embodiment of this application, as shown below. Figure 1 As shown in the figure, taking a video containing an athlete's movement scene as an example, the target object is an athlete (which can be called the target athlete). The current frame is any frame image that needs to be detected. The first frame template is the region where the target object is located in the first frame image of the video, i.e., the initial target template, which is usually manually labeled by the user and considered as the true value. The memory pool is a storage space used to store target memories. This memory pool stores each target template (i.e., the target memory shown in the figure). For the frame shown in the figure, the memory pool stores the initial target template and the target templates corresponding to each preceding image of this frame.

[0041] When detecting the target object in the current frame shown in the figure, the current frame can be input into the backbone network to obtain the image features corresponding to the current frame, namely the search region features W shown in the figure. H C (that is, the feature map, where W and H represent the width and height of the feature map respectively, and C represents the number of channels in the feature map, i.e., the number of feature maps).

[0042] In order to obtain multiple or one candidate regions of target objects, the memory of each target in the memory pool can be input into the backbone network to obtain the image features corresponding to each target template (i.e., the feature information described above). The image features of the image regions corresponding to each target template can be fused through the feature fusion network (i.e., the memory network shown in the figure) to obtain the fused features (which can be simply referred to as target fusion features). Based on the target fusion features and search region features, the correlation between the search region features and the target fusion features is calculated through a region candidate network, generating multiple anchor points (i.e., one or more target object candidate regions). The classification results (i.e., the classification (confidence) shown in the figure) and regression results (i.e., the regression position (x,y,w,h) shown in the figure) of these multiple anchor points are output. In (x,y,w,h), x and y represent the x and y coordinates of the center point of the target object candidate region (or any vertex of the target object candidate region, such as the top left vertex), and w and h represent the width and height of the target object candidate region. Based on the classification and regression results of these multiple anchor points, the final anchor point is selected from these multiple anchor points. Based on the classification and regression results of the selected anchor point, the target position of the target object in the current frame and the corresponding detection confidence (i.e., the tracking result output shown in the figure) are obtained. In addition, the target memory can be updated according to the tracking result corresponding to the current frame. For example, the tracking result is added to the target memory (i.e., the tracking result output and memory update shown in the figure) so that the updated target memory can be used for the detection of the target object in the next frame of the current frame.

[0043] In one optional embodiment, detecting target objects in an image based on a set of target templates to obtain detection results of target objects in the image includes: detecting target objects in the image based on the set of target templates to obtain one or more target object candidate regions; determining a target region from the one or more target object candidate regions based on the set of interference templates; and determining the detection results of the target object based on the target region.

[0044] Optionally, when detecting target objects in an image, one or more candidate regions for target objects can be obtained from the target template set (i.e., target memory) and the interference template set (i.e., interference memory). The specific process is as follows: The target template set (target memory) and interference template set (interference memory) in the memory pool are processed by the backbone network to extract image features. These features are then fused using the recall network to obtain a fused template. When detecting target objects in the image, multiple regions are searched to obtain multiple search bounding boxes. Each search bounding box is then matched against the corresponding fused template to obtain one or more candidate regions for the target object with high matching scores. Finally, the final target region is selected from these candidate regions, and the detection result for the target object in the image is obtained based on this target region.

[0045] Optionally, in the process of target tracking using a neural network model, in addition to using target memory, target memory and interference memory can also be maintained based on the detection results of the target object to make the tracking and detection of the target object more accurate. The specific steps are as follows: Step 1: Input the first frame of the video sequence; Step 2: The user performs target annotation. Based on the annotation information, the image features of the image region where the target object is located in the first frame image are extracted to obtain the initial target template, which is then recorded in the target memory; the interference memory is empty.

[0046] Step 3: Input the second frame of the video sequence, and based on the initial target template of the first frame, predict the target position of the target object in the second frame to obtain the prediction result; Step 4: Based on the prediction results, output tracking information such as the target location and prediction confidence of the target object in the second frame; Step 5: Determine whether the prediction result of the second frame meets the conditions for updating the memory; if it does, update the memory; otherwise, do not update. Step 6: Input the third frame. Based on the target memory and interfering memory at this time, use the recall network to fuse the characteristics of the target memory and interfering memory in the memory pool to obtain the fused memory. Step 7: Using the fused memory as a template, predict the target location and output the result; the same applies to subsequent frames.

[0047] For a clearer description of the above process, please refer to... Figure 2 The example shown, Figure 2 This is a schematic diagram of the network structure of another target tracking method provided in the embodiments of this application, such as... Figure 2As shown, taking a video containing an athlete's movement scene as an example, the target object is an athlete (which can be called the target athlete). As shown in the figure, the current frame is any frame image that needs to be detected. The area where the target object is located in the first frame image of the first template video is the initial target template. The memory pool is a storage space used to store target memories and interference memories. The memory pool stores each target template (i.e., the target memory shown in the figure) and each interference template (i.e., the interference memory shown in the figure). For the current frame shown in the figure, the memory pool stores the initial target template, the target templates corresponding to each preceding image of this frame, and the interference templates corresponding to the interference objects in the preceding images of the current frame.

[0048] For the current frame, the current frame image is input into the backbone network to obtain the search region feature W corresponding to the current frame image. H C.

[0049] To obtain multiple candidate regions for one or more target objects, the target memories and interference memories in the memory pool can be input into the backbone network. The image features of each target memory and interference memory are extracted to obtain the image features corresponding to each target template and each interference template. The image features corresponding to each target template can be fused through a feature fusion network (i.e., the recall network shown in the figure) to obtain the target feature kernel (i.e., the target fusion feature) corresponding to the target memory. The image features corresponding to each interference template can also be fused through a feature fusion network to obtain the fused interference features (which can be called interference feature kernel or interference fusion feature). To obtain several candidate anchor points (i.e., target object candidate regions), the correlation between the search region features and the target feature kernel can be calculated using a region candidate network based on the target feature kernel (i.e., target fusion features) and the search region features. This yields multiple anchor points, denoted as Top-K anchor points (i.e., one or more target object candidate regions). The classification result (i.e., the classification (confidence) shown in the figure) and regression result (i.e., the regression position (x, y, w, h) shown in the figure) of the Top-K anchor points are obtained. The corresponding feature vectors of the target object candidate regions are calculated with the interference feature kernels, and the anchor point with the lowest match to the interference memory is selected, denoted as the Bottom-1 anchor point. Based on the selected anchor point, the target position and confidence of the target object in the current frame are determined (i.e., the tracking result output shown in the figure). The target memory and interference memory can also be updated according to the tracking results. For example, the tracking results can be added to the target memory, and some or all of the image regions corresponding to the anchor points other than the finally selected anchor point can be added to the interference memory, i.e., the tracking result output and memory update shown in the figure.

[0050] In an optional embodiment, based on the aforementioned target template set, target objects in the aforementioned image are detected to obtain one or more target object candidate regions, including: Determine the target fusion features corresponding to the above target template set; Based on the above target fusion features, the target objects in the above images are detected to obtain one or more target object candidate regions; Based on the above set of interference templates, a target region is determined from one or more candidate target object regions, including: The interference fusion features of the above interference template set are determined, and the target region is determined from one or more target object candidate regions based on the above interference fusion features.

[0051] The following explanation Figure 2 The example shown illustrates the memory matching process; see [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a target object tracking method provided in an embodiment of this application. The specific steps are as follows: Step S1: Extract image features of the target memory and the interfering memory.

[0052] Step S2: Based on the extracted image features of the target memory and the interference memory, target template fusion is performed to fuse them into a target feature kernel (i.e., target fusion feature), and interference template fusion is performed to fuse them into an interference feature kernel (i.e., interference fusion feature).

[0053] Step S3: Calculate the correlation between the search region features, target feature kernel, and interference features corresponding to the current frame image to obtain the correlation calculation result.

[0054] Step S4: Based on the relevance calculation results, obtain one or more candidate regions for the target object.

[0055] Step S5: Determine the final detection result from the one or more candidate regions of the target object, that is, the target region of the target object in the image.

[0056] Step S6: Update the target memory and interference memory based on the obtained detection results.

[0057] Specifically, see Figure 4 , Figure 4 This is a schematic diagram illustrating the principle of memory matching provided in an embodiment of this application. For example... Figure 4 As shown, the memory pool stores target memory and interference memory. The target memory and interference memory extract image features through the backbone network, and then the memory network performs target template fusion to form a target feature kernel (w). h C), namely the target fusion feature, is fused with the interference template through the recall network to form the interference feature kernel (w). h C), namely, interference fusion features. The search region image of the current frame is processed by the backbone network to extract search region features, and correlation is calculated with the target feature kernel. Several candidate anchor points are generated through the region candidate network. The K anchor points with the highest matching degree obtained from the correlation calculation are selected and denoted as Top-K anchor points (that is, representing one or more target object candidate regions). The region alignment feature corresponding to this Top-K anchor point is correlated with the interference feature kernel (wherein, the region alignment feature is the region alignment feature shown in the figure, which is to determine the feature corresponding to several target object candidate regions in the search region features), and the matching information of this Top-K anchor point with the interference memory is obtained. The anchor point with the lowest matching with the interference memory is selected and denoted as Bottom-1 anchor point. The tracking result is output based on this Bottom-1 anchor point, including the target position of the target object (i.e., the target region) and the confidence of the target position, that is, the output tracking result (confidence and target position) shown in the figure.

[0058] It should be noted that, Figure 1 and Figure 2 The target template and interference template in the memory pool are not limited to the template corresponding to the first frame template shown in the figure, and can also be updated according to subsequent frames.

[0059] Understandable Figure 1 and Figure 2 The example shown is merely one illustration, and the network structure of the target object detection method in this application embodiment is not limited. Figure 1 and Figure 2 The difference between the two is that, Figure 1 When detecting target objects, only target memory is used. Figure 2 When detecting a target object, both target memory and interference memory are taken into account.

[0060] Through the embodiments of this application, target memory can be considered, or both target memory and interference memory can be considered simultaneously, to detect target objects. This method of using a memory mechanism can more comprehensively record various changes in the target object during the movement process, effectively improving the accuracy of detection.

[0061] In one alternative implementation, based on the aforementioned set of interference templates, a target region is determined from one or more candidate target object regions, including: For each candidate region of a target object, determine the first matching degree between the candidate region of the target object and each target template in the above target template set, and the second matching degree between the candidate region of the target object and each interference template in the above interference template set; Based on the first matching degree and the second matching degree mentioned above, the first target matching degree of the candidate region of the target object is determined; Based on the first target matching degree corresponding to each target object candidate region, the target region is determined from each target object candidate region.

[0062] Optionally, the matching degree between each candidate region of the target object and each target template can be obtained in the manner described above, and denoted as the first matching degree. For any candidate region of the target object, the matching degree between the candidate region and each interference template in each interference template set can also be calculated to obtain multiple matching degrees. The average (or median) of these multiple matching degrees is taken as the matching degree between the candidate region of the target object and each interference template, and the matching degree between each candidate region of the target object and each interference template can be obtained, denoted as the second matching degree. Subtracting each value in the second matching degree from the first matching degree one by one, the first target matching degree can be obtained. The candidate region of the target object corresponding to the maximum value in the first target matching degree is taken as the target region.

[0063] Through the embodiments of this application, the candidate region of the target object with a high matching degree with the target memory and a low matching degree with the interfering memory can be selected as the target region based on the matching degree. In this way, the currently tracked target object can get as close as possible to the target memory and as far away from the interfering memory as possible, thereby improving the accuracy of detection.

[0064] In an optional embodiment, based on the aforementioned target template set, target objects in the aforementioned image are detected to obtain one or more target object candidate regions, including: Based on at least one target template in the target template set, the image is used to detect the target object as described above, and one or more target object candidate regions are obtained.

[0065] Optionally, the aforementioned target template set contains at least one target template. When detecting a target object, one or more target templates can be used. Alternatively, a fused template of all target templates in the target template set can be used for detection, and so on; no further limitations are specified here.

[0066] In one embodiment, the detection result of the target object can be obtained using an initial target template plus target memory (or target memory and interference memory). Specifically, the initial target template in the target template set can be matched one-to-one with multiple search region boxes corresponding to the image to obtain one or more target object candidate regions. Then, based on the target memory and interference memory, or based on the target memory, the final target region corresponding to the target object is selected from the one or more target object candidate regions, and the detection result of the target object is obtained based on the target region.

[0067] In one embodiment, the detection result of the target object can be obtained using each target template (i.e., target memory) in the target template set. Specifically, each target template in the target template set can be matched with multiple search region boxes corresponding to the image to obtain one or more target object candidate regions. Then, the target region corresponding to the final target object is selected from the one or more target object candidate regions, and the detection result of the target object is obtained based on the target region.

[0068] In one embodiment, the detection result of the target object can be obtained using each target template (i.e., target memory) of the target template set and each interference template (i.e., interference memory) of the interference template set. Specifically, each target template in the target template set and each interference template in the interference template set can be matched with multiple search region boxes corresponding to the image to obtain one or more target object candidate regions. Then, the target region corresponding to the final target object is selected from the one or more target object candidate regions, and the detection result of the target object is obtained based on the target region.

[0069] The above is merely an example, and this embodiment is not intended to be limiting.

[0070] Through the embodiments of this application, one or more candidate regions of a target object can be obtained in a variety of ways, which improves the diversity and flexibility of determining one or more candidate regions of a target object.

[0071] In one optional embodiment, the target object in the image is detected based on the target template set to obtain the detection result of the target object in the image, including: Based on the initial target template in the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Based on the above target template set and the above interference template set, a target region is determined from one or more target object candidate regions. Based on the aforementioned target area, the detection results for the aforementioned target objects are determined.

[0072] In one embodiment of this application, an initial target template from the target template set can be used to detect target objects in an image, obtain one or more target object candidate regions, and then target memory + interference memory can be used to obtain the final target object detection result.

[0073] In one optional embodiment, determining a target region from the one or more target object candidate regions based on the target template set and the interference template set includes: For each candidate region of a target object, determine the third matching degree between the candidate region of the target object and each target template in the target template set, and the fourth matching degree between the candidate region of the target object and each interference template in the interference template set; Based on the third matching degree and the fourth matching degree, the second target matching degree of the candidate region of the target object is determined; Based on the second target matching degree corresponding to each target object candidate region, the target region is determined from each target object candidate region.

[0074] Optionally, the matching degree between each candidate region of the target object and each target template can be obtained in the manner described above, denoted as the third matching degree. For any candidate region of the target object, the matching degree between the candidate region and each interference template in each interference template set can also be calculated to obtain multiple matching degrees. The average (or median) of these multiple matching degrees is taken as the matching degree between the candidate region of the target object and each interference template, which is denoted as the fourth matching degree. Subtracting each value in the third matching degree from each value in the fourth matching degree one by one, the second target matching degree can be obtained. The candidate region of the target object corresponding to the maximum value in the second target matching degree is taken as the target region.

[0075] See Figure 5 , Figure 5 This is a schematic diagram of the network structure of a lightweight target object detection method provided in an embodiment of this application, as shown below. Figure 5 As shown, the memory pool retains both target memory and interference memory. Template images labeled from the first frame are processed through a backbone network and adjustment layers to extract template features (i.e., the initial target template). The current frame image is processed through the backbone network and adjustment layers to extract search region features. A region candidate network is used to calculate the correlation between template features (i.e., the initial target template) and search region features (optionally, this correlation can be calculated using deep correlation, as shown in the figure), generating multiple candidate anchor points. The K anchor points with the highest matching degree are selected and denoted as Top-K anchor points (i.e., one or more target object candidate regions). From the target memory and interference memory, the retained calculated region alignment features (i.e., the region alignment features shown in the figure) are extracted. ... ), calculate any anchor point in the Top-K anchor points (i.e., the following The matching degree of ), where, for ... The formula for the k-th anchor point in the array is as follows:

[0076] in, Represents the i-th target template in target memory. eigenvectors, Represents the i-th interference template in the interference memory. eigenvectors; The feature vector representing the k-th anchor point in the Top-K anchor points; Representative calculation and Relevance matching value; Represents the i-th target template in target memory. The weights; Represents the i-th interference template in the interference memory. The weights are calculated using a formula. The final anchor point is selected based on this formula, representing the candidate anchor point that is most similar to the current target memory and least similar to the interfering memory. This means the target region is selected as the detection result for that target object, as shown in the diagram: final anchor point = argmax. k (score( Output the top-1 anchor point based on the result. Let m1 be the number of target templates in the target memory and m2 be the number of interference templates in the interference memory. It is understood that the number of target templates and interference templates may vary for different frame images; that is, the number of target templates and interference templates in different frame images may be the same or different. The specific method for determining the weight of each target template in the target memory is not limited in this embodiment. Optionally, the sum of the weights of each target template in the target memory is 1. The initial target template may have a higher weight value, while the weights of the remaining target templates may be lower, etc., which are not limited here. The specific method for determining the weight of each interference template in the interference memory is not limited in this embodiment. The sum of the weights of each interference template in the interference memory is 1. The weight of each interference template may be set based on the distance between the image region where each interference template is located and the image region where the target object is located, etc., which are not limited here.

[0077] In one optional embodiment, the target object in the image is detected based on the target template set to obtain the detection result of the target object in the image, including: Based on the initial target template in the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Based on the above set of target templates, a target region is determined from one or more candidate regions of target objects. Based on the aforementioned target area, the detection results for the aforementioned target objects are determined.

[0078] In one embodiment of this application, an initial target template in the target template set can be used to detect target objects in an image to obtain one or more target object candidate regions, and then the target memory (i.e., the target template set) can be used to obtain the final target object detection result.

[0079] See Figure 6 , Figure 6 This is a schematic diagram of the network structure of another lightweight target object detection method provided in the embodiments of this application, as shown below. Figure 6 As shown, when only the target memory is retained in the memory pool, the specific process can be found in [reference needed]. Figure 5 The implementation shown retrieves the retained calculated region alignment features (i.e., the following) from the target memory. The matching degree of any anchor point in the Top-K anchor points is calculated using the following formula:

[0080] Using the above formula, the final anchor point is selected as the anchor point most similar to the current target memory, that is, the target region is selected as the detection result of the target object, as shown in the figure: final anchor point = argmax. k (score( Output the top-1 anchor point based on the result.

[0081] It should be noted that the specific implementation method for detecting target objects in an image based on each target template in the target template set corresponding to the image, and obtaining one or more candidate regions for target objects, can be described as follows: Figure 5 and Figure 6 The initial target template is replaced with the fusion template of each target template in the target template set. For the specific implementation details of other parts, please refer to [reference needed]. Figure 5 and Figure 6 This will not be elaborated upon here.

[0082] Through the embodiments of this application, a lightweight implementation method can be adopted to obtain several target candidate regions through an initial target template or a fusion template of various target templates, and then select the final target region from these several target candidate regions, thereby reducing the amount of computation and improving efficiency.

[0083] In one optional embodiment, determining the target region from one or more of the target object candidate regions includes: Based on the matching degree between each candidate region of the target object and each target template in the target template set, the target region is determined from one or more of the above candidate regions of the target object; Based on the matching degree between each candidate region of the target object and each interference template in the corresponding interference template set of the image, the target region is determined from one or more of the above candidate regions of the target object.

[0084] Optionally, when determining the target region from one or more candidate target object regions, at least one of the following implementation methods may be included: In one optional implementation, for any candidate region of a target object, a matching degree can be calculated with each target template in the target template set to obtain multiple matching degrees. The average of these multiple matching degrees is taken as the matching degree between the candidate region of the target object and each target template. Finally, the candidate region of the target object with the highest matching degree among all the candidate regions of the target object is taken as the final target region.

[0085] In an optional embodiment, after obtaining the detection result of the target object in the above image, the method further includes: Determine the target template corresponding to the image; The target template set is updated based on the target template corresponding to the image.

[0086] Optionally, after obtaining the detection result of the target object in the image, the target template corresponding to the image can be obtained based on the detection result, and the target template set can be updated based on the target template. For example, if the similarity between the target template corresponding to the image and any template in the target template set is low, the target template can be added to the target template set, which is not limited here.

[0087] Specifically, in one optional embodiment, updating the target template set based on the target template corresponding to the image includes: Determine the target template corresponding to the image and the similarity between each target template in the target template set; If all of the above similarities are less than or equal to the set threshold, or if the fusion similarity of the above similarities is less than or equal to the set threshold, then the target template is added to the target template set.

[0088] Optionally, the similarity between the target template corresponding to the image and each target template in the target template set can be calculated and recorded as the fifth similarity. If each similarity in the fifth similarity is less than or equal to a set threshold (such as 0.9), or if the fusion similarity of each similarity in the fifth similarity (such as the average of each similarity in the fifth similarity) is less than or equal to the set threshold (such as 0.9), then the target template corresponding to the image can be added to the target template set as the target template set for the next frame image.

[0089] Through the embodiments of this application, a target template can be obtained based on the detection results of the image, and the target template set can be updated based on the target template, so that the target template set can record the latest features of the target object and improve the integrity of the target template set.

[0090] In an optional embodiment, after obtaining the detection result of the target object in the above image, the method further includes: Identify the interference template corresponding to the image; The set of interference templates is updated based on the interference templates corresponding to the image.

[0091] Optionally, after obtaining the detection result of the target object in the image, the corresponding interference template can be determined based on the detection result, and the interference template set can be updated. For example, the interference template can be determined by taking some or all of the image regions corresponding to the target object candidate regions other than the target region in one or more target object candidate regions as interference templates, and adding the interference template to the interference template set.

[0092] Specifically, in one optional embodiment, determining the interference template corresponding to the image includes: Obtain the target region in the image where the target object is located, and the image regions where each candidate interference object of the target object is located; For any image region containing a candidate interference object, determine the first correlation between the image region and the target region, and the second correlation between the image region and each interference template in the interference template set corresponding to the image. Based on the first correlation and the second correlation corresponding to the image region where each candidate interference object is located, the interference template corresponding to the image is determined from the image region where each candidate interference object is located.

[0093] Optionally, the target region where the target object is located in the image, and the image regions where each candidate interference object of the target object is located, can be obtained. The image regions where each candidate interference object is located can be image regions other than the target region within one or more of the aforementioned target object candidate regions, for example, the image regions corresponding to the Top-K anchor points other than the Bottom-1 anchor point.

[0094] For any image region containing a candidate interference object, a first correlation (characterizing the distance between the two regions) can be determined between the image region and the target region, and a second correlation (characterizing the similarity between the two regions) can be determined between the image region and each interference template in the interference template set. Then, based on the first and second correlations, the corresponding interference template for the image is determined from the image regions containing each candidate interference object. For example, an image region containing an interference object with a high first correlation and a low second correlation can be selected as the corresponding interference template.

[0095] The memory update process is illustrated below with an accompanying diagram. See also... Figure 7 , Figure 7 This is a schematic diagram of a memory update process provided in an embodiment of this application, such as... Figure 7 As shown, the final selected Bottom-1 anchor point is the final target position of the target object tracked in the current frame (i.e., the image region corresponding to this anchor point is the image region where the target object is located). Based on the target object's detection result, i.e., the target position and confidence level, it is determined whether the correlation between the target object's detection result and the existing target memory is less than a certain threshold, that is, whether the target object has undergone a certain change. If so, the target template corresponding to the target object's detection result is updated to the target memory; otherwise, it is not updated. Among the Top-K candidate anchor points, excluding the selected Bottom-1 anchor point as the target position, Bottom-K2 is selected as the candidate interference anchor point. It is determined whether the distance between these interference anchor points and the target position of the current target object is greater than a certain threshold. If so, this anchor point is a possible interference; otherwise, it is considered not an interference, and the relevant information of the anchor point can be discarded. Furthermore, it is determined whether the matching degree between the interference and some or all of the interference templates in the existing interference memory is less than a certain threshold, that is, whether a new interference has been generated. If so, this interference is updated to the interference memory; otherwise, it is not updated.

[0096] Through the embodiments of this application, the target template set and the interference template set can be maintained and updated according to the detection results of the target object, that is, the target memory and interference memory can be maintained and updated to keep the target memory and interference memory up-to-date, providing richer and more diverse information for subsequent target object detection and improving the accuracy of detection.

[0097] The benefits of the memory mechanism of this application are explained below. As shown in Table 1, the success rate and accuracy of tracking the target object are compared for video sequence 1, video sequence 2, and video sequence 3 after applying no memory mechanism, implementing target memory, and implementing both target memory and interference memory. It can be seen that the success rate and accuracy of tracking the target object are improved after implementing target memory, and the effect of implementing both target memory and interference memory is the best.

[0098] Table 1

[0099] See Figure 8 , Figure 8 This is a schematic diagram of a target object detection device provided in an embodiment of this application. The target object detection device provided in this embodiment is used for tracking target objects in a video. The target object detection device 1 includes: The target object processing module 11 is used to detect target objects in the image based on the target template set, and obtain the detection results of the target objects in the image.

[0100] In one feasible embodiment, the target object processing module described above is configured to: Based on the target template set and interference template set mentioned above, the target objects in the above images are detected.

[0101] In one feasible embodiment, the target object processing module described above is configured to: Based on the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Based on the above set of interference templates, the target region is determined from one or more candidate target object regions. Based on the aforementioned target area, the detection results for the aforementioned target objects are determined.

[0102] In one feasible embodiment, the target object processing module described above is configured to: Determine the target fusion features corresponding to the above target template set, and determine the interference fusion features of the above interference template set; Based on the above target fusion features, the target objects in the above images are detected to obtain one or more target object candidate regions; Based on the aforementioned interference fusion characteristics, the target region is determined from one or more candidate regions of the target object.

[0103] In one feasible embodiment, the target object processing module described above is configured to: For each candidate region of a target object, determine the first matching degree between the candidate region of the target object and each target template in the above target template set, and the second matching degree between the candidate region of the target object and each interference template in the above interference template set; Based on the first matching degree and the second matching degree mentioned above, the first target matching degree of the candidate region of the target object is determined; Based on the first target matching degree corresponding to each target object candidate region, the target region is determined from each target object candidate region.

[0104] In one feasible embodiment, the target object processing module described above is configured to: Based on the initial target template in the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Based on the above target template set and the above interference template set, a target region is determined from one or more target object candidate regions. Based on the aforementioned target area, the detection results for the aforementioned target objects are determined.

[0105] In one feasible embodiment, the target object processing module described above is configured to: Based on the initial target template in the above target template set, the target objects in the above image are detected to obtain one or more target object candidate regions; Based on the above set of target templates, a target region is determined from one or more candidate regions of target objects. Based on the aforementioned target area, the detection results for the aforementioned target objects are determined.

[0106] In one feasible embodiment, the above-described apparatus further includes a template updating module, which is used for: After obtaining the detection results of the target objects in the above images, the target template corresponding to the images is determined; The target template set is updated based on the target template corresponding to the image.

[0107] In one feasible embodiment, the template update module described above is used for: Determine the target template corresponding to the image and the similarity between each target template in the target template set; If all of the above similarities are less than or equal to the set threshold, or if the fusion similarity of the above similarities is less than or equal to the set threshold, then the target template is added to the target template set.

[0108] In one feasible embodiment, the template update module described above is used for: Identify the interference template corresponding to the image; The set of interference templates is updated based on the interference templates corresponding to the image.

[0109] In one feasible embodiment, the template update module described above is used for: Obtain the target region in the image where the target object is located, and the image regions where each candidate interference object of the target object is located; For any image region containing a candidate interference object, determine the first correlation between the image region and the target region, and the second correlation between the image region and each interference template in the interference template set corresponding to the image. Based on the first correlation and the second correlation corresponding to the image region where each candidate interference object is located, the interference template corresponding to the image is determined from the image region where each candidate interference object is located.

[0110] The apparatus provided in this application embodiment can implement at least one of multiple modules through an AI model. AI-related functions can be executed through non-volatile memory, volatile memory, and a processor.

[0111] The processor may include one or more processors. In this case, the one or more processors may be general-purpose processors, such as central processing unit (CPU), application processor (AP), etc., or pure graphics processing unit, such as graphics processing unit (GPU), visual processing unit (VPU), and / or AI-specific processors, such as neural processing unit (NPU).

[0112] The one or more processors control the processing of input data based on predefined operating rules or artificial intelligence (AI) models stored in non-volatile and volatile memory. These predefined operating rules or AI models are provided through training or learning.

[0113] Here, "providing through learning" refers to obtaining predefined operating rules or an AI model with desired characteristics by applying a learning algorithm to multiple learning datasets. This learning can be performed within the device itself, in which the AI ​​is executed according to the embodiment, and / or can be implemented via a separate server / system.

[0114] This AI model can contain multiple neural network layers. Each layer has multiple weight values, and the computation of a layer is performed using the computation results of the previous layer and the multiple weights of the current layer. Examples of neural networks include, but are not limited to, Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), Recurrent Neural Networks (RNNs), Restricted Boltzmann Machines (RBMs), Deep Belief Networks (DBNs), Bidirectional Recurrent Deep Neural Networks (BRDNNs), Generative Adversarial Networks (GANs), and Deep Q-Networks.

[0115] A learning algorithm is a method of training a predetermined target device (e.g., a robot) using multiple learning data sets to enable, allow, or control the target device to make determinations or predictions. Examples of such learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0116] This AI model can obtain the detection result of a target object in any frame of a video by using image data as input data. This AI model can be obtained through training. Here, "obtained through training" means obtaining a predefined operating rule or AI model configured to perform desired features (or purposes) by training a basic AI model with multiple training data using a training algorithm. The AI ​​model may include multiple neural network layers. Each of the multiple neural network layers includes multiple weight values, and neural network computation is performed by calculating the results of the previous layer and the multiple weight values.

[0117] The methods in this application embodiment can be applied to visual understanding, which is a technology for recognizing and processing things like human vision, and includes, for example, object recognition, object tracking, image retrieval, human recognition, scene recognition, 3D reconstruction / localization, or image enhancement.

[0118] In specific implementation, the AI ​​model can be the one described in the embodiments of this application. Figure 1-2 The neural network model shown in 4-7 is not limited here.

[0119] In specific implementation, the aforementioned device 1 can perform the above-described actions through its built-in functional modules. Figure 3 The implementation methods provided for each step are detailed in the above-mentioned implementation methods and will not be repeated here.

[0120] See Figure 9 , Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 9 As shown, the electronic device 1000 in this embodiment may include: a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the electronic device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 9 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0121] exist Figure 9 In the electronic device 1000 shown, the network interface 1004 provides network communication functions; the user interface 1003 is mainly used to provide an input interface for users; and the processor 1001 can be used to call the device control application stored in the memory 1005.

[0122] It should be understood that in some feasible implementations, the processor 1001 described above may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0123] In specific implementation, the aforementioned electronic device 1000 can perform the above-described actions through its built-in functional modules. Figure 1 The implementation methods provided for each step are detailed in the above-mentioned implementation methods and will not be repeated here.

[0124] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor to perform... Figure 3 The methods provided in each step are detailed in the implementation methods provided in the above steps, and will not be repeated here.

[0125] The aforementioned computer-readable storage medium can be an internal storage unit of the task processing apparatus provided in any of the foregoing embodiments, such as a hard disk or memory of an electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device. The aforementioned computer-readable storage medium can also include magnetic disks, optical disks, read-only memory (ROM), or random access memory (RAM), etc. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the electronic device. The computer-readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0126] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform... Figure 3 The methods provided for each step in the process.

[0127] The terms "first," "second," etc., used in the claims, description, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or electronic devices. References to "embodiment" herein mean that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The presentation of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. The term "and / or" as used in this application's description and appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0128] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Those skilled in the art can implement the described functions using different methods for each specific application, but such implementations should not be considered beyond the scope of this application.

[0129] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method performed by an electronic device, characterized in that, The method includes: Based on the target template set and the interference template set, target objects in the frame images of the video are detected; Output the detection results for the target object; Wherein, the target template set includes one or more target templates of the target object, and the interference template set includes one or more interference templates; The step of detecting target objects in video frame images based on the target template set and the interference template set includes: Based on the target template set and the interference template set, one or more target candidate regions in the frame image are determined to include the target object; Determining a target region from the one or more target candidate regions, wherein determining the target region from the one or more target candidate regions includes: For each target candidate region, determine the first matching degree between the target candidate region and each target template in the target template set, and the second matching degree between the target candidate region and each interference template in the interference template set; For each target candidate region, the first target matching degree of the target candidate region is determined based on the first matching degree and the second matching degree corresponding to the target candidate region; The target region is determined from each target candidate region based on the first target matching degree of each target candidate region.

2. The method according to claim 1, characterized in that, Each of the interference templates includes interference object information about the interference object, which interferes with the detection of the target object in other frames prior to the detected frame image. The interference object information is the image region where the interference object is located, or image feature information of the image region where the interference object is located.

3. The method according to claim 1, characterized in that, Each of the one or more target templates includes frame object information of the target object in each frame image, wherein each frame image is a frame image that includes the target object in other frame images preceding the detected frame image, and the frame object information is the image region where the target object is located, or the image feature information of the image region where the target object is located.

4. The method according to claim 3, characterized in that, The target template set includes initial target templates; The initial target template includes initial object information of the target object in the first frame image. The first frame image is a frame image in the video that includes the target object, determined by the user, or a separate image that includes the target object, independent of the video.

5. The method according to claim 1, characterized in that, Obtain one or more target candidate regions, including: Based on the target template set, determine the fusion target features; Based on the fused target features, one or more target candidate regions in the frame image are identified as including the target object.

6. The method according to claim 5, characterized in that, The acquired frame image is determined to contain one or more target candidate regions, including: Determine multiple search regions for the frame image; The search region features are obtained by extracting the image features of each of the multiple search regions; Calculate the correlation between the search region features of the multiple search regions and the fusion target features; Based on the correlation, one or more target candidate regions are determined from the plurality of search regions.

7. The method according to claim 5, characterized in that, Also includes: In response to the target region meeting preset conditions, the target template set is updated.

8. The method according to claim 7, characterized in that, The step of updating the target template set in response to the target region meeting preset conditions includes: Calculate the similarity between the target region and the fused target features of the target template set; In response to the similarity being less than a first threshold, the target region is added as a target template to the target template set.

9. The method according to claim 7, characterized in that, The step of updating the target template set in response to the target region meeting preset conditions includes: Calculate the similarity between the target region and all target templates in the target template set; In response to all similarities less than the second threshold, the target region is added as a target template to the target template set.

10. The method according to claim 1, characterized in that, The obtained frame image is determined to contain one or more target candidate regions, including: Based on the target template set, determine the fusion target features; Based on the set of interference templates, the fusion interference features are determined; Based on the fusion target features and the fusion interference features, one or more target candidate regions are obtained.

11. The method according to claim 5 or 10, characterized in that, Based on the target template set, the fusion target features are determined, including: The fused target features are determined by fusing the image features of the image regions corresponding to each target template included in the target template set.

12. The method according to claim 10, characterized in that, Based on the set of interference templates, the fused interference features are determined, including: The fusion interference features are determined by using the image features of the image regions corresponding to each interference template in the fusion interference template set.

13. The method according to claim 10, wherein determining the fused target features based on the target template set includes: Based on all target templates included in the target template set, determine the fusion target features; Based on the set of interference templates, the fused interference features are determined, including: Based on all the interference templates included in the interference template set, the fusion interference features are determined.

14. The method according to claim 10, characterized in that, Also includes: Update the interference template set using the following method: Some or all of the other target candidate regions, excluding the target region, are added to the interference template set as interference templates.

15. The method according to claim 1, characterized in that, Determine the first target match degree of the target candidate region, including any of the following: Based on the first mean and the second mean, the first target matching degree corresponding to the target candidate region is determined; Based on the first median value and the second median value, the first target matching degree corresponding to the target candidate region is determined; Wherein, the first mean and the first median are the mean and median of the first matching degree between the target candidate region and each target template in the target template set, and the second mean and the second median are the mean and median of the second matching degree between the target candidate region and each interference template in the interference template set.

16. The method according to claim 4, characterized in that, Based on the target template set, one or more target candidate regions in the frame image that are determined to include the target object are obtained, including: Based on the initial target template, the target object in the frame image is detected to obtain one or more target candidate regions.

17. The method according to any one of claims 1 to 10 and 12-16, characterized in that, Also includes: Determine the interference template corresponding to the frame image; The set of interference templates is updated based on the interference templates corresponding to the frame images.

18. The method according to claim 17, characterized in that, Determine the interference template corresponding to the frame image, including: Obtain the target region where the target object is located in the frame image, and the image regions where each candidate interference object of the target object is located; For any image region containing a candidate interference object, determine the first correlation between the image region and the target region, and the second correlation between the image region and each interference template in the interference template set corresponding to the frame image; Based on the first correlation and the second correlation corresponding to the image region where each candidate interference object is located, the interference template corresponding to the frame image is determined from the image region where each candidate interference object is located.

19. An electronic device, characterized in that, Including processor and memory; The memory is used to store computer programs; The processor is configured to implement the method as described in any one of claims 1 to 18 when the computer program is invoked.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the method as described in any one of claims 1 to 18.

Images