A training method and apparatus for a detection model based on active learning

Abstract

This application relates to the field of computer vision technology and provides a training method and apparatus for a detection model based on active learning. The method includes: inputting a sample video into a detection model Mi to obtain a detection result Ni; scoring the detection result Ni according to an active learning algorithm to obtain an information content score Vi for each frame of the sample video; obtaining an image set Ci based on the information content score Vi; training the detection model Mi based on the image set Ci to obtain a detection model Mi+1; inputting a sample video into the detection model Mi+1 to obtain a detection result Ni+1; scoring the detection result Ni+1 to obtain an information content score Vi+1; obtaining an image set Ci+1 based on the information content score Vi+1; and determining a detection model Mi+2 based on the image set Ci and the image set Ci+1. This reduces the training time of the detection model and yields a detection model with high recognition accuracy.

Description

Technical Field

[0001] This application belongs to the field of computer vision technology, and in particular relates to a training method and apparatus for a detection model based on active learning. Background Technology

[0002] In video security scenarios, detection models are often used to process and analyze video data. Before using a detection model for video analysis, it needs to be trained.

[0003] Currently, one method for training detection models is to extract frames from a video at certain frame intervals, resulting in a discrete image set. This discrete image set is then input into the detection model to obtain hard-to-classify samples. These hard-to-classify samples are those whose features the detection model struggles to learn. These hard-to-classify samples are then manually labeled, and the model is trained using these labeled samples to obtain the trained detection model.

[0004] However, the process of training the detection model described above requires frame extraction from the video, which is tedious and time-consuming. Hard-to-classify samples may contain a large number of highly similar adjacent frame images. Training the detection model using these highly similar adjacent frame images can lead to overfitting. Summary of the Invention

[0005] This application provides a training method and apparatus for a detection model based on active learning, which can solve the problem of overfitting in the training of the detection model.

[0006] In a first aspect, embodiments of this application provide a training method for a detection model based on active learning, the method comprising:

[0007] S101: Obtain sample video;

[0008] S102: Input the sample video into the detection model Mi to obtain the detection result Ni of each frame in the sample video, and score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame in the sample video. The detection model is used to identify the target object, and the information content score value of any frame is used to indicate the credibility of the detection result of the target object in any frame. i is a positive integer greater than or equal to 1.

[0009] S103: Based on the information content score Vi of each frame in the sample video, obtain the image set Ci, which includes the images in the sample video where the information content score Vi is the peak value;

[0010] S104: Train the detection model Mi based on the image set Ci to obtain the detection model Mi+1;

[0011] S105: Input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame in the sample video, and score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame in the sample video. The detection result is used to indicate the predicted position and predicted category of the target in each frame of the sample video.

[0012] S106: Based on the information content score value Vi+1 of each frame in the sample video, obtain the image set Ci+1, which includes the images in the sample video whose information content score value Vi+1 is the peak value.

[0013] S107: Determine the detection model Mi+2 based on image set Ci and image set Ci+1.

[0014] In one possible implementation, the detection model Mi+2 is determined based on image set Ci and image set Ci+1, including:

[0015] S201: From image set Ci+1, delete images that are the same as images in image set Cj to obtain an updated image set Ci+1, where j takes all positive integers greater than or equal to 1 and less than or equal to i.

[0016] S202: Based on the updated image set Ci+1, the detection model Mi+1 is trained to obtain the detection model Mi+2;

[0017] S203: When the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of times, the detection model Mi+2 is determined to be the target detection model.

[0018] S204: When the accuracy of the detection model Mi+2 in identifying the target is less than a preset threshold, or when i+2 is less than a preset number of times, update i = i+1, and repeat S105, S106, S201 and S202 until the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to the preset threshold, or when i+2 is greater than or equal to the preset number of times.

[0019] In one possible implementation, the detection model is trained based on an image set to obtain a trained detection model, including:

[0020] Annotate an image set to obtain annotation results, which are used to represent the true category and true location of the target object in each frame of the image set;

[0021] The detection model is trained based on an image set and annotation results to obtain the trained detection model.

[0022] In one possible implementation, an image set is obtained based on the information content score of each frame in a video, including:

[0023] Plot a relationship curve based on the information content score and frame number of each frame in a video;

[0024] From the relationship curve, obtain all frame numbers whose information content score value is the peak value;

[0025] Determine an image set that contains all images corresponding to all frame numbers.

[0026] In one possible implementation, before acquiring all frame numbers whose information content score value is the peak value, the method of this application embodiment further includes:

[0027] The relationship curve is denoised to make it smoother.

[0028] In one possible implementation, the noise reduction process includes either Fourier transform filtering or wavelet filtering.

[0029] This application embodiment solves the technical problem of overfitting in detection model training caused by the complex process of extracting frames from sample videos and using adjacent frames with highly similar content as training samples in the obtained images through iterative training of the detection model with fewer high-value frame images. It achieves the technical effect of reducing the training time of the detection model, improving the training effect of the detection model, avoiding overfitting of the detection model, and improving the detection and recognition accuracy of the detection model.

[0030] Secondly, embodiments of this application provide a target detection method, the method comprising:

[0031] Acquire the image to be detected;

[0032] The image to be detected is input into the trained detection model to obtain the detection result. The detection model is trained based on the training method in the first aspect or any possible implementation of the first aspect.

[0033] Thirdly, embodiments of this application provide a training apparatus for a detection model based on active learning, the apparatus comprising:

[0034] The acquisition module is used to acquire sample videos;

[0035] The prediction module is used to input the sample video into the detection model Mi to obtain the detection result Ni of each frame in the sample video, and to score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame in the sample video. The detection model is used to identify the target object, and the information content score value of any frame is used to indicate the confidence of the detection result of the target object in any frame. i is a positive integer greater than or equal to 1.

[0036] The processing module is used to obtain an image set Ci based on the information content score Vi of each frame of the sample video. The image set Ci includes images in the sample video where the information content score Vi is the peak value.

[0037] The training module is used to train the detection model Mi based on the image set Ci to obtain the detection model Mi+1.

[0038] The prediction module is used to input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame in the sample video, and to score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame in the sample video. The detection result is used to indicate the predicted position and predicted category of the target in each frame of the sample video.

[0039] The processing module is used to obtain an image set Ci+1 based on the information content score Vi+1 of each frame of the sample video. The image set Ci+1 includes images in the sample video where the information content score Vi+1 is the peak value. Based on the image set Ci and the image set Ci+1, the detection model Mi+2 is determined.

[0040] Fourthly, embodiments of this application provide an electronic device including a memory and a processor. The memory stores instructions; the processor executes the instructions stored in the memory, causing the electronic device to perform the training method of the detection model based on active learning in the first aspect or any possible implementation of the first aspect.

[0041] Fifthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform a training method for a detection model based on active learning in the first aspect or any possible implementation thereof.

[0042] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart illustrating a training method for a detection model based on active learning, provided in an embodiment of this application.

[0045] Figure 2 This is a flowchart illustrating a training method for a detection model based on active learning, provided in an embodiment of this application.

[0046] Figure 3 This is a schematic diagram of the detection results and relationship curves of a training method for a detection model based on active learning provided in an embodiment of this application;

[0047] Figure 4 This is a structural block diagram of the object detection model generated in the training method of the detection model based on active learning provided in the embodiments of this application;

[0048] Figure 5 This is an application block diagram of a training method for a detection model based on active learning, provided in an embodiment of this application.

[0049] Figure 6 This is a schematic block diagram of a training device for a detection model based on active learning, provided in an embodiment of this application. Detailed Implementation

[0050] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0051] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0052] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0053] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0054] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0056] Please see Figure 1 , Figure 1 This illustration shows a flowchart of a training method for a detection model based on active learning, as provided in an embodiment of this application. Figure 1 As shown, the training method for the detection model based on active learning in this application embodiment may include:

[0057] S101. Obtain sample video.

[0058] Electronic devices can access sample videos stored in storage devices.

[0059] Electronic devices may include servers or computers, etc.

[0060] Storage devices can be used to store sample videos. Sample videos can be captured using devices such as mobile phones or cameras.

[0061] The sample video includes multiple frames, and there is continuity between adjacent frames.

[0062] S102. Input the sample video into the detection model Mi to obtain the detection result Ni of each frame in the sample video, and score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame in the sample video. The detection model is used to identify the target object, and the information content score value of any frame is used to indicate the credibility of the detection result of the target object in any frame. i is a positive integer greater than or equal to 1.

[0063] The electronic device can input the acquired sample video into the detection model Mi to obtain the detection result Ni for each frame of the sample video. The electronic device can then score the detection result Ni using an active learning algorithm to obtain the information content score Vi for each frame of the sample video. Thus, the electronic device can obtain the information content score Vi for each frame of the sample video using the detection model Mi and the active learning algorithm.

[0064] The detection model can be used to identify objects in each frame of a video. For example, the detection model can identify objects in any frame and label the identified objects as predicted bounding boxes.

[0065] In this context, the target object is the object detected in each frame of the video. The target object can include one or more objects, and these objects can belong to the same or different categories. For example, the target object can include one or more categories such as vehicles, ships, people, faces, full-body images of people, or animals.

[0066] Specifically, active learning algorithms can employ active learning for deep detection neural networks (ALDD) or consistency-based active learning for object detection (CALD) to effectively measure the information content of each frame in a sample video. Through measurement mechanisms such as ALDD or CALD, active learning algorithms can convert the information content of each frame in a sample video into an information content score Vi. Thus, the electronic device can obtain the information content score Vi for each frame in the sample video.

[0067] The amount of information content is related to factors such as the number of target objects in each frame of the video and the confidence level of the target objects.

[0068] Among them, the information content score Vi of any frame image can be used to indicate the credibility of the detection result of the target object in any frame image.

[0069] The term 'i' mentioned in this application can be used to distinguish between the detection model before each training iteration and the detection model after each training iteration.

[0070] In summary, when electronic devices input sample videos into the detection model Mi, there is no need to perform frame extraction on the sample videos, nor is it necessary to store the extracted multi-frame images. This saves storage space and ensures the continuity between adjacent frames in the sample video.

[0071] S103. Based on the information content score Vi of each frame in the sample video, obtain the image set Ci, which includes the images in the sample video where the information content score Vi is the peak value.

[0072] There is a positive correlation between the information content score Vi of any frame and the information content of that frame; that is, the higher the information content score, the higher the information content of that frame, and the higher its training value for the active learning model. Therefore, electronic devices can determine the images with the highest information content score Vi in the sample video as the image set Ci based on the information content score Vi of each frame.

[0073] The training requirements for the detection model Mi can only be met if each frame in the image set Ci contains corresponding annotations. Since each frame in the image set Ci does not contain corresponding annotations, the detection model Mi has difficulty learning the features of each frame in the image set Ci.

[0074] As can be seen, the image set Ci is a collection of images for which the detection model Mi has difficulty learning the features of the target body.

[0075] The image set Ci contains high-value frame images. High-value frame images are images that have high training value for the detection model.

[0076] S104. Based on the image set Ci, train the detection model Mi to obtain the detection model Mi+1.

[0077] The electronic device can use the image set Ci as training samples and input it into the detection model Mi to train the detection model Mi. Thus, the electronic device can obtain the detection model Mi+1 after the detection model Mi has been trained.

[0078] Specifically, the electronic device can annotate the image set Ci to obtain the annotation result Ri. The annotation result Ri is used to represent the true category and true location of the target object in each frame of the image set Ci. Thus, the electronic device can train the detection model Mi based on the image set Ci and the annotation result Ri to obtain the detection model Mi+1.

[0079] For example, electronic devices can use image annotation tools to annotate image sets.

[0080] The image annotation tool is used to identify, detect, and label targets in any frame of an image, ensuring that the targets in any frame of an image are located within the labeled detection box.

[0081] The marked detection box is a truth box that is manually labeled and conforms to the actual situation of the target object.

[0082] Image annotation tools can include rectangular annotation tools and polygon annotation tools. For example, a rectangular annotation tool could be labeledimg, and a polygon annotation tool could be labeledme.

[0083] Because the detection model's recognition accuracy improves with each training iteration, detection model Mi+1 is the model obtained after training detection model Mi.

[0084] It is evident that the recognition accuracy of detection model Mi+1 is higher than that of detection model Mi.

[0085] S105. Input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame in the sample video, and score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame in the sample video. The detection result is used to indicate the predicted position and predicted category of the target in each frame of the sample video.

[0086] The electronic device can input the sample video again into the detection model Mi+1 to obtain the detection result Ni+1 for each frame of the sample video. The electronic device can then score the detection result Ni+1 again using an active learning algorithm to obtain the information content score Vi+1 for each frame of the sample video.

[0087] The above implementation method can be found in the description of the electronic device in S102 inputting the sample video into the detection model Mi and obtaining the information content score value Vi of each frame of the sample video, which will not be elaborated here.

[0088] Since different detection models yield different information content in each frame of the sample video, the information content score obtained by the active learning algorithm after transforming the information content of each frame is also different.

[0089] Therefore, the information content score value Vi+1 of each frame in the sample video is different from the information content score value Vi of each frame in the sample video in S102.

[0090] S106. Based on the information content score value Vi+1 of each frame in the sample video, obtain the image set Ci+1, which includes the images in the sample video whose information content score value Vi+1 is the peak value.

[0091] The electronic device can determine the image set Ci+1 based on the information content score Vi+1 of each frame in the sample video, where the information content score Vi+1 is the peak value.

[0092] The above implementation method can be found in S103, where the electronic device determines the image set Ci as the image with the peak information content score value Vi in the sample video based on the information content score value Vi of each frame of the sample video. This will not be elaborated here.

[0093] The training requirements for the detection model Mi+1 can only be met if each frame in the image set Ci+1 contains corresponding annotation results. Since each frame in the image set Ci+1 does not contain corresponding annotation results, the detection model Mi+1 has difficulty learning the features of each frame in the image set Ci+1.

[0094] It is evident that the image set Ci+1 is a collection of images for which the detection model Mi+1 struggles to learn the features of the target object.

[0095] The image set Ci+1 contains high-value frame images.

[0096] Because the active learning algorithm assigns different information content scores to each frame of the sample image, the electronic device obtains different images from the sample video based on the different information content scores.

[0097] Therefore, the images contained in image set Ci+1 are different from those contained in image set Ci in S103.

[0098] S107. Determine the detection model Mi+2 based on image set Ci and image set Ci+1.

[0099] The electronic device can determine the training samples for the detection model Mi+1 based on the image set Ci and the image set Ci+1. The electronic device can then input the determined training samples into the detection model Mi+1 to train it. Thus, the electronic device can obtain the detection model Mi+2 after training the detection model Mi+1.

[0100] Specifically, the electronic device can delete images identical to those in image set Ci+1 from image set Ci+1, resulting in an updated image set Ci+1. The electronic device can then annotate the updated image set Ci+1, obtaining an annotation result Ri+1. The annotation result Ri+1 represents the true category and true location of the target object in each frame of the updated image set Ci+1. Therefore, the electronic device can train the detection model Mi+1 based on the updated image set Ci+1 and the annotation result Ri+1, resulting in a detection model Mi+2.

[0101] Because the recognition accuracy of the detection model improves with each training iteration, and detection model Mi+2 is obtained after training detection model Mi+1, it is evident that the recognition accuracy of detection model Mi+2 is higher than that of detection model Mi+1.

[0102] The active learning-based detection model training method of this application acquires sample videos and inputs them into the detection model Mi to obtain the detection result Ni for each frame of the sample video. The active learning algorithm scores the detection result Ni to obtain the information content score Vi for each frame of the sample video. Based on the information content score Vi, an image set Ci is obtained, which includes images in the sample video where the information content score Vi is the peak value. This image set Ci can be used as training samples for the detection model Mi, achieving the goal of training the detection model with fewer images, reducing the training time, eliminating the need for frame extraction from the complete sample video, simplifying the process, and resulting in a collection of high-value frames with high training value for the detection model. The detection model Mi is then trained using the image set Ci to obtain a detection model Mi+1, which can be used as a model for retraining, thereby improving the recognition accuracy of the detection model.

[0103] Subsequently, the sample video is input again into the determined detection model Mi+1 to obtain the detection result Ni+1 for each frame of the sample video. The detection result Ni+1 is scored using an active learning algorithm to obtain the information content score Vi+1 for each frame of the sample video. Based on the information content score Vi+1 for each frame of the sample video, an image set Ci+1 is obtained. Image set Ci+1 includes images in the sample video whose information content score Vi+1 is the peak value. The resulting image set Ci+1 is a set of high-value frame images with high training value for the detection model. Based on the obtained image set Ci and image set Ci+1, the detection model Mi+2 can be determined. The training samples for the detection model Mi+1 can be determined based on image set Ci and image set Ci+1, achieving the goal of training the detection model with fewer images. Furthermore, there are no highly similar adjacent frame images in image sets Ci and Ci+1.

[0104] This allows for iterative training of the detection model using fewer images, reducing training time. Using high-value frame images as training samples improves the training performance. Furthermore, the absence of highly similar adjacent frames in the training samples prevents overfitting and enhances the model's recognition accuracy.

[0105] exist Figure 1 Based on the embodiments, the following, in conjunction with Figure 2 This paper details a feasible method for obtaining a target detection model with high recognition accuracy using electronic devices.

[0106] Figure 2 This illustration shows a flowchart of a training method for a detection model based on active learning, as provided in an embodiment of this application. Figure 2 As shown, the training method of the detection model based on active learning in this application may include: S101-S106 and S201-S204.

[0107] The specific implementation methods of S101-S106 can be found in the description above, and will not be repeated here.

[0108] Among them, S201-S204 is a feasible implementation of S107.

[0109] The specific implementation process of S201-S204 includes the following:

[0110] S201. From image set Ci+1, delete images that are the same as images in image set Cj to obtain an updated image set Ci+1, where j is a positive integer greater than or equal to 1 and less than or equal to i.

[0111] Here, image set Cj is the set of images in the sample video that the electronic device has labeled before obtaining image set Ci+1.

[0112] Before training the detection model Mi+1, the resulting image set Ci+1 may contain some images from the image set Cj. Therefore, the electronic device can compare the true location of the target object in each frame of image set Ci+1 with the true location of the target object in each frame of image set Cj, and also compare the true category of the target object in each frame of image set Ci+1 with the true category of the target object in each frame of image set Cj.

[0113] Thus, the electronic device can obtain an image in image set Ci+1 that is identical to an image in image set Cj.

[0114] In summary, the electronic device can delete images from image set Ci+1 that are identical to images in image set Cj, thus obtaining an updated image set Ci+1. This ensures that the updated image set Ci+1 does not contain any images identical to those in image set Cj, avoiding duplicate annotation of identical images in the sample video.

[0115] When i = 1, j can be 1, image set Ci+1 is image set C2, and image set Cj is image set C1.

[0116] Based on this, the electronic device can delete images from image set C2 that are identical to images in image set C1, thereby obtaining an updated image set C2.

[0117] When i = 2, j can be either 1 or 2, meaning image set Ci+1 is image set C3, and image set Cj can be image set C1 or image set C2. Based on this, the electronic device can delete images from image set C3 that are identical to images in image set C1, and also delete images from image set C2, thus obtaining an updated image set C3.

[0118] When i > 2, j can take any positive integer greater than or equal to 1 and less than or equal to i, that is, the image set Cj is the image set C1, image set C2, ..., and image set Ci-1.

[0119] Based on this, the electronic device can delete images from image set Ci that are the same as images in image set C1, images that are the same as images in image set C2, ..., and images that are the same as images in image set Ci-1, thereby obtaining an updated image set Ci.

[0120] For example, when i = 3, j can be 1, 2, or 3, that is, image set Ci+1 is image set C4, and image set Cj can be image set C1, image set C2, or image set C3.

[0121] The electronic device can delete images from image set C4 that are identical to images in image set C1, images that are identical to images in image set C2, and images that are identical to images in image set C3, thereby obtaining an updated image set C4.

[0122] S202. Based on the updated image set Ci+1, train the detection model Mi+1 to obtain the detection model Mi+2.

[0123] The electronic device can use the updated image set Ci+1 as training samples and input it into the detection model Mi+1 to train the detection model Mi+1. Thus, the electronic device can obtain the detection model Mi+2 after the detection model Mi+1 has been trained.

[0124] The required recognition accuracy of the detection model varies across different application scenarios. Therefore, electronic devices need to assess the recognition accuracy of the detection model. In one feasible implementation, the electronic device can determine whether the recognition accuracy of the detection model Mi+2 meets the requirements by judging whether the accuracy of the detection model Mi+2 in recognizing the target is greater than or equal to a preset threshold, or by judging whether i+2 is greater than or equal to a preset number of attempts.

[0125] The preset threshold is a value for the accuracy rate that is pre-set in the electronic device.

[0126] The preset number of times refers to the number of times the training and detection model is pre-set in the electronic device.

[0127] When the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of times, the electronic device may execute S203.

[0128] When the accuracy of the detection model Mi+2 in identifying the target is less than a preset threshold, or when i+2 is less than a preset number of times, the electronic device may execute S204.

[0129] S203. When the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of times, the detection model Mi+2 is determined to be the target detection model.

[0130] When the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of attempts, the electronic device can determine that the detection model Mi+2 has high recognition accuracy. At this point, the electronic device can stop training the detection model Mi+2. Thus, the electronic device can determine that the detection model Mi+2 is a target detection model.

[0131] S204. When the accuracy of the detection model Mi+2 in identifying the target is less than a preset threshold, or when i+2 is less than a preset number of times, update i = i+1, and repeat S105, S106, S201 and S202 until the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to the preset threshold, or when i+2 is greater than or equal to the preset number of times.

[0132] When the accuracy of the detection model Mi+2 in recognizing the target is less than a preset threshold, or when i+2 is less than a preset number of times, the electronic device can determine that the recognition accuracy of the detection model Mi+2 is low.

[0133] At this point, the electronic device can update i to i+1 and repeatedly execute S105, S106, S201 and S202 to iteratively train the detection model Mi+2.

[0134] In summary, electronic devices can ensure that they obtain a target detection model with high recognition accuracy by executing S201-S204.

[0135] In another feasible implementation, the electronic device does not need to judge the accuracy of the detection model Mi+2 in identifying the target.

[0136] Therefore, after executing S202, the electronic device does not need to execute S203 and S204, meaning it does not need to iteratively train the detection model Mi+1. The electronic device can directly determine the detection model Mi+2 as the target detection model.

[0137] The following details a feasible implementation method for an electronic device to obtain an image set based on the information content score of each frame in a video.

[0138] In some embodiments, the electronic device can plot a relationship curve based on the information content score and frame number of each frame in a video. The electronic device can then extract all frame numbers with peak information content scores from the relationship curve. Based on these peak information content scores, the electronic device can extract images corresponding to all the frame numbers from the video. The extracted images corresponding to all the frame numbers can be compiled into an image set. Thus, the electronic device obtains an image set.

[0139] The frame number is the sequence number of any frame in the video.

[0140] The information content score of any frame in the relationship curve corresponds to the frame number of that frame.

[0141] Since the relationship curves obtained by electronic devices may contain noise interference, the electronic devices need to perform noise reduction processing such as Fourier transform filtering or wavelet filtering on the obtained relationship curves to remove the noise.

[0142] Furthermore, before acquiring all frames with peak information content scores, the electronic device can perform noise reduction processing on the relationship curve to smooth it out. This reduces the number of adjacent frames with highly similar content that the electronic device receives, avoiding overfitting of the detection model when training it with such images.

[0143] Based on the above description, in S103, the electronic device can plot a first relationship curve based on the information content score Vi and frame number of each frame in the sample video. The electronic device can then obtain all frame numbers where the information content score Vi reaches its peak from the first relationship curve. Therefore, the electronic device can determine that the image set Ci contains images corresponding to all frame numbers.

[0144] Based on the above description, in S106, the electronic device can plot a second relationship curve based on the information content score value Vi+1 and the frame number of each frame in the sample video. The electronic device can then obtain all frame numbers where the information content score value Vi+1 reaches its peak from the second relationship curve. Therefore, the electronic device can determine that the image set Ci+1 contains images corresponding to all frame numbers.

[0145] In addition, the electronic device can save the information content score value Vi of each frame of the sample video output by the detection model Mi, and the information content score value Vi+1 of each frame of the sample video output by the detection model Mi+1, in a preset output document.

[0146] Based on the description of the above embodiments, after determining the detection model Mi+2 according to the image set Ci and the image set Ci+1, the electronic device can use the detection model Mi+2 to detect and identify the sample video.

[0147] In some embodiments, the electronic device can input a sample video into a detection model Mi+2 to obtain the detection results for each frame of the sample video. The detection results are used to indicate the predicted location and predicted category of the target object in each frame of the sample video. The electronic device can display the detection results for each frame of the sample video.

[0148] The detection result of any frame is used to indicate the predicted location and category of the target object in any frame of the sample video. For example, the predicted location of the target object is on the left side of any frame, and the predicted category of the target object is a vehicle. Furthermore, the electronic device can process the detection results of each frame in the obtained sample video.

[0149] For ease of explanation, Figure 3 In the sample video, all target objects are represented in text form. However, in real-world applications, the target objects in the detection results of each frame are the corresponding physical objects.

[0150] Based on the description of the above embodiments, and in conjunction with Figure 3 Provide a detailed description of the detection results and relationship curves displayed by the electronic device.

[0151] like Figure 3 As shown, the electronic device can display the relationship curve between the detection result of the image with frame number x4 in the sample video and the sample video.

[0152] x4 is the frame number of the image corresponding to the progress bar playback position in the sample video.

[0153] The detection results for image frame x4 include: target 1, target 2, target 3, and target 4. Target 1 is vehicle 1, target 2 is a person, target 3 is vehicle 2, and target 4 is a cat. The positions of the four targets are displayed using rectangular detection boxes.

[0154] The information content score of the image with frame number x4 in the relationship curve is y9.

[0155] In addition, the relationship curve also shows the information content score values ​​corresponding to other frame numbers. The information content score value corresponding to the image with frame number x1 is y3, that of frame number x2 is y4, that of frame number x3 is y8, that of frame number x5 is y6, that of frame number x6 is y7, that of frame number x7 is y1, that of frame number x8 is y5, that of frame number x9 is y8, and that of frame number x10 is y2.

[0156] Based on this, when viewing the detection results of images other than frame number x4 in the sample video, users can adjust the progress bar. The electronic device responds to the user's adjustment and displays the detection result of the frame at the corresponding playback position on the progress bar. This ensures that the detection result and information content score of any frame in the sample video are arranged vertically and correspondingly, allowing users to easily verify the detection result and information content score of any frame.

[0157] Below, in conjunction with Figure 4 and Figure 5 The following example illustrates the specific implementation process of a training method for a detection model based on active learning provided in an embodiment of this application.

[0158] The electronic device includes a sample video acquisition unit, an active learning unit, a relationship curve plotting unit, a peak frame acquisition unit, a data mining unit, an image frame acquisition unit, a labeling unit, a dataset management unit, and a result visualization unit.

[0159] The active learning unit includes a detection model and an active learning algorithm.

[0160] The dataset management unit includes labeled datasets and unlabeled datasets.

[0161] Assuming a preset threshold of 95%, a preset number of iterations of 2, i = 1, j = 1, then Mi is M1, Mi+1 is M2, Mi+2 is M3. Vi is V1, Vi+1 is V2. Cj and Ci are both C1, Ci+1 is C2. Ri is R1, Ri+1 is R2.

[0162] Figure 5 This diagram illustrates an application block diagram of a training method for a detection model based on active learning, provided in an embodiment of this application. For example... Figure 5 As shown, the training method for the detection model based on active learning in this application embodiment may include:

[0163] Step 11: The sample video acquisition unit acquires the sample video.

[0164] Step 12: The sample video acquisition unit can input the sample video into the detection model M1. The detection model M1 outputs the detection result N1 of each frame in the sample video. The active learning algorithm is used to score the detection result N1. The active learning unit outputs the information content score value V1 of each frame in the sample video.

[0165] Step 13: The detection model M1 can input the information content score value V1 and frame number of each frame in the sample video into the relation curve drawing unit, and the relation curve drawing unit draws the first relation curve.

[0166] In addition, the relationship curve plotting unit can input the first relationship curve into the result visualization unit for display. The result visualization unit can also display the detection results output by the detection model M1.

[0167] Step 14: The relationship curve drawing unit can input the first relationship curve into the peak frame acquisition unit. The peak frame acquisition unit performs noise reduction processing on the first relationship curve and obtains the frame number of the peak value with information content score V1 based on the noise-reduced first relationship curve. Thus, interference between adjacent frame images with highly similar content can be eliminated.

[0168] Step 15: The peak frame acquisition unit can input the frame number of the peak information content score value V1 into the data mining unit, and the data mining unit outputs the frame number of the peak information content score value V1.

[0169] Since the detection model M1 has not yet started training, the dataset management unit has not yet stored the frame numbers of images with peak information content scores. Therefore, the data mining unit does not need to delete the frame numbers of images with peak information content scores V1.

[0170] Step 16: The data mining unit can input the frame number where the information content score V1 is the peak value into the image frame extraction unit. The image frame extraction unit can extract the image set C1 corresponding to the frame number where the information content score V1 is the peak value from the sample video. At the same time, the image frame extraction unit can input the image set C1 into the unlabeled dataset of the dataset management unit for storage.

[0171] Step 17: The image frame extraction unit can input the image set C1 into the annotation unit for annotation, and obtain the annotation result R1.

[0172] It should be understood that the annotation of the annotation units in step 17 can be done manually or by using algorithm model-assisted annotation, and this application does not impose specific restrictions on the annotation method.

[0173] Step 18: The annotation unit can use the annotation result R1 and image set C1 as training samples and input them into the detection model M1. Figure 4 The detection model M1 shown is trained to obtain... Figure 4 The detection model M2 is shown. Simultaneously, the annotation unit can input the annotation results R1 and image set C1 into the labeled dataset of the dataset management unit for storage.

[0174] When the accuracy of detection model M2 in identifying targets is greater than or equal to 95%, detection model M2 does not need to be trained again. Therefore, detection model M2 is a target detection model.

[0175] In steps 11-18 above, the detection model M1 was trained only once. At this time, the number of training iterations of the detection model M1 is less than the preset number of iterations.

[0176] If the number of training iterations for detection model M1 is less than the preset number of iterations, or if the accuracy of detection model M2 in recognizing the target is less than 95%, steps 19-25 can be continued to retrain detection model M2, thereby obtaining detection model M3 with higher recognition accuracy.

[0177] Step 19: The sample video acquisition unit can input the sample video into the detection model M2. The detection model M2 outputs the detection result N2 of each frame in the sample video. The active learning algorithm is used to score the detection result N2. The active learning unit outputs the information content score value V2 of each frame in the sample video.

[0178] Step 20: The detection model M2 can input the information content score value V2 and frame number of each frame in the sample video into the relation curve drawing unit, and the relation curve drawing unit draws the second relation curve.

[0179] In addition, the relationship curve plotting unit can input a second relationship curve into the result visualization unit for display. The result visualization unit can also display the detection results output by the detection model M2.

[0180] Step 21: The relationship curve drawing unit can input the second relationship curve into the peak frame acquisition unit. The peak frame acquisition unit performs noise reduction processing on the second relationship curve and obtains the frame number of the information content score value V2 as the peak value based on the noise-reduced second relationship curve.

[0181] Step 22: The peak frame acquisition unit can input the frame number of the information content score value V2 as the peak value into the data mining unit. The data mining unit deletes the frame number that is the same as the frame number of the labeled information content score value V1 as the peak value stored in the dataset management unit from the frame number of the information content score value V2 as the peak value, and obtains the updated frame number of the information content score value V2 as the peak value. The labeled frame number of the information content score value V1 as the peak value is the frame number of each image in the image set C1.

[0182] Since the detection model M1 has already completed one training iteration, the dataset management unit already stores the frame numbers of images with peak information content score V1. Therefore, the data mining unit needs to remove frames with the same frame numbers as images with peak information content score V1 from the frame numbers of images with peak information content score V2. This avoids the electronic device repeatedly labeling the same images in the sample video.

[0183] Step 23: The data mining unit can input the frame number with the peak value of the updated information content score V2 into the image frame extraction unit. The image frame extraction unit can extract the updated image set C2 corresponding to the frame number with the peak value of the updated information content score V2 from the sample video. At the same time, the image frame extraction unit can input the updated image set C2 into the unlabeled dataset of the dataset management unit for storage.

[0184] If the detection model M2 does not require retraining, the data mining unit can arrange the information content score of each frame in the updated image set C2 in descending order, and take the k frames corresponding to the first k information content scores as the high-value frames in the sample video mined by the detection model M2. Thus, the electronic device can obtain the globally optimal frame of the sample video.

[0185] Step 24: The image frame extraction unit can input the updated image set C2 into the annotation unit for annotation, and obtain the annotation result R2.

[0186] Step 25: The annotation unit can use the annotation result R2 and the updated image set C2 as training samples and input them into the detection model M2. Figure 4 The detection model M2 shown is trained to obtain the following results: Figure 4 The detection model M3 is shown.

[0187] The annotation unit can input the annotation result R2 and the updated image set C2 into the labeled dataset of the dataset management unit for storage.

[0188] In steps 11-25 above, detection model M1 was trained twice. At this point, the number of training iterations of detection model M1 is equal to the preset number of iterations. Therefore, detection model M3 is the target detection model.

[0189] If the accuracy of detection model M3 in identifying targets is less than 95%, detection model M3 can be retrained. The training process of detection model M3 is the same as the training process of detection model M2 described above, and will not be repeated here.

[0190] This application also provides a target detection method, the method comprising:

[0191] Acquire the image to be detected; input the image to be detected into the trained detection model to obtain the detection result. The detection model is trained based on the training method in the above embodiments.

[0192] Please see Figure 6 , Figure 6 This diagram illustrates a schematic block representation of a training device for a detection model based on active learning, as provided in an embodiment of this application. Figure 6 As shown, the training device 300 for the detection model based on active learning in this application embodiment may include:

[0193] Module 301 is used to acquire sample videos;

[0194] The prediction module 302 is used to input the sample video into the detection model Mi to obtain the detection result Ni of each frame of the sample video, and to score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame of the sample video. The detection model is used to identify the target object, and the information content score value of any frame of the image is used to indicate the confidence of the detection result of the target object in any frame of the image. i is a positive integer greater than or equal to 1.

[0195] Processing module 303 is used to obtain image set Ci based on the information content score value Vi of each frame image in the sample video. The image set Ci includes images in the sample video where the information content score value Vi is the peak value.

[0196] The training module 304 is used to train the detection model Mi based on the image set Ci to obtain the detection model Mi+1;

[0197] The prediction module 302 is used to input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame of the sample video, and to score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame of the sample video. The detection result is used to indicate the predicted position and predicted category of the target object in each frame of the sample video.

[0198] Processing module 303 is used to obtain image set Ci+1 based on the information content score value Vi+1 of each frame image in the sample video. Image set Ci+1 includes images in the sample video where the information content score value Vi+1 is the peak value.

[0199] The processing module 303 is used to determine the detection model Mi+2 based on the image set Ci and the image set Ci+1.

[0200] In some embodiments, the training module 304 is specifically used to delete images identical to those in image set Cj from image set Ci+1 to obtain an updated image set Ci+1, where j is a positive integer greater than or equal to 1 and less than or equal to i; to train the detection model Mi+1 based on the updated image set Ci+1 to obtain the detection model Mi+2; when the accuracy of the detection model Mi+2 in recognizing the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of times, the detection model Mi+2 is determined to be the target detection model; when the accuracy of the detection model Mi+2 in recognizing the target is less than the preset threshold, or when i+2 is less than the preset number of times, i is updated to i+1, and S105, S106, S201 and S202 in the method embodiment are repeatedly executed until the accuracy of the detection model Mi+2 in recognizing the target is greater than or equal to the preset threshold, or when i+2 is greater than or equal to the preset number of times.

[0201] In some embodiments, the training module 304 is specifically used to train a detection model based on an image set to obtain a trained detection model, including: labeling an image set to obtain labeling results, the labeling results being used to represent the true category and true location of the target object in each frame of an image set; and training the detection model based on an image set and the labeling results to obtain a trained detection model.

[0202] In some embodiments, the processing module 303 is specifically configured to obtain an image set based on the information content score value of each frame image in a video, including: drawing a relationship curve based on the information content score value and frame number of each frame image in a video; obtaining all frame numbers with information content score values ​​as peak values ​​from the relationship curve; and determining that an image set contains images corresponding to all frame numbers.

[0203] In some embodiments, the processing module 303 is specifically used to perform noise reduction processing on the relationship curve before acquiring all frame numbers whose information content score value is the peak value, so as to smooth the relationship curve.

[0204] In some embodiments, the noise reduction process includes: Fourier transform filtering or wavelet filtering.

[0205] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to perform the steps in the above-described method embodiments.

[0206] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0207] This application provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to perform the steps described in the various method embodiments above.

[0208] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0209] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0210] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0211] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0212] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0213] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0214] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0215] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training method for a detection model based on active learning, characterized in that, The method includes: S101: Obtain sample video; S102: Input the sample video into the detection model Mi to obtain the detection result Ni of each frame in the sample video, and score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame in the sample video. The detection model is used to identify the target object, and the information content score value of any frame is used to indicate the credibility of the detection result of the target object in any frame. i is a positive integer greater than or equal to 1. S103: Based on the information content score Vi of each frame in the sample video, obtain an image set Ci, wherein the image set Ci includes images in the sample video whose information content score Vi is the peak value; S104: Train the detection model Mi based on the image set Ci to obtain the detection model Mi+1; S105: Input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame in the sample video, and score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame in the sample video. The detection result is used to indicate the predicted position and predicted category of the target body in each frame of the sample video. S106: Based on the information content score value Vi+1 of each frame in the sample video, obtain the image set Ci+1, wherein the image set Ci+1 includes the images in the sample video whose information content score value Vi+1 is the peak value; S107: Determine the detection model Mi+2 based on the image set Ci and the image set Ci+1; The step of determining the detection model Mi+2 based on the image set Ci and the image set Ci+1 includes: S201: From the image set Ci+1, delete the images that are the same as the images in the image set Cj to obtain the updated image set Ci+1, where j takes all positive integers greater than or equal to 1 and less than or equal to i. S202: Based on the updated image set Ci+1, the detection model Mi+1 is trained to obtain the detection model Mi+2; S203: When the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to a preset threshold, or when i+2 is greater than or equal to a preset number of times, the detection model Mi+2 is determined to be the target detection model. S204: When the accuracy of the detection model Mi+2 in identifying the target is less than a preset threshold, or when i+2 is less than a preset number of times, update i=i+1, and repeat S105, S106, S201 and S202 until the accuracy of the detection model Mi+2 in identifying the target is greater than or equal to the preset threshold, or when i+2 is greater than or equal to the preset number of times.

2. The method as described in claim 1, characterized in that, The detection model is trained based on an image set to obtain the trained detection model, which includes: The image set is labeled to obtain the labeling results, which are used to represent the true category and true location of the target object in each frame of the image set; The detection model is trained based on the image set and the annotation results to obtain the trained detection model.

3. The method as described in claim 1, characterized in that, Based on the information content score of each frame in a video, an image set is obtained, including: Based on the information content score and frame number of each frame in the video, a relationship curve is plotted. From the relationship curve, obtain all frame numbers whose information content score value is the peak value; It is determined that the image set contains images corresponding to all the frame numbers.

4. The method as described in claim 3, characterized in that, Before obtaining all frame numbers whose information content score value is the peak value, the method further includes: The relationship curve is subjected to noise reduction processing to make it smooth.

5. The method as described in claim 4, characterized in that, The noise reduction process includes: Fourier transform filtering or wavelet filtering.

6. A target detection method, characterized in that, The method includes: Acquire the image to be detected; The image to be detected is input into the trained detection model to obtain the detection result. The detection model is trained based on the training method described in any one of claims 1-5.

7. A training apparatus for a detection model based on active learning, used to perform the method as described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire sample videos; The prediction module is used to input the sample video into the detection model Mi to obtain the detection result Ni of each frame in the sample video, and to score the detection result Ni according to the active learning algorithm to obtain the information content score value Vi of each frame in the sample video. The detection model is used to identify the target object, and the information content score value of any frame is used to indicate the confidence of the detection result of the target object in any frame, where i is a positive integer greater than or equal to 1. The processing module is configured to obtain an image set Ci based on the information content score Vi of each frame of the sample video, wherein the image set Ci includes images in the sample video whose information content score Vi is the peak value; The training module is used to train the detection model Mi based on the image set Ci to obtain the detection model Mi+1; The prediction module is used to input the sample video into the detection model Mi+1 to obtain the detection result Ni+1 of each frame in the sample video, and to score the detection result Ni+1 according to the active learning algorithm to obtain the information content score value Vi+1 of each frame in the sample video. The detection result is used to indicate the predicted position and predicted category of the target object in each frame of the sample video. The processing module is used to obtain an image set Ci+1 based on the information content score Vi+1 of each frame of the sample video, wherein the image set Ci+1 includes images in the sample video whose information content score Vi+1 is the peak value; and to determine a detection model Mi+2 based on the image set Ci and the image set Ci+1.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images