Supervised moving target retrieval method and system for limited training data and storage medium
By extracting the common features of motion targets in the feature network and using the loss function for category to judge, the problem of misunderstanding of deep learning under limited training data is solved, and the accuracy of exercise target retrieval is improved.
Patent Information
- Application Number
- CN202411352561.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-26
- Publication Date
- 2025-01-21
AI Technical Summary
With limited training data, existing deep learning methods are difficult to effectively distinguish different categories of sports targets, resulting in false detection.
Through the characteristic network, the different exercise time of the same exercise target is extracted, learn the common features in different motion states, and use the loss function of the computing feature similarity to make the category judgment.
It reduces the dependence of deep learning on the amount of data, improves the accuracy of moving target retrieval in the case of a small amount of data, and reduces the false detection rate.
Smart Images
Figure CN119336929A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of deep learning, and relates to a supervised moving target retrieval method, system and storage medium for limited training data. Background Art
[0002] Image retrieval based on deep learning feature matching is superior to retrieval using traditional feature matchers (such as Template Matching, BFMatching, FLANN, etc.) in terms of accuracy and generalization. Traditional feature matcher retrieval often cannot match when the target moves (such as the front and side of a vehicle, pedestrians turning around, etc.), while deep learning can track each operation and parameter through the forward propagation and back propagation of the model, and build a calculation graph to record the relationship between them, which is more conducive to learning the rules hidden behind the data, so as to find similar targets for matching.
[0003] Most of the current deep learning methods are highly dependent on the amount of data and need to be trained based on a large amount of data. In some scenarios with small amounts of data, similar targets often lead to similar network feature extraction, and different target categories have small and uneven category intervals in the network output expression, which leads to false detection. Summary of the invention
[0004] Therefore, in order to solve the above-mentioned shortcomings, the present invention provides a supervised moving target retrieval method, system and storage medium for limited training data; the present invention provides assistance for moving target retrieval in the case of small amount of data, and reduces the dependence of deep learning on the amount of data during target search.
[0005] In one aspect, the present invention provides a supervised moving object retrieval method for limited training data, the method comprising the following steps: Get the image of the moving target and pass it into the feature network; In the feature network, the same moving target is established as a category regardless of the changes caused by its movement, and features are extracted at different movement moments of the moving target, so that the feature network can learn the common features under different movement states and merge them into the same category of objects; The similarity between the target and other matched targets is calculated by calculating the loss function of feature similarity, and the network is used to converge the loss to determine whether they are of the same class. Optionally, the features of the same target in different motion states are made more similar, that is, they are of the same class, and the features of different targets are not affected by motion changes and thus appear to be more distant.
[0006] Specifically, the method of using the network to converge the loss to determine whether it is the same class is: Set the loss function's boundary value for the category; According to this defined value, the loss function converges to it; During reasoning, if the judgment value is less than the defined value, they are the same type of moving target; if the judgment value is greater than the defined value, they are not the same type of moving target; The judgment value is a cosine angle or an Euclidean distance, wherein the loss function corresponding to the cosine angle is a cosine familiarity loss function, and the loss function corresponding to the Euclidean distance is a Euclidean distance loss function.
[0007] Optionally, the supervised moving target retrieval method for limited training data also includes Feature limiting and color mean limiting are performed in post-processing of network inference.
[0008] Optionally, the feature restriction is specifically manifested as calculating the target with the highest similarity and checking whether it is within the range of the Euclidean distance restriction or the angle restriction of the same category.
[0009] Optionally, the color mean limits are: Perform secondary screening on the targets within the feature restriction range, take the color mean of the three channels r, g, and b in turn, and limit the mean range according to the background ratio of the entire training data set (for example, if the background ratio of vehicle data is less than that of pedestrian data, the mean range of vehicle data will be smaller than that of pedestrians. For example, the mean of vehicles may be within ±10 of the target, while that of pedestrians may be within ±15. The specific value is related to the fit and the background ratio).
[0010] Optionally, a method for obtaining an image of a moving target is to use a detection network or manually select and crop the moving target, and the cropping process controls the fit of the moving target.
[0011] In the second aspect, a supervised moving object retrieval system for limited training data comprises The target module is used to obtain the image of the moving target and pass it into the feature network; The classification module is used to classify the same moving target as a category in the feature network regardless of the changes caused by its movement, extract features of the moving target at different movement moments, and let the feature network learn the common features under different movement states and classify them into the same category of objects; The discriminant module is used to calculate the similarity between the target and other matched targets by calculating the loss function of feature similarity, and use the network to converge the loss to determine whether they are of the same category.
[0012] In a third aspect, a storage medium stores a computer program thereon, characterized in that the program is executed by a processor to implement the supervised moving target retrieval method for limited training data.
[0013] The present invention has the following advantages: The present invention classifies each moving target in the case of a small amount of data, establishes the same moving target as one category regardless of the changes caused by its movement, extracts features of the target at different movement moments, and allows the network to learn the common features under different movement states and merge them into the same category of objects, performs post-processing on subsequent network reasoning, and screens out false detections caused by small and uneven intervals between categories due to too few data sets. It can provide assistance for moving target retrieval in the case of a small amount of data, and reduce the dependence of deep learning on the amount of data when searching for targets. For example, the present invention can be used for vehicle and pedestrian detection of drones and image comparison, retrieval of target people or vehicles, and even target recapture. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 It is a flowchart of the supervised moving target retrieval method for limited training data according to the present invention; Figure 2 It is a systematic block diagram of the supervised moving object retrieval system for limited training data according to the present invention; In the figure: 100, target module; 200, classification module; 300, discrimination module; 400, post-processing module. DETAILED DESCRIPTION
[0015] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0016] In this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises", or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article, or apparatus.
[0017] As described in the background technology, image retrieval based on deep learning feature matching is superior to retrieval using traditional feature matchers (such as Template Matching, BFMatching, FLANN, etc.) in terms of accuracy and generalization. Traditional feature matcher retrieval often cannot match when the target moves (such as the front and side of a vehicle, pedestrians turning around, etc.), while deep learning can track each operation and parameter through the forward propagation and back propagation of the model, and build a calculation graph to record the relationship between them, which is more conducive to learning the rules hidden behind the data, so as to find similar targets for matching.
[0018] Most of the current deep learning methods are highly dependent on the amount of data and need to be trained based on a large amount of data. In some scenarios with small amounts of data, similar targets often lead to similar network feature extraction, and different target categories have small and uneven category intervals in the network output expression, which leads to false detection.
[0019] Based on the above problems, this embodiment provides a supervised moving target retrieval method for limited training data, such as Figure 1 As shown, the method comprises the following steps: Step S100, obtaining an image of a moving target and passing it into a feature network; Step S200, in the feature network, the same moving target is established as a category regardless of the changes caused by its movement, and features are extracted at different movement moments of the moving target (i.e., picture frames of the target's movement changes at different moments are input), so that the feature network learns the common features under different movement states and merges them into the same category of objects; Step S300, by calculating the loss function of feature similarity (such as cosine similarity, Euclidean distance and other loss functions that use angle information and position information), the similarity between the target (which can be understood as the target target) and other matched targets is calculated pairwise, and the network is used to converge the loss to determine whether they are of the same category, so that the features of the same target in its different motion states are more similar, that is, they are of the same category, and the features of non-same targets are not affected by motion changes and thus appear to be more distant.
[0020] The matched target is a moving target. Target and its features of the same category and Target and its features of different categories are all matched targets. A brief description is all the targets in the data waiting to be compared with target.
[0021] The above technical features classify each moving target in the case of a small amount of data, establish the same moving target as a category regardless of the changes caused by its movement, extract features of the target at different movement moments, and let the network learn the common features in different motion states and merge them into the same category of objects. This can help with the retrieval of moving targets in the case of a small amount of data and reduce the dependence of deep learning on the amount of data when searching for targets.
[0022] In one embodiment, the method of using the network to converge the loss to determine whether they are of the same type is: Set the loss function's boundary value for the category; According to this bounded value, the loss function converges to it; the convergence of the loss function to it refers to the convergence of the calculated output and the given label (that is, the meaning of the convergence of the loss function itself), which is mainly to normalize the calculated output values of the same type of targets to within the range of the bounded value, and the output values of different targets to outside the range of the bounded value. For specific reference, please refer to centerloss, contrastiveloss, etc.
[0023] During reasoning, if the judgment value is less than the defined value, they are the same type of moving target; if the judgment value is greater than the defined value, they are not the same type of moving target; The judgment value is a cosine angle or an Euclidean distance, wherein the loss function corresponding to the cosine angle is a cosine familiarity loss function, and the loss function corresponding to the Euclidean distance is a Euclidean distance loss function.
[0024] Since the detection network inevitably has some background factors, in one embodiment, the supervised moving target retrieval method for limited training data also includes: Feature limiting and color mean limiting are performed in post-processing of network inference.
[0025] Exemplarily, the feature restriction is specifically manifested as calculating the target with the highest similarity and checking whether it is within the range of the Euclidean distance restriction or the angle restriction of the same category.
[0026] Exemplary color mean limits are: Perform secondary screening on the targets within the feature restriction range, take the color mean of the three channels r, g, and b in turn, and limit the mean range according to the background ratio of the entire training data set (for example, if the background ratio of vehicle data is less than that of pedestrian data, the mean range of vehicle data will be smaller than that of pedestrians. For example, the mean of vehicles may be within ±10 of the target, while that of pedestrians may be within ±15. The specific value is related to the fit and the background ratio).
[0027] In one embodiment, a method for acquiring an image of a moving target is to use a detection network (such as YOLO, SDD, RCN, etc.) or manual crop to select and crop the moving target, requiring that the degree of fit with the moving target during the cropping process is as high as possible to eliminate the impact of background changes on feature extraction.
[0028] The above technical features solve the problem of false detection caused by poor convergence of each category in distance or angle due to a small data set. Double restrictions are also more conducive to reducing false detection when background differences are eliminated. It can realize post-processing of subsequent network reasoning and eliminate false detection caused by small and uneven intervals between categories due to too few data sets.
[0029] In another embodiment, a supervised moving object retrieval system for limited training data includes: The target module 100 is used to obtain the image of the moving target and pass it into the feature network; The classification module 200 is used to establish the same moving object as a category in the feature network regardless of the changes caused by its movement, extract features of the moving object at different movement moments, and let the feature network learn the common features under different movement states and classify them into the same category of objects; The discrimination module 300 is used to calculate the similarity between the target and other matched targets by calculating the loss function of feature similarity, and use the network to converge the loss to determine whether they are of the same category. The post-processing module 400 is used to perform feature restriction and color mean restriction in the post-processing of network inference.
[0030] The above technical features are based on the supervised moving target retrieval method for limited training data, which can provide assistance for moving target retrieval in the case of a small amount of data, and reduce the dependence of deep learning on the amount of data during target search.
[0031] It should be noted that, in the above embodiments, the description of each embodiment has its own focus, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0032] It should be understood by those skilled in the art that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program codes.
[0033] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0034] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0035] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0036] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the above-mentioned supervised moving target retrieval method for limited training data.
[0037] If the module / unit integrated in the supervised moving target retrieval device / terminal device for limited training data is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.
[0038] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer readable media do not include electric carrier signals and telecommunication signals.
[0039] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0040] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0041] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0042] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application. The above embodiments are only used to illustrate the technical solution of the present application, not to limit it; although the present application is described in detail with reference to the above embodiments, ordinary technicians in this field should understand that: it is still possible to modify the technical solutions recorded in the above embodiments, or to replace some of the technical features therein by equivalent; and these modifications or replacements do not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of each embodiment of the present application.
[0043] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A supervised moving object retrieval method for limited training data, characterized in that: The steps include: Get the image of the moving target and pass it into the feature network; In the feature network, the same moving target is established as a category regardless of the changes caused by its movement, and features are extracted at different movement moments of the moving target, so that the feature network can learn the common features under different movement states and merge them into the same category of objects; The similarity between the target and other matched targets is calculated pairwise by calculating the loss function of feature similarity, and the network is used to converge the loss to determine whether they are of the same category.
2. A supervised moving target retrieval method for limited training data according to claim 1, characterized in that: The characteristics of the same target in different motion states are made more similar, that is, they are of the same category. The characteristics of different targets are not affected by motion changes and thus appear to be more distant.
3. A supervised moving object retrieval method for limited training data according to claim 2, characterized in that: The method of using the network to converge the loss to determine whether it is the same class is: Set the loss function's boundary value for the category; According to this bounded value, the loss function converges to itself; During reasoning, if the judgment value is less than the defined value, they are the same type of moving target; if the judgment value is greater than the defined value, they are not the same type of moving target; The judgment value is a cosine angle or an Euclidean distance, wherein the loss function corresponding to the cosine angle is a cosine familiarity loss function, and the loss function corresponding to the Euclidean distance is a Euclidean distance loss function.
4. A supervised moving object retrieval method for limited training data according to claim 1, characterized in that: Also includes Feature limiting and color mean limiting are performed in post-processing of network inference.
5. A supervised moving object retrieval method for limited training data according to claim 4, characterized in that: The feature restriction is specifically manifested in calculating the target with the highest similarity and checking whether it is within the range of the Euclidean distance restriction or the angle restriction of the same category.
6. A supervised moving object retrieval method for limited training data according to claim 4, characterized in that: The color mean limits are: The targets within the feature restriction range are screened twice, and the color mean of the three channels r, g, and b are taken in turn, and the mean range is limited according to the background ratio of the overall training data set.
7. A supervised moving object retrieval method for limited training data according to claim 4, characterized in that: The method for obtaining an image of a moving target is to use a detection network or manually select and crop the moving target, and the cropping process controls the fit of the moving target.
8. A supervised moving object retrieval system for limited training data, characterized by: include The target module is used to obtain the image of the moving target and pass it into the feature network; The classification module is used to classify the same moving target as a category in the feature network regardless of the changes caused by its movement, extract features of the moving target at different movement moments, and let the feature network learn the common features under different movement states and classify them into the same category of objects; The discriminant module is used to calculate the similarity between the target and other matched targets by calculating the loss function of feature similarity, and use the network to converge the loss to determine whether they are of the same category.
9. A storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement a supervised moving target retrieval method for limited training data as described in any one of claims 1 to 7.