Method and device for detecting interested target
By selecting the resolution and subnet depth of the multi-level interest object detection model sequentially, the problem that the Motion Detection module of the battery-type camera cannot effectively judge the interest object is solved, which improves detection efficiency and accuracy and reduces power consumption.
Patent Information
- Application Number
- CN202310864478.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-14
- Publication Date
- 2025-01-14
AI Technical Summary
The Motion Detection module of existing battery-type cameras cannot effectively determine whether the moving objects in the screen are targets of interest, resulting in false detection and frequent calls to increase the power consumption of the entire machine.
Using a multi-level object detection model of interest, the image and subnet are selected in sequence from low to high resolution and subnet depth for detection. If the object of interest is detected by either resolution and subnet, the process will be completed, otherwise the process will be continued. High resolution and deeper subnets for detection.
The balance between detection effect and calculation consumption improves the efficiency of the target detection of interest and reduces false detection and power consumption.
Smart Images
Figure CN119313863A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for detecting an object of interest and a method and device for establishing an object of interest detection model. Background Art
[0002] Battery-powered cameras have gradually entered people's field of vision due to their excellent portability and easy deployment. In most cases, the videos recorded by cameras are not of interest to people, and long-term recording is also a huge consumption of the camera's battery.
[0003] Existing battery-powered camera solutions generally use an MD (Motion Detection) module to determine whether there are any moving objects in the current picture. If a moving object is found, the entire system is called. However, the MD module cannot determine whether the current moving object is a target of interest. For example, false detections may occur when the wind blows flowers, plants, and trees, or when light changes. If the entire system is called frequently, the power consumption of the entire machine will increase. Summary of the invention
[0004] The embodiments of the present invention provide a method and device for detecting an object of interest and a method and device for establishing an object of interest detection model, so as to improve the efficiency of detecting an object of interest.
[0005] The technical solution of the embodiment of the present invention is achieved as follows:
[0006] A method for detecting an object of interest, the method comprising:
[0007] A1. From among the preset multiple levels of resolution, select one resolution for the current image to be detected in order from low to high as the current resolution; from among the multiple levels of subnets of the trained object detection model of interest, select one subnet in order from low to high as the current subnet; wherein each level of the subnet of the object detection model of interest contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules;
[0008] B1. Obtain the image to be detected of the current resolution of the current picture to be detected, input the image to be detected of the current resolution into the current subnet, obtain the current interesting target detection result, and determine whether the current interesting target detection result indicates that there is an interesting target in the current picture to be detected. If so, end this process; otherwise, return to step A1.
[0009] The subnetworks at multiple levels of the object detection model of interest satisfy:
[0010] For two subnets at adjacent levels, the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended from the higher-level subnet.
[0011] Step B1, inputting the image to be detected at the current resolution into the current subnet, includes:
[0012] The image to be detected of the current resolution is input into the current subnet, and the features extracted by the subnet of the previous level to which the current subnet belongs are input into the current subnet.
[0013] When the subnet of the previous level of the level to which the current subnet belongs contains multiple feature extraction modules,
[0014] The step of inputting the features extracted from the subnet of the previous level to which the current subnet belongs into the current subnet comprises:
[0015] The features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs are respectively input into the feature extraction module at the same position of the current subnet.
[0016] After the features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs are respectively input into the feature extraction module at the same position of the current subnet, the method further comprises:
[0017] For any feature extraction module in the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the feature extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the feature is upsampled, and the upsampled feature is fused with the feature from the current subnet or with the image to be detected input into the current subnet.
[0018] When there are multiple feature extraction modules extended by the higher-level subnet,
[0019] The determining whether the current interesting target detection result indicates that the interesting target exists in the current to-be-detected picture includes:
[0020] When the last non-extended feature extraction module or any extended feature extraction module in the higher-level subnet outputs an extracted feature, it is determined whether the feature indicates that there is an object of interest in the current image to be detected;
[0021] And, if so, end this process, otherwise, continue to input the feature into the next feature extraction module in the higher-level subnet, or, if the feature extraction module is the last feature extraction module of the higher-level subnet, return to step A1.
[0022] A method for establishing a target of interest detection model, the method comprising:
[0023] A2. Obtain a set of training sample images, wherein each training picture corresponds to multiple frames of training sample images, and the resolution of each frame of the training sample image is one of the preset multiple levels of resolution, and obtain a true detection result of the target of interest for each frame of the training sample image;
[0024] B2. Select one resolution from the preset multiple levels of resolutions in order from low to high as the current resolution; select one subnet from the multiple levels of subnets of the target detection model to be trained in order from low to high as the current subnet; wherein each level of the subnet of the target detection model contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules;
[0025] C2. Select training sample images in turn from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image;
[0026] D2, calculating the loss function based on the predicted detection result of the target of interest of the current training sample image and the actual detection result of the target of interest, updating the neural network parameters of the current subnet based on the loss function, and returning to step C2 until the current subnet converges;
[0027] E2. Return to step B2 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.
[0028] The subnetworks at multiple levels of the object detection model of interest satisfy:
[0029] For two subnets at adjacent levels, the higher-level subnet is composed of: the feature extraction module of the lower-level subnet and the feature extraction module extended by the higher-level subnet;
[0030] Step C2, inputting the selected training sample image into the current subnet, includes:
[0031] The selected training sample image is input into the current subnet, and at the same time, the features of the same training picture extracted by the subnet of the previous level to which the current subnet belongs are input into the current subnet, wherein the features of the same training picture are: the features of the training sample image of the previous resolution of the training picture corresponding to the selected training sample image.
[0032] An interesting object detection device, the device comprising:
[0033] A preparation and judgment module, used to select one resolution for the current image to be detected in sequence from low to high among multiple levels of preset resolutions as the current resolution; select one subnet in sequence from low to high among multiple levels of subnets of the trained object detection model of interest as the current subnet; wherein each level of the subnet of the object detection model of interest includes at least one feature extraction module, and the higher the level of the subnet, the more feature extraction modules it includes; and, determine whether the current object detection result output by the current subnet indicates that the current image to be detected has an object of interest, if so, end the current object detection process of interest, otherwise, return to the action of selecting one resolution for the current image to be detected in sequence from low to high among multiple levels of preset resolutions as the current resolution;
[0034] The detection module is used to input the image to be detected at the current resolution into the current subnet according to the current resolution and current subnet selected by the preparation and judgment module, and send the detection result of the target of interest output by the current subnet to the preparation and judgment module.
[0035] A device for establishing a model for detecting an object of interest, the device comprising:
[0036] A training preparation module is used to obtain a set of training sample images, wherein each training picture corresponds to multiple frames of training sample images, and the resolution of each frame of the training sample image is one of the preset multiple levels of resolution, and obtain the real detection result of the target of interest of each frame of the training sample image;
[0037] The training module is used to perform B2 to E2 as follows:
[0038] B2. Select one resolution from the preset multiple levels of resolutions in order from low to high as the current resolution; select one subnet from the multiple levels of subnets of the target detection model to be trained in order from low to high as the current subnet; wherein each level of the subnet of the target detection model contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules;
[0039] C2. Select training sample images in turn from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image;
[0040] D2, calculating the loss function based on the predicted detection result of the target of interest of the current training sample image and the actual detection result of the target of interest, updating the neural network parameters of the current subnet based on the loss function, and returning to step C2 until the current subnet converges;
[0041] E2. Return to step B2 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.
[0042] In an embodiment of the present invention, for the same picture, images and subnets for detecting the target of interest are selected in sequence according to the resolution of the image of the picture from low to high and the depth of the subnet of the target of interest detection model from shallow to deep (i.e., the number of feature extraction modules contained in the subnet from few to many). If the target of interest is detected by an image of any resolution and a subnet of any level, the target of interest detection process is terminated. Otherwise, a higher resolution image and a deeper subnet are continued to be used for target of interest detection, thereby achieving a better balance between detection effect (such as detection accuracy and / or detection speed) and computational consumption, and improving the efficiency of target of interest detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0044] Figure 1 A flowchart of a method for detecting an object of interest provided by an embodiment of the present invention;
[0045] Figure 2 A schematic diagram of the structure of an object of interest detection model used in an application example of the present invention;
[0046] Figure 3 A flowchart of a method for establishing a detection model of an object of interest provided by an embodiment of the present invention;
[0047] Figure 4 A schematic diagram of the structure of an object of interest detection device provided by an embodiment of the present invention;
[0048] Figure 5 A schematic diagram of the structure of an apparatus for establishing a detection model of an object of interest provided in one embodiment of the present invention. DETAILED DESCRIPTION
[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0050] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0051] The inventor of the present application has found that the segmentation method based on input text can segment the area of interest by inputting words or phrases such as "car" or "blue sky". This method can improve the flexibility and interpretability of the segmentation method, because people only pay attention to specific categories in many scenarios. The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0052] Figure 1 This is a flowchart of a method for detecting an object of interest provided by an embodiment of the present invention, and the specific steps are as follows:
[0053] Step 101: Among the preset multiple levels of resolution, select one resolution for the current image to be detected in order from low to high as the current resolution; among the multiple levels of subnets of the trained object detection model of interest, select one subnet in order from low to high as the current subnet; wherein each level of the subnet of the object detection model of interest contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules.
[0054] When a subnet contains multiple feature extraction modules, the multiple feature extraction modules are connected in series, that is, the output of the previous feature extraction module serves as the input of the next feature extraction module.
[0055] Step 102: Obtain the image to be detected of the current resolution of the current picture to be detected, input the image to be detected of the current resolution into the current subnet, obtain the current detection result of the target of interest, and determine whether the current detection result of the target of interest indicates that there is a target of interest in the current picture to be detected. If so, end this process; otherwise, return to step 101.
[0056] Figure 2 FIG. 1 is a schematic diagram of the structure of the object detection model used in an application example of the present invention. Figure 2 As shown, the object detection model of interest includes three levels of subnets, among which the first-level subnet includes two feature extraction modules: feature extraction modules A and B, the second-level subnet includes four feature extraction modules: feature extraction modules A, B, C, and D, and the third-level subnet includes six feature extraction modules: feature extraction modules A, B, C, D, E, and F. That is, the first-level subnet has the lowest level, the second-level subnet is the second, and the third-level subnet has the highest level. Three levels of resolution are preset: 56*56, 112*112, and 224*224, among which, the lower the resolution, the lower the level. 56*56 is the first-level resolution, 112*112 is the second-level resolution, and 224*224 is the third-level resolution. The image with the first-level resolution is input to the feature extraction module A of the first-level subnet, the image with the second-level resolution is input to the feature extraction module A of the second-level subnet, and the image with the third-level resolution is input to the feature extraction module A of the third-level subnet. For a picture to be detected, the process of object detection of interest is as follows:
[0057] First, obtain the first-level resolution corresponding to the picture to be detected: 56*56 image, then input the 56*56 image to be detected into the first-level subnet of the trained target detection model of interest, and judge whether the target detection result output by the first-level subnet indicates the existence of the target of interest. If so, it is determined that the current picture to be detected has the target of interest, and the process is terminated. Otherwise, obtain the second-level resolution corresponding to the picture to be detected: 112*112 image, then input the 112*112 image to be detected into the second-level subnet of the trained target detection model of interest, and judge whether the target detection result output by the second-level subnet indicates the existence of the target of interest. If so, it is determined that the current picture to be detected has the target of interest, and the process is terminated. If not, obtain the third-level resolution corresponding to the picture to be detected: 224*224 image, then input the 224*224 image to be detected into the third-level subnet of the trained target detection model of interest, and judge whether the target detection result output by the third-level subnet indicates the existence of the target of interest. If so, it is determined that the current picture to be detected has the target of interest. Otherwise, it is determined that the current picture to be detected does not have the target of interest.
[0058] In the above embodiment, for the same picture, images and subnets for detecting the target of interest are selected in sequence according to the resolution of the image of the picture from low to high and the depth of the subnet of the target of interest detection model from shallow to deep (i.e., the number of feature extraction modules contained in the subnet from few to many). If the target of interest is detected by an image of any resolution and a subnet of any level, the target of interest detection process is terminated. Otherwise, a higher resolution image and a deeper subnet are continued to be used for target of interest detection, thereby achieving a better balance between detection effect (such as detection accuracy and / or detection speed) and computational consumption, and improving the efficiency of target of interest detection.
[0059] In an optional embodiment, multiple levels of subnets of the target of interest detection model satisfy: for two subnets of adjacent levels, the subnet with a higher level among the two subnets is called a higher-level subnet, and the subnet with a lower level among the two subnets is called a lower-level subnet, then the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended from the higher-level subnet.
[0060] like Figure 2 The target detection model of interest shown in the figure, wherein the first-level subnet and the second-level subnet are two subnets of adjacent levels, among which the second-level subnet is a higher-level subnet and the first-level subnet is a lower-level subnet, and the first-level subnet contains two feature extraction modules: feature extraction modules A and B; the second-level subnet contains four feature extraction modules, then the first two of these four feature extraction modules are feature extraction modules A and B in the first-level subnet, and the last two are newly added feature extraction modules C and D; the second-level subnet and the third-level subnet are two subnets of adjacent levels, among which the third-level subnet is a higher-level subnet and the second-level subnet is a lower-level subnet, and the third-level subnet contains six feature extraction modules, then the first four of these six feature extraction modules are feature extraction modules A, B, C, and D in the second-level subnet, and the last two are newly added feature extraction modules E and F.
[0061] In an optional embodiment, in step 102, inputting the image to be detected at the current resolution into the current subnet includes: inputting the image to be detected at the current resolution into the current subnet, and simultaneously inputting features extracted from a subnet of a previous level to which the current subnet belongs into the current subnet.
[0062] like Figure 2 The object detection model of interest shown in the figure, wherein the input of the secondary subnetwork includes not only the image to be detected with the secondary resolution, but also the features extracted by the primary subnetwork; the input of the tertiary subnetwork includes not only the image to be detected with the tertiary resolution, but also the features extracted by the secondary subnetwork.
[0063] In an optional embodiment, when a subnet of a level previous to the level to which the current subnet belongs contains multiple feature extraction modules, in step 102, the features extracted by the subnet of a level previous to the level to which the current subnet belongs are input into the current subnet, including: inputting the features extracted by each feature extraction module of the subnet of a level previous to the level to which the current subnet belongs into the feature extraction modules at the same position of the current subnet.
[0064] like Figure 2 The object detection model of interest shown in the figure, the feature extraction module A of the first-level subnet extracts the feature F A1 Output to the feature extraction module A of the secondary subnet, feature F extracted by the feature extraction module B of the primary subnet B1 Output to the feature extraction module B of the secondary subnet; feature F extracted by the feature extraction module A of the secondary subnet A2 Output to the feature extraction module A of the third-level subnet, feature F extracted by the feature extraction module B of the second-level subnet B2 Output to the feature extraction module B of the third-level subnet, feature F extracted by the feature extraction module C of the second-level subnet C2 Output to the feature extraction module C of the third-level subnet, and the feature extraction module D of the second-level subnet extracts the feature F D2 Output to the feature extraction module D of the three-level subnetwork.
[0065] In an optional embodiment, the "feature extraction modules at the same position" in different subnets refer to the feature extraction modules at the same depth position in different subnets, such as: feature extraction modules A and B in the first, second and third level subnets, and feature extraction modules C and D in the second and third level subnets.
[0066] In an optional embodiment, in step 102, after the features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs are respectively input into the feature extraction module at the same position of the current subnet, it further includes: for any feature extraction module in the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the features extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the features are upsampled, and the upsampled features are fused with the features from the current subnet or with the image to be detected input into the current subnet.
[0067] like Figure 2 The object detection model shown in the figure, when the feature extraction module A of the secondary subnet receives the image to be detected with a secondary resolution of 112*112 and the feature F sent by the feature extraction module A of the primary subnet A1 When A1 Upsample and get feature F after upsampling A1 ', where F A1'The size is 112*112, and then the feature extraction module A of the secondary subnet is used for F A1 ' and the 112*112 image to be detected to obtain feature F A2 Similarly, when the feature extraction module B of the secondary subnet receives the feature F output by the feature extraction module A of the secondary subnet A2 and feature F sent by feature extraction module B of the first-level subnet B1 When B1 Upsample and get feature F after upsampling B1 ', where F B1 The size of F A2 The same, then the feature extraction module B of the secondary subnet is used for F B1 ' and F A2 Perform feature fusion to obtain feature F B2 Similarly, the feature extraction modules A, B, C, and D of the third-level subnet also need to perform the same upsampling and feature fusion process as the feature extraction modules A and B of the second-level subnet.
[0068] In practical applications, in step 102, after upsampling the feature, the upsampled feature may be convolved in 1*1 manner, and then the convolved feature may be fused with the feature from the current subnet or with the image to be detected input into the current subnet.
[0069] In the above embodiment, each level of subnet absorbs the features extracted by the previous level of subnet and fuses the features of different scales, thereby improving the accuracy of detecting the object of interest.
[0070] In an optional embodiment, when there are multiple feature extraction modules extended in the higher-level subnet, in step 102, it is determined whether the current target of interest detection result indicates that there is a target of interest in the current screen to be detected, including: when the last non-extended feature extraction module or any extended feature extraction module in the higher-level subnet outputs an extracted feature, it is determined whether the feature indicates that there is a target of interest in the current screen to be detected; and, if so, the present process is terminated, otherwise, the feature is continued to be input into the next feature extraction module in the higher-level subnet, or, if the feature extraction module is the last feature extraction module of the higher-level subnet, the process returns to step 101.
[0071] Still Figure 2 The object detection model shown illustrates the above process:
[0072] When the last non-extended feature extraction module in the secondary subnet: feature extraction module B outputs feature F B2 When F B2Whether it indicates that there is an interesting target, if so, it is determined that there is an interesting target in the current image to be detected, and the process ends, otherwise, F B2 Input the feature extraction module C of the secondary subnet;
[0073] When the feature extraction module C of the secondary subnet outputs the feature F C2 When F C2 Whether it indicates that there is an interesting target, if so, it is determined that there is an interesting target in the current image to be detected, and the process ends, otherwise, F C2 Input the feature extraction module D of the secondary subnet;
[0074] When the feature extraction module D of the secondary subnet outputs the feature F D2 When F D2 Whether it indicates that there is an object of interest, if so, it is determined that there is an object of interest in the current image to be detected, and the process ends; if not, the image to be detected with the third-level resolution continues to be input into the third-level subnet;
[0075] Afterwards, when the last non-extended feature extraction module in the three-level subnet: feature extraction module D outputs feature F D3 When F D3 Whether it indicates that there is an interesting target, if so, it is determined that there is an interesting target in the current image to be detected, and the process ends, otherwise, F D3 Input the feature extraction module E of the three-level subnet;
[0076] When the feature extraction module E of the three-level subnet outputs the feature F E When F E Whether it indicates that there is an interesting target, if so, it is determined that there is an interesting target in the current image to be detected, and the process ends, otherwise, F E Input the feature extraction module F of the three-level subnet;
[0077] When the feature extraction module F outputs the feature F F When F F Whether it indicates that there is an object of interest, if so, it is determined that there is an object of interest in the current picture to be detected, otherwise, it is determined that there is no object of interest in the current picture to be detected.
[0078] In practical applications, judging whether the feature indicates that there is an object of interest in the current screen to be detected includes: calculating the probability that the object of interest exists in the current screen to be detected based on the feature, judging whether the probability is greater than a preset threshold, and if so, determining that there is an object of interest in the current screen to be detected. Specifically, for example: performing global average pooling processing on the feature to obtain a global average pooling value, performing sigmoid calculation on the global average pooling value to obtain the probability that there is an object of interest in the current screen to be detected, judging whether the probability is greater than a preset threshold, and if so, determining that there is an object of interest in the current screen to be detected. Among them, the specific value of the preset threshold can be determined based on experience, etc.
[0079] The sigmoid calculation formula is as follows:
[0080]
[0081] Among them, X is the global average pooling value of the feature; S(X) is the sigmoid calculation result, that is, the probability that the target of interest exists in the current image to be detected.
[0082] In an optional embodiment, the feature extraction module includes: one or any combination of a convolution module, a residual module, an inverse residual module, and a depth separable module. In actual use, the most suitable network structure of the target detection model of interest can be searched based on the performance of the chip used in the current target detection task of interest, the task computing power and the parameter quantity limitation, and combined with Neural Architecture Search (NAS).
[0083] The following adopts Figure 2 The detailed steps of the object detection process when the object detection model is shown:
[0084] Step 01: Get the first-level resolution corresponding to the image to be detected: 56*56 image, then input the 56*56 image to be detected into the first-level subnet of the trained object detection model of interest. When the feature extraction module A of the first-level subnet outputs feature F A1 When , save the feature F A1 , and the feature F A1 Output to the feature extraction module B of the first-level subnet. When the feature extraction module B of the first-level subnet outputs feature F B1 When the feature F B1 Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater, it is determined that there is an object of interest in the image to be detected, and the process ends; otherwise, save the feature F B1 , and go to step 02.
[0085] Step 02: Get the secondary resolution of the image to be detected: 112*112, then input the 112*112 image to be detected into the secondary subnet of the trained object detection model, and save the feature F A1 Output to the feature extraction module A of the secondary subnet, the feature extraction module A of the secondary subnet is A1 Upsample to get feature F of size 112*112 A1 ', and the feature extraction module A of the secondary subnet is A1 ' and the 112*112 image to be detected to obtain feature F A2 , save feature F A2 , and the feature F A2 Output to the feature extraction module B of the secondary subnet, and at the same time, the saved feature F B1 Output to the feature extraction module B of the secondary subnet;
[0086] The feature extraction module B of the secondary subnet is F B1 Upsample to get feature F B1 ', F B1 The size of F A2 The same, and the feature extraction module B of the secondary subnet is B1 ' and F A2 Perform feature fusion to obtain feature F B2 , and for the feature F B2 Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater, it is determined that there is an object of interest in the image to be detected, and the process ends. Otherwise, save the feature F B2 , and the feature F B2 Output to the feature extraction module C of the secondary subnet;
[0087] When the feature extraction module C of the secondary subnet outputs the feature F C2 When the feature F C2 Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater, it is determined that there is an object of interest in the image to be detected, and the process ends. Otherwise, save the feature F C2 , and the feature F C2 Output to the feature extraction module D of the secondary subnet;
[0088] When the feature extraction module D of the secondary subnet outputs the feature F D2 When the feature F D2Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater, it is determined that there is an object of interest in the image to be detected, and the process ends. Otherwise, save the feature F D2 , and go to step 03.
[0089] Step 03: Get the third-level resolution corresponding to the image to be detected: 224*224, then input the 224*224 image to be detected into the third-level subnet of the trained object detection model of interest, and save the feature F A2 Output to the feature extraction module A of the three-level subnet, the feature extraction module A of the three-level subnet is A2 Upsample to get feature F of size 224*224 A2 ', and the feature extraction module A of the three-level subnet is A2 ' and the 224*224 image to be detected to obtain feature F A3 , the feature F A3 Output to the feature extraction module B of the three-level subnetwork;
[0090] The saved feature F B2 Output to the feature extraction module B of the three-level subnet, the feature extraction module B of the three-level subnet is B2 Upsample to get feature F B2 ', F B2 The size of F A3 The same, and the feature extraction module B of the three-level subnet is F B2 ' and F A3 Perform feature fusion to obtain feature F B3 , F B3 Output to the feature extraction module C of the three-level subnetwork;
[0091] The saved feature F C2 Output to the feature extraction module C of the three-level subnet, the feature extraction module C of the three-level subnet C2 Upsample to get feature F C2 ', F C2 The size of F B3 The same, and the feature extraction module C of the three-level subnet is C2 ' and F B3 Perform feature fusion to obtain feature F C3 , F C3 Output to the feature extraction module D of the three-level subnetwork;
[0092] The saved feature F D2 Output to the feature extraction module D of the three-level subnet, the feature extraction module D of the three-level subnetD2 Upsample to get feature F D2 ', F D2 The size of F C3 The same, and the feature extraction module D of the three-level subnet is F D2 ' and F C3 Perform feature fusion to obtain feature F D3 , for feature F D3 Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater, it is determined that there is an object of interest in the image to be detected, and the process ends. Otherwise, F D3 Output to the feature extraction module E of the three-level subnet;
[0093] When the feature extraction module E of the three-level subnet outputs the feature F E When the feature F E Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than the preset threshold. If it is greater than, it is determined that there is an object of interest in the image to be detected, and the process ends; otherwise, F E Output to the feature extraction module F of the three-level subnet;
[0094] When the feature extraction module F of the three-level subnet outputs the feature F F When the feature F F Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, and determine whether the sigmoid calculation result is greater than a preset threshold. If so, it is determined that there is an object of interest in the picture to be detected, otherwise, it is determined that there is no object of interest in the picture to be detected.
[0095] In an optional embodiment, in step 102, if thereafter, it further includes: if a moving object is detected in the current image to be detected at the same time, waking up the camera's video recording function and / or intelligent analysis function, the intelligent analysis function is for example: human shape detection, face detection, or face recognition.
[0096] Figure 3 The flowchart of the method for establishing a target detection model of interest provided by an embodiment of the present invention is as follows:
[0097] Step 301: Obtain a set of training sample images, wherein each training picture corresponds to multiple frames of training sample images, and the resolution of each frame of training sample images is one of multiple preset levels of resolution, and obtain the actual detection result of the target of interest for each frame of training sample images.
[0098] There are two real interesting target detection results: interesting target detected and interesting target not detected, which can be represented by 1 and 0 respectively.
[0099] Step 302: Among the preset multiple levels of resolution, select one resolution in order from low to high as the current resolution; among the multiple levels of subnets of the target detection model of interest to be trained, select one subnet in order from low to high as the current subnet; wherein each level of the subnet of the target detection model of interest contains at least one feature extraction module, and a subnet with a higher level contains more feature extraction modules.
[0100] Step 303: Select training sample images in sequence from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image.
[0101] Step 304: Calculate the loss function based on the predicted detection result of the target of interest and the actual detection result of the target of interest of the current training sample image, update the neural network parameters of the current subnet based on the loss function, and return to step 303 until the current subnet converges;
[0102] Step 305: Return to step 302 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.
[0103] In the above embodiment, for the same training picture, multiple frames of training sample images are set according to the resolution of the image of the picture from low to high. At the same time, for each level of resolution from low to high, multiple levels of subnets of the target of interest detection model are set from shallow to deep (that is, the feature extraction modules contained in the subnet are from few to many). Then, each level of resolution and the corresponding subnet are trained respectively, and finally an target of interest detection model composed of multiple subnets adapted to different resolutions is obtained, so that the target of interest model can perform detection with different detection effects and different computational consumption on images of different resolutions, and a good balance is achieved between detection effect (such as: detection accuracy and / or detection speed) and computational consumption, thereby improving the efficiency of target of interest detection.
[0104] In an optional embodiment, the subnets of multiple levels of the object detection model of interest satisfy: for two subnets at adjacent levels, the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended from the higher-level subnet.
[0105] In an optional embodiment, in step 303, the selected training sample image is input into the current subnet, including: inputting the selected training sample image into the current subnet, and at the same time inputting the features of the same training picture extracted by the subnet of the previous level of the level to which the current subnet belongs into the current subnet, wherein the features of the same training picture are: the features of the training sample image of the previous resolution of the training picture corresponding to the selected training sample image.
[0106] by Figure 2 Take the object detection model of interest shown as an example:
[0107] For the secondary subnet, a 112*112 training sample image of a training picture is input into the feature extraction module A of the secondary subnet. At the same time, the feature F of the 56*56 training sample image of the same training picture extracted by the feature extraction module A of the primary subnet is input into the feature extraction module A of the secondary subnet. A1 Input the feature extraction module A of the secondary subnet; and when the feature extraction module A of the secondary subnet outputs feature F A2 When F A2 Input the feature extraction module B of the secondary subnet, and at the same time, the feature F output by the feature extraction module B of the primary subnet B1 Input feature extraction module B of the secondary subnet;
[0108] For the third-level subnet, a 224*224 training sample image of a training picture is input into the feature extraction module A of the third-level subnet. At the same time, the feature F of the 112*112 training sample image of the same training picture extracted by the feature extraction module A of the second-level subnet is A2 Input the feature extraction module A of the three-level subnet; and when the feature extraction module A of the three-level subnet outputs feature F A3 When F A3 Input the feature extraction module B of the third-level subnet, and at the same time, the feature F output by the feature extraction module B of the second-level subnet B2 Input the feature extraction module B of the three-level subnet; and when the feature extraction module B of the three-level subnet outputs feature F B3 When F B3 Input the feature extraction module C of the third-level subnet, and at the same time, the feature F output by the feature extraction module C of the second-level subnet C2 Input the feature extraction module C of the three-level subnet; and when the feature extraction module C of the three-level subnet outputs the feature F C3 When F C3 Input the feature extraction module D of the third-level subnet, and at the same time, the feature F output by the feature extraction module D of the second-level subnet D2 Input the feature extraction module D of the three-level subnetwork.
[0109] In an optional embodiment, when a subnet of a level previous to that of the current subnet contains multiple feature extraction modules, in step 303, features of the same training picture extracted by the subnet of a level previous to that of the current subnet are input into the current subnet, including: features of the same training picture extracted by each feature extraction module of the subnet of a level previous to that of the current subnet are respectively input into feature extraction modules at the same position of the current subnet.
[0110] In an optional embodiment, in step 303, after the features of the same training picture extracted by the subnet of the previous level to which the current subnet belongs are input into the current subnet, the method further includes: for any feature extraction module in the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the features of the same training picture extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the features are upsampled, and the upsampled features are feature fused with the features from the current subnet or the training sample images input into the current subnet.
[0111] In an optional embodiment, the feature extraction module includes: one or a combination of any multiple of a convolution module, a residual module, an inverse residual module, and a depthwise separable module.
[0112] The following adopts Figure 2 The detailed steps of the process of establishing an object detection model of interest are as follows:
[0113] Step 01: Obtain a set of training sample images, where for each training picture, obtain images of three levels of resolution of the training picture respectively. The three levels of resolution are: 56*56, 112*112, and 224*224, and obtain the actual detection result of the target of interest for each frame of the training sample image: 1 or 0.
[0114] Step 02: First train the first-level subnet, which is the subnet composed of feature modules A and B. The training process is as follows:
[0115] The training sample images are read from the 56*56 training sample images in the training sample image set in sequence, and the read 56*56 training sample images are input into the first-level subnet. When the feature extraction module B of the first-level subnet outputs the feature F B1 When F B1 Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, the sigmoid calculation result is the probability that the predicted training sample image has the target of interest, calculate the loss function based on the probability and the actual target of interest detection result (1 or 0) of the training sample image, and adjust the neural network parameters of the first-level subnet based on the loss function;
[0116] Repeat the above training process until the first-level subnet converges.
[0117] It should be pointed out that for each training sample image extracted by the feature extraction modules A and B of the first-level subnet, A1 and F B1 , all need to be saved for use in training the secondary subnet.
[0118] Step 03: When the first-level subnet converges, start training the second-level subnet. The training process is as follows:
[0119] The training sample images are read out one by one from the 112*112 training sample images in the training sample image set, and the read 112*112 training sample images are input into the secondary subnet. At the same time, the feature F corresponding to the 56*56 training sample image of the same training screen saved in step 02 is input into the secondary subnet. A1 、F B1 Input the feature extraction modules A and B of the secondary subnet respectively;
[0120] Feature extraction module A to F of the secondary subnet A1 Upsample to get feature F of size 112*112 A1 ', and the feature extraction module A of the secondary subnet is A1 ' and 112*112 training sample image to obtain feature F A2 , the feature F A2 Output to the feature extraction module B of the secondary subnet;
[0121] The feature extraction module B of the secondary subnet is F B1 Upsample to get the size of F A2 The same feature F B1 ', and the feature extraction module B of the secondary subnet is B1 ' and F A2 Perform feature fusion to obtain feature F B2 , the feature F B2 Output to the feature extraction module C of the secondary subnet;
[0122] The feature extraction module C of the secondary subnet extracts the features F C2 Output to the feature extraction module D of the secondary subnet;
[0123] When the feature extraction module D of the secondary subnet outputs the feature F D2 When F D2Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, the sigmoid calculation result is the predicted probability that the target of interest exists in the 112*112 training sample image, calculate the loss function based on the probability and the actual target of interest detection result (1 or 0) of the 112*112 training sample image, and adjust the neural network parameters of the secondary subnet based on the loss function;
[0124] Repeat the above training process until the secondary subnet converges.
[0125] It should be noted that for each 112*112 training sample image extracted by the feature extraction modules A, B, C and D of the secondary subnet, the feature F A2 、F B2 、F C2 and F D2 , all need to be saved for use in training the three-level subnet.
[0126] Step 04: When the secondary subnet converges, start training the tertiary subnet. The training process is as follows:
[0127] The training sample images are read out one by one from the 224*224 training sample images in the training sample image set, and the read 224*224 training sample images are input into the third-level subnet. At the same time, the feature F corresponding to the 112*112 training sample image of the same training screen saved in step 03 is input into the third-level subnet. A2 、F B2 、F C2 and F D2 Input the feature extraction modules A, B, C, and D of the three-level subnetwork respectively;
[0128] Feature extraction module A to F of the three-level subnet A2 Upsample to get feature F of size 224*224 A2 ', and the feature extraction module A of the three-level subnet is A2 ' and the 224*224 training sample image to obtain feature F A3 , the feature F A3 Output to the feature extraction module B of the three-level subnetwork;
[0129] The feature extraction module B of the three-level subnet is F B2 Upsample to get the size of F A3 The same feature F B2 ', and the feature extraction module B of the three-level subnet is B2 ' and F A3 Perform feature fusion to obtain feature F B3 , the feature F B Output to the feature extraction module C of the three-level subnetwork;
[0130] The feature extraction module C of the three-level subnet is F C2 Upsample to get the size of F B3 The same feature F C2 ', and the feature extraction module C of the three-level subnet is C2 ' and F B3 Perform feature fusion to obtain feature F C3 , the feature F C3 Output to the feature extraction module D of the three-level subnetwork;
[0131] The feature extraction module D of the three-level subnet is F D2 Upsample to get the size of F C3 The same feature F D2 ', and the feature extraction module D of the three-level subnet is D2 ' and F C3 Perform feature fusion to obtain feature F D3 , the feature F D3 Output to the feature extraction module E of the three-level subnet;
[0132] When the feature extraction module E of the three-level subnet outputs the feature F E When the feature F E Output to the feature extraction module F of the three-level subnet;
[0133] When the feature extraction module F of the three-level subnet outputs the feature F F When F F Perform global average pooling calculation, perform sigmoid calculation on the obtained global average pooling value, the sigmoid calculation result is the predicted probability that the target of interest exists in the 224*224 training sample image, calculate the loss function based on the probability and the actual target of interest detection result (1 or 0) of the 224*224 training sample image, and adjust the neural network parameters of the three-level subnetwork based on the loss function;
[0134] Repeat the above training process until the three-level subnetwork converges. At this point, the object detection model of interest converges.
[0135] The execution subject of the embodiment of the present invention may be a GPU (Graphics Processing Unit).
[0136] Figure 4 This is a schematic diagram of the structure of an interesting target detection device provided by an embodiment of the present invention. The device mainly includes: a preparation and judgment module 41 and a detection module 42, wherein:
[0137] The preparation and judgment module 41 is used to select one resolution for the current picture to be detected in sequence from low to high among the preset multiple levels of resolution as the current resolution; and select one subnet in sequence from low to high among the multiple levels of subnets of the trained target detection model of interest as the current subnet; wherein each level of the subnet of the target detection model of interest includes at least one feature extraction module, and the higher the level of the subnet, the more feature extraction modules it contains; and, determine whether the current target detection result of interest output by the current subnet indicates that there is a target of interest in the current picture to be detected, and if so, end the current target detection process, otherwise, return to the action of selecting one resolution for the current picture to be detected in sequence from low to high among the preset multiple levels of resolution as the current resolution.
[0138] The detection module 42 is used to input the image to be detected at the current resolution into the current subnet according to the current resolution and the current subnet selected by the preparation and judgment module 41, and send the detection result of the target of interest output by the current subnet to the preparation and judgment module 41.
[0139] In an optional embodiment, the subnets of the object detection model of interest at multiple levels satisfy: for two subnets at adjacent levels, the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended by the higher-level subnet;
[0140] The detection module 42 inputs the image to be detected of the current resolution into the current subnet, including: inputting the image to be detected of the current resolution into the current subnet, and simultaneously inputting the features extracted from the subnet of the previous level to which the current subnet belongs into the current subnet.
[0141] In an optional embodiment, when the subnet of the previous level to which the current subnet belongs contains multiple feature extraction modules, the detection module 42 inputs the features extracted by the subnet of the previous level to which the current subnet belongs into the current subnet, including: inputting the features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs into the feature extraction modules at the same position of the current subnet.
[0142] In an optional embodiment, after the detection module 42 inputs the features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs into the feature extraction module at the same position of the current subnet, it further includes: for any feature extraction module among the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the features extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the feature is upsampled, and the upsampled features are fused with the features from the current subnet or with the image to be detected input into the current subnet.
[0143] In an optional embodiment, when there are multiple feature extraction modules extended in the higher-level subnet, the preparation and judgment module 41 judges whether the current target of interest detection result indicates that there is a target of interest in the current picture to be detected, including: when the last non-extended feature extraction module or any extended feature extraction module in the higher-level subnet outputs an extracted feature, judge whether the feature indicates that there is a target of interest in the current picture to be detected; and, if so, end the present process, otherwise, continue to input the feature into the next feature extraction module in the higher-level subnet, or, if the feature extraction module is the last feature extraction module of the higher-level subnet, return to the action of selecting a resolution as the current resolution for the current picture to be detected in the preset multiple levels of resolution in order from low to high.
[0144] In an optional embodiment, the feature extraction module includes: one or a combination of any multiple of a convolution module, a residual module, an inverse residual module, and a depthwise separable module.
[0145] Figure 5 This is a schematic diagram of the structure of an apparatus for establishing a detection model of an object of interest provided by an embodiment of the present invention. The apparatus mainly includes: a training preparation module 51 and a training module 52, wherein:
[0146] The training preparation module 51 is used to obtain a set of training sample images, wherein each training screen corresponds to multiple frames of training sample images, and the resolution of each frame of the training sample image is one of the preset multiple levels of resolution, so as to obtain the real detection result of the target of interest of each frame of the training sample image.
[0147] The training module 52 is used to perform the following steps B2 to E2:
[0148] B2. Select one resolution from the preset multiple levels of resolutions in order from low to high as the current resolution; select one subnet from the multiple levels of subnets of the target detection model to be trained in order from low to high as the current subnet; wherein each level of the subnet of the target detection model contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules;
[0149] C2. Select training sample images in turn from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image;
[0150] D2, calculating the loss function based on the predicted detection result of the target of interest of the current training sample image and the actual detection result of the target of interest, updating the neural network parameters of the current subnet based on the loss function, and returning to step C2 until the current subnet converges;
[0151] E2. Return to step B2 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.
[0152] In an optional embodiment, the subnets of the object detection model of interest at multiple levels satisfy: for two subnets at adjacent levels, the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended by the higher-level subnet;
[0153] The training module 52 inputs the selected training sample image into the current subnet, including: inputting the selected training sample image into the current subnet, and at the same time inputting the features of the same training picture extracted by the subnet of the previous level of the level to which the current subnet belongs into the current subnet, wherein the features of the same training picture are: the features of the training sample image of the previous resolution of the training picture corresponding to the selected training sample image.
[0154] In an optional embodiment, when the subnet of the previous level of the level to which the current subnet belongs contains multiple feature extraction modules,
[0155] The training module 52 inputs the features of the same training picture extracted by the subnet of the previous level of the level to which the current subnet belongs into the current subnet, including: inputting the features of the same training picture extracted by each feature extraction module of the subnet of the previous level of the level to which the current subnet belongs into the feature extraction modules at the same position of the current subnet respectively;
[0156] After the training module 52 inputs the features of the same training picture extracted by the subnet of the previous level to which the current subnet belongs into the current subnet, it further includes: for any feature extraction module in the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the features of the same training picture extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the features are upsampled, and the upsampled features are feature fused with the features from the current subnet or the training sample images input into the current subnet.
[0157] In an optional embodiment, the feature extraction module includes: one or a combination of any multiple of a convolution module, a residual module, an inverse residual module, and a depthwise separable module.
[0158] An embodiment of the present invention also provides a non-transitory computer-readable storage medium, which stores instructions. When the instructions are executed by a processor, the processor causes the processor to perform the steps of the method for detecting an object of interest described in any embodiment of the present invention, or to perform the steps of the method for establishing a model of interest detection described in any embodiment of the present invention.
[0159] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.
[0160] Specific embodiments are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not used to limit the present application. For those skilled in the art, changes can be made in the specific implementation methods and application scopes according to the ideas, spirits and principles of the present invention, and any modifications, equivalent substitutions, improvements, etc. made therein should be included in the scope of protection of this application.
Claims
1. A method for detecting an object of interest, characterized in that: The method includes: A1. From among the preset multiple levels of resolution, select one resolution for the current image to be detected in order from low to high as the current resolution; from among the multiple levels of subnets of the trained object detection model of interest, select one subnet in order from low to high as the current subnet; wherein each level of the subnet of the object detection model of interest contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules; B1. Obtain the image to be detected of the current resolution of the current picture to be detected, input the image to be detected of the current resolution into the current subnet, obtain the current interesting target detection result, and determine whether the current interesting target detection result indicates that there is an interesting target in the current picture to be detected. If so, end this process; otherwise, return to step A1.
2. The method according to claim 1, characterized in that The subnetworks at multiple levels of the object detection model of interest satisfy: For two subnets at adjacent levels, the higher-level subnet is composed of: a feature extraction module of the lower-level subnet and a feature extraction module extended from the higher-level subnet.
3. The method according to claim 2, characterized in that Step B1, inputting the image to be detected at the current resolution into the current subnet, includes: The image to be detected of the current resolution is input into the current subnet, and the features extracted by the subnet of the previous level to which the current subnet belongs are input into the current subnet.
4. The method according to claim 3, characterized in that When the subnet of the previous level of the level to which the current subnet belongs contains multiple feature extraction modules, The step of inputting the features extracted from the subnet of the previous level to which the current subnet belongs into the current subnet comprises: The features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs are respectively input into the feature extraction module at the same position of the current subnet.
5. The method according to claim 4, characterized in that After the features extracted by each feature extraction module of the subnet of the previous level to which the current subnet belongs are respectively input into the feature extraction module at the same position of the current subnet, the method further comprises: For any feature extraction module in the feature extraction modules at the same position of the current subnet, when the feature extraction module receives the feature extracted by the feature extraction module at the same position of the subnet of the previous level to which the current subnet belongs, the feature is upsampled, and the upsampled feature is fused with the feature from the current subnet or with the image to be detected input into the current subnet.
6. The method according to any one of claims 2 to 5, characterized in that: When there are multiple feature extraction modules extended by the higher-level subnet, The determining whether the current interesting target detection result indicates that the interesting target exists in the current to-be-detected picture includes: When the last non-extended feature extraction module or any extended feature extraction module in the higher-level subnet outputs an extracted feature, it is determined whether the feature indicates that there is an object of interest in the current image to be detected; And, if so, end this process, otherwise, continue to input the feature into the next feature extraction module in the higher-level subnet, or, if the feature extraction module is the last feature extraction module of the higher-level subnet, return to step A1.
7. A method for establishing a target of interest detection model, characterized in that: The method includes: A2. Obtain a set of training sample images, wherein each training picture corresponds to multiple frames of training sample images, and the resolution of each frame of the training sample image is one of the preset multiple levels of resolution, and obtain a true detection result of the target of interest for each frame of the training sample image; B2. Select one resolution from the preset multiple levels of resolutions in order from low to high as the current resolution; select one subnet from the multiple levels of subnets of the target detection model to be trained in order from low to high as the current subnet; wherein each level of the subnet of the target detection model contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules; C2. Select training sample images in turn from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image; D2, calculating the loss function based on the predicted detection result of the target of interest of the current training sample image and the actual detection result of the target of interest, updating the neural network parameters of the current subnet based on the loss function, and returning to step C2 until the current subnet converges; E2. Return to step B2 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.
8. The method according to claim 7, characterized in that The subnetworks at multiple levels of the object detection model of interest satisfy: For two subnets at adjacent levels, the higher-level subnet is composed of: the feature extraction module of the lower-level subnet and the feature extraction module extended by the higher-level subnet; Step C2, inputting the selected training sample image into the current subnet, includes: The selected training sample image is input into the current subnet, and at the same time, the features of the same training picture extracted by the subnet of the previous level to which the current subnet belongs are input into the current subnet, wherein the features of the same training picture are: the features of the training sample image of the previous resolution of the training picture corresponding to the selected training sample image.
9. An interesting object detection device, characterized in that: The device includes: A preparation and judgment module, used to select one resolution for the current image to be detected in sequence from low to high among multiple levels of preset resolutions as the current resolution; select one subnet in sequence from low to high among multiple levels of subnets of the trained object detection model of interest as the current subnet; wherein each level of the subnet of the object detection model of interest includes at least one feature extraction module, and the higher the level of the subnet, the more feature extraction modules it includes; and, determine whether the current object detection result output by the current subnet indicates that the current image to be detected has an object of interest, if so, end the current object detection process of interest, otherwise, return to the action of selecting one resolution for the current image to be detected in sequence from low to high among multiple levels of preset resolutions as the current resolution; The detection module is used to input the image to be detected at the current resolution into the current subnet according to the current resolution and current subnet selected by the preparation and judgment module, and send the detection result of the target of interest output by the current subnet to the preparation and judgment module.
10. A device for establishing a model for detecting an object of interest, characterized in that: The device includes: A training preparation module is used to obtain a set of training sample images, wherein each training picture corresponds to multiple frames of training sample images, and the resolution of each frame of the training sample image is one of the preset multiple levels of resolution, and obtain the real detection result of the target of interest of each frame of the training sample image; The training module is used to perform B2 to E2 as follows: B2. Select one resolution from the preset multiple levels of resolutions in order from low to high as the current resolution; select one subnet from the multiple levels of subnets of the target detection model to be trained in order from low to high as the current subnet; wherein each level of the subnet of the target detection model contains at least one feature extraction module, and a higher level of the subnet contains more feature extraction modules; C2. Select training sample images in turn from the training sample images of the current resolution in the training sample set, input the selected training sample images into the current subnet, and obtain the detection prediction result of the target of interest of the current training sample image; D2, calculating the loss function based on the predicted detection result of the target of interest of the current training sample image and the actual detection result of the target of interest, updating the neural network parameters of the current subnet based on the loss function, and returning to step C2 until the current subnet converges; E2. Return to step B2 and enter the training process of the next level subnet until all levels of the subnets of the target detection model of interest to be trained converge, and use the target detection model of interest at this time as the target detection model of interest to be finally used.