Method for removing duplicate frames for detecting slices, storage medium and electronic device

Abstract

The present invention discloses a method for removing duplicate frames for detecting slices, a storage medium and an electronic device, and relates to the field of image processing technology. The method for removing duplicate frames for detecting slices comprises: obtaining a target image, and performing slice processing on the target image to obtain a plurality of sub-images of the target image; performing target detection on each sub-image to obtain a sub-image detection result of each sub-image and a first feature map of each sub-image; performing target detection on the target image to obtain a full-image detection result of the target image and a second feature map of the target image; obtaining a target frame whose side length is greater than a preset side length threshold from each sub-image detection result and the full-image detection result, and determining a second target frame from the obtained target frame; inputting the first set of each sub-image and the second set of the target image into a pre-trained deduplication model to obtain a third target frame output by the deduplication model. The scheme provided by the embodiment of the present invention can remove duplicate target frames.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method for removing duplicate frames for detecting slices, a storage medium and an electronic device. Background Art

[0002] Drone inspections are fast and safe. Currently, drone inspections have been used in many practical scenarios, such as distribution line inspections and transmission line inspections in the power industry.

[0003] The resolution of images taken by drones is usually large. In order to ensure the detection of small targets in the image, the original image needs to be sliced ​​during inference to obtain multiple sub-images of the original image and detect the target in the sub-images. In this case, if the same target is divided into more than two sub-images during the slicing process, multiple target frames of the same target will be detected in different sub-images during subsequent target detection, affecting the accuracy of target detection.

[0004] Therefore, a technical solution is needed to remove duplicate target frames. Summary of the invention

[0005] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, one object of the present invention is to provide a method for removing duplicate frames for detecting slices, a storage medium and an electronic device to achieve the removal of duplicate target frames.

[0006] According to a first aspect of an embodiment of the present invention, a method for removing duplicate frames for detecting slices is provided, the method comprising:

[0007] Acquire a target image, and perform slicing processing on the target image to obtain multiple sub-images of the target image;

[0008] Performing target detection on each sub-image to obtain a sub-image detection result of each sub-image and a first feature map of each sub-image;

[0009] Performing target detection on the target image to obtain a full-image detection result of the target image and a second feature map of the target image;

[0010] Acquire a target frame whose side length is greater than a preset side length threshold from each sub-image detection result and the full-image detection result, and determine a second target frame from the acquired target frame;

[0011] Inputting the first set of sub-images and the second set of target images into a pre-trained deduplication model to obtain a third target frame output by the deduplication model;

[0012] Among them, for each sub-image, the first set of the sub-image is: a set formed by the first target frame of the sub-image and the first feature map of the sub-image, the first target frame is: a target frame that does not belong to the second target frame in the sub-image detection result of the sub-image, and a target frame that does not belong to the second target frame in the full-image detection result and is located in the sub-image; the second set is a set formed by the second target frame and the second feature map; the deduplication model is: a neural network model trained using a sample feature map and a sample target frame of a sample image with annotated boxes, and the neural network model is used to output a deduplicated target frame.

[0013] According to a second aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for removing duplicate frames for detecting slices is implemented.

[0014] According to a third aspect of an embodiment of the present invention, there is provided an electronic device comprising: a memory and a processor; a computer program is stored on the memory, and when the computer program is executed by the processor, the above-mentioned method for removing duplicate frames for detecting slices is implemented.

[0015] In the solution provided by the embodiment of the present invention, a first set of first target frames and a second set of second target frames are obtained, and the first target frames corresponding to small targets and the second target frames corresponding to large targets are distinguished by edge length thresholds, and detection is performed on the sub-image and the full target image, respectively, so that targets in various ranges in the target image can be detected. Moreover, when training a deduplication model, the annotated frame is used as supervision information, and the training model selects and outputs the target frame that is close to the annotated frame from the input sample target frame according to the sample feature map, that is, the target frame that best represents the actual target position, thereby excluding input frames with inaccurate representations of other positions, so that the model is trained to deduplicate the sample target frames. Accordingly, the trained deduplication model can also deduplicate the first target frame according to the first feature map of the sub-image and deduplicate the second target frame according to the second feature map of the full image feature.

[0016] During the deduplication process, target detection is performed on the target image and the sub-image obtained after slicing, which refers to both the full image information and the local information of the sub-image. In this way, the obtained feature map and the position information of the target frame are more comprehensive, which can improve the accuracy of target detection based on the feature map and the target frame.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1It is a flowchart of a method for removing duplicate frames for detecting slices provided by an embodiment of the present invention;

[0019] Figure 2 It is a flowchart of a model training method provided by an embodiment of the present invention;

[0020] Figure 3 It is a flowchart of a model reasoning method provided by an embodiment of the present invention;

[0021] Figure 4 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0023] The following describes a method for removing duplicate frames, a storage medium, and an electronic device for detecting slices according to an embodiment of the present invention with reference to the accompanying drawings.

[0024] In one embodiment of the present invention, see Figure 1 , a method for removing duplicate frames for detecting slices is provided, the method comprising the following steps S101-S104.

[0025] S101: Acquire a target image, and perform slicing processing on the target image to obtain a plurality of sub-images of the target image.

[0026] The target image may be an image taken by a drone or an image taken by a camera at a fixed position, and this is not limited in the embodiment of the present invention.

[0027] In one embodiment, the slicing process can be performed in the following manner to obtain sub-images of equal size. The size of the slice (h, w) is set, and the sliced ​​sub-images are obtained from the target image in the order of (h, w) from left to right and from top to bottom, with a horizontal axis interval of w and a vertical axis interval of h. Specifically, h can be 0.4*height, and w can be 0.4*width.

[0028] S102: Performing target detection on each sub-image to obtain a sub-image detection result of each sub-image and a first feature map of each sub-image.

[0029] In a specific scenario, the target in the target image may be a bird's nest on a distribution line, line damage, etc.

[0030] In one implementation, a pre-trained target detection model may be used for target detection.

[0031] In another implementation, the image features of the preset target may be compared with the image features of the sub-image to determine whether there are image features with high similarity in the sub-image to determine whether the target exists.

[0032] The sub-image detection result may include a target box, which is used to indicate the position of the detected target and may be a rectangular box surrounding the detected target. When the target box is output by a target detection model, the sub-image detection result may also include the confidence level corresponding to the target box output by the target detection model.

[0033] In one embodiment, the backbone network of the target detection model may perform feature extraction on the sub-image to obtain a first feature map. Specifically, feature extraction may be performed using ResNet (residual network).

[0034] S103: Performing target detection on the target image to obtain a full-image detection result of the target image and a second feature map of the target image.

[0035] The implementation method of target detection on the target image is similar to step S102, the only difference is the replacement of the name concepts such as the full-image detection result and the sub-image detection result, the second feature map and the first feature map, etc., which will not be described in detail here.

[0036] S104: Obtain a target frame whose side length is greater than a preset side length threshold from the detection results of each sub-image and the detection result of the entire image, and determine a second target frame from the obtained target frames.

[0037] Among them, the edge length threshold can be set manually.

[0038] For example, in the detection results of each sub-image, a target frame whose side length is greater than a preset side length threshold and whose distance from the boundary of the sub-image to which it belongs is less than a preset distance threshold is selected, and in the detection results of the entire image, a target frame whose side length is greater than the side length threshold and is located in two or more sub-images is selected, and the selected target frame is used as the second target frame.

[0039] When calculating the side length, the longest side of the target box can be used for calculation. For any sub-image, the boundary distance of the target box belonging to the sub-image can be the minimum value of the distance from the center of the target box to the four boundaries of the sub-image.

[0040] Since larger objects are more likely to be segmented into multiple sub-images during slicing, that is, located in two or more sub-images, in this case, for the sub-images, the segmented large objects have the characteristics of longer side lengths and being close to the sub-image boundaries, so they can be screened by the side length threshold and boundary distance.

[0041] In this way, the selected second target box and the second feature Figure 1 That is, the target position represented by the feature map of the entire image is predicted, so as to avoid treating the same target as multiple targets in different sub-maps for target frame prediction when the large target is sliced ​​and segmented, and reduce the possibility of multiple target frames appearing for the same target. This can handle the situation where the non-maximum suppression (NMS) method in SAHI (Slicing Aided Hyper Inference and Fine-tuning for SmallObject Detection) is used for the detection results of the block edge and the target is large and the duplicate frames cannot be filtered out.

[0042] In other embodiments, all boxes whose side lengths are greater than a preset threshold value may be selected as target boxes.

[0043] S105: Input the first set of sub-images and the second set of target images into a pre-trained deduplication model to obtain a third target frame output by the deduplication model.

[0044] Among them, for each sub-image, the first set of the sub-image is: a set formed by the first target frame of the sub-image and the first feature map of the sub-image, the first target frame is: a target frame that does not belong to the second target frame in the sub-image detection result of the sub-image, and a target frame that does not belong to the second target frame in the full-image detection result and is located in the sub-image; the second set is a set formed by the second target frame and the second feature map; the deduplication model is: a neural network model trained using a sample feature map and a sample target frame of a sample image with annotated boxes, and the neural network model is used to output a deduplicated target frame.

[0045] The above-mentioned annotation box may be a target box manually marked in advance for the target in the image.

[0046] The following example illustrates how to obtain the first set and the second set.

[0047] For the target image I, an image sequence (I0, I1, ..., IN, I) is obtained through slicing, where I0, I1, ..., IN are N+1 sub-images of the target image I.

[0048] Perform target detection on the image sequence (I0, I1, ..., IN, I) to obtain the detection results (P0, P1, ..., PN, P) and the corresponding feature maps (F0, F1, ..., FN, F). P0, P1, ..., PN are the sub-image detection results corresponding to I0, I1, ..., IN, P is the full-image detection result, F0, F1, ..., FN are the first feature maps corresponding to the sub-image detection results, and F is the second feature map.

[0049] For the sub-image detection results (P0, P1, ..., PN), the sets of boxes (Pb0, Pb1, ..., PbN) close to the boundary and with the longest side greater than L are obtained respectively, which are the second target boxes selected from the sub-image detection results; the remaining target box sets (Pf0, Pf1, ..., PfN) of the sub-image detection results are the first target boxes that do not belong to the second target box in the sub-image detection results of the sub-image.

[0050] Convert the coordinates of the sub-images in (Pb0, Pb1, ..., PbN) to the corresponding coordinates in the target image ( , ,..., ). The coordinates of the sub-image may be the coordinates of a coordinate system established with the upper left corner of the sub-image as the zero point, and the coordinates of the target image may be the coordinates of a coordinate system established with the upper left corner of the target image as the zero point.

[0051] For the full-image detection result P, the position of the target frame in P is compared with the image area included in the sub-image (I0, I1, ..., IN), and the second target frame whose coordinates span the two sub-images and whose longest side is greater than L is combined to obtain the set P'. In the full-image detection result, the target frame that does not belong to P' is the first target frame of each sub-image ( , ,..., ),in, is the first target frame of I0, is the first target box of I1… and so on.

[0052] Will( , ,..., ) is converted to the coordinates in the corresponding cut image ( , ,..., ); L is the edge length threshold, and in this embodiment, the smaller value of (0.15*height, 0.15*width) can be taken, where height is the height of the target image and width is the width of the target image.

[0053] Recombining the results into {(Pf0, ,F0), (Pf1, ,F1),...,(PfN, ,FN), ( , ,..., , , F)}, which are input into H respectively to obtain the final detection results {r0, r1, ..., rN, r} after deduplication, where r0, r1, ..., rN are the target boxes detected in the N sub-images and r is the target box detected in the whole image.

[0054] Among them, (Pf0, ,F0), (Pf1, ,F1),...,(PfN, , FN) is the first set of each sub-image, ( , ,..., , , F) is the second set.

[0055] In the solution provided by the embodiment of the present invention, a first set of first target frames and a second set of second target frames are obtained, and the first target frames corresponding to small targets and the second target frames corresponding to large targets are distinguished by edge length thresholds, and detection is performed on the sub-image and the full target image, respectively, so that targets in various ranges in the target image can be detected. Moreover, when training a deduplication model, the annotated frame is used as supervision information, and the training model selects and outputs the target frame that is close to the annotated frame from the input sample target frame according to the sample feature map, that is, the target frame that best represents the actual target position, thereby excluding input frames with inaccurate representations of other positions, so that the model is trained to deduplicate the sample target frames. Accordingly, the trained deduplication model can also deduplicate the first target frame according to the first feature map of the sub-image and deduplicate the second target frame according to the second feature map of the full image feature.

[0056] During the deduplication process, target detection is performed on the target image and the sub-image obtained after slicing, which refers to both the full image information and the local information of the sub-image. In this way, the obtained feature map and the position information of the target frame are more comprehensive, which can improve the accuracy of target detection based on the feature map and the target frame.

[0057] In one embodiment of the present invention, the deduplication model is trained in the following manner:

[0058] Input the sample image with the annotated box into the trained target detection model, obtain the sample feature map of the sample image output by the backbone network of the target detection model, and obtain the sample target box output by the prediction branch of the target detection model;

[0059] The sample feature map and the sample target frame are input into a preset neural network model to obtain a fourth target frame output by the neural network model, the target loss is calculated based on the fourth target frame and the annotation frame, and the model parameters of the neural network model are adjusted based on the calculated target loss to obtain a deduplication model.

[0060] In the target detection model, the backbone network is used to extract features from the input image to obtain a feature map, and the prediction branch is used to predict the position of the target box based on the extracted feature map.

[0061] The sample image may be an image with a similar scene and a similar target object as the target image. The method of extracting the sample feature map is similar to the first feature map and the second feature map, and will not be described in detail here.

[0062] In one embodiment of the present invention, the neural network model is a neural network model with a decoder of a DETR-like model, and the target detection model is a YOLOv8 model.

[0063] The DETR model is a complete detector, including a backbone network and a prediction head, which are used for classification and regression respectively. The output of the prediction head of the DETR model is combined with the Hungarian matching algorithm, which limits the output of the prediction head to a collective output, that is, one target will not generate multiple predictions, so that deduplication can be achieved. The prediction head plus the Hungarian matching algorithm is used for training and optimization, so that the original model has the ability of collective prediction.

[0064] Specifically, the DETR model can be migrated for use, such as migrating the prediction head part of DINO (DETR with ImprovedDeNoising Anchor Boxes for End-to-End Object Detection), or a custom network structure.

[0065] In addition, the target detection model can also be a YOLOv5 model, a Faster RCNN model, a DETR model, etc.

[0066] In addition, when the target detection model is a DETR model, as shown in the above embodiment, the prediction head of the DETR model has the ability to deduplicate, so the prediction head of the target detection model can be directly used for deduplication.

[0067] When the target detection model is a DETR model, in one embodiment of the present invention, each sub-image may be input into the target detection model to obtain a sub-image detection result output by the target detection model and a first feature map of each sub-image.

[0068] That is, the model for object detection can be used to output sample feature maps and sample object boxes for training the deduplication model. The trained deduplication model is equivalent to adding a new prediction branch to the existing object detection model.

[0069] In this case, when training the deduplication model, you can first freeze the target detection model, that is, when training the deduplication model, you will no longer adjust the model parameters belonging to the target detection model.

[0070] In addition, when training the deduplication model, the sample images used when training the target detection model can be used, thereby saving the cost of obtaining training samples.

[0071] In one embodiment of the present invention, the target loss is calculated as follows.

[0072] The Hungarian matching algorithm is used to match the positive and negative samples of the fourth target box to obtain the fifth target box as the positive sample and the sixth target box as the negative sample; the fifth classification loss and the regression loss are calculated based on the fifth target box and the annotation box, and the sixth classification loss of the sixth target box is calculated; the target loss is determined based on the fifth classification loss, the regression loss and the sixth classification loss.

[0073] Among them, the fifth classification loss can be the cross entropy loss calculated based on the fifth target box and the annotation box, and the regression loss is the L1 loss; the sixth classification loss is the classification loss between the sixth target box and the custom background class. In the image, the foreground part that does not belong to the target object will be classified as the background.

[0074] For example, the fourth target box includes 300 targets output by the model, and there are 5 targets in the actual image. The Hungarian matching algorithm will divide these 300 targets into 5 positive samples and 295 negative samples. For the 5 positive samples, the classification loss (cross entropy loss) and regression loss (L1 loss) need to be calculated, and the 295 negative samples need to calculate the sixth classification loss.

[0075] The target loss can be the sum of the fifth classification loss, the regression loss, and the sixth classification loss.

[0076] In one embodiment of the present invention, the deduplication model includes a plurality of decoding layers connected in sequence; each decoding layer outputs a corresponding target box and feature map after receiving an input.

[0077] For example, when the backbone network of the target detection model outputs the sample feature map out_b and the sample target box {(box0, cls0),...,(boxk1,clsk1)}, box0, cls0 represent the position of the target box of sample No. 0, which can be represented by the coordinates of the four corners of the rectangular box, cls0 represents the confidence of the target box of sample No. 0; boxk1, clsk1 represent the position of the target box of sample No. k1, clsk1 represents the confidence of the target box of sample No. k1, and so on.

[0078] Input the sample feature map out_b and the sample target frame into the first decoding layer of H to obtain the first step prediction result of the first decoding layer { } and the updated features , respectively represent the updated prediction boxes 0 to K1 and the corresponding feature maps; the first step prediction results and the updated features are input to the second decoding layer to obtain the second step prediction results { } and the updated features , represents the target box and feature map output by the second decoding layer; and so on, the prediction result of the m-1 step and the updated features are input to the m-th decoding layer to obtain the final prediction result { )} and the updated features ; For each layer of prediction results, the Hungarian matching algorithm can be used to match positive and negative samples, and the cross entropy and L1 loss functions can be used for optimization to finally obtain a trained model.

[0079] Specifically, m can be set to 6, 7, 8, etc.

[0080] In one embodiment of the present invention, before inputting the sample image with the labeled box into the trained object detection model, the method further includes:

[0081] Process the sample image in any one or more of the following ways:

[0082] Performing a scale transformation operation on the sample image;

[0083] Flip the sample image;

[0084] Perform a color shift operation on the sample image.

[0085] When training a target detection model, the above processing can also be performed when inputting a sample image into the target detection model to achieve image enhancement.

[0086] Figure 2A flow chart of a model training method is shown, wherein the original training data, i.e., sample images, are collected; a detector trained by a mobile phone, i.e., the trained target detection model in the aforementioned embodiment, is added with a new prediction branch as a deduplication model, and the model training process is as shown in the aforementioned embodiment of training a deduplication model, so that the model parameters of the detector are frozen, and the deduplication model is trained to output a target box, and the loss is calculated with the annotation box, and only the model parameters of the deduplication model as the prediction branch are adjusted. It is determined whether the deduplication model converges. If not, the training is continued in the above manner, otherwise the training is completed.

[0087] Figure 3 A flow chart of a model inference method is shown, wherein a picture to be predicted, i.e., a target image in step S101, is input; the original picture is cut into blocks to obtain a plurality of sub-images; the M1 model is used for prediction, and the prediction results are post-processed and grouped to obtain a first set and a second set, wherein the M1 model is the target detection model mentioned above, and the grouping results are respectively input into the H deduplication model to obtain deduplication results.

[0088] In one embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned methods for removing duplicate frames for detecting slices is implemented.

[0089] In one embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, any of the above-mentioned methods for removing duplicate frames for detecting slices is implemented.

[0090] Figure 4 It is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0091] like Figure 4 As shown, the electronic device 400 includes: a processor 401 and a memory 403. The processor 401 and the memory 403 are connected, such as through a bus 402. Optionally, the electronic device 400 may further include a transceiver 404. It should be noted that in actual applications, the transceiver 404 is not limited to one, and the structure of the electronic device 400 does not constitute a limitation on the embodiments of the present invention.

[0092] Processor 401 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. Processor 401 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0093] The bus 402 may include a path to transmit information between the above components. The bus 402 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 402 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] The memory 403 is used to store a computer program corresponding to the method for removing duplicate frames for detecting slices according to the above embodiment of the present invention, and the computer program is controlled and executed by the processor 401. The processor 401 is used to execute the computer program stored in the memory 403 to implement the contents shown in the above method embodiment.

[0095] Among them, the electronic device 400 includes but is not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device 400 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0096] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0097] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0099] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0100] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0101] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0102] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0103] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for removing duplicate frames for detecting slices, characterized in that: The method comprises: Acquire a target image, and perform slicing processing on the target image to obtain multiple sub-images of the target image; Performing target detection on each sub-image to obtain a sub-image detection result of each sub-image and a first feature map of each sub-image; Performing target detection on the target image to obtain a full-image detection result of the target image and a second feature map of the target image; Acquire a target frame whose side length is greater than a preset side length threshold from each sub-image detection result and the full-image detection result, and determine a second target frame from the acquired target frame; Inputting the first set of sub-images and the second set of target images into a pre-trained deduplication model to obtain a third target frame output by the deduplication model; Wherein, for each sub-image, the first set of the sub-image is: a set formed by the first target frame of the sub-image and the first feature map of the sub-image, the first target frame is: a target frame that does not belong to the second target frame in the sub-image detection result of the sub-image, and a target frame that does not belong to the second target frame in the full-image detection result and is located in the sub-image; the second set is a set formed by the second target frame and the second feature map; the deduplication model is: a neural network model trained using a sample feature map and a sample target frame of a sample image with annotated frames, and the neural network model is used to output a deduplicated target frame; The step of acquiring a target frame having a side length greater than a preset side length threshold from each sub-image detection result and the full-image detection result, and determining a second target frame from the acquired target frame includes: Select a target frame whose side length is greater than a preset side length threshold and whose distance from the boundary of the corresponding sub-image is less than a preset distance threshold in the detection results of each sub-image, and select a target frame whose side length is greater than the side length threshold and is located in two or more sub-images in the detection results of the whole image, and use the selected target frame as the second target frame; or, All boxes whose side lengths are greater than a preset threshold are selected as target boxes, and the selected target boxes are used as second target boxes.

2. The method according to claim 1, characterized in that The deduplication model is trained in the following manner: Inputting a sample image with a labeled box into a trained target detection model, obtaining a sample feature map of the sample image output by a backbone network of the target detection model, and obtaining a sample target box output by a prediction branch of the target detection model; The sample feature map and the sample target frame are input into a preset neural network model to obtain a fourth target frame output by the neural network model, the target loss is calculated based on the fourth target frame and the annotation frame, and the model parameters of the neural network model are adjusted based on the calculated target loss to obtain a deduplication model.

3. The method according to claim 2, characterized in that The neural network model is a neural network model with a decoder of a DETR-like model, and the target detection model is a YOLOv8 model.

4. The method according to claim 2, characterized in that: The performing target detection on each sub-image to obtain a sub-image detection result of each sub-image and a first feature map of each sub-image includes: Each sub-image is input into the target detection model to obtain a sub-image detection result output by the target detection model and a first feature map of each sub-image.

5. The method according to claim 2, characterized in that: The deduplication model includes a plurality of decoding layers connected in sequence; each decoding layer outputs a corresponding target frame and feature map after receiving input.

6. The method according to claim 2, characterized in that Before inputting the sample image with the marked box into the trained target detection model, the method further includes: The sample image is processed by any one or more of the following methods: Performing a scale transformation operation on the sample image; Performing a flip operation on the sample image; A color conversion operation is performed on the sample image.

7. The method according to claim 2, characterized in that The calculating the target loss based on the fourth target box and the annotation box includes: Using the Hungarian matching algorithm to perform positive and negative sample matching on the fourth target frame, to obtain a fifth target frame as a positive sample and a sixth target frame as a negative sample; Calculating a fifth classification loss and a regression loss based on the fifth target box and the annotation box, and calculating a sixth classification loss of a sixth target box; A target loss is determined based on the fifth classification loss, the regression loss, and the sixth classification loss.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for removing duplicate frames for detecting slices according to any one of claims 1 to 7 is implemented.

9. An electronic device, characterized in that: include: Memory, processor; The memory stores a computer program, and when the computer program is executed by the processor, the method for removing duplicate frames for detecting slices according to any one of claims 1 to 7 is implemented.

Images