Target detection network training candidate box matching method and device, equipment and medium

Abstract

The application provides a candidate box matching method and device in target detection network training, equipment and medium. The method comprises the following steps: obtaining a target detection training data set containing an original image, and setting an initial candidate box in the original image; performing an initialization operation on a training degree dictionary, inputting an image containing a target box into a target detection network for training, matching the initial candidate box with the target box, and updating the training degree dictionary according to the matching result; in each round of training, selecting a preset number of low-training-degree candidate boxes from the updated training degree dictionary, and performing cropping matching on the low-training-degree candidate boxes and the target boxes to be matched, and updating the training degree dictionary according to the cropping matching result; inputting the image corresponding to the cropped target box to be matched into the target detection network for training until the training number of the target detection network reaches a preset requirement. The application improves the target detection capability of the candidate box and improves the precision of the target detection network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and particularly relates to a candidate box matching method and device in target detection network training, equipment and medium. BACKGROUND

[0002] In the target detection network training process, most of the prior boxes are laid. The prior box is the starting point and basis of network classification and regression. The target detection takes the training of neural network as the starting point, and actually trains the classification and regression ability of the prior box.

[0003] In the prior art, the target detection is innovated and improved around the neural network structure, loss, label assignment and the like. Although these innovations and improvements are valuable, they ignore the attention to whether the prior box itself can be fully trained. Some technologies only focus on setting a fixed scale and size for the prior box.

[0004] Therefore, in the target detection network training process, many prior boxes actually cannot match any positive samples, and the prior boxes cannot be fully trained and focused. In the inference application, the target detection ability of these prior boxes is low, and errors are prone to occur in inference, which reduces the accuracy of the target detection network. SUMMARY

[0005] Therefore, the embodiments of the present application provide a candidate box matching method, device, equipment and medium in target detection network training, to solve the problem that the prior box has low target detection ability in the prior art, is prone to errors in inference, and reduces the accuracy of the target detection network.

[0006] The first aspect of the embodiments of the present application provides a candidate box matching method in target detection network training, comprising: obtaining a target detection training data set containing an original image, and setting an initial candidate box in the original image; performing an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary to an initial value; inputting an image containing a target box into a target detection network for training, and matching the initial candidate box with the target box, and updating the training degree dictionary according to the matching result; in each round of training of the target detection network, selecting a preset number of low training degree candidate boxes from the updated training degree dictionary, and performing cropping matching on the low training degree candidate boxes and the target box to be matched, and updating the training degree dictionary according to the cropping matching result; inputting the image corresponding to the cropped target box to be matched into the target detection network for training, until the training round number of the target detection network reaches a preset requirement.

[0007] In a second aspect, the embodiment of the present application provides a candidate box matching device in target detection network training, which comprises: an acquisition module configured to acquire a target detection training data set containing an original image, and set initial candidate boxes in the original image; an initialization module configured to perform an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary as an initial value; an updating module configured to input an image containing a target box into a target detection network for training, match the initial candidate boxes with the target box, and update the training degree dictionary according to a matching result; a clipping and matching module configured to select a preset number of low-training-degree candidate boxes from the updated training degree dictionary in each round of training of the target detection network, clip and match the low-training-degree candidate boxes with a target box to be matched, and update the training degree dictionary according to a clipping and matching result; and a training module configured to input an image corresponding to a clipped target box to be matched into the target detection network for training until the number of training rounds of the target detection network reaches a preset requirement.

[0008] In a third aspect, the embodiment of the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the above method when executing the program.

[0009] In a fourth aspect, the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0010] The above at least one technical scheme adopted by the embodiment of the present application can achieve the following beneficial effects:

[0011] By acquiring a target detection training data set containing an original image, setting initial candidate boxes in the original image, performing an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary as an initial value, inputting an image containing a target box into a target detection network for training, matching the initial candidate boxes with the target box, updating the training degree dictionary according to a matching result, selecting a preset number of low-training-degree candidate boxes from the updated training degree dictionary in each round of training of the target detection network, clipping and matching the low-training-degree candidate boxes with a target box to be matched, updating the training degree dictionary according to a clipping and matching result, and inputting an image corresponding to a clipped target box to be matched into the target detection network for training until the number of training rounds of the target detection network reaches a preset requirement, all candidate boxes can be fully trained, the target detection capability of the candidate boxes is improved, and the accuracy of the target detection network is higher. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0013] Figure 1 is a flowchart of a candidate box matching method in target detection network training provided by the embodiments of the present application;

[0014] Figure 2 is a structural diagram of a candidate box matching device in target detection network training provided by the embodiments of the present application;

[0015] Figure 3 is a structural diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0016] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application, but it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0017] As described in the background, most of the existing target detection technologies will lay prior boxes (i.e. candidate boxes, referred to as anchors), which are the starting point and basis of network classification and regression. Target detection takes training neural network as the starting point, which is actually training the classification and regression ability of anchor. The existing technologies are currently innovating and improving around neural network structure, loss, label assignment, etc. These innovations and improvements are valuable, but they ignore the concern for whether the anchor itself can be fully trained. Some technologies only focus on setting fixed scales and sizes for anchors.

[0018] Therefore, in the target detection network training process, many anchors actually cannot match any positive samples, and the anchors cannot be fully trained and focused. In the inference application, the target detection ability of these anchors that have not been trained is low, and errors are prone to occur in inference, thereby reducing the accuracy of the target detection network.

[0019] The core of a target detection algorithm based on deep learning is to build a neural network (including a backbone network and a detection head, which is divided into a classification head and a regression head) and set prior boxes, which are generally more than 2,000 and are tiled on the input image of the neural network. These prior boxes can be regarded as candidate boxes. Each image is input into the target detection network, and the network needs to determine which candidate boxes anchor are foreground and which candidate boxes anchor are background. For candidate boxes anchor belonging to the foreground, the network will regress the bounding boxes (bbox) of these candidate boxes anchor. Finally, after post-processing such as non-maximum suppression, the final target detection result is obtained.

[0020] The inference process of the existing target detection network can be simplified as follows:

[0021] 1) Network inference: input the image into the neural network to obtain a feature map;

[0022] 2) Classification: which candidate boxes anchor belong to the foreground;

[0023] 3) Regression: regress the bounding boxes of the candidate boxes anchor;

[0024] 4) Post-processing of the bounding boxes belonging to the foreground to obtain the target detection result.

[0025] As can be seen, the existing target detection technology takes training the neural network as the starting point, and actually trains the classification and regression capabilities of the candidate boxes anchor.

[0026] In one example, assume that there are 2,000 anchors in the network, and one of the anchors (referred to as a1) has a position of (200, 200, 10, 10), where the four values represent the horizontal coordinate of the center point, the vertical coordinate, the height of the box, and the width, respectively. In the training process, a1 has never matched a positive sample, which means that a1 has never been trained. That is, an anchor must match a target box GT to be trained, otherwise it cannot be trained. Therefore, no matter how good the network structure, loss, and label assignment technology is, a1 cannot be trained, and it is easy to make mistakes in inference.

[0027] In another example, assume that one of the anchors in the network (referred to as a2) has a position of (312, 200, 40, 37). In the network training process, a2 matches fewer positive samples, or only matches positive samples of a certain class, and does not match positive samples of other classes. At this time, a2 also cannot be well trained, resulting in low target detection capability of a2, which is easy to make mistakes in inference.

[0028] Therefore, the embodiment of the present application provides a candidate box matching method in target detection network training to solve the above problems. The present application records the training degree of each candidate box anchor in the target detection network training process, matches the candidate box anchor with the target box GT by using the label assignment technology, and updates the training degree dictionary according to the matching result in each training round. In addition, the low training degree candidate box anchor is also matched to the target box GT as much as possible by using the clipping matching technology, so as to improve the training degree of the low training degree anchor, so as to ensure that each anchor in the network can be matched to the positive sample and can be fully trained and focused, so as to improve the target detection ability of the candidate box and promote the higher accuracy of the target detection network.

[0029] The technical solutions of the present application will be described in detail below in combination with the drawings and specific embodiments.

[0030] Figure 1 is a flowchart of the candidate box matching method in target detection network training provided by the embodiment of the present application. Figure 1 The candidate box matching method in target detection network training can be executed by a server. As shown in Figure 1 , the candidate box matching method in target detection network training can specifically include:

[0031] S101, obtaining a target detection training data set containing an original image, and setting an initial candidate box in the original image;

[0032] S102, performing an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary to an initial value;

[0033] S103, inputting an image containing a target box into a target detection network for training, and matching the initial candidate box with the target box, and updating the training degree dictionary according to the matching result;

[0034] S104, in each training round of the target detection network, selecting a preset number of low training degree candidate boxes from the updated training degree dictionary, and clipping and matching the low training degree candidate boxes with the target box to be matched, and updating the training degree dictionary according to the clipping and matching result;

[0035] S105, inputting the image corresponding to the clipped target box to be matched into the target detection network for training until the training round number of the target detection network reaches a preset requirement.

[0036] In some embodiments, the initialization operation is performed on the predetermined training degree dictionary, including: setting the character length corresponding to the training degree in the training degree dictionary according to the number of initial candidate boxes, and setting the initial value of the training degree corresponding to each initial candidate box in the training degree dictionary to 0.

[0037] Specifically, the candidate box in the embodiments of the present application can also be referred to as a prior box, and the candidate box is simply referred to as an anchor in the following embodiments. The value corresponding to the training degree of each anchor is recorded in the training degree dictionary of the present application, for example, there are 2,000 anchors in the target detection network, and the training degree dictionary after initialization will contain the training degrees of characters from 0 to 2000, that is, the character length of the initialized training degree dictionary is 2,000, and the initial value of the training degree corresponding to each anchor is 0.

[0038] In some embodiments, the image containing the target box is input into the target detection network for training, and the initial candidate box is matched with the target box, including: obtaining a batch of images, the batch of images containing a plurality of target boxes, inputting the batch of images into the target detection network previously built for training; in each round of training, calculating the intersection over union between the initial candidate box and the target box, when the intersection over union is greater than a preset threshold, matching the initial candidate box and the target box when the intersection over union is greater than the preset threshold, and increasing the value of the training degree corresponding to the initial candidate box by a first score when the intersection over union is greater than the preset threshold; wherein the initial candidate box can match one or more target boxes, and each time the initial candidate box matches a target box, the value of the training degree corresponding to the initial candidate box is increased by a first score in the training degree dictionary.

[0039] Specifically, in the training process of the target detection network, a batch of pictures will be input into the target detection network, and the target boxes GT are pre-labeled in the input batch of pictures, for example: 16 pictures are input into the target detection network at a time, and it is assumed that there are 50 target boxes (also referred to as actual boxes, referred to as GT for short) in the 16 pictures, then the target detection network will use a preset label assignment technology to match the initial candidate box anchor with the target box.

[0040] Further, the label assignment technology adopted in the embodiments of the present application is to calculate the intersection over union, that is, by calculating the intersection over union between the initial candidate box anchor and the target box, when the intersection over union is greater than a preset threshold, the initial candidate box anchor is matched with the target box, and a first score is added to the training degree of the initial candidate box anchor to obtain the updated training degree.

[0041] For example, in one specific embodiment, when the intersection over union between the initial candidate box anchor and the target box is greater than 0.5, the initial candidate box anchor is matched with the target box, and the value of the training degree corresponding to the initial candidate box anchor is increased by 1. After the assignment is completed, the training degree of each initial candidate box anchor can be known. For example, there are 2,000 anchors and 50 target boxes GT in the network for matching, and one target box GT can be matched with one anchor, so when one anchor a1 can match two target boxes GT, the training degree of the anchor is 2, and if a1 cannot match any target box GT, the value of the training degree is 0.

[0042] Obviously, assuming that 10,000 training pictures are input into the network, and these pictures have 50,000 target boxes GT, during the entire training process, it is assumed that some anchors can match 100 target boxes GT, and after one round of training, the training degree corresponding to these anchors is 100, and the training degree of the anchor that does not match the target box GT is 0, and the training degree dictionary is updated according to the result.

[0043] It should be noted that the present application can first use the above method to train the target detection network for multiple rounds (such as 10 rounds), and then perform pruning matching and fusion matching after the target detection network converges. The contents of pruning matching and fusion matching will be described in detail below in combination with specific embodiments.

[0044] In some embodiments, a preset number of low-training-degree candidate boxes are selected from the updated training degree dictionary, and the low-training-degree candidate boxes are subjected to pruning matching with the target boxes to be matched, including: sorting the initial candidate boxes in the updated training degree dictionary according to the values of the training degrees corresponding to the initial candidate boxes, and selecting a preset number of low-training-degree candidate boxes according to the sorting result; scaling and pruning the image corresponding to the target box to be matched, pasting the pruned image of the target box to be matched onto the low-training-degree candidate box, and making the intersection over union between the target box to be matched and the low-training-degree candidate box 50%-100%; training the target detection network using the pasted image, and in each round of training, increasing the value of the training degree corresponding to the low-training-degree candidate box matched with the target box to be matched by a second value.

[0045] Specifically, in order to further improve the training degree of the anchor, the present application also provides a pruning matching technology for low-training-degree anchors. After the above label matching, the number of anchors that can match the target box GT this time is recorded as k, the training degree dictionary is updated according to the matching result, and the training degree of the anchor that matches the target box GT is increased by 1. In each step of each round of training in the subsequent pruning matching, n pictures are input into the target detection network, and there are m target boxes GT.

[0046] Further, in the updated training degree dictionary, based on the training degree, 2*k anchors with the lowest training degree are found, which are referred to as low training degree anchors. And 2*k target boxes (GTs) to be matched (to be matched with the low training degree anchors) are randomly selected from m GTs, m is generally less than 2*k, so the selection can be repeated. The 2*k low training degree anchors and the 2*k target boxes to be matched are matched by using the cropping matching technology.

[0047] The cropping matching technology method provided by the embodiment of the application is that a picture g corresponding to a target box GT to be matched is cropped out after being expanded by 20%, and then is pasted on a low training degree anchor a. The picture g needs to be randomly scaled, and the scaling scale is 80%-120% of a. Then the picture g is randomly pasted, and the intersection over union of the pasting position and a is 50%-100%. Through the above cropping matching technology, in each step of training, the low training degree anchor is matched to the corresponding GT, so that the low training degree anchor can also be trained to a certain extent, and the training degree of the low training degree anchor is also increased. Meanwhile, the training degree dictionary is updated according to the cropping matching result, and the training degree of the low training degree anchor in the cropping matching is increased by 0.75.

[0048] In some embodiments, after the training degree dictionary is updated according to the cropping matching result, the method further includes: selecting a preset number of calibration candidate boxes from the updated training degree dictionary, and traversing the original image to select a fusion base image and a fusion additional image; randomly scaling the fusion additional image so that the size of the target box in the fusion additional image is scaled to 80%-120% of the size of the calibration candidate box; adding the randomly scaled fusion additional image to the fusion base image to obtain a fusion image, and making the intersection over union between the target box and the calibration candidate box be 50%-100%; inputting the fusion image into the target detection network to train the calibration candidate box, and increasing the value of the training degree corresponding to the calibration candidate box matched to the target box in the training by a third score value.

[0049] Specifically, after the training degree dictionary is updated according to the cropping matching result, the training degrees in the updated training degree dictionary are reordered, and a certain number of calibration candidate boxes are selected according to the sorting result, for example, based on the training degree sorting, 2*n anchors with the lowest training degree are found from n pictures, and these anchors are used as calibration anchors.

[0050] Further, traverse n pictures in the target detection training data set, each time, select one picture as a "fusion base map", and randomly select other two pictures as "fusion auxiliary maps". For each "fusion auxiliary map", randomly scale it. In order to make the calibration anchor match the target box GT as much as possible, the scaling principle is to scale a certain target box b to 80%-120% of a certain "calibration anchor" a.

[0051] Then, add the "fusion auxiliary map" to the "fusion base map" by random whole image addition. The addition principle is that the target box b is calibrated to the "calibration anchor" a, that is, the scaled fusion auxiliary map is moved to the position where b and a overlap, and then added to the fusion base map, and the intersection over union of b and a is 50%-100%. Since the above operation is performed on the n pictures, n new fusion pictures are generated. The new target boxes of these pictures will match some low training degree calibration anchors, so that the low training degree calibration anchors can match the corresponding target box GT, so that the low training degree calibration anchors can also be trained to a certain extent, and their training degree will also rise. At the same time, update the "training degree dictionary" according to the fusion matching result, and add 0.5 to the training degree of the calibration anchor matched by the fusion.

[0052] In some embodiments, a preset number of calibration candidate boxes are selected from the updated training degree dictionary, and the original image is traversed to select a fusion base map and a fusion auxiliary map, including: based on the sorting result of the training degree in the updated training degree dictionary, selecting a preset number of low training degree candidate boxes from low to high as calibration candidate boxes; traverse the original image in the target detection training data set, and select one original image as a fusion base map each time, and randomly select other two original images as fusion auxiliary maps.

[0053] In some embodiments, the randomly scaled fusion auxiliary map is added to the fusion base map to obtain a fusion image, including: moving the target box in the randomly scaled fusion auxiliary map to the position of the calibration candidate box, so that the target box is calibrated to the calibration candidate box, and adding the randomly scaled fusion auxiliary map to the fusion base map to obtain a fusion image.

[0054] The present application can be used in the training process of any multi-target detection technology, does not increase any inference cost, solves the problem that many candidate boxes anchors cannot be trained, ensures that all prior boxes are fully trained, and promotes higher model precision of multi-target detection.

[0055] The following is an embodiment of the device of the present application, which can be used to execute the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0056] Figure 2 is a structural schematic diagram of a candidate box matching device in target detection network training provided by an embodiment of the present application. As shown in the figure, the candidate box matching device in target detection network training comprises: Figure 2

[0057] The acquisition module 201 is configured to acquire a target detection training data set containing an original image, and set an initial candidate box in the original image.

[0058] The initialization module 202 is configured to perform an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary as an initial value.

[0059] The updating module 203 is configured to input an image containing a target box into a target detection network for training, match the initial candidate box with the target box, and update the training degree dictionary according to the matching result.

[0060] The cropping and matching module 204 is configured to select a preset number of low-training-degree candidate boxes from the updated training degree dictionary in each round of training of the target detection network, crop and match the low-training-degree candidate boxes with a target box to be matched, and update the training degree dictionary according to the cropping and matching result.

[0061] The training module 205 is configured to input the image corresponding to the cropped target box to be matched into the target detection network for training until the number of training rounds of the target detection network reaches a preset requirement.

[0062] In some embodiments, Figure 2 The initialization module 202 of the above sets the character length corresponding to the training degree in the training degree dictionary according to the number of initial candidate boxes, and sets the initial value of the training degree corresponding to each initial candidate box in the training degree dictionary as 0.

[0063] In some embodiments, Figure 2 The updating module 203 of the above acquires batch images containing multiple target boxes, inputs the batch images into the pre-built target detection network for training, calculates the intersection over union between the initial candidate box and the target box in each round of training, matches the initial candidate box and the target box when the intersection over union is greater than a preset threshold, and increases the value of the training degree corresponding to the initial candidate box by a first score when the intersection over union is greater than the preset threshold. Wherein, the initial candidate box can match one or more target boxes, and the value of the training degree corresponding to the initial candidate box is increased by the first score in the training degree dictionary every time the initial candidate box matches a target box.

[0064] In some embodiments, Figure 2 ​The cropping matching module 204 sorts the updated training degree dictionary according to the value of the training degree corresponding to the initial candidate box, and selects a preset number of low training degree candidate boxes according to the sorting result; scales and crops the image corresponding to the target box to be matched, pastes the cropped image of the target box to be matched onto the low training degree candidate box, and makes the intersection over union between the target box to be matched and the low training degree candidate box 50%-100%; the target detection network is trained using the pasted image, and in each round of training, the value of the training degree corresponding to the low training degree candidate box matched to the target box to be matched is increased by a second score.

[0065] In some embodiments, Figure 2 The fusion matching module 206 selects a preset number of calibration candidate boxes from the updated training degree dictionary after updating the training degree dictionary according to the cropping matching result, and traverses the original image to select a fusion base image and a fusion auxiliary image; the target box in the fusion auxiliary image is randomly scaled to scale the size of the target box in the fusion auxiliary image to 80%-120% of the size of the calibration candidate box; the fusion auxiliary image after random scaling is added to the fusion base image to obtain a fusion image, and the intersection over union between the target box and the calibration candidate box is 50%-100%; the calibration candidate box is trained by inputting the fusion image into the target detection network, and the value of the training degree corresponding to the calibration candidate box matched to the target box in the training is increased by a third score.

[0066] In some embodiments, Figure 2 The fusion matching module 206 selects a preset number of low training degree candidate boxes as calibration candidate boxes from low to high based on the sorting result of the training degree in the updated training degree dictionary; traverse the original image in the target detection training data set, and select one original image as a fusion base image and randomly select two other original images as fusion auxiliary images in each traversal.

[0067] In some embodiments, Figure 2 The fusion matching module 206 moves the target box in the fusion auxiliary image after random scaling to the position of the calibration candidate box to calibrate the target box to the calibration candidate box, and adds the fusion auxiliary image after random scaling to the fusion base image to obtain a fusion image.

[0068] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0069] Figure 3 is a structural schematic diagram of an electronic device 3 provided by the embodiments of the present application. As Figure 3As shown, the electronic device 3 of this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. The processor 301 implements the steps in each of the above method embodiments when executing the computer program 303. Alternatively, the processor 301 implements the functions of each module / unit in each of the above apparatus embodiments when executing the computer program 303.

[0070] By way of example, the computer program 303 can be divided into one or more modules / units, which are stored in the memory 302 and executed by the processor 301 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 303 in the electronic device 3.

[0071] The electronic device 3 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The electronic device 3 can include but is not limited to the processor 301 and the memory 302. Those skilled in the art can understand that the electronic device 3 can include more or fewer components, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like. Figure 3 The electronic device 3 is merely an example and does not constitute a limitation on the electronic device 3, which can include more or fewer components than those shown, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, and the like.

[0072] The processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0073] The memory 302 can be an internal storage unit of the electronic device 3, for example, a hard disk or a memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 3. Further, the memory 302 can include both the internal storage unit and the external storage device of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0074] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0075] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0076] Those of ordinary skill in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0077] In the embodiments of the present application, it should be understood that the disclosed apparatus / computer device and method can be implemented in other manners. For example, the described apparatus / computer device embodiments are merely schematic. For example, the division of the modules or units is merely logical function division. There can be another division manner for the actual implementation, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0078] The units described as separated components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments.

[0079] In addition, each functional unit in the embodiments of the present application can be integrated in a processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of a software functional unit.

[0080] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method in the above embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can be executed by a processor to implement the steps of each method embodiment. The computer program can include computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer readable medium can include or exclude contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to the legislation and patent practice, the computer readable medium does not include electric carrier signal and telecommunication signal.

[0081] The above examples are only used to illustrate the technical solutions of the present application, but not limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for candidate box matching in target detection network training, characterized in that, The method comprises the following steps: obtain a target detection training data set containing original images, and set initial candidate boxes in the original images; perform an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary to an initial value; input an image containing a target box into a target detection network for training, match the initial candidate box with the target box, and update the training degree dictionary according to the matching result; in each round of training of the target detection network, select a preset number of low-training-degree candidate boxes from the updated training degree dictionary, and perform cropping and matching on the low-training-degree candidate boxes and the target boxes to be matched, and update the training degree dictionary according to the cropping and matching result; input the image corresponding to the cropped target box to be matched into the target detection network for training until the number of training rounds of the target detection network reaches a preset requirement.

2. The method of claim 1, wherein, The initialization operation on the predetermined training degree dictionary comprises the following steps: set the character length corresponding to the training degree in the training degree dictionary according to the number of initial candidate boxes, and set the initial value of the training degree corresponding to each initial candidate box in the training degree dictionary to 0.

3. The method of claim 1, wherein, The inputting of the image containing the target box into the target detection network for training and the matching of the initial candidate box with the target box comprise the following steps: obtain a batch of images, the batch of images containing a plurality of target boxes, and input the batch of images into a pre-built target detection network for training; in each round of training, calculate the intersection over union between the initial candidate box and the target box, when the intersection over union is greater than a preset threshold, match the initial candidate box and the target box when the intersection over union is greater than the preset threshold, and increase the value of the training degree corresponding to the initial candidate box whose intersection over union is greater than the preset threshold by a first score; wherein the initial candidate box can match one or more target boxes, and each time the initial candidate box matches a target box, the value of the training degree corresponding to the initial candidate box in the training degree dictionary is increased by a first score.

4. The method of claim 1, wherein, The selection of a preset number of low-training-degree candidate boxes from the updated training degree dictionary and the cropping and matching of the low-training-degree candidate boxes and the target boxes to be matched comprise the following steps: sort the initial candidate boxes in the updated training degree dictionary according to the value of the training degree corresponding to the initial candidate boxes, and select a preset number of low-training-degree candidate boxes according to the sorting result; scale and crop the image corresponding to the target box to be matched, paste the cropped image of the target box to be matched onto the low-training-degree candidate box, and make the intersection over union between the target box to be matched and the low-training-degree candidate box 50%-100%; use the pasted image to train the target detection network, and in each round of training, increase the value of the training degree corresponding to the low-training-degree candidate box matched to the target box to be matched by a second score.

5. The method of claim 1, wherein, After the updating of the training degree dictionary according to the cropping and matching result, the method further comprises the following steps: selecting a preset number of calibration candidate boxes from the updated training degree dictionary, and traversing the original images to select a fusion base image and a fusion auxiliary image; randomly scaling the fusion auxiliary image so as to scale the size of a target box in the fusion auxiliary image to 80%-120% of the calibration candidate box size; adding the randomly scaled fusion auxiliary image to the fusion base image to obtain a fusion image, and making the intersection over union between the target box and the calibration candidate box to be 50%-100%; training the calibration candidate box in the target detection network by inputting the fusion image into the target detection network, and increasing the value of the training degree corresponding to the calibration candidate box matched to the target box in the training by a third score.

6. The method of claim 5, wherein, The selecting a preset number of calibration candidate boxes from the updated training degree dictionary, and traversing the original images to select a fusion base image and a fusion auxiliary image, comprises: selecting a preset number of low training degree candidate boxes as the calibration candidate boxes from low to high based on the sorting result of the training degrees in the updated training degree dictionary; traversing the original images in the target detection training data set, and selecting one original image as a fusion base image and randomly selecting two other original images as fusion auxiliary images in each traversal.

7. The method of claim 5, wherein, The adding the randomly scaled fusion auxiliary image to the fusion base image to obtain a fusion image, comprises: moving the target box in the randomly scaled fusion auxiliary image to the position of the calibration candidate box so as to calibrate the target box to the calibration candidate box, and adding the randomly scaled fusion auxiliary image to the fusion base image to obtain a fusion image.

8. A candidate box matching device in target detection network training, characterized in that, comprises: an acquisition module configured to acquire a target detection training data set containing original images, and set initial candidate boxes in the original images; an initialization module configured to perform an initialization operation on a predetermined training degree dictionary, so as to set the training degree corresponding to each initial candidate box in the training degree dictionary to an initial value; an updating module configured to input an image containing a target box into a target detection network for training, match the initial candidate box with the target box, and update the training degree dictionary according to the matching result; a cropping and matching module configured to select a preset number of low training degree candidate boxes from the updated training degree dictionary in each round of training of the target detection network, crop and match the low training degree candidate boxes with a target box to be matched, and update the training degree dictionary according to the cropping and matching result; a training module configured to input the image corresponding to the cropped target box to be matched into the target detection network for training until the number of training rounds of the target detection network reaches a preset requirement. 9.An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method of any one of claims 1 to 7. The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images