Candidate box processing method and device, equipment and medium
By determining the overlap value of the candidate box in the target detection around the vehicle and judging according to the threshold, the candidate box splitting problem is solved, improving the accuracy of the candidate box and the correctness of subsequent functional processing.
Patent Information
- Application Number
- CN202311020676.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-14
- Publication Date
- 2025-02-21
AI Technical Summary
In the detection of targets around the vehicle, candidate boxes obtained through the BEV model are prone to splitting problems, resulting in multiple candidate boxes generated by the same object, and it is difficult to determine which candidate box is split, affecting subsequent functional processing.
By determining a first target candidate box and at least one second target candidate box among the multiple candidate boxes, overlap calculation is performed, the first overlap value and the second overlap value are obtained, and whether the second target candidate box exists based on the preset threshold value is determined whether the second target candidate box exists. Split issue.
Effectively filter split candidate boxes to improve the accuracy of candidate boxes and ensure the correctness of subsequent functional processing.
Smart Images
Figure CN119494949A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image recognition technology, and in particular to a candidate frame processing method, device, equipment and medium. Background Art
[0002] With the widespread use of on-board cameras in modern vehicle equipment, the detection and recognition of various targets encountered by vehicles during driving based on on-board visual sensors has become increasingly mature. Its role has demonstrated its important value for both current assisted driving and future autonomous driving, and has become an indispensable core technology for modern high-end intelligent vehicles. The bird's eye view (BEV) is constructed from top to bottom with the vehicle coordinate system as the center. The range covers the area around the vehicle with a width of tens to hundreds of meters. Its function is to facilitate the driver to obtain the surrounding environment of the vehicle from a bird's eye view through perspective conversion, thereby improving the ability to control the vehicle.
[0003] In the related art, when obtaining candidate frames of targets around the vehicle through the BEV model, there will be many target splitting problems. For example, multiple candidate frames will be generated for the same identified object. If it is impossible to determine which candidate frame is a split candidate frame generated for the same object and which candidate frame is the only candidate frame generated for a certain object, it will bring obstacles to the subsequent processing of other functions and even cause serious impact. Therefore, how to select the candidate frames generated by splitting from many candidate frames for processing becomes an urgent problem to be solved. Summary of the invention
[0004] In order to solve the above technical problems, the present disclosure provides a candidate frame processing method, device, equipment and medium.
[0005] In a first aspect, the present disclosure provides a candidate frame processing method, comprising:
[0006] Determine a first target candidate frame and at least one second target candidate frame from the multiple candidate frames;
[0007] For each second target candidate frame, respectively perform a first overlap calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value for each second target candidate frame;
[0008] Perform a second overlap calculation on the second target candidate frame whose first overlap value is less than the first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value;
[0009] A second target candidate box whose second overlap value is greater than or equal to a second preset threshold is determined as a candidate box with a splitting problem.
[0010] In a second aspect, the present disclosure provides a candidate frame processing device, including:
[0011] A first processing module, configured to determine a first target candidate frame and at least one second target candidate frame from among the multiple candidate frames;
[0012] A first calculation module is used to perform a first overlap calculation on the first target candidate frame and the second target candidate frame for each second target candidate frame, so as to obtain a first overlap value of each second target candidate frame;
[0013] A second calculation module is used to perform a second overlap calculation on the second target candidate frame whose first overlap value is less than the first preset threshold and the first target candidate frame after the rotation frame translation operation, to obtain a second overlap value;
[0014] The second processing module is used to determine a second target candidate frame whose second overlap value is greater than or equal to a second preset threshold as a candidate frame with a split problem.
[0015] In a third aspect, the present disclosure provides a candidate frame processing device, including:
[0016] processor;
[0017] A memory for storing executable instructions;
[0018] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the candidate frame processing method of the first aspect.
[0019] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the candidate frame processing method of the first aspect.
[0020] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:
[0021] The candidate frame processing method, apparatus, device and medium of the disclosed embodiments can determine a first target candidate frame and at least one second target candidate frame from multiple candidate frames, and then for each second target candidate frame, perform a first overlap calculation on the first target candidate frame and the second target candidate frame respectively to obtain a first overlap value for each second target candidate frame, and then perform a second overlap calculation on the second target candidate frame whose first overlap value is less than a first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value, and finally determine the second target candidate frame whose second overlap value is greater than or equal to the second preset threshold as a candidate frame with a split problem, thereby being able to determine the second target candidate frame with a split problem from multiple candidate frames through the first overlap value and the second overlap value, thereby effectively filtering the split second target candidate frames and improving the accuracy of the candidate frames. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0023] Figure 1 A schematic diagram of a process flow of a candidate frame processing method provided by an embodiment of the present disclosure;
[0024] Figure 2 A schematic diagram of the structure of a candidate frame processing device provided in an embodiment of the present disclosure;
[0025] Figure 3 A schematic diagram of the structure of a candidate frame processing device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0026] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0027] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0028] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0029] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0030] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0031] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0032] In order to solve the above problems, the present disclosure provides a candidate frame processing method, device, equipment and medium. Figure 1 The candidate frame processing method provided in the embodiment of the present disclosure is described in detail.
[0033] Figure 1 A schematic flow chart of a candidate frame processing method provided by an embodiment of the present disclosure is shown.
[0034] In the embodiment of the present disclosure, the candidate frame processing method may be executed by an electronic device. Specifically, the electronic device may include but is not limited to a mobile terminal such as a computer device, a mobile phone, a vehicle-mounted device, a vehicle controller, a tablet computer, a wearable device, etc.
[0035] like Figure 1 As shown, the candidate frame processing method may include the following steps.
[0036] S110: Determine a first target candidate frame and at least one second target candidate frame from multiple candidate frames.
[0037] In an embodiment of the present disclosure, the electronic device may determine a first target candidate frame and at least one second target candidate frame from among multiple candidate frames.
[0038] Optionally, the candidate box may be an image box obtained by the BEV model. For example, the candidate box may be an image box corresponding to a vehicle, an obstacle, a pedestrian, etc., which is not limited here. Each candidate box may include a confidence level.
[0039] Optionally, the confidence level may be used to characterize the credibility of the candidate box.
[0040] Optionally, the first target candidate box may be a candidate box with the highest confidence among multiple candidate boxes.
[0041] Optionally, the second target candidate box may be a candidate box among the multiple candidate boxes except the first target candidate box.
[0042] Specifically, the electronic device can obtain multiple candidate boxes output by the BEV model (such as multiple candidate boxes generated for several objects around the vehicle), each candidate box can include a corresponding confidence level, and the electronic device can sort the multiple candidate boxes in descending order according to the confidence level, and then determine the candidate box with the highest confidence level as the first target candidate box, and determine at least one candidate box other than the first target candidate box as the second target candidate box.
[0043] S120 . For each second target candidate frame, perform a first overlap calculation on the first target candidate frame and the second target candidate frame respectively to obtain a first overlap value for each second target candidate frame.
[0044] In the embodiment of the present disclosure, for each second target candidate frame, the electronic device may perform a first overlap calculation on the first target candidate frame and the second target candidate frame respectively to obtain a first overlap value corresponding to each second target candidate frame.
[0045] Optionally, the first overlap calculation may be used to calculate the overlap degree between two candidate frames.
[0046] Optionally, the first overlap value may be used to characterize the degree of overlap between two candidate frames. For example, the first overlap value may be 0.5, 0.6, etc., a value less than or equal to 1.0, which is not limited here.
[0047] Specifically, the electronic device may perform a first overlap calculation on the first target candidate frame and each second target candidate frame to obtain an overlap degree between the first target candidate frame and each second target candidate frame, that is, obtain a first overlap value.
[0048] Optionally, S120 may specifically include: performing rotation intersection-and-union calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value.
[0049] Optionally, the rotation IOU calculation may be to calculate the IOU of two rotated candidate frames, wherein the IOU (Intersection over Union, IOU) may be to calculate the ratio of the intersection and the union of the two candidate frames.
[0050] Specifically, the electronic device can perform a rotational intersection-and-union ratio calculation on the first target candidate box and the second target candidate box, such as by using the four vertex coordinates and side length data of the candidate boxes, and performing a rotational intersection-and-union ratio calculation according to the rotation IOU calculation formula to obtain a first overlap value, wherein the rotational intersection-and-union ratio calculation can be any achievable method and is not limited here.
[0051] S130, performing a second overlap calculation on the second target candidate frame whose first overlap value is less than the first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value.
[0052] In an embodiment of the present disclosure, after obtaining the first overlap value, the electronic device may perform a second overlap calculation on the first target candidate frame and the second target candidate frame undergoing the rotation frame translation operation to obtain a second overlap value if the first overlap value is less than a first preset threshold.
[0053] Optionally, the first preset threshold may be a pre-set threshold, for example, the first preset threshold may be 0.2, 0.3, etc., which is not limited here.
[0054] Optionally, the rotation frame translation operation may be an operation of translating the candidate frame.
[0055] Optionally, the second overlap calculation may be used to calculate the overlap degree between two candidate frames.
[0056] Optionally, the second overlap value may be used to characterize the degree of overlap between two candidate frames. For example, the second overlap value may be 0.3, 0.4, etc., a value less than or equal to 1.0, which is not limited here.
[0057] Specifically, after obtaining the first overlap value, the electronic device can judge the first overlap value. If the first overlap value is less than the first preset threshold, the electronic device performs a second overlap calculation on the first target candidate frame and the second target candidate frame for the rotation frame translation operation to obtain a second overlap value. The specific implementation method is referred to below.
[0058] S140: Determine a second target candidate box whose second overlap value is greater than or equal to a second preset threshold as a candidate box with a split problem.
[0059] In the embodiment of the present disclosure, after obtaining the second overlap value, the electronic device may determine the second target candidate frame whose second overlap value is greater than or equal to the second preset threshold as a candidate frame with a splitting problem.
[0060] Optionally, the second preset threshold may be a pre-set threshold, for example, the second preset threshold may be 0.3, 0.4, etc., which is not limited here.
[0061] Optionally, the to-be-output list may include candidate boxes with high confidence and without splitting problems.
[0062] Specifically, after the electronic device obtains the second overlap value, if the second overlap value is greater than or equal to the second preset threshold, it means that the second target candidate box has a high degree of overlap with the first target candidate box. Therefore, it can be determined that the second target candidate box is a candidate box with a splitting problem, so that the second target candidate box can be removed and the first target candidate box can be added to the list to be output, and finally the list to be output including the candidate boxes with high confidence and no splitting problem is output.
[0063] Therefore, in the embodiment of the present disclosure, a first target candidate frame and at least one second target candidate frame can be determined from multiple candidate frames, and then for each second target candidate frame, a first overlap calculation is performed on the first target candidate frame and the second target candidate frame respectively to obtain a first overlap value for each second target candidate frame, and then a second overlap calculation is performed on the second target candidate frame whose first overlap value is less than a first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value, and finally the second target candidate frame whose second overlap value is greater than or equal to the second preset threshold is determined as a candidate frame with a splitting problem, thereby being able to determine the second target candidate frame with a splitting problem from multiple candidate frames through the first overlap value and the second overlap value, thereby effectively filtering the split second target candidate frames and improving the accuracy of the candidate frames.
[0064] Optionally, after S120, the candidate frame processing method may further include: determining a second target candidate frame whose first overlap value is greater than or equal to a first preset threshold as a candidate frame with a splitting problem.
[0065] In an embodiment of the present disclosure, after obtaining the first overlap value, the electronic device may determine the second target candidate frame whose first overlap value is greater than or equal to the first preset threshold as a candidate frame with a splitting problem.
[0066] Specifically, after obtaining the first overlap value, the electronic device can determine the second target candidate box whose first overlap value is greater than or equal to the first preset threshold as a candidate box with a split problem, so that the electronic device can remove the second target candidate box and add the first target candidate box to the to-be-output list that does not have a split problem.
[0067] In this way, the second target candidate frame that has a high degree of overlap with the first target candidate frame can be removed, and the split candidate frames can be effectively filtered.
[0068] Optionally, when the first overlap value is less than a first preset threshold, the candidate frame processing method may further include: calculating the distances between the center point of the first target candidate frame and the center point of the second target candidate frame and the coordinate origin respectively to obtain a first distance value and a second distance value; performing a first parameter calculation on the first distance value, a preset longitudinal tolerance ratio and a preset minimum longitudinal tolerance distance to obtain a maximum tolerance distance.
[0069] In an embodiment of the present disclosure, when the first overlap value is less than a first preset threshold, the electronic device can calculate the distances between the center points of the first target candidate frame and the second target candidate frame and the coordinate origin respectively to obtain a first distance value and a second distance value.
[0070] Optionally, the coordinate origin may be an origin in a rectangular coordinate system, for example, a rectangular coordinate system constructed with the center point of the vehicle as the origin.
[0071] Optionally, the first distance value may be a distance value between a center point of the first target candidate box and a coordinate origin.
[0072] Optionally, the second distance value may be a distance value between a center point of the second target candidate frame and a coordinate origin.
[0073] Specifically, when the first overlap value is less than a first preset threshold, the electronic device can calculate the distances between the center point of the first target candidate box and the center point of the second target candidate box and the coordinate origin respectively to obtain a first distance value and a second distance value. For example, the electronic device can determine the coordinates of the center point of the first target candidate box and the coordinates of the center point of the second target candidate box in a rectangular coordinate system constructed with the center point of the vehicle as the origin, and calculate the first distance value to the coordinate origin based on the coordinates of the center point of the first target candidate box, and calculate the second distance value to the coordinate origin based on the coordinates of the center point of the second target candidate box.
[0074] Furthermore, the electronic device may perform a first parameter calculation on the first distance value, the preset longitudinal tolerance ratio and the preset minimum longitudinal tolerance distance to obtain a maximum tolerance distance.
[0075] Optionally, the preset longitudinal tolerance ratio may be a pre-set longitudinal tolerance ratio.
[0076] Optionally, the preset minimum longitudinal tolerance distance may be a pre-set minimum longitudinal tolerance distance.
[0077] Optionally, the first parameter calculation may be a calculation for calculating a maximum tolerance distance parameter.
[0078] Specifically, after obtaining the first distance value, the electronic device may perform a first parameter calculation on the first distance value, the preset longitudinal tolerance ratio, and the preset minimum longitudinal tolerance distance to obtain a maximum tolerance distance. For example, the electronic device may obtain the first distance value D1 and the second distance value D2, and pre-set the preset longitudinal tolerance ratio L and the preset minimum longitudinal tolerance distance M, and then perform a first parameter calculation on the first distance value D1, the preset longitudinal tolerance ratio L, and the preset minimum longitudinal tolerance distance M, such as calculating through the first parameter calculation formula T=max(D1*L,M) to obtain the maximum tolerance distance T.
[0079] Optionally, when the first overlap value is less than a first preset threshold, the candidate frame processing method may further include: projecting the center point of the second target candidate frame onto the line connecting the center point of the first target candidate frame and the coordinate origin to obtain the first target point; calculating the distance between the first target point and the center point of the first target candidate frame to obtain a third distance value; performing a second parameter calculation on the third distance value, the maximum tolerance distance and the preset distance value to obtain a vertical penalty parameter.
[0080] In an embodiment of the present disclosure, the electronic device may calculate the first target point projected from the center point of the second target candidate frame to the line connecting the center point of the first target candidate frame and the coordinate origin.
[0081] Optionally, the first target point may be a point on a line connecting the center point of the first target candidate frame and the coordinate origin projected from the center point of the second target candidate frame.
[0082] Specifically, the electronic device may take a perpendicular line to the center point of the second target candidate frame, and take the intersection of the perpendicular line and a line connecting the center point of the first target candidate frame and the coordinate origin as the first target point, thereby obtaining the corresponding first target point.
[0083] Furthermore, after obtaining the first target point, the electronic device may calculate the distance between the first target point and the center point of the first target candidate frame to obtain a third distance value.
[0084] Optionally, the third distance value may be a distance value used to represent the distance between the first target point and the center point of the first target candidate frame.
[0085] Specifically, the electronic device may calculate the distance between the first target point and the center point of the first target candidate frame, for example, by calculating the coordinates of the center point of the first target candidate frame and the coordinates of the first target point to obtain a third distance value, recorded as lon_err.
[0086] Furthermore, the electronic device may perform a second parameter calculation on the third distance value, the maximum tolerance distance, and the preset distance value to obtain a longitudinal penalty parameter.
[0087] Optionally, the second parameter calculation may be calculation of a longitudinal penalty parameter.
[0088] Optionally, the preset distance value may be a pre-set distance value. For example, the preset distance value may be 1.0.
[0089] Specifically, the electronic device can perform a second parameter calculation on the third distance value, the maximum tolerance distance and the preset distance value. For example, the electronic device can perform a second parameter calculation on the third distance value lon_err, the maximum tolerance distance T and the preset distance value 1.0 through the second parameter calculation formula alpha_l=min(lon_err / T,1.0) to obtain the longitudinal penalty parameter alpha_l.
[0090] Optionally, S130 may specifically include: projecting the center point of the first target candidate box onto the line connecting the center point of the second target candidate box and the coordinate origin to obtain the second target point; determining the third target candidate box based on the second target point and the side length data of the second target candidate box, and the side length data of the second target candidate box includes a second distance value; performing rotation intersection and union ratio calculation on the first target candidate box and the third target candidate box, and multiplying them by the longitudinal penalty parameter to obtain a second overlap value.
[0091] In an embodiment of the present disclosure, the electronic device may project the center point of the first target candidate frame onto a line connecting the center point of the second target candidate frame and the coordinate origin to obtain a second target point.
[0092] Optionally, the second target point may be a point on a line connecting the center point of the second target candidate box and the coordinate origin projected from the center point of the first target candidate box.
[0093] Specifically, the electronic device can make a perpendicular line from the center point of the first target candidate frame to the line connecting the center point of the second target candidate frame and the coordinate origin, and use the intersection of the perpendicular line and the line connecting the center point of the second target candidate frame and the coordinate origin as the second target point. For example, the center point of the first target candidate frame can be X1, and the center point of the second target candidate frame can be X2. The center point X1 of the first target candidate frame is projected onto the line connecting the center point X2 of the second target candidate frame and the coordinate origin to obtain the second target point X2', wherein the coordinates of the second target point X2' can be calculated according to the corresponding coordinates of X1 and X2, and the second distance value D2, through the corresponding coordinate calculation formula: X2'=(X1·X2 / D2)*X2 / D2.
[0094] Furthermore, the electronic device may determine a third target candidate frame based on the second target point and the side length data of the second target candidate frame, where the side length data of the second target candidate frame includes the second distance value.
[0095] Optionally, the side length data may be length and width data of the candidate box.
[0096] Optionally, the third target candidate box may be a candidate box with the second target point as its origin and the side length data of the second target candidate box as its side length data.
[0097] Optionally, the side length data of the second target candidate box may include a second distance value.
[0098] Specifically, the electronic device can determine the third target candidate frame using the second target point as the origin and the side length data of the second target candidate frame as the side length data.
[0099] Furthermore, the electronic device may perform rotational intersection-over-union calculation on the first target candidate frame and the third target candidate frame, and multiply the calculation by a longitudinal penalty parameter to obtain a second overlap value.
[0100] Specifically, the electronic device may perform rotational intersection-and-union calculation on the first target candidate frame and the third target candidate frame, and multiply the result by the vertical penalty parameter alpha_l to obtain the second overlap value.
[0101] Figure 2 A schematic diagram of the structure of a candidate frame processing device provided by an embodiment of the present disclosure is shown.
[0102] In some embodiments of the present disclosure, Figure 2 The candidate frame processing device shown can be arranged in an electronic device. Specifically, the electronic device can include but is not limited to a mobile terminal such as a computer device, a mobile phone, a vehicle-mounted device, a vehicle controller, a tablet computer, a wearable device, etc.
[0103] like Figure 2 As shown, the candidate frame processing device 200 may include a first processing module 210 , a first calculation module 220 , a second calculation module 230 and a second processing module 240 .
[0104] The first processing module 210 may be configured to determine a first target candidate frame and at least one second target candidate frame from among multiple candidate frames.
[0105] The first calculation module 220 can be used to perform a first overlap calculation on the first target candidate frame and the second target candidate frame for each second target candidate frame, to obtain a first overlap value for each second target candidate frame.
[0106] The second calculation module 230 can be used to perform a second overlap calculation on the second target candidate frame whose first overlap value is less than the first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value.
[0107] The second processing module 240 can be used to determine a second target candidate frame whose second overlap value is greater than or equal to a second preset threshold as a candidate frame with a split problem.
[0108] Therefore, in the embodiment of the present disclosure, a first target candidate frame and at least one second target candidate frame can be determined from multiple candidate frames, and then for each second target candidate frame, a first overlap calculation is performed on the first target candidate frame and the second target candidate frame respectively to obtain a first overlap value for each second target candidate frame, and then a second overlap calculation is performed on the second target candidate frame whose first overlap value is less than a first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value, and finally the second target candidate frame whose second overlap value is greater than or equal to the second preset threshold is determined as a candidate frame with a splitting problem, thereby being able to determine the second target candidate frame with a splitting problem from multiple candidate frames through the first overlap value and the second overlap value, thereby effectively filtering the split second target candidate frames and improving the accuracy of the candidate frames.
[0109] In some embodiments of the present disclosure, the first target candidate box may be a candidate box with the highest confidence among multiple candidate boxes, and the second target candidate box may be a candidate box other than the first target candidate box among the multiple candidate boxes.
[0110] In some embodiments of the present disclosure, the first computing module 220 may specifically include a first computing unit.
[0111] The first calculation unit can be used to calculate the rotation intersection-and-union ratio of the first target candidate frame and the second target candidate frame to obtain a first overlap value.
[0112] In some embodiments of the present disclosure, the candidate frame processing device 200 may further include a third processing module.
[0113] The third processing module can be used to perform a first overlap calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value, and then determine the second target candidate frame whose first overlap value is greater than or equal to a first preset threshold as a candidate frame with a splitting problem.
[0114] In some embodiments of the present disclosure, the candidate box processing device 200 may further include a third calculation module and a fourth calculation module.
[0115] The third calculation module can be used to calculate the distances between the center point of the first target candidate frame and the center point of the second target candidate frame and the coordinate origin respectively to obtain a first distance value and a second distance value when the first overlap value is less than a first preset threshold.
[0116] The fourth calculation module can be used to perform a first parameter calculation on the first distance value, the preset longitudinal tolerance ratio and the preset minimum longitudinal tolerance distance to obtain a maximum tolerance distance.
[0117] In some embodiments of the present disclosure, the candidate box processing device 200 may further include a fifth calculation module, a sixth calculation module and a seventh calculation module.
[0118] The third calculation module can be used to project the center point of the second target candidate frame onto a line connecting the center point of the first target candidate frame and the coordinate origin to obtain a first target point when the first overlap value is less than a first preset threshold.
[0119] The sixth calculation module can be used to calculate the distance between the first target point and the center point of the first target candidate box to obtain a third distance value.
[0120] The seventh calculation module can be used to perform a second parameter calculation on the third distance value, the maximum tolerance distance and the preset distance value to obtain a longitudinal penalty parameter.
[0121] In some embodiments of the present disclosure, the second computing module 230 may specifically include a second computing unit, a target determination unit and a third computing unit.
[0122] The second calculation unit can be used to project the center point of the first target candidate box onto a line connecting the center point of the second target candidate box and the coordinate origin to obtain a second target point.
[0123] The target determination unit may be configured to determine a third target candidate frame based on the side length data of the second target point and the second target candidate frame, wherein the side length data of the second target candidate frame includes a second distance value.
[0124] The third calculation unit can be used to calculate the rotation intersection-over-union ratio of the first target candidate box and the third target candidate box, and multiply the vertical penalty parameter to obtain a second overlap value.
[0125] It should be noted that Figure 2 The candidate frame processing device 200 shown can execute Figure 1 The various steps in the method embodiment shown in the figure are implemented Figure 1 The various processes and effects in the method embodiment shown are not described in detail here.
[0126] Figure 3 A schematic diagram of the structure of a candidate frame processing device provided by an embodiment of the present disclosure is shown.
[0127] In some embodiments of the present disclosure, Figure 3 The candidate frame processing device shown may be an electronic device that wants to perform candidate frame processing. Specifically, the electronic device may include but is not limited to mobile terminals such as mobile phones, vehicle-mounted devices, vehicle controllers, tablet computers, wearable devices, smart home devices, etc.
[0128] like Figure 3As shown, the candidate frame processing device may include a processor 301 and a memory 302 storing computer program instructions.
[0129] Specifically, the processor 301 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0130] The memory 302 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 302 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 302 may be inside or outside the integrated gateway device. In a particular embodiment, the memory 302 is a non-volatile solid-state memory. In a particular embodiment, the memory 302 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically rewritable ROM (EAROM) or a flash memory, or a combination of two or more of these.
[0131] The processor 301 reads and executes the computer program instructions stored in the memory 302 to perform the steps of the candidate box processing method provided in the embodiment of the present disclosure.
[0132] In one example, the candidate frame processing device may further include a transceiver 303 and a bus 304. Figure 3 As shown, the processor 301, the memory 302 and the transceiver 303 are connected via a bus 304 and communicate with each other.
[0133] The bus 304 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 304 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.
[0134] The embodiments of the present disclosure further provide a computer-readable storage medium, which may store a computer program. When the computer program is executed by a processor, the processor implements the candidate frame processing method provided by the embodiments of the present disclosure.
[0135] The above-mentioned storage medium may, for example, include a memory 302 of computer program instructions, and the above-mentioned instructions may be executed by the processor 301 of the candidate frame processing device to complete the candidate frame processing method provided by the embodiment of the present disclosure. Optionally, the storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, a random access memory (Random Access Memory, RAM), a compact disc read-only memory (Compact Disc ROM, CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.
[0136] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0137] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A candidate frame processing method, characterized in that: include: Determine a first target candidate frame and at least one second target candidate frame from the multiple candidate frames; For each of the second target candidate frames, respectively perform a first overlap calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value for each of the second target candidate frames; Performing a second overlap calculation on the second target candidate frame whose first overlap value is less than a first preset threshold and the first target candidate frame after the rotation frame translation operation, to obtain a second overlap value; The second target candidate box whose second overlap value is greater than or equal to a second preset threshold is determined as a candidate box with a splitting problem.
2. The method according to claim 1, characterized in that The first target candidate box is a candidate box with the highest confidence among the multiple candidate boxes, and the second target candidate box is a candidate box among the multiple candidate boxes except the first target candidate box.
3. The method according to claim 1, characterized in that: The performing a first overlap calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value includes: A rotation intersection-union ratio is calculated for the first target candidate box and the second target candidate box to obtain the first overlap value.
4. The method according to claim 1, characterized in that After performing a first overlap calculation on the first target candidate frame and the second target candidate frame to obtain a first overlap value, the method further includes: The second target candidate box whose first overlap value is greater than or equal to a first preset threshold is determined as a candidate box with a splitting problem.
5. The method according to claim 1, characterized in that When the first overlap value is less than a first preset threshold, the method further includes: Calculating the distances between the center point of the first target candidate frame and the center point of the second target candidate frame and the coordinate origin respectively to obtain a first distance value and a second distance value; A first parameter calculation is performed on the first distance value, the preset longitudinal tolerance ratio and the preset minimum longitudinal tolerance distance to obtain a maximum tolerance distance.
6. The method according to claim 5, characterized in that When the first overlap value is less than a first preset threshold, the method further includes: Projecting the center point of the second target candidate frame onto a connecting line between the center point of the first target candidate frame and the coordinate origin to obtain a first target point; Calculating the distance between the first target point and the center point of the first target candidate box to obtain a third distance value; A second parameter calculation is performed on the third distance value, the maximum tolerance distance and the preset distance value to obtain a longitudinal penalty parameter.
7. The method according to claim 5, characterized in that The performing a second overlap calculation on the first target candidate frame and the second target candidate frame that are subjected to the rotation frame translation operation to obtain a second overlap value includes: Projecting the center point of the first target candidate frame onto the line connecting the center point of the second target candidate frame and the coordinate origin to obtain a second target point; determining a third target candidate box according to the side length data of the second target point and the second target candidate box, wherein the side length data of the second target candidate box includes the second distance value; A rotational intersection-over-union ratio is calculated for the first target candidate box and the third target candidate box, and the result is multiplied by the vertical penalty parameter to obtain the second overlap value.
8. A candidate frame processing device, characterized in that: include: A first processing module, configured to determine a first target candidate frame and at least one second target candidate frame from among the multiple candidate frames; A first calculation module is used to perform a first overlap calculation on the first target candidate box and the second target candidate box for each second target candidate box, to obtain a first overlap value for each second target candidate box; A second calculation module is used to perform a second overlap calculation on the second target candidate frame whose first overlap value is less than a first preset threshold and the first target candidate frame after the rotation frame translation operation to obtain a second overlap value; The second processing module is used to determine the second target candidate box whose second overlap value is greater than or equal to a second preset threshold as a candidate box with a split problem.
9. A candidate frame processing device, characterized in that: include: processor; A memory for storing executable instructions; The processor is used to read the executable instructions from the memory and execute the executable instructions to implement the candidate frame processing method described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the candidate frame processing method described in any one of claims 1 to 7.