A method, apparatus, device, and storage medium for detecting and identifying dense targets.

Abstract

This invention relates to the field of vehicle visual target detection technology, and provides a method, apparatus, device, and storage medium for detecting and recognizing dense targets. The method includes: acquiring a test image; identifying target objects in the test image to obtain pre-recognition results; wherein the pre-recognition results include a set of pre-detection boxes for each pre-identified target in the test image; selecting the optimal detection box corresponding to the pre-identified target and the optimal detection box for any missed targets from the pre-recognition results according to a preset screening criterion; and summarizing the optimal detection boxes for the pre-identified targets and the missed targets as the target recognition result of the test image. This method for detecting and recognizing dense targets can accurately identify the detection boxes of highly overlapping targets within stacked detection boxes, thereby avoiding the loss of recognition of dense targets and increasing the accuracy of target recognition in images.

Description

Technical Field

[0001] This application relates to the field of vehicle visual target detection technology, specifically to a method, apparatus, device, and storage medium for detecting and recognizing dense targets. Background Technology

[0002] The main function of advanced driver assistance systems (ADAS) is to perceive the road environment and make corresponding decisions. Vehicle target detection is a core module of ADAS in road environment perception. Vehicle target detection involves analyzing images captured by the vehicle's vision camera sensors to perceive the location information of specific targets in the road environment. When detecting and identifying target objects in the image, the target detection algorithm generates multiple bounding boxes. Existing technologies typically use the Non-Maximum Suppression (NMS) algorithm to remove redundant bounding boxes generated by target detection and retain the optimal bounding box for different target objects. However, when applied to dense target recognition filtering, this algorithm filters out bounding boxes of similar targets with high overlap. Therefore, this method is prone to loss of recognition when identifying dense targets in an image, significantly reducing the accuracy of target recognition.

[0003] Therefore, there is a need to provide a method, apparatus, device, and storage medium for detecting and recognizing dense targets, which can accurately identify the detection boxes of highly overlapping targets when recognizing dense targets on an image, so as to avoid erroneous filtering and thus solve the above problems. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the present invention provides a method, apparatus, device and storage medium for detecting and recognizing dense targets. When filtering redundant detection boxes on an image, it can accurately identify the detection boxes of each highly overlapping target in the stacked detection boxes by combining the NMS algorithm and IOU calculation (Intersection over Union). This solves the technical problem in the prior art that the target detection algorithm easily filters out the detection boxes of highly overlapping targets when recognizing dense targets in the image, resulting in recognition loss.

[0005] The method for detecting and identifying dense targets provided by this invention includes:

[0006] Acquire the image to be tested;

[0007] The target objects in the image under test are identified to obtain a pre-identification result; wherein, the pre-identification result includes a set of pre-detection boxes in the image under test for identifying each pre-identified target;

[0008] According to the preset screening criteria, the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target are obtained from the pre-identification results. The optimal detection box of the missed target is the false detection result of the pre-identification box set for the pre-identified target.

[0009] The optimal detection bounding boxes of the pre-identified targets and the missed targets are summarized as the target recognition result of the tested image.

[0010] In one embodiment of the present invention, the step of identifying the target object in the tested image and obtaining a pre-identification result includes:

[0011] The image under test is input into the target recognition model;

[0012] The target object in the image under test is identified by the target recognition model, and multiple detection boxes for the target object are obtained.

[0013] Based on the coordinate and size information of the detection boxes, the multiple detection boxes are classified to obtain the pre-identification result of the target object recognition in the tested image, wherein the pre-identification result includes a set of pre-detection boxes for each pre-identified target in the tested image.

[0014] In one embodiment of the present invention, classifying the plurality of detection boxes based on the coordinate and size information of the detection boxes to obtain a pre-recognition result of the target object recognition in the tested image includes:

[0015] Based on the coordinate information of the detection boxes, the multiple detection boxes are classified and summarized into multiple container sets;

[0016] For each container set, the container set is classified into at least one sub-container set based on the size information of the detection frame;

[0017] In the image under test, the pre-identification target corresponding to each of the sub-container sets is determined, and the sub-container sets are used as the pre-detection box set corresponding to the pre-identification target.

[0018] In one embodiment of the present invention, the step of selecting the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target from the pre-identification results according to a preset screening criterion includes:

[0019] For each set of pre-detection boxes, the optimal detection box in the set is retained, and the detection boxes in the set that do not meet the screening criteria are deleted. The detection boxes that meet the screening criteria are transferred to other existing sets of pre-detection boxes or a new set of pre-detection boxes corresponding to a missed target is created.

[0020] The process iterates through each set of pre-detection boxes until only the optimal detection box remains in the set of pre-detection boxes corresponding to each pre-identified target and the missed target. The optimal detection box is the detection box with the highest confidence in the set of pre-detection boxes.

[0021] In one embodiment of the present invention, the step of deleting detection boxes that do not meet the screening criteria from each set of pre-detection boxes, and transferring detection boxes that meet the screening criteria from the set of pre-detection boxes to other existing sets of pre-detection boxes or creating a new set of pre-detection boxes corresponding to a missed target, includes:

[0022] For each of the aforementioned pre-detection box sets:

[0023] Determine the optimal detection box and other detection boxes from the pre-detection box set;

[0024] Calculate the intersection-union ratio (IUU) of the other detection boxes with the optimal detection box;

[0025] Combining the preset screening criteria and the intersection-union ratio, the optimal detection box is retained in the pre-detection box set, and other detection boxes in the pre-detection box set that do not meet the screening criteria are deleted. Other detection boxes in the pre-detection box set that meet the screening criteria are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created.

[0026] In one embodiment of the present invention, determining the optimal detection box and other detection boxes in the pre-detection box set includes:

[0027] The pre-detection boxes are sorted in descending order based on their confidence level.

[0028] In the set of pre-detection boxes, the detection box with the highest confidence level is selected as the optimal detection box, and the detection boxes other than the optimal detection box are selected as other detection boxes.

[0029] In one embodiment of the present invention, the step of combining a preset screening criterion and the intersection-union ratio (IUU) to retain the optimal detection box in the pre-detection box set and delete other detection boxes in the pre-detection box set that do not meet the screening criterion, and transferring other detection boxes in the pre-detection box set that meet the screening criterion to other existing pre-detection box sets or creating a new pre-detection box set corresponding to a missed target, includes:

[0030] If the intersection-union ratio of the other detection boxes with the optimal detection box is greater than or equal to the preset confidence threshold, then the other detection boxes are determined to not meet the screening criteria and are deleted from the pre-detection box set.

[0031] If the intersection-union ratio of the other detection boxes and the optimal detection box is less than the preset confidence threshold, then the other detection boxes are determined to meet the screening criteria, and the other detection boxes are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created.

[0032] If all other detection boxes meet the screening criteria in all existing pre-detection box sets, then the other detection boxes are transferred to a newly created pre-detection box set corresponding to a missed target.

[0033] The present invention provides a dense target detection and identification device, comprising:

[0034] The image acquisition module is used to acquire the image under test.

[0035] The image pre-detection module identifies target objects in the image under test and obtains pre-detection results; wherein, the pre-detection results include a set of pre-detection boxes in the image under test that identify each pre-detection target;

[0036] The pre-detection filtering module, according to a preset screening standard, filters the pre-identification results to obtain the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target, wherein the optimal detection box of the missed target is the false detection result of the pre-detection box set for the pre-identified target;

[0037] The result generation module is used to summarize the optimal detection boxes of the pre-identified targets and the missed targets as the target recognition results of the tested image.

[0038] The present invention provides an electronic device comprising: a storage device and one or more processors; the storage device is used to store one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the dense target detection and identification method as described in any of the preceding embodiments.

[0039] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer processor, causes the computer to perform the dense target detection and identification method described in any of the above embodiments.

[0040] The beneficial effects of the present invention are as follows: The method, apparatus, device, and storage medium for detecting and recognizing dense targets provided by the present invention can first classify multiple detection frames for target recognition based on position and size when recognizing dense targets in an image, so as to obtain a set of pre-detection frames for each pre-identified target. Then, based on the combination of NMS algorithm and IOU calculation, the set of pre-detection frames corresponding to each pre-identified target is traversed to filter out the detection frames of the pre-identified targets with high overlap, thereby accurately recognizing the detection frames of each highly overlapping target in the stacked detection frames, so as to avoid the loss of recognition of dense targets and increase the accuracy of target recognition in the image.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0043] Figure 1 This is a schematic diagram illustrating the logical process of a method for detecting and identifying dense targets, as shown in an exemplary embodiment of this application.

[0044] Figure 2 This is a flowchart illustrating a method for detecting and identifying dense targets, as shown in an exemplary embodiment of this application;

[0045] Figure 3 This is a flowchart illustrating step S2 as shown in an exemplary embodiment of this application;

[0046] Figure 4 This is a flowchart illustrating step S23 in an exemplary embodiment of this application;

[0047] Figure 5 This is a flowchart illustrating step S3 in an exemplary embodiment of this application;

[0048] Figure 6 This is a flowchart illustrating step S31 in an exemplary embodiment of this application;

[0049] Figure 7 This is a block diagram illustrating a dense target detection and identification device in an exemplary embodiment of this application;

[0050] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0051] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0052] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0053] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0054] Please see Figures 1 to 8 This invention provides a method, apparatus, device, and storage medium for detecting and recognizing dense targets. When filtering redundant detection boxes on an image, it combines the NMS algorithm and IOU calculation to accurately identify the detection boxes of each highly overlapping target in the stacked detection boxes. This solves the technical problem in the prior art where target detection algorithms easily filter out the detection boxes of highly overlapping targets when recognizing dense targets in an image, resulting in recognition loss.

[0055] like Figure 1 As shown, the present invention provides a method for detecting and recognizing dense targets. This recognition method is applied to the field of target recognition and can mark the identified target objects in the image under test with detection boxes so that users can visually distinguish the marked targets in the image under test.

[0056] In one embodiment of the present invention, the dense target detection and recognition method is applied to the visual recognition of vehicle-mounted assisted driving. It can perform target recognition analysis on the test image captured by the vision camera sensor on the vehicle to perceive the position information of specific targets in the road environment, such as front and rear wheels and road obstacles. When recognizing dense targets (such as dense cones) in the road environment, multiple detection boxes marking dense targets are stacked together and difficult to distinguish and mark. The dense target detection and recognition method first classifies multiple detection boxes for pre-identification based on the position and size information of the detection boxes to obtain a set of pre-detection boxes for each pre-identified target. Then, based on the combination of NMS algorithm and IOU calculation, it traverses the set of pre-detection boxes corresponding to each pre-identified target to filter out the optimal detection boxes of the pre-identified targets and the missed targets with high overlap with the pre-identified targets. This enables the marking of detection boxes for each dense target in the image to assist the user in visually observing the road conditions.

[0057] Please see Figure 1 and Figure 2 , Figure 1 and Figure 2 The flowchart shown is a method for detecting and identifying dense targets provided in an embodiment of the present invention, including the following steps:

[0058] S1. Acquire the image to be tested.

[0059] In one embodiment of the present invention, the dense target detection and recognition method is applied to the visual recognition module of vehicle-mounted assisted driving. Therefore, in step S1, the acquired test image is a road environment image captured by the visual camera sensor on the vehicle, such as a road condition image in front of the vehicle or a road condition image behind the vehicle. The target object to be detected and recognized for the test image is usually a vehicle or a road obstacle, such as dense cones on the road.

[0060] Next, step S2 is performed to identify the target objects in the image under test and obtain pre-identification results; wherein, the pre-identification results include a set of pre-detection boxes for identifying each pre-identified target in the image under test.

[0061] In step S2, the dense target detection and recognition method, when recognizing specified target objects in the test image, can pre-identify and label specified types of target objects in the test image using a target recognition model, thereby marking detection boxes for all pre-identified target objects in the test image. Then, based on the position and size information of the detection boxes in the test image, the obtained multiple detection boxes are classified to obtain pre-identification results, so as to initially determine the pre-identified targets corresponding to each region on the image. The pre-identification results include a set of pre-detection boxes corresponding to each pre-identified target, and the set of pre-detection boxes includes multiple detection boxes, the center coordinates and sizes of which are similar.

[0062] When detecting and recognizing a specified target object in the test image, multiple detection boxes with different confidence levels are generated for the regions of the pre-identified target objects in the test image. However, when pre-identifying dense targets with high overlap, multiple detection boxes marking different target object regions will be stacked together (for example, detection boxes of targets that are close to or partially overlap each other in the image will partially intersect). The stacked detection boxes cannot visually identify and distinguish densely distributed target objects in the image. It is necessary to classify the detection boxes to determine the region range of the corresponding pre-identified target marked by the detection boxes in different pre-selection box sets in the test image, so as to facilitate further selection and identification of the target objects actually marked by each detection box in the same region.

[0063] Specifically, such as Figure 3 As shown, in step S2 of one embodiment of the present invention, that is, identifying the target object in the image under test and obtaining a pre-identification result, the following steps are included:

[0064] S21. Input the image under test into the target recognition model;

[0065] It should be noted that the target recognition model used in step S21 is an image recognition model trained for a specified type of target. In one embodiment of the present invention, the target recognition model can be a convolutional neural network model trained for cone-shaped target recognition.

[0066] S22. The target object in the image under test is identified by the target recognition model, and multiple detection boxes for the target object are obtained; wherein, the multiple detection boxes are the markings drawn by the target recognition model for all regions in the image under test where the target object appears.

[0067] S23. Classify the multiple detection boxes based on the coordinate and size information of the detection boxes to obtain the pre-recognition result of the target object recognition in the tested image. The pre-recognition result includes a set of pre-detection boxes in the tested image corresponding to each pre-recognition target. The detection boxes in each set of pre-detection boxes are marked as target areas in the tested image within the same coordinate threshold range.

[0068] Specifically, such as Figure 4 As shown, in step S23, that is, classifying the multiple detection boxes based on the coordinate and size information of the detection boxes to obtain the pre-recognition result of the target object recognition in the tested image, the steps include:

[0069] S231. Based on the coordinate information of the detection boxes, the multiple detection boxes are classified and grouped into multiple container sets, wherein the coordinate information of the detection boxes includes the center coordinates of the detection box body in the image under test. Specifically, detection boxes whose center coordinate spacing in the image under test is less than a coordinate threshold are classified and grouped into the same container set, so that the center coordinates of the detection boxes contained in the same container set are all within the same center coordinate threshold range.

[0070] S232. For each container set, the container set is classified into at least one sub-container set according to the size information of the detection box, wherein the size information includes the side length information of the detection box in the image being tested.

[0071] In a container set, the center coordinates of all detection boxes contained in the container set are within the same center coordinate threshold range. That is to say, the detection boxes in the container set all define the range of the target object in the same location area of ​​the image being tested. However, the target objects marked by detection boxes of different sizes in this location area are not the same. Therefore, it is necessary to further classify the detection boxes in the container set according to the size information of the detection boxes in order to make a preliminary judgment on the pre-identified target marked by each detection box.

[0072] Specifically, for each container set, detection boxes whose side length differences are less than the side length threshold are classified and grouped into the same sub-container set. All detection boxes contained in each sub-container set are marked with the same size frame at the same position in the image being tested as the pre-identified target.

[0073] S233. Determine the pre-identification target corresponding to each of the sub-container sets in the image under test, and use the sub-container sets as the pre-detection box set corresponding to the pre-identification target.

[0074] Next, step S3 is executed: according to the preset screening criteria, the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target are obtained from the pre-identification results, wherein the optimal detection box of the missed target is the false detection result of the pre-identification box set for the pre-identified target.

[0075] In the pre-identification results, the detection boxes contained in each pre-detection box set are targets marked with boxes of similar size in the same location area of ​​the tested image. However, when marking dense targets, even different detection boxes of similar size in the same area may mark different targets with high overlap. If only the NMS algorithm is used to filter out detection boxes with low confidence, there is a high probability of mistakenly deleting detection boxes that have missed identifying targets. Therefore, the filtering rule used in step S3 combines the NMS algorithm and IOU calculation. When filtering redundant detection boxes, the cross-union ratio (IU) value between detection boxes is used to determine whether each detection box in each pre-detection box set has misidentified the pre-identified target, and whether the misidentified detection box is marking other pre-identified targets or a misidentified target that was missed. Thus, when filtering redundant detection boxes, detection boxes that have missed identifying targets are identified and retained, thereby improving the accuracy of dense target identification.

[0076] Specifically, such as Figure 1 and 5 As shown, in step S3 of one embodiment of the present invention, that is, according to a preset screening criterion, the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target are obtained from the pre-identification results, including the following steps:

[0077] S31. For each set of pre-detection boxes, retain the most suitable detection box in the set of pre-detection boxes, delete the detection boxes in the set of pre-detection boxes that do not meet the screening criteria, and transfer the detection boxes that meet the screening criteria to other existing sets of pre-detection boxes or create a new set of pre-detection boxes corresponding to a missed target; wherein, the most suitable detection box is the detection box with the highest confidence in the set of pre-detection boxes.

[0078] In step S31, based on the screening criteria, it is determined whether other detection boxes in the pre-detection box set are marked with the same pre-identification target relative to the optimal detection box. If it is determined that the other detection box does not meet the screening criteria (i.e., it is marked with the same pre-identification target as the optimal detection box), the detection box is deleted from the pre-detection box set. If it is determined that the other detection box meets the screening criteria (i.e., it is not marked with the same pre-identification target as the optimal detection box), the detection box is transferred to other existing pre-detection box sets for further screening. When the detection box meets the screening criteria of other existing pre-detection boxes, it means that the target marked by the detection box does not belong to any pre-identification target. Therefore, the detection box is transferred to a newly created pre-detection box set, which corresponds to a missed target.

[0079] S32. Iterate through each of the pre-detection box sets until only the optimal detection box remains in the pre-detection box sets corresponding to each pre-identified target and the missed target, wherein the optimal detection box is the detection box with the highest confidence in the pre-detection box set.

[0080] Furthermore, such as Figure 1 and 6 As shown, in step S31, that is, for each set of pre-detection boxes, the most suitable detection box in the set is retained, the detection boxes in the set that do not meet the screening criteria are deleted, and the detection boxes that meet the screening criteria are transferred to other existing sets of pre-detection boxes or to a newly created set of pre-detection boxes corresponding to a missed target, including the following steps:

[0081] For each of the aforementioned pre-detection box sets:

[0082] S311. Determine the optimal detection box and other detection boxes in the pre-detection box set. Specifically, sort all detection boxes in the pre-detection box set in descending order according to their confidence level; select the detection box with the highest confidence level in the pre-detection box set as the optimal detection box, and select the other detection boxes in the pre-detection box set as the other detection boxes.

[0083] S312. Calculate the intersection-union ratio (IUU) of the other detection frames and the optimal detection frame.

[0084] S313. Combining the preset screening criteria and the intersection-union ratio, retain the optimal detection box in the pre-detection box set, delete other detection boxes in the pre-detection box set that do not meet the screening criteria, and transfer other detection boxes in the pre-detection box set that meet the screening criteria to other existing pre-detection box sets or create a new pre-detection box set corresponding to a missed target.

[0085] Specifically, if the intersection-union ratio of the other detection boxes and the optimal detection box is greater than or equal to a preset confidence threshold, then the other detection boxes are determined to not meet the screening criteria (a higher intersection-union ratio indicates that the other detection boxes and the optimal detection boxes mark the same pre-identification target), and the other detection boxes are deleted from the pre-detection box set;

[0086] If the intersection-union ratio (IU) of the other detection boxes and the optimal detection box is less than a preset confidence threshold, then the other detection boxes are determined to meet the screening criteria (a lower IU indicates that the other detection boxes and the optimal detection boxes do not mark the same pre-identified target), and the other detection boxes are transferred to other existing pre-detection box sets so that the other detection boxes can continue to traverse other existing pre-detection box sets, thereby determining whether the other detection boxes mark the target corresponding to other pre-detection box sets;

[0087] When the other detections meet the screening criteria in all existing pre-detection box sets, it means that the target marked by the detection box does not belong to any of the identified pre-identified targets or missed targets. Therefore, the detection box is transferred to a new pre-detection box set, which corresponds to a new missed target.

[0088] Next, step S4 is executed to summarize the optimal detection boxes of the pre-identified targets and the missed targets as the target recognition results of the tested image.

[0089] In step S4, the optimal detection boxes of the pre-identified targets and the missed targets selected from the pre-identification results are retained as the target identification results. The target identification results are displayed on the image under test to assist the user in visually identifying densely distributed target objects in the image.

[0090] The beneficial effects of the present invention are as follows: The method, apparatus, device, and storage medium for detecting and recognizing dense targets provided by the present invention can first classify multiple detection frames for target recognition based on position and size when recognizing dense targets in an image, so as to obtain a set of pre-detection frames for each pre-identified target. Then, based on the combination of NMS algorithm and IOU calculation, the set of pre-detection frames corresponding to each pre-identified target is traversed to filter out the detection frames of the pre-identified targets with high overlap, thereby accurately recognizing the detection frames of each highly overlapping target in the stacked detection frames, so as to avoid the loss of recognition of dense targets and increase the accuracy of target recognition in the image.

[0091] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0092] Figure 7 This is a block diagram illustrating a dense target detection and recognition device 1100 according to an exemplary embodiment of this application. The device is specifically configured in a vehicle-assisted driving vision recognition unit, but it can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0093] like Figure 7 As shown, the exemplary dense target detection and recognition device 1100 includes: an image acquisition module 1101, an image pre-detection module 1102, a pre-detection filtering module 1103, and a result generation module 1104. Detailed descriptions of each functional module are as follows:

[0094] Image acquisition module 1101 is used to acquire the image under test;

[0095] The image pre-detection module 1102 identifies target objects in the image under test and obtains pre-detection results; wherein, the pre-detection results include a set of pre-detection boxes in the image under test for identifying each pre-detection target;

[0096] The pre-detection filtering module 1103 filters the pre-identification results according to preset screening criteria to obtain the optimal detection box corresponding to the pre-identification target and the optimal detection box of the missed target, wherein the optimal detection box of the missed target is the false detection result of the pre-detection box set for the pre-identification target;

[0097] The result generation module 1104 is used to summarize the optimal detection boxes of the pre-identified targets and the missed targets as the target recognition results of the tested image.

[0098] In one embodiment, the image pre-detection module 1102 is specifically used for:

[0099] The image under test is input into the target recognition model;

[0100] The target object in the image under test is identified by the target recognition model, and multiple detection boxes for the target object are obtained.

[0101] Based on the coordinate and size information of the detection boxes, the multiple detection boxes are classified to obtain the pre-identification result of the target object recognition in the tested image, wherein the pre-identification result includes a set of pre-detection boxes for each pre-identified target in the tested image.

[0102] In one embodiment, the image pre-detection module 1102 is specifically used for:

[0103] Based on the coordinate information of the detection boxes, the multiple detection boxes are classified and summarized into multiple container sets;

[0104] For each container set, the container set is classified into at least one sub-container set based on the size information of the detection frame;

[0105] In the image under test, the pre-identification target corresponding to each of the sub-container sets is determined, and the sub-container sets are used as the pre-detection box set corresponding to the pre-identification target.

[0106] In one embodiment, the pre-screening filtering module 1103 is specifically used for:

[0107] For each set of pre-detection boxes, the optimal detection box in the set is retained, and the detection boxes in the set that do not meet the screening criteria are deleted. The detection boxes that meet the screening criteria are transferred to other existing sets of pre-detection boxes or a new set of pre-detection boxes corresponding to a missed target is created.

[0108] The process iterates through each set of pre-detection boxes until only the optimal detection box remains in the set of pre-detection boxes corresponding to each pre-identified target and the missed target. The optimal detection box is the detection box with the highest confidence in the set of pre-detection boxes.

[0109] In one embodiment, the pre-screening filtering module 1103 is specifically used for:

[0110] For each of the aforementioned pre-detection box sets:

[0111] Determine the optimal detection box and other detection boxes from the pre-detection box set;

[0112] Calculate the intersection-union ratio (IUU) of the other detection boxes with the optimal detection box;

[0113] Combining the preset screening criteria and the intersection-union ratio, the optimal detection box is retained in the pre-detection box set, and other detection boxes in the pre-detection box set that do not meet the screening criteria are deleted. Other detection boxes in the pre-detection box set that meet the screening criteria are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created.

[0114] In one embodiment, the pre-screening filtering module 1103 is specifically used for:

[0115] The detection boxes are sorted in descending order according to their confidence level in the pre-detection box set; the detection box with the highest confidence level in the pre-detection box set is selected as the optimal detection box, and the detection boxes other than the optimal detection box are selected as other detection boxes.

[0116] In one embodiment, the pre-screening filtering module 1103 is specifically used for:

[0117] If the intersection-union ratio of the other detection boxes with the optimal detection box is greater than or equal to the preset confidence threshold, then the other detection boxes are determined to not meet the screening criteria and are deleted from the pre-detection box set.

[0118] If the intersection-union ratio of the other detection boxes and the optimal detection box is less than the preset confidence threshold, then the other detection boxes are determined to meet the screening criteria, and the other detection boxes are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created.

[0119] If all other detection boxes meet the screening criteria in all existing pre-detection box sets, then the other detection boxes are transferred to a newly created pre-detection box set corresponding to a missed target.

[0120] It should be noted that the dense target detection and identification device and the dense target detection and identification method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the dense target detection and identification device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0121] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the streaming data sequence quality detection method provided in the above embodiments.

[0122] Figure 8 A schematic diagram of a computer system suitable for implementing the embodiments of this application is shown. It should be noted that... Figure 8 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0123] like Figure 8 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203, such as performing the methods described in the above embodiments. The RAM 1203 also stores various programs and data required for system operation. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via a bus 1204. An Input / Output (I / O) interface 1205 is also connected to the bus 1204.

[0124] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0125] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0126] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0129] Another aspect of this application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer's processor, causes the computer to perform a method for detecting the quality of a current stream of data sequences. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0130] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the streaming data sequence quality detection method provided in the various embodiments described above.

[0131] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting and identifying dense targets, characterized in that, include: Acquire the image to be tested; The target objects in the image under test are identified to obtain a pre-identification result; wherein, the pre-identification result includes a set of pre-detection boxes in the image under test for identifying each pre-identified target; According to the preset screening criteria, the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target are obtained from the pre-identification results. The optimal detection box of the missed target is the false detection result of the pre-identification box set for the pre-identified target. The optimal detection bounding boxes of the pre-identified targets and the missed targets are summarized as the target recognition result of the tested image; The step of identifying the target object in the tested image and obtaining a pre-identification result includes: The image under test is input into the target recognition model; The target object in the image under test is identified by the target recognition model, and multiple detection boxes for the target object are obtained. Based on the coordinate and size information of the multiple detection boxes, the multiple detection boxes are classified to obtain the pre-identification result of the target object recognition in the tested image. The pre-identification result includes the set of pre-detection boxes of each pre-identified target in the tested image. The step of classifying the multiple detection boxes based on their coordinate and size information to obtain a pre-recognition result for the target object in the tested image includes: Based on the coordinate information of the multiple detection boxes, the multiple detection boxes are classified and summarized into multiple container sets; For each container set, based on the size information of multiple detection frames in the container set, the container set is classified and summarized into at least one sub-container set; In the image under test, the pre-identification target corresponding to each of the sub-container sets is determined, and the sub-container sets are used as the pre-detection box set corresponding to the pre-identification target.

2. The method for detecting and identifying dense targets according to claim 1, characterized in that, The step of selecting the optimal detection box corresponding to the pre-identified target and the optimal detection box for the missed target from the pre-identification results according to the preset screening criteria includes: For each set of pre-detection boxes, the optimal detection box in the set is retained, and the detection boxes in the set that do not meet the screening criteria are deleted. The detection boxes that meet the screening criteria are transferred to other existing sets of pre-detection boxes or a new set of pre-detection boxes corresponding to a missed target is created. The process is repeated until only the optimal detection box remains in the pre-detection box set corresponding to each pre-identified target and the missed target. The optimal detection box is the detection box with the highest confidence in the pre-detection box set.

3. The method for detecting and identifying dense targets according to claim 2, characterized in that, For each of the pre-detection box sets, the optimal detection box in the pre-detection box set is retained, and detection boxes that do not meet the screening criteria are deleted. Furthermore, detection boxes that meet the screening criteria are transferred to other existing pre-detection box sets or to newly created pre-detection box sets corresponding to a missed target. This includes: For each of the aforementioned pre-detection box sets: Determine the optimal detection box and other detection boxes from the pre-detection box set; Calculate the intersection-union ratio (IUU) of the other detection boxes with the optimal detection box; Combining the preset screening criteria and the intersection-union ratio, the optimal detection box is retained in the pre-detection box set, and other detection boxes in the pre-detection box set that do not meet the screening criteria are deleted. Other detection boxes in the pre-detection box set that meet the screening criteria are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created.

4. The method for detecting and identifying dense targets according to claim 3, characterized in that, The step of determining the optimal detection box and other detection boxes in the pre-detection box set includes: The pre-detection boxes are sorted in descending order based on their confidence level. In the pre-detection box set, the detection box with the highest confidence level is selected as the optimal detection box, and the detection boxes other than the optimal detection box are selected as other detection boxes.

5. The method for detecting and identifying dense targets according to claim 3, characterized in that, The step of combining the preset screening criteria and the intersection-union ratio (IUU) to retain the optimal detection box in the pre-detection box set and delete other detection boxes in the pre-detection box set that do not meet the screening criteria, and transferring other detection boxes in the pre-detection box set that meet the screening criteria to other existing pre-detection box sets or creating a new pre-detection box set corresponding to a missed target, includes: If the intersection-union ratio of the other detection boxes with the optimal detection box is greater than or equal to the preset confidence threshold, then the other detection boxes are determined to not meet the screening criteria and are deleted from the pre-detection box set. If the intersection-union ratio of the other detection boxes and the optimal detection box is less than the preset confidence threshold, then the other detection boxes are determined to meet the screening criteria, and the other detection boxes are transferred to other existing pre-detection box sets or a new pre-detection box set corresponding to a missed target is created. If all other detection boxes meet the screening criteria in all existing pre-detection box sets, then the other detection boxes are transferred to a newly created pre-detection box set corresponding to a missed target.

6. A device for detecting and identifying dense targets, characterized in that, include: Image acquisition module, used to acquire the image to be tested; The image pre-detection module identifies target objects in the image under test and obtains pre-detection results; wherein, the pre-detection results include a set of pre-detection boxes in the image under test that identify each pre-detection target; The pre-detection filtering module, according to a preset screening standard, filters the pre-identification results to obtain the optimal detection box corresponding to the pre-identified target and the optimal detection box of the missed target, wherein the optimal detection box of the missed target is the false detection result of the pre-detection box set for the pre-identified target; The result generation module is used to summarize the optimal detection boxes of the pre-identified targets and the missed targets as the target recognition results of the tested image; The image pre-detection module is used to input the image under test into the target recognition model; to identify target objects in the image under test through the target recognition model, and to obtain multiple detection boxes for identifying target objects; to classify the multiple detection boxes based on the coordinate and size information of the multiple detection boxes, and to obtain the pre-recognition result of the image under test for identifying target objects, wherein the pre-recognition result includes a set of pre-detection boxes for each pre-identified target in the image under test; The image pre-detection module classifies and categorizes the multiple detection boxes into multiple container sets based on their coordinate information. For each container set, it classifies and categorizes the container set into at least one sub-container set based on the size information of the multiple detection boxes in the container set. It then determines the pre-identification target corresponding to each of the sub-container sets in the image under test and uses the sub-container set as the pre-detection box set corresponding to the pre-identification target.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images