Detection frame filtering method and device and electronic equipment

Abstract

The invention is applied to the field of intelligent driving, and relates to a detection frame filtering method and device and electronic equipment, and the method comprises the steps: carrying out the target detection processing of a target image, and obtaining a candidate detection frame of a target object in the target image; performing image semantic segmentation processing on the target image to obtain a tag graph of the target image; based on the label graph and the candidate detection box, determining score information of the candidate detection box; and filtering the candidate detection box according to the scoring information to obtain a detection box of the target object. The accuracy of the detection frame of the target object is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving technology, and in particular to a detection frame filtering method, device and electronic equipment. Background Art

[0002] Perception is an important part of autonomous driving technology. It uses cameras and other devices to collect images, and obtains the detection frame of the target object in the image based on deep neural network detection to determine the location of the target object.

[0003] In the related art, due to environmental reasons such as weather, the quality of the collected image is poor, which leads to inaccurate detection frames obtained based on the collected image, that is, inaccurate target objects are detected. Therefore, a method that can filter the detection frame and improve the accuracy of target detection is urgently needed. Summary of the invention

[0004] One of the purposes of the present invention is to provide a method for filtering a detection frame to solve the problem of low accuracy of the detection frame of the target object in the prior art; the second purpose is to provide a device for filtering a detection frame; and the third purpose is to provide an electronic device.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for filtering a detection frame, comprising:

[0007] Performing target detection processing on the target image to obtain a candidate detection frame of the target object in the target image;

[0008] Performing image semantic segmentation processing on the target image to obtain a label map of the target image;

[0009] Determining score information of the candidate detection frame based on the label map and the candidate detection frame;

[0010] The candidate detection frames are filtered according to the scoring information to obtain a detection frame of the target object.

[0011] Further, based on the label map and the candidate detection frame, determining the score information of the candidate detection frame includes:

[0012] Based on the label map and the candidate detection frame, determining an evaluation factor of the candidate detection frame; wherein the evaluation factor is used to evaluate the accuracy of the candidate detection frame based on at least one dimension;

[0013] According to the evaluation factor, score information of the candidate detection box is determined.

[0014] Further, the evaluation factor includes a first factor, and the first factor is used to evaluate the accuracy of the candidate detection frame based on the category; based on the label map and the candidate detection frame, determining the evaluation factor of the candidate detection frame includes:

[0015] Determine at least one first pixel point of the candidate detection frame;

[0016] Determine a second pixel point corresponding to the first pixel point in the label image;

[0017] A first factor of the candidate detection box is determined according to the category of the first pixel and the category of the second pixel.

[0018] Further, determining at least one first pixel point of the candidate detection box includes:

[0019] Divide the candidate detection frame into a preset number of grids, and use the intersection of the grids as the first pixel point; or,

[0020] Based on the pixel coordinates of the center point of the candidate detection frame, points are taken along a preset direction to obtain at least one first pixel point.

[0021] Further, determining a first factor of the candidate detection frame according to the category of the first pixel point and the category of the second pixel point includes:

[0022] Determining a score matrix for the candidate detection frame according to the category of the first pixel point and the category of the second pixel point; a first value in the score matrix indicates that the category of the first pixel point is consistent with the category of the second pixel point, and a second value in the score matrix indicates that the category of the first pixel point is inconsistent with the category of the second pixel point;

[0023] A first factor of the candidate detection box is determined according to the score matrix.

[0024] Further, determining a first factor of the candidate detection box according to the score matrix includes:

[0025] Determine an intermediate matrix according to the score matrix and a preset weight matrix;

[0026] A first factor of the candidate detection box is determined according to the sum of the elements in the intermediate matrix.

[0027] Further, the evaluation factor includes a second factor, and the second factor is used to evaluate the accuracy of the candidate detection frame based on the position; based on the label map and the candidate detection frame, determining the evaluation factor of the candidate detection frame includes:

[0028] Determine a projection area of ​​the candidate detection box in the label image based on vertex coordinates of the candidate detection box;

[0029] In the label map, determining a target area corresponding to the category corresponding to the candidate detection frame;

[0030] Determine the intersection area and the union area of ​​the projection area and the target area;

[0031] A second factor of the candidate detection box is obtained based on a ratio of the intersection area to the union area.

[0032] Further, performing target detection processing on the target image to obtain a candidate detection frame of the target object in the target image includes:

[0033] Inputting the target image into a preset target detection model to obtain an initial candidate detection frame of the target object in the target image; wherein each initial candidate detection frame has a confidence level;

[0034] Initial candidate detection frames whose confidences are less than or equal to a preset confidence threshold are removed to obtain candidate detection frames of the target object in the target image.

[0035] Further, performing image semantic segmentation processing on the target image to obtain a label map of the target image includes:

[0036] The target image is input into a preset image semantic segmentation model to obtain a label map of the target image.

[0037] Further, filtering the candidate detection frames according to the scoring information to obtain the detection frame of the target object includes:

[0038] The candidate detection frames whose score information is less than or equal to a preset score threshold are deleted to obtain the detection frame of the target object.

[0039] A detection frame filtering device, comprising:

[0040] A first processing module is used to perform target detection processing on the target image to obtain a candidate detection frame of the target object in the target image;

[0041] A second processing module is used to perform image semantic segmentation processing on the target image to obtain a label map of the target image;

[0042] A determination module, configured to determine score information of the candidate detection frame based on the label map and the candidate detection frame;

[0043] The third processing module is used to filter the candidate detection frames according to the scoring information to obtain the detection frame of the target object.

[0044] An electronic device, comprising: a memory, a processor;

[0045] The memory stores computer-executable instructions;

[0046] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the above-mentioned detection frame filtering method.

[0047] A computer-readable storage medium stores computer-executable instructions, which are used to implement a detection frame filtering method when executed by a processor.

[0048] A computer program product includes a computer program, and when the computer program is executed by a processor, a method for filtering a detection frame is implemented.

[0049] Beneficial effects of the present invention:

[0050] (1) The present invention enables an electronic device to determine the scoring information of a candidate detection frame based on a label map, and then use the scoring information to screen or filter the candidate detection frames, thereby obtaining a reliable and highly accurate detection frame of the target object, thereby ensuring the accuracy of the detection frame and improving the accuracy of target detection;

[0051] (2) The present invention enables the electronic device to use the label map and the candidate detection frame to obtain evaluation factors from multiple dimensions, and then obtain scoring information that takes multiple dimensions into consideration, so that the filtering of the candidate detection frame is more accurate and comprehensive, and the accuracy of the detection frame is guaranteed, thereby improving the accuracy of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention;

[0053] Figure 2 A flowchart of a method for filtering a detection frame provided by an embodiment of the present invention;

[0054] Figure 3 A flowchart of a method for filtering a detection frame provided by an embodiment of the present invention;

[0055] Figure 4 A schematic diagram of a candidate detection frame provided by an embodiment of the present invention;

[0056] Figure 5 A schematic diagram of the structure of a filtering device for a detection frame provided by an embodiment of the present invention;

[0057] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0059] The perception process in the field of autonomous driving is inseparable from target detection, which mainly uses the target detection model to process the images taken by the camera and identify various objects in the image, such as lane lines, vehicles, pedestrians, etc. Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present invention. Figure 1 As shown in the figure, during the object detection process, the object detection model will identify the location and category of the object in the image, such as using the detection box to represent the location of the object and output the category of the object. Figure 1 As shown, each object can be marked by a rectangular box, and the category of each object is marked ( Figure 1 The target detection model may be, for example, a YOLOx series model, etc., which is not limited in the embodiment of the present invention. However, in this method, due to environmental reasons such as weather, the quality of the collected image is poor, and thus the detection frame may be misdetected, resulting in inaccurate object recognition.

[0060] In view of this, an embodiment of the present invention provides a method for filtering detection frames, which uses the label map of the image after image semantic segmentation processing to score the detection frames of objects in the image, and then uses the scores to filter the detection frames of the objects, filtering out detection frames with low scores, thereby ensuring the accuracy of the detection frames and improving the accuracy of target detection.

[0061] The execution subject of the embodiment of the present invention may be an electronic device with processing capabilities, such as a server, a computer, a tablet computer, etc., and the embodiment of the present invention is not limited here. The present invention can be applied to the field of intelligent driving, and in particular, when the vehicle needs to detect the target object when performing the intelligent driving function, the detection frame filtering method provided by the embodiment of the present invention can be implemented to improve the accuracy of target detection. In some embodiments, the execution subject of the embodiment of the present invention may be an electronic device with processing capabilities such as an electronic control unit or a domain controller on the vehicle.

[0062] Figure 2This is a flow chart of a method for filtering a detection frame provided by an embodiment of the present invention. Figure 2 As shown, the method includes:

[0063] S101, performing target detection processing on a target image to obtain a candidate detection frame of a target object in the target image.

[0064] Exemplarily, the above-mentioned target image represents an image for target detection processing. It should be noted that the embodiment of the present invention does not limit the type, size and content of the target image, and can be set according to actual needs. The above-mentioned target object represents the object that needs to be identified in the target image, and the object is related to the content included in the target image. For example, if the target image is a road image captured by a camera on a vehicle, the object or object category can be elements such as roads, vehicles, lane lines, pedestrians, traffic lights, obstacles, etc. that need to be identified. It should be noted that the embodiment of the present invention does not limit the category and number of target objects, and can be set based on actual needs.

[0065] The candidate detection frame represents a possible detection frame belonging to the target object. The detection frame may be in the form of a 2D rectangular frame or other forms, which are not limited in the embodiments of the present invention. It should be understood that the target detection process can obtain the position information of the object (represented by the position information of the detection frame) and the object category. For example, there are 3 detection frames belonging to the category of vehicle (object), and these three detection frames can be used as candidate detection frames for the vehicle object.

[0066] In one possible implementation, the electronic device may input the target image into a preset target detection model to directly obtain a candidate detection frame of the target object in the target image. In another possible implementation, the electronic device may also input the target image into a preset target detection model to obtain multiple initial candidate detection frames of the target object in the target image, and each initial candidate detection frame has a confidence level. The electronic device may delete the initial candidate detection frame whose confidence level is less than or equal to the preset confidence level threshold based on the confidence level of each initial candidate detection frame and the preset confidence level threshold to obtain a candidate detection frame of the target object. The preset target detection model may be, for example, a YOLOx series model, etc., which is not limited in the embodiments of the present invention.

[0067] S102: Perform image semantic segmentation processing on the target image to obtain a label map of the target image.

[0068] Exemplarily, the label map is used to represent the object category of each pixel in the target image. In other words, the label map can identify and classify the objects included in the target image at a finer pixel level. For example, the label map can use different colors to distinguish different object categories, that is, use the color of the pixel to represent the object category; it can also use different values ​​to distinguish different object categories, that is, use the value of the pixel to represent the object category.

[0069] In some possible implementations, the electronic device may input the target image into a preset image semantic segmentation model to obtain a label map of the target image. The preset image semantic segmentation model may be, for example, a U-net (U-shaped Network), a variable group convolutional neural network (Vargnet), etc., which is not limited in the embodiments of the present invention.

[0070] S103: Determine score information of the candidate detection frame based on the label map and the candidate detection frame.

[0071] Exemplarily, the above-mentioned scoring information is used to evaluate the accuracy of the candidate detection box.

[0072] In some possible implementations, the electronic device determines, for each candidate detection frame, whether the category of the candidate detection frame is consistent with the category of each pixel point on the label map corresponding to the candidate detection frame, and determines the scoring information of the candidate detection frame based on the number of consistent pixels. For example, the number of consistent pixels can be used as the scoring information of the candidate detection frame.

[0073] In some possible implementations, the electronic device determines, for each candidate detection frame, a projection area of ​​the candidate detection frame projected onto the label map, and determines a target area of ​​the category corresponding to the candidate detection frame on the label map, and determines score information of the candidate detection frame based on the degree of overlap between the projection area and the target area. For example, the degree of overlap may be used as the score information.

[0074] In some possible implementations, the electronic device may determine initial scoring information based on the two implementations described above, and obtain scoring information for the candidate detection frame by multiplying the initial scoring information by a preset weight coefficient.

[0075] S104: Filter the candidate detection frames according to the scoring information to obtain a detection frame of the target object.

[0076] Exemplarily, the detection box of the target object represents a finally determined detection box, which may indicate the category and position information of the target object.

[0077] In some possible implementations, the electronic device may filter out candidate detection frames whose scoring information is less than or equal to the preset scoring threshold based on the scoring information and the preset scoring threshold, and obtain the detection frame of the target object. Alternatively, the electronic device may filter out candidate detection frames whose scoring information is greater than the preset scoring threshold based on the scoring information and the preset scoring threshold, and obtain the detection frame of the target object.

[0078] In the method for filtering detection frames provided by the embodiments of the present invention, the electronic device can perform image semantic segmentation processing on the target image to obtain a label map of the target image, and then determine the score information of the candidate detection frames of the target object in the acquired target image based on the label map, and finally filter the candidate detection frames based on the score information to obtain the detection frame of the target object. In this way, the electronic device can determine the score information of the candidate detection frames based on the label map, and then use the score information to screen or filter the candidate detection frames to obtain a reliable and highly accurate detection frame of the target object, thereby ensuring the accuracy of the detection frame and improving the accuracy of target detection.

[0079] Figure 3 This is a flow chart of a method for filtering a detection frame provided by an embodiment of the present invention. Figure 3 As shown, in this embodiment Figure 2 Based on the embodiment shown, the method for filtering the detection frame provided by the present invention is further described. The method includes:

[0080] S201, inputting a target image into a preset target detection model to obtain an initial candidate detection box of the target object in the target image.

[0081] Exemplarily, the electronic device may input the target image into a preset target detection model, and output the position information, category, and confidence of the initial candidate detection frame of the target object in the target image, that is, the initial candidate detection frame has confidence; the confidence may represent the probability of the object category to which the initial candidate detection frame belongs. For example, the position information of the initial detection frame may be the vertex coordinates of a rectangle.

[0082] S202: Remove initial candidate detection frames whose confidence is less than or equal to a preset confidence threshold, and obtain candidate detection frames of the target object in the target image.

[0083] Exemplarily, the electronic device may remove the initial candidate detection frames whose confidence is less than or equal to a preset confidence threshold based on the confidence of each initial candidate detection frame, to obtain the candidate detection frame of the target object in the target image.

[0084] S203: Input the target image into a preset image semantic segmentation model to obtain a label map of the target image.

[0085] Exemplarily, the electronic device may input the target image into a preset image semantic segmentation model and output a label map of the target image.

[0086] S204: Determine an evaluation factor of the candidate detection box based on the label map and the candidate detection box.

[0087] Exemplarily, the above evaluation factor is used to evaluate the accuracy of the candidate detection box based on at least one dimension.

[0088] In some possible implementations, the evaluation factor may include a first factor, and the first factor is used to evaluate the accuracy of the candidate detection box based on the category; in this implementation, step S204 may include the following steps:

[0089] S2041. Determine at least one first pixel point of a candidate detection box.

[0090] Exemplarily, the first pixel point represents a pixel point in the candidate detection box area in the target image.

[0091] In a possible implementation, the electronic device may divide the candidate detection frame into a preset number of grids, and use the intersection of the grids as the first pixel point. Figure 4 As shown, Figure 4 A schematic diagram of a candidate detection frame provided in an embodiment of the present invention, wherein the candidate detection frame may be divided into N*N grids, wherein N is an integer greater than 1, and N may be set based on actual needs, and the embodiment of the present invention is not limited thereto. Figure 4 It is just an example of dividing into 3*3; then the intersection of the grid is used as the first pixel point (refer to Figure 4 dots as shown).

[0092] In another possible implementation, the electronic device may select points along a preset direction based on the pixel coordinates of the center point of the candidate detection frame to obtain at least one first pixel point. For example, based on the pixel coordinates of the center point of the candidate detection frame, a pixel point may be determined as the first pixel point at a preset interval along a straight line passing through the center point.

[0093] In another possible implementation, the electronic device may randomly determine a preset number of pixel points in the candidate detection frame as the first pixel points.

[0094] S2042: Determine the second pixel point corresponding to the first pixel point in the label image.

[0095] Exemplarily, the second pixel point represents the projection of the first pixel point to the corresponding pixel point in the label image.

[0096] In a possible implementation, the electronic device may determine the coordinates of the second pixel point based on a preset transformation matrix and the coordinates of the first pixel point.

[0097] S2043. Determine a first factor of the candidate detection box according to the category of the first pixel and the category of the second pixel.

[0098] Exemplarily, the category of the first pixel is consistent with the object category corresponding to the candidate detection frame. As mentioned above, each pixel in the label map has a category, so the category of the second pixel can be determined based on the label map.

[0099] In some possible implementations, the electronic device may first determine a score matrix of the candidate detection frame according to the category of the first pixel and the category of the second pixel; wherein the elements in the score matrix correspond one-to-one with the first pixel or the second pixel in position, for example, Figure 4 As shown, there are 2*2 first pixel points, and the dimension of the score matrix can be 2*2.

[0100] Specifically, the electronic device can assign values ​​to the elements in the matrix based on whether the category of the first pixel point is consistent with the category of the second pixel point. For example, the first value in the score matrix represents that the category of the first pixel point is consistent with the category of the second pixel point, and the second value represents that the category of the first pixel point is inconsistent with the category of the second pixel point. The numerical values ​​of the first value and the second value are not limited in this embodiment of the present invention, for example, they can be 1 or 0.

[0101] Furthermore, the electronic device can determine the first factor of the candidate detection frame according to the score matrix. Specifically, the electronic device can use the sum of the values ​​of the elements in the score matrix as the first factor; or, determine an intermediate matrix according to the score matrix and a preset weight matrix; and then determine the first factor of the candidate detection frame according to the sum of the elements in the intermediate matrix.

[0102] Among them, the dimension of the weight matrix is ​​consistent with the dimension of the score matrix. In other words, the weight matrix can characterize the weight coefficient of each first pixel point. It should be noted that the embodiment of the present invention does not limit the content of the weight matrix, and can be set based on actual needs. For example, the weight matrix can be set to a Gaussian distribution matrix. The weight matrix can reflect the weight distribution in the spatial position, so that the pixels closer to the target center have greater influence. Then, the Hadamard product of the score matrix and the weight matrix can be calculated to obtain the intermediate matrix, and finally the numerical values ​​of all elements in the intermediate matrix are summed to obtain the first factor.

[0103] In this way, it is possible to determine whether the candidate detection frame obtained by target detection is accurate from the perspective of category, that is, from the perspective of category, obtain the first factor for evaluating the accuracy of the candidate detection frame.

[0104] In some possible implementations, the above-mentioned evaluation factor may include a second factor, which is used to evaluate the accuracy of the candidate detection frame based on the position; in this implementation, step S204 may include: determining the projection area of ​​the candidate detection frame in the label image based on the vertex coordinates of the candidate detection frame; in the label image, determining the target area corresponding to the category corresponding to the candidate detection frame; determining the intersection area and the union area of ​​the projection area and the target area; based on the ratio of the intersection area to the union area, obtaining the second factor of the candidate detection frame.

[0105] Exemplarily, as mentioned above, the target detection model can output the position information of the candidate detection frame, that is, the vertex coordinates of the candidate detection frame. The electronic device can calculate the projection point coordinates of the vertex coordinates of the candidate detection frame projected onto the label image based on the vertex coordinates of the candidate detection frame and a preset transformation matrix, and then obtain the projection area corresponding to the candidate detection frame in the label image, and the target area corresponding to the category corresponding to the candidate detection frame on the label image, that is, the area of ​​pixels consistent with the category corresponding to the candidate detection frame. The intersection area and union area of ​​the projection area and the target area can be calculated, and the ratio of the intersection area to the union area is used as the second factor of the candidate detection frame.

[0106] S205: Determine score information of the candidate detection box according to the evaluation factors.

[0107] Exemplarily, the electronic device may directly use the first factor or the second factor as the scoring information of the candidate detection frame, or may set a weight coefficient to jointly determine the scoring information of the candidate detection frame based on the first factor, the second factor and the weight coefficient.

[0108] S206 , deleting candidate detection frames whose score information is less than or equal to a preset score threshold, and obtaining a detection frame of the target object.

[0109] For example, the electronic device may filter out candidate detection frames whose scoring information is less than or equal to the preset scoring threshold based on the scoring information and the preset scoring threshold, and obtain the detection frame of the target object.

[0110] In the method for filtering detection frames provided by the embodiment of the present invention, the electronic device can obtain candidate detection frames based on the initial candidate detection frames of the target object output by the target detection model, and determine the evaluation factors of the candidate detection frames based on the label map after the image semantic segmentation processing, and then obtain the scoring information of the candidate detection frames according to the evaluation factors, and filter the detection frames of the target object using the scoring information. In this way, the label map and the candidate detection frames can be used to obtain evaluation factors from multiple dimensions, and then obtain scoring information considering multiple dimensions, so that the filtering of the candidate detection frames is more accurate and comprehensive, the accuracy of the detection frames is guaranteed, and the accuracy of target detection is improved.

[0111] Figure 5 A schematic diagram of the structure of a filtering device for a detection frame provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the detection frame filtering device 300 of the embodiment of the present invention includes:

[0112] The first processing module 301 is used to perform target detection processing on the target image to obtain a candidate detection frame of the target object in the target image;

[0113] The second processing module 302 is used to perform image semantic segmentation processing on the target image to obtain a label map of the target image;

[0114] A determination module 303 is used to determine the score information of the candidate detection box based on the label map and the candidate detection box;

[0115] The third processing module 304 is used to filter the candidate detection frames according to the scoring information to obtain the detection frame of the target object.

[0116] In some possible implementations, the determining module 303 is specifically configured to:

[0117] Based on the label map and the candidate detection frame, determining an evaluation factor of the candidate detection frame; wherein the evaluation factor is used to evaluate the accuracy of the candidate detection frame based on at least one dimension;

[0118] According to the evaluation factor, score information of the candidate detection box is determined.

[0119] In some possible implementations, the evaluation factor includes a first factor, and the first factor is used to evaluate the accuracy of the candidate detection box based on the category; the determination module 303 is specifically used to:

[0120] Determine at least one first pixel point of the candidate detection frame;

[0121] Determine a second pixel point corresponding to the first pixel point in the label image;

[0122] A first factor of the candidate detection box is determined according to the category of the first pixel and the category of the second pixel.

[0123] In some possible implementations, the determining module 303 is specifically configured to:

[0124] Divide the candidate detection frame into a preset number of grids, and use the intersection of the grids as the first pixel point; or,

[0125] Based on the pixel coordinates of the center point of the candidate detection frame, points are taken along a preset direction to obtain at least one first pixel point.

[0126] In some possible implementations, the determining module 303 is specifically configured to:

[0127] Determining a score matrix for the candidate detection frame according to the category of the first pixel point and the category of the second pixel point; a first value in the score matrix indicates that the category of the first pixel point is consistent with the category of the second pixel point, and a second value in the score matrix indicates that the category of the first pixel point is inconsistent with the category of the second pixel point;

[0128] A first factor of the candidate detection box is determined according to the score matrix.

[0129] In some possible implementations, the determining module 303 is specifically configured to:

[0130] Determine an intermediate matrix according to the score matrix and a preset weight matrix;

[0131] A first factor of the candidate detection box is determined according to the sum of the elements in the intermediate matrix.

[0132] In some possible implementations, the evaluation factor includes a second factor, and the second factor is used to evaluate the accuracy of the candidate detection box based on the position; the determination module 303 is specifically used to:

[0133] Determine a projection area of ​​the candidate detection box in the label image based on vertex coordinates of the candidate detection box;

[0134] In the label map, determining a target area corresponding to the category corresponding to the candidate detection frame;

[0135] Determine the intersection area and the union area of ​​the projection area and the target area;

[0136] A second factor of the candidate detection box is obtained based on a ratio of the intersection area to the union area.

[0137] In some possible implementations, the first processing module 301 is specifically configured to:

[0138] Inputting the target image into a preset target detection model to obtain an initial candidate detection frame of the target object in the target image; wherein each initial candidate detection frame has a confidence level;

[0139] Initial candidate detection frames whose confidences are less than or equal to a preset confidence threshold are removed to obtain candidate detection frames of the target object in the target image.

[0140] In some possible implementations, the second processing module 302 is specifically configured to:

[0141] The target image is input into a preset image semantic segmentation model to obtain a label map of the target image.

[0142] In some possible implementations, the third processing module 304 is specifically configured to:

[0143] The candidate detection frames whose score information is less than or equal to a preset score threshold are deleted to obtain the detection frame of the target object.

[0144] The device of the embodiment of the present invention can be used to execute the technical solution of any of the above-mentioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0145] Figure 6 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 6 As shown, the electronic device 500 may include: at least one processor 501 and a memory 502 .

[0146] The memory 502 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer-executable instructions.

[0147] The memory 502 may include a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0148] The processor 501 is used to execute the computer-executable instructions stored in the memory 502 to implement the method for filtering the detection frame described in the aforementioned method embodiment. Among them, the processor 501 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. Specifically, when implementing the method for filtering the detection frame described in the aforementioned method embodiment, the electronic device can be, for example, an electronic device with processing functions such as a server or a computer, or an electronic device with processing functions such as an electronic control unit or a domain controller on a vehicle.

[0149] Optionally, the electronic device 500 may further include a communication interface 503. In a specific implementation, if the communication interface 503, the memory 502 and the processor 501 are implemented independently, the communication interface 503, the memory 502 and the processor 501 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0150] Optionally, in a specific implementation, if the communication interface 503, the memory 502 and the processor 501 are integrated on a chip, the communication interface 503, the memory 502 and the processor 501 can communicate through an internal interface.

[0151] The present invention also provides a computer-readable storage medium, in which computer program instructions are stored. When a processor executes the computer program instructions, the above-mentioned detection frame filtering method is implemented.

[0152] The present invention also provides a computer program product, including a computer program, which implements the above detection frame filtering method when executed by a processor.

[0153] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium may be any available medium that can be accessed by a general or special purpose computer.

[0154] An exemplary readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in a dedicated integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in the filtering device of the detection frame.

[0155] The present invention also provides a car, including an electronic device for implementing the above detection frame filtering method.

[0156] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0157] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A method for filtering a detection frame, characterized in that: The method comprises: Performing target detection processing on the target image to obtain a candidate detection frame of the target object in the target image; Performing image semantic segmentation processing on the target image to obtain a label map of the target image; Determining score information of the candidate detection frame based on the label map and the candidate detection frame; The candidate detection frames are filtered according to the scoring information to obtain a detection frame of the target object.

2. The method according to claim 1, characterized in that Determining score information of the candidate detection frame based on the label map and the candidate detection frame includes: Based on the label map and the candidate detection frame, determining an evaluation factor of the candidate detection frame; wherein the evaluation factor is used to evaluate the accuracy of the candidate detection frame based on at least one dimension; According to the evaluation factor, score information of the candidate detection box is determined.

3. The method according to claim 2, characterized in that The evaluation factor includes a first factor, and the first factor is used to evaluate the accuracy of the candidate detection frame based on the category; Determining an evaluation factor of the candidate detection frame based on the label map and the candidate detection frame includes: Determine at least one first pixel point of the candidate detection frame; Determine a second pixel point corresponding to the first pixel point in the label image; A first factor of the candidate detection box is determined according to the category of the first pixel and the category of the second pixel.

4. The method according to claim 3, characterized in that Determining at least one first pixel point of the candidate detection frame includes: Divide the candidate detection frame into a preset number of grids, and use the intersection of the grids as the first pixel point; or, Based on the pixel coordinates of the center point of the candidate detection frame, points are taken along a preset direction to obtain at least one first pixel point.

5. The method according to claim 3, characterized in that: Determining a first factor of the candidate detection frame according to the category of the first pixel and the category of the second pixel includes: Determining a score matrix for the candidate detection frame according to the category of the first pixel point and the category of the second pixel point; a first value in the score matrix indicates that the category of the first pixel point is consistent with the category of the second pixel point, and a second value in the score matrix indicates that the category of the first pixel point is inconsistent with the category of the second pixel point; A first factor of the candidate detection box is determined according to the score matrix.

6. The method according to claim 5, characterized in that Determining a first factor of the candidate detection frame according to the score matrix includes: Determine an intermediate matrix according to the score matrix and a preset weight matrix; A first factor of the candidate detection box is determined according to the sum of the elements in the intermediate matrix.

7. The method according to claim 2, characterized in that The evaluation factor includes a second factor, and the second factor is used to evaluate the accuracy of the candidate detection frame based on the position; Determining an evaluation factor of the candidate detection frame based on the label map and the candidate detection frame includes: Determine a projection area of ​​the candidate detection box in the label image based on vertex coordinates of the candidate detection box; In the label map, determining a target area corresponding to the category corresponding to the candidate detection frame; Determine the intersection area and the union area of ​​the projection area and the target area; A second factor of the candidate detection box is obtained based on a ratio of the intersection area to the union area.

8. The method according to any one of claims 1 to 7, characterized in that: Performing target detection processing on the target image to obtain a candidate detection frame of the target object in the target image includes: Inputting the target image into a preset target detection model to obtain an initial candidate detection frame of the target object in the target image; wherein each initial candidate detection frame has a confidence level; Initial candidate detection frames whose confidences are less than or equal to a preset confidence threshold are removed to obtain candidate detection frames of the target object in the target image.

9. A detection frame filtering device, characterized in that: The device comprises: A first processing module is used to perform target detection processing on the target image to obtain a candidate detection frame of the target object in the target image; A second processing module is used to perform image semantic segmentation processing on the target image to obtain a label map of the target image; A determination module, configured to determine score information of the candidate detection frame based on the label map and the candidate detection frame; The third processing module is used to filter the candidate detection frames according to the scoring information to obtain the detection frame of the target object.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

Images