Object detection method, device, electronic device and storage medium

Abstract

Embodiments of the present invention relate to an object detection method, device, electronic device, and storage medium. The method comprises: obtaining an image and distance information of a target scene captured simultaneously by a camera and a distance measuring device, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the distance measuring device; determining a target suppression threshold of a non-maximum suppression algorithm based on the two or more distance information, and determining multiple predicted object frames from the image; and determining a target object frame from the multiple predicted object frames using the non-maximum suppression algorithm and the target suppression threshold. This method can accurately identify overlapping objects within the target scene and reduce the rate of missed object recognition.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of target detection technology, and in particular to an object detection method, device, electronic device, and storage medium. Background Art

[0002] In the prior art, computer vision technology is used to determine whether an object exists in an image or video sequence and accurately locate the object to achieve object detection. Here, the object is, for example, a pedestrian, an animal, etc.

[0003] Taking pedestrian detection as an example, in practice, it's very likely that a scene will contain a large number of pedestrians. When there are a large number of pedestrians, they tend to cluster together. This can easily lead to missing identifications when capturing the scene from a single angle. This is especially true when two or more people overlap significantly, making them easily misidentified as a single person. Capturing the scene from multiple angles, however, is more expensive. Summary of the Invention

[0004] In view of this, in order to solve the problem that scene images are captured from a single angle during object detection and object detection based on the captured images is prone to missed recognition, an embodiment of the present invention provides an object detection method and device.

[0005] In a first aspect, an embodiment of the present invention provides an object detection method, the method comprising:

[0006] Acquire images and distance information of a target scene captured by a camera and a ranging device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the ranging device;

[0007] Determining a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and determining a plurality of predicted object frames from the image;

[0008] The non-maximum suppression algorithm and the target suppression threshold are used to determine a target object frame from the multiple predicted object frames.

[0009] In one possible implementation, determining the target suppression threshold of the non-maximum suppression algorithm based on two or more distance information includes:

[0010] determining an adjustment coefficient based on two or more distance information, wherein the adjustment coefficient is used to characterize the density of objects in the target scene;

[0011] A target suppression threshold of a non-maximum suppression algorithm is determined based on the adjustment coefficient.

[0012] In one possible implementation, determining the adjustment coefficient based on two or more pieces of distance information includes:

[0013] Determine a maximum distance value and a minimum distance value from two or more distance information;

[0014] determining a difference between the maximum distance and the minimum distance;

[0015] Dividing the difference by a first value to obtain a second value, where the first value is a value obtained by subtracting 1 from the number of prediction object frames;

[0016] Substitute the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient.

[0017] In one possible implementation, determining a target suppression threshold of a non-maximum suppression algorithm based on the adjustment coefficient includes:

[0018] The adjustment coefficient is set and operated with a preset standard suppression threshold to obtain a target suppression threshold of the non-maximum suppression algorithm.

[0019] In one possible implementation, in the step of substituting the second value and the preset standard suppression threshold into a preset formula to obtain the adjustment coefficient, the preset formula includes:

[0020]

[0021] Wherein, d is the second value, and thresh is the standard suppression threshold.

[0022] In one possible implementation, determining the maximum distance value and the minimum distance value from two or more pieces of distance information includes:

[0023] Determining whether there is target distance information among two or more distance information that meets a first set condition, where the first set condition includes that the distance information is greater than or equal to a preset distance threshold;

[0024] If the distance information exists and is greater than or equal to a preset distance threshold, a maximum distance value and a minimum distance value are determined from other distance information except the target distance information.

[0025] In one possible implementation, the determining the target object frame from the plurality of predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold includes:

[0026] Determine the prediction object frame with the highest prediction score among the current plurality of prediction object frames as the reference prediction object frame;

[0027] determining, for each prediction target frame other than the reference prediction target frame, a degree of overlap between the other prediction target frame and the reference prediction target frame;

[0028] The other predicted object frames that meet a second setting condition are determined as target object frames, where the second setting condition includes that the overlap degree corresponding to the predicted object frames is greater than the target suppression threshold.

[0029] In a second aspect, an embodiment of the present invention provides an object detection device, the device comprising:

[0030] A distance information acquisition module is used to acquire images and distance information of a target scene captured by a camera and a distance measuring device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the distance measuring device;

[0031] a target suppression threshold determination module, configured to determine a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and to determine a plurality of predicted object frames from the image;

[0032] The target object frame determination module is used to determine the target object frame from the multiple predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold.

[0033] In one possible implementation, the target suppression threshold determination module is specifically configured to:

[0034] determining an adjustment coefficient based on two or more distance information, wherein the adjustment coefficient is used to characterize the density of objects in the target scene;

[0035] A target suppression threshold of a non-maximum suppression algorithm is determined based on the adjustment coefficient.

[0036] In one possible implementation, the target suppression threshold determination module is specifically configured to:

[0037] Determine a maximum distance value and a minimum distance value from two or more distance information;

[0038] determining a difference between the maximum distance and the minimum distance;

[0039] Dividing the difference by a first value to obtain a second value, where the first value is a value obtained by subtracting 1 from the number of prediction object frames;

[0040] Substitute the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient.

[0041] In one possible implementation, the target suppression threshold determination module is specifically configured to:

[0042] The adjustment coefficient is set and operated with a preset standard suppression threshold to obtain a target suppression threshold of the non-maximum suppression algorithm.

[0043] In one possible implementation, the target suppression threshold determination module is specifically configured to:

[0044] In the step of substituting the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient, the preset formula includes:

[0045]

[0046] Wherein, d is the second value, and thresh is the standard suppression threshold.

[0047] In one possible implementation, the target suppression threshold determination module is further configured to:

[0048] Determining whether there is target distance information among two or more distance information that meets a first set condition, where the first set condition includes that the distance information is greater than or equal to a preset distance threshold;

[0049] If the distance information exists and is greater than or equal to a preset distance threshold, a maximum distance value and a minimum distance value are determined from other distance information except the target distance information.

[0050] In a possible implementation, the target object frame determination module is specifically configured to:

[0051] Determine the prediction object frame with the highest prediction score among the current plurality of prediction object frames as the reference prediction object frame;

[0052] determining, for each prediction target frame other than the reference prediction target frame, a degree of overlap between the other prediction target frame and the reference prediction target frame;

[0053] The other predicted object frames that meet a second setting condition are determined as target object frames, where the setting condition includes that the overlap degree corresponding to the predicted object frames is greater than the target suppression threshold.

[0054] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the processor is configured to execute an object detection method stored in the memory to implement the object detection method described in any one of the first aspects.

[0055] In a fourth aspect, an embodiment of the present invention provides a storage medium, characterized in that the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the object detection method described in any one of the first aspects.

[0056] The technical solution provided by an embodiment of the present invention obtains images and distance information of a target scene collected by a camera and a ranging device at the same time, wherein the target scene includes two or more objects, and the distance information is used to characterize the distance between the objects and the ranging device. Then, based on the two or more distance information, a target suppression threshold of a non-maximum suppression algorithm is determined, and multiple predicted object frames are determined from the image. Finally, the target object frame is determined from the multiple predicted object frames using the non-maximum suppression algorithm and the target suppression threshold. Since the suppression threshold of the non-maximum suppression algorithm is dynamically determined based on the distance information between the object and the ranging device, and the distance information between the object and the ranging device can characterize the number of objects at the same distance from the ranging device, that is, the density of the objects, the suppression threshold of the non-maximum suppression algorithm is dynamically adjusted according to the density of the objects in the target scene, thereby accurately identifying overlapping objects in the target scene and reducing the missed recognition rate of objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A schematic diagram of an application scenario illustrating an embodiment of the present invention;

[0058] Figure 2 A flowchart of an object detection method according to an exemplary embodiment of the present invention;

[0059] Figure 3 A schematic diagram of a target scene image captured by a camera according to an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of target scene distance information collected by a distance measuring device according to an embodiment of the present invention;

[0061] Figure 5 A flowchart of another object detection method provided by an exemplary embodiment of the present invention;

[0062] Figure 6 A flow chart of another object detection method provided by an exemplary embodiment of the present invention;

[0063] Figure 7 Schematic diagram of the relationship between the adjustment coefficient f(d) and d shown in an embodiment of the present invention;

[0064] Figure 8A block diagram of an object detection device according to an exemplary embodiment of the present invention;

[0065] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0067] See also Figure 1 , which is a schematic diagram of an application scenario shown in an embodiment of the present invention.

[0068] Figure 1 The application scenario shown includes: a device 101 , a camera 102 , a distance measuring device 103 , and a target scene 104 .

[0069] The device 101 may be a hardware device or software that supports network connection and provides various network services. When the device 101 is hardware, it may support various electronic devices with display screens, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, servers, etc. Figure 1 Only a desktop computer is used as an example. When the device 101 is software, it can be installed in the electronic devices listed above.

[0070] The camera 102 is used to capture images of the target scene 104. In practice, the camera 102 can be a dome camera, a gun camera, a fisheye camera, etc.

[0071] The distance measuring device 103 is used to collect the distance between the pedestrians in the target scene 104 and the distance measuring device 103. In practice, the distance measuring device 103 can be a TOF (Time of Flight) sensor, a microwave radar, or the like.

[0072] For example, if ranging device 103 is a TOF sensor, the TOF sensor emits modulated near-infrared light, which is reflected by an object. The sensor calculates the time difference or phase difference between light emission and reflection to convert the distance of the captured object into depth information. Combined with the image captured by camera 102, this information can be used to obtain three-dimensional profile data of the pedestrian. Device 101 clusters the three-dimensional profile data returned by the TOF sensor, grouping data corresponding to the same pedestrian into the same cluster. The distance between the pedestrian and the ranging device can then be determined based on the values ​​in each cluster.

[0073] The camera 102 and the ranging device 103 are respectively connected to the device 101 for communication. In this way, the camera 102 and the ranging device 103 can respectively send the collected images and distance information to the device 101, and then the device 101 executes the object detection method provided by the embodiment of the present invention to detect objects in the target scene 104, that is, pedestrians.

[0074] The object detection method provided by the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. The embodiments do not limit the embodiments of the present invention.

[0075] See also Figure 2 , is a flow chart of an embodiment of an object detection method provided by an exemplary embodiment of the present invention. As an embodiment, Figure 2 The process shown can be applied to Figure 1 On the device 101 shown in the example, that is, taking pedestrian detection as an example, Figure 2 As shown, the process may include the following steps:

[0076] Step 201: Acquire images and distance information of a target scene captured by a camera and a ranging device at the same time. The target scene includes two or more objects, and the distance information is used to represent the distance between the object and the ranging device.

[0077] In the embodiment of the present invention, the camera 102 and the distance measuring device 103 can respectively periodically collect images and distance information of the target scene 104, and send the collected images and distance information to the device 101 in real time. In this way, the device 101 can obtain the images and distance information of the target scene 104 collected by the camera 102 and the distance measuring device 103. Here, the distance information is used to characterize the object (in Figure 1 In the application scenario shown, the object is a pedestrian) and the distance between the distance measuring device 103. Figure 3 , which is a schematic diagram of the target scene 104 image captured by the camera 102, see Figure 4 , is a schematic diagram of the distance information of the target scene 104 collected by the ranging device. Wherein, d1 represents Figure 3The distance between pedestrian A and the distance measuring device 103, d2 represents Figure 3 The distance between pedestrian B and the ranging device 103.

[0078] Furthermore, in order to realize pedestrian detection in the target scene 104 based on the image of the target scene 104 captured by the camera 102 and the distance information of the target scene 104 captured by the ranging device 103, the device 101 obtains in step 201 the image and distance information of the target scene 104 captured by the camera 102 and the ranging device 103 at the same time.

[0079] Specifically, as an optional implementation method, the camera 102 and the ranging device 103 can collect images and distance information of the target scene 104 according to the same period, so that the device 101 receives the images and distance information of the target scene 104 collected by the camera 102 and the ranging device 103 at the same time each time.

[0080] As another optional implementation method, the camera 102 and the ranging device 103 can collect images and distance information of the target scene 104 according to different periods. Then, after receiving the image and distance information of the target scene 104, the device 101 can obtain the image and distance information corresponding to the same moment according to their respective timestamps, that is, obtain the image and distance information of the target scene 104 collected by the camera 102 and the ranging device 103 at the same moment.

[0081] Step 202: Determine a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information.

[0082] As can be seen from the description of step 202, in the embodiment of the present invention, when the non-maximum suppression algorithm is used to detect objects from an image, a preset and fixed suppression threshold is not used. Instead, the target suppression threshold of the non-maximum suppression algorithm is determined based on two or more distance information obtained in step 201. Through this processing, the suppression threshold of the non-maximum suppression algorithm can be dynamically adjusted according to the density of objects in the target scene, thereby accurately identifying overlapping objects in the target scene and reducing the missed recognition rate of objects.

[0083] Specifically, in one embodiment, Figure 5 As shown, determining the target suppression threshold of the non-maximum suppression algorithm based on two or more distance information may include the following steps:

[0084] Step 501: Determine an adjustment coefficient based on two or more distance information.

[0085] First, the adjustment coefficient is used to represent the density of pedestrians in the target scene 104. Specifically, a larger adjustment coefficient indicates a higher density of pedestrians in the target scene 104, and a smaller adjustment coefficient indicates a lower density of pedestrians in the target scene 104.

[0086] In one embodiment, the density of pedestrians in the target scene 104 can be characterized by determining the average value of the distances between pedestrians at different distances relative to the ranging device 103. That is, the adjustment coefficient is determined by determining the average value of the distances between pedestrians at different distances relative to the ranging device 103.

[0087] Specifically, when obtaining the average value of the distances between pedestrians at different distances from the distance measuring device 103, it is assumed that the distances of the pedestrians from the distance measuring device 103 are d1, d2, d3, ..., d n-1 d n , where n is the number of pedestrians, n ≥ 2, then the average value d can be calculated by the following formula (1):

[0088]

[0089] Afterwards, the above average value d and the preset standard suppression threshold value may be substituted into the preset formula shown in the following formula (2) to obtain the adjustment coefficient.

[0090]

[0091] In the above formula (2), thresh is the standard suppression threshold.

[0092] Furthermore, the following formula (3) can be derived from formula (1):

[0093]

[0094] It can be seen that determining the adjustment coefficient based on two or more distance information includes: determining the maximum distance and the minimum distance from two or more distance information, and determining the difference between the maximum distance and the minimum distance; dividing the difference by the first value to obtain a second value, wherein the first value is the value after the number of predicted pedestrian frames is reduced by 1, and the second value is the above-mentioned average value d, and the second value and the preset standard suppression threshold are substituted into the preset formula shown in the above formula (three) to obtain the adjustment coefficient.

[0095] For ease of understanding, Figure 7 The diagram below shows the relationship between the adjustment coefficient f(d) and d. Figure 7As shown, as the value of d increases, it means that the density of pedestrians in the target scene 104 is greater, so the adjustment coefficient increases; when the distance approaches 0, it means that the density of pedestrians in the target scene 104 is smaller, so the adjustment coefficient approaches 1.

[0096] In addition, in one embodiment, when determining the maximum distance and the minimum distance from two or more distance information, it can be first determined whether there is target distance information that meets the first set condition in the two or more distance information. If so, the maximum distance and the minimum distance are determined from the other distance information other than the target distance information.

[0097] Here, the above-mentioned first setting condition can be that the distance information is greater than a preset first distance threshold, and / or the distance information is less than a preset second distance threshold. The first distance threshold and the second distance threshold can be set by the operator or limited by the test distance of the device itself. The present invention does not impose any restrictions on this.

[0098] It can be understood that when the distance information is greater than the preset first distance threshold, or the distance information is less than the preset second distance threshold, that is, when the distance information is too large or too small, it is very likely that it means that the distance information is not accurate. Therefore, the maximum distance value and the minimum distance value can be determined from other distance information except the distance information greater than the first distance threshold and / or the distance information other than the distance information less than the second distance threshold.

[0099] Step 502: Determine a target suppression threshold of the non-maximum suppression algorithm based on the adjustment coefficient.

[0100] In one embodiment, the adjustment coefficient is subjected to a setting operation with a preset standard suppression threshold to obtain a target suppression threshold of the non-maximum suppression algorithm. Specifically, the setting operation may be a multiplication operation.

[0101] It can be seen that in the embodiment of the present invention, the larger the adjustment coefficient, that is, the greater the density of pedestrians in the target scene, the larger the target suppression threshold; the smaller the adjustment coefficient, that is, the smaller the density of pedestrians in the target scene, the smaller the target suppression threshold.

[0102] Step 203: Determine multiple predicted object boxes from the image.

[0103] In one embodiment, the device 101 may input the acquired image into a pre-trained classifier to obtain a plurality of pedestrian frames (hereinafter referred to as predicted pedestrian frames for ease of description). Figure 3 The image shown is input into a pre-trained classifier, and three predicted pedestrian boxes can be obtained (for the convenience of description, they are referred to as predicted pedestrian box 1, predicted pedestrian box 2 and predicted pedestrian box 3 below)

[0104] In addition, the output result of the above-mentioned classifier may also include a prediction score corresponding to the predicted pedestrian box, which is used to represent the probability that the object in the predicted pedestrian box is a pedestrian. Specifically, the higher the prediction score, the higher the probability that the object in the predicted pedestrian box is a pedestrian; the lower the prediction score, the lower the probability that the object in the predicted pedestrian box is a pedestrian.

[0105] Step 204: Determine a target object frame from multiple predicted object frames using a non-maximum suppression algorithm and a target suppression threshold.

[0106] In one embodiment, see Figure 6 , using a non-maximum suppression algorithm and a target suppression threshold, determining a target object frame from multiple predicted object frames may include the following steps:

[0107] Step 601: Determine the prediction object frame with the highest corresponding prediction score among the current multiple prediction object frames as the reference prediction object frame.

[0108] In one embodiment, the predicted pedestrian frame with the highest prediction score, that is, the predicted pedestrian frame with the highest probability that the object is a pedestrian, is determined as the reference predicted pedestrian frame. Figure 3 Among them, the prediction score of the predicted pedestrian box 1 is the highest, so the predicted pedestrian box 1 is determined as the benchmark predicted pedestrian box.

[0109] Step 602: For each prediction target frame other than the reference prediction target frame, determine the degree of overlap between the other prediction target frame and the reference prediction target frame.

[0110] In one embodiment, for each other prediction object frame except the baseline prediction object frame, the degree of overlap between the other prediction object frame and the baseline prediction object frame is determined, and the degree of overlap is used to characterize the probability that the object in the other prediction object frame is a pedestrian. Specifically, the greater the degree of overlap, the lower the probability that the object in the other prediction object frame is a pedestrian; the smaller the degree of overlap, the higher the probability that the object in the other prediction object frame is a pedestrian.

[0111] As an optional implementation, the degree of overlap can be calculated by dividing the area of ​​the overlapping region between the other prediction object frame and the reference prediction object frame by the total area of ​​the other prediction object frame and the reference prediction object frame. The total area here refers to the sum of the area of ​​the other prediction object frame and the area of ​​the reference prediction object frame minus the area of ​​the overlapping region between the other prediction object frame and the reference prediction object frame.

[0112] Step 603: Determine other predicted object frames that meet a second set condition as target object frames. The second set condition includes that the overlap degree corresponding to the predicted object frames is greater than a target suppression threshold.

[0113] In one embodiment, as can be seen from step 602, a higher degree of overlap means a lower probability that the object within the predicted object frame is a pedestrian. Therefore, in step 603, the predicted object frames outside the predicted object frame that meet the second set condition, that is, the corresponding degree of overlap is greater than the target suppression threshold, are selected as target object frames. The number of target object frames constitutes the final pedestrian detection result.

[0114] For ease of understanding, the following specific examples are shown:

[0115] Assumptions Figure 3 The prediction score of the predicted pedestrian frame 1 is the highest. Then, according to the above description, the predicted pedestrian frame 1 is determined as the benchmark predicted pedestrian frame. Then, the overlap between the predicted pedestrian frame 2 and the benchmark predicted pedestrian frame 1 is determined (assuming it is 0.9), and the overlap between the predicted pedestrian frame 3 and the benchmark predicted pedestrian frame 1 is determined (assuming it is 0.85). Continuing to assume that the standard suppression threshold is 0.8, according to the above description, the predicted pedestrian frame 2 and the predicted pedestrian frame 3 whose overlap is greater than the standard suppression threshold are excluded, and the predicted pedestrian frame 1 is determined as the target pedestrian frame. It can be seen that according to the processing method in the prior art, Figure 3 The two pedestrians originally detected were detected as one pedestrian, resulting in pedestrian misidentification.

[0116] By applying the method provided by the embodiment of the present invention, assuming Figure 3 The distance between pedestrian A and pedestrian B is 2.7. According to the above description, d is 2.7. Substituting d into the above formula (3), we can get the adjustment coefficient f(d) (assuming it is 1.1). Further, the adjustment coefficient is set and calculated with the standard suppression threshold (assuming it is a multiplication operation), and the target suppression threshold is 0.88. At this time, according to the non-maximum suppression algorithm, the predicted pedestrian frame 2 with an overlap greater than the target suppression threshold is excluded, and the predicted pedestrian frame 1 and the predicted pedestrian frame 3 are determined to be the target pedestrian frame, that is, the detection result is 2 people, and they are correctly detected. Figure 3 No pedestrians were missed in the recognition.

[0117] It can be seen that the technical solution provided by the embodiment of the present invention is to obtain the image and distance information of the target scene collected by the camera and the ranging device at the same time, the target scene includes two or more objects, and the distance information is used to characterize the distance between the object and the ranging device. Then, based on the two or more distance information, the target suppression threshold of the non-maximum suppression algorithm is determined, and multiple predicted object frames are determined from the image. Finally, the target object frame is determined from the multiple predicted object frames using the non-maximum suppression algorithm and the target suppression threshold. Since the suppression threshold of the non-maximum suppression algorithm is dynamically determined based on the distance information between the object and the ranging device, and the distance information between the object and the ranging device can characterize the density of objects in the target scene, the suppression threshold of the non-maximum suppression algorithm is dynamically adjusted according to the density of objects in the target scene, thereby achieving accurate recognition of overlapping objects in the target scene and reducing the missed recognition rate of objects.

[0118] See also Figure 8 , is a block diagram of an embodiment of an object detection device provided by an exemplary embodiment of the present invention. Figure 8 As shown, the device includes:

[0119] A distance information acquisition module 81 is configured to acquire images and distance information of a target scene captured by a camera and a distance measuring device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the distance measuring device;

[0120] an object suppression threshold determination module 82, configured to determine an object suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and to determine a plurality of predicted object frames from the image;

[0121] The target object frame determination module 83 is configured to determine a target object frame from the plurality of predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold.

[0122] In one possible implementation, the target suppression threshold determination module 82 is specifically configured to:

[0123] determining an adjustment coefficient based on two or more distance information, wherein the adjustment coefficient is used to characterize the density of objects in the target scene;

[0124] A target suppression threshold of a non-maximum suppression algorithm is determined based on the adjustment coefficient.

[0125] In one possible implementation, the target suppression threshold determination module 82 is specifically configured to:

[0126] Determine a maximum distance value and a minimum distance value from two or more distance information;

[0127] determining a difference between the maximum value and the minimum value;

[0128] Dividing the difference by a first value to obtain a second value, where the first value is a value obtained by subtracting 1 from the number of prediction object frames;

[0129] Substitute the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient.

[0130] In one possible implementation, the target suppression threshold determination module 82 is specifically configured to:

[0131] The adjustment coefficient is set and operated with a preset standard suppression threshold to obtain a target suppression threshold of the non-maximum suppression algorithm.

[0132] In one possible implementation, the target suppression threshold determination module 82 is specifically configured to:

[0133] In the step of substituting the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient, the preset formula includes:

[0134]

[0135] Wherein, d is the second value, and thresh is the standard suppression threshold.

[0136] In one possible implementation, the target suppression threshold determination module 82 is further configured to:

[0137] Determining whether there is target distance information among two or more distance information that meets a first set condition, where the first set condition includes that the distance information is greater than or equal to a preset distance threshold;

[0138] If the distance information exists and is greater than or equal to a preset distance threshold, a maximum distance value and a minimum distance value are determined from other distance information except the target distance information.

[0139] In a possible implementation, the target object frame determination module 83 is specifically configured to:

[0140] Determine the prediction object frame with the highest prediction score among the current plurality of prediction object frames as the reference prediction object frame;

[0141] determining, for each prediction target frame other than the reference prediction target frame, a degree of overlap between the other prediction target frame and the reference prediction target frame;

[0142] The other predicted object frames that meet a second setting condition are determined as target object frames, where the second setting condition includes that the overlap degree corresponding to the predicted object frames is greater than the target suppression threshold.

[0143] Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. Figure 9 The electronic device 900 shown includes: at least one processor 901, a memory 902, at least one network interface 904 and another user interface 903. The various components in the electronic device 900 are coupled together via a bus system 905. It is understood that the bus system 905 is used to achieve connection and communication between these components. In addition to including a data bus, the bus system 905 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 905 is not described in detail. Figure 9 Various buses are labeled as bus system 905.

[0144] The user interface 903 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touchpad, or a touch screen).

[0145] It is understood that the memory 902 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 902 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0146] In some embodiments, the memory 902 stores the following elements, executable units, or data structures, or a subset thereof, or an extended set thereof: an operating system 9021 and application programs 9022 .

[0147] The operating system 9021 includes various system programs, such as a framework layer, a core library layer, and a driver layer, for implementing various basic services and handling hardware-based tasks. Application programs 9022 include various application programs, such as a media player (MediaPlayer) and a browser (Browser), for implementing various application services. Programs implementing the methods of the embodiments of the present invention may be included in application programs 9022.

[0148] In an embodiment of the present invention, by calling a program or instruction stored in the memory 902, specifically, a program or instruction stored in the application 9022, the processor 901 is configured to execute the method steps provided in each method embodiment, for example, including:

[0149] Acquire images and distance information of a target scene captured by a camera and a ranging device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the ranging device;

[0150] Determining a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and determining a plurality of predicted object frames from the image;

[0151] The non-maximum suppression algorithm and the target suppression threshold are used to determine a target object frame from the multiple predicted object frames.

[0152] The methods disclosed in the above embodiments of the present invention can be applied to or implemented by processor 901. Processor 901 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in processor 901 or by software instructions. The above processor 901 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software units in the decoding processor. The software units can be located in storage media well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 902 , and the processor 901 reads the information in the memory 902 and completes the steps of the above method in combination with its hardware.

[0153] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof.

[0154] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0155] The electronic device provided in this embodiment may be Figure 9The electronic device shown in FIG. 1 can perform the following operations: Figure 2 All steps of the pedestrian detection method in the Figure 2 For details on the technical effects of the pedestrian detection method shown, please refer to Figure 2 For the sake of brevity, the relevant description will not be repeated here.

[0156] An embodiment of the present invention further provides a storage medium (computer-readable storage medium). The storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and the memory may also include a combination of the aforementioned types of memory.

[0157] When one or more programs in the storage medium can be executed by one or more processors, the object detection method executed on the object detection device side can be implemented.

[0158] Acquire images and distance information of a target scene captured by a camera and a ranging device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the ranging device;

[0159] Determining a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and determining a plurality of predicted object frames from the image;

[0160] The non-maximum suppression algorithm and the target suppression threshold are used to determine a target object frame from the multiple predicted object frames.

[0161] Professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0162] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0163] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An object detection method, characterized in that: The method comprises: Acquire images and distance information of a target scene captured by a camera and a ranging device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the ranging device; Determining a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and determining a plurality of predicted object frames from the image; Determining a target object frame from the plurality of predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold; Determining the target suppression threshold of the non-maximum suppression algorithm based on two or more distance information includes: An adjustment coefficient is determined based on two or more distance information, the adjustment coefficient being used to characterize the density of objects in the target scene; the adjustment coefficient is determined by determining an average value of distances between objects at different distances relative to a ranging device, and calculating the average value and a preset standard suppression threshold using the following formula to obtain the adjustment coefficient: Wherein, d is the average value, and thresh is the standard suppression threshold; A target suppression threshold of a non-maximum suppression algorithm is determined based on the adjustment coefficient.

2. The method according to claim 1, characterized in that The determining of the adjustment coefficient based on two or more distance information includes: Determine a maximum distance value and a minimum distance value from two or more distance information; determining a difference between the maximum distance and the minimum distance; Dividing the difference by a first value to obtain a second value, where the first value is a value obtained by subtracting 1 from the number of prediction object frames; Substitute the second value and the preset standard suppression threshold into a preset formula to obtain an adjustment coefficient.

3. The method according to claim 1, characterized in that Determining a target suppression threshold of a non-maximum suppression algorithm based on the adjustment coefficient includes: The adjustment coefficient is set and operated with a preset standard suppression threshold to obtain a target suppression threshold of the non-maximum suppression algorithm.

4. The method according to claim 2, characterized in that Determining the maximum distance value and the minimum distance value from two or more distance information includes: Determining whether there is target distance information among two or more distance information that meets a first set condition, where the first set condition includes that the distance information is greater than or equal to a preset distance threshold; If the distance information exists and is greater than or equal to a preset distance threshold, a maximum distance value and a minimum distance value are determined from other distance information except the target distance information.

5. The method according to claim 1, wherein The determining the target object frame from the plurality of predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold comprises: Determine the prediction object frame with the highest prediction score among the current plurality of prediction object frames as the reference prediction object frame; determining, for each prediction target frame other than the reference prediction target frame, a degree of overlap between the other prediction target frame and the reference prediction target frame; The other predicted object frames that meet a second setting condition are determined as target object frames, where the second setting condition includes that the overlap degree corresponding to the predicted object frames is greater than the target suppression threshold.

6. An object detection device, characterized in that The device comprises: A distance information acquisition module is used to acquire images and distance information of a target scene captured by a camera and a distance measuring device at the same time, wherein the target scene includes two or more objects, and the distance information is used to represent the distance between the objects and the distance measuring device; a target suppression threshold determination module, configured to determine a target suppression threshold of a non-maximum suppression algorithm based on two or more distance information, and to determine a plurality of predicted object frames from the image; a target object frame determination module, configured to determine a target object frame from a plurality of predicted object frames by using the non-maximum suppression algorithm and the target suppression threshold; Determining the target suppression threshold of the non-maximum suppression algorithm based on two or more distance information includes: An adjustment coefficient is determined based on two or more distance information, the adjustment coefficient being used to characterize the density of objects in the target scene; the adjustment coefficient is determined by determining an average value of distances between objects at different distances relative to a ranging device, and calculating the average value and a preset standard suppression threshold using the following formula to obtain the adjustment coefficient: Wherein, d is the average value, and thresh is the standard suppression threshold; A target suppression threshold of a non-maximum suppression algorithm is determined based on the adjustment coefficient.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is configured to execute an object detection method stored in the memory to implement the object detection method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the object detection method according to any one of claims 1 to 5.

Images