Target detection method, apparatus, device, and storage medium

Abstract

The present disclosure provides a target detection method and device, equipment and storage medium, relates to the technical field of artificial intelligence, specifically relates to the technical field of image processing, computer vision, virtual reality and deep learning, and can be applied to 3D vision, smart city, intelligent transportation and the like. The method comprises the following steps: acquiring a to-be-detected image; dividing the to-be-detected image into at least two image blocks based on depth information, wherein there is an overlap between the at least two image blocks; for each image block in the at least two image blocks, performing target detection on the image block to obtain a detection box corresponding to a target object in the image block; and in response to determining that at least one detection box corresponds to a same target object, determining a target detection box of the target object based on the position of the at least one detection box. The target detection method provided by the present disclosure improves the efficiency and accuracy of target detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of image processing, computer vision, virtual reality and deep learning, and can be applied to 3D vision, smart city, intelligent transportation and the like. BACKGROUND

[0002] Monocular 3D object detection is one of the most widely used computer vision technologies, which can be applied to vehicle automatic driving systems, intelligent robots, intelligent transportation and the like. At present, monocular 3D object detection strongly depends on a single model to estimate the 3D properties of target objects in the entire scene, while different models have different scene perception capabilities at different distances, and without special division, it is easy to overfit to a distance range of the scene, thereby affecting the performance of 3D object detection. SUMMARY

[0003] The present disclosure provides a target detection method, device, equipment and storage medium.

[0004] According to a first aspect of the present disclosure, a target detection method is provided, comprising: acquiring a to-be-detected image; dividing the to-be-detected image into at least two image blocks based on depth information, wherein there is an overlap between the at least two image blocks; for each image block in the at least two image blocks, performing target detection on the image block to obtain a detection box corresponding to a target object in the image block; and in response to determining that the same target object corresponds to at least one detection box, determining a target detection box of the target object based on a position of the at least one detection box.

[0005] According to a second aspect of the present disclosure, a target detection device is provided, comprising: an acquisition module configured to acquire a to-be-detected image; a division module configured to divide the to-be-detected image into at least two image blocks based on depth information, wherein there is an overlap between the at least two image blocks; a detection module configured to, for each image block in the at least two image blocks, perform target detection on the image block to obtain a detection box corresponding to a target object in the image block; and a determination module configured to, in response to determining that the same target object corresponds to at least one detection box, determine a target detection box of the target object based on a position of the at least one detection box.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in any one of the implementations of the first aspect.

[0007] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in any implementation form of the first aspect.

[0008] According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method described in any implementation form of the first aspect.

[0009] It should be understood that the details described in this section are not intended to identify key or critical features of the embodiments of the present disclosure, nor are they used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0011] Figure 1 is an exemplary system architecture diagram to which the present disclosure can be applied;

[0012] Figure 2 is a flowchart of one embodiment of the target detection method according to the present disclosure;

[0013] Figure 3 is a flowchart of another embodiment of the target detection method according to the present disclosure;

[0014] Figure 4 is a flowchart of yet another embodiment of the target detection method according to the present disclosure;

[0015] Figure 5 is a diagram of one application scenario of the target detection method according to the present disclosure;

[0016] Figure 6 is a structural schematic diagram of one embodiment of the target detection apparatus according to the present disclosure;

[0017] Figure 7 is a block diagram of an electronic device for implementing the target detection method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0019] It should be noted that the embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] Figure 1 An exemplary system architecture 100 is shown, to which embodiments of the object detection method or the object detection apparatus of the present disclosure can be applied.

[0021] As shown in Figure 1 The system architecture 100 can include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium to provide a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0022] A user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send information, etc. Various client applications can be installed on the terminal devices 101, 102, 103.

[0023] The terminal devices 101, 102, 103 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc. When the terminal devices 101, 102, 103 are software, they can be installed in the above-mentioned electronic devices. They can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made herein.

[0024] The server 105 can provide various services. For example, the server 105 can analyze and process a to-be-detected image obtained from the terminal devices 101, 102, 103, and generate a processing result (e.g. an object detection frame of a target object).

[0025] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (e.g. to provide distributed services), or as a single software or software module. No specific limitation is made herein.

[0026] It should be noted that the object detection method provided by the embodiments of the present disclosure is generally executed by the server 105, and correspondingly, the object detection apparatus is generally arranged in the server 105.

[0027] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0028] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a target detection method according to the present disclosure. The target detection method includes the following steps:

[0029] Step 201: Obtain the image to be detected.

[0030] In this embodiment, the execution body of the target detection method (e.g.) Figure 1 The server 105 shown will acquire the image to be detected. In this embodiment, the image to be detected can be acquired by a monocular camera. The monocular camera can send the acquired image to be detected to the execution entity in real time, or the monocular camera can send the acquired image to be detected to the execution entity at preset time intervals.

[0031] Step 202: Divide the image to be detected into at least two image blocks based on depth information.

[0032] In this embodiment, the execution entity divides the image to be detected into at least two image blocks based on depth information, wherein there is overlap between the at least two image blocks. Here, depth information refers to the depth range, which is also the perceptual range of the image to be detected, such as 0-30 meters, 30-50 meters, etc. For example, the execution entity can divide the image to be detected based on a distance adaptive strategy to obtain multiple image blocks with different depth ranges. The execution entity will divide the image to be detected along the vertical axis of the image. Generally, the execution entity will divide the image to be detected according to three depth ranges: near, middle, and far.

[0033] For example, if the execution entity determines that the overall perception range of the image to be detected is 70 meters, it will divide the entire space of the image into three image blocks representing near, medium, and far distances: 0-30 meters, 30-50 meters, and 50-70 meters. These three image blocks will overlap, ensuring that edge information is not lost during image segmentation and maintaining the integrity of the image information. It should be noted that because objects located closer to the image occupy a larger pixel space, the lower image blocks (i.e., the blocks with the smallest depth range of 0-30 meters) will be larger.

[0034] Optionally, in some scenarios, in order to ensure the consistency of the pixel ratio of the divided image blocks and the pixel ratio of the undivided image, the execution body divides the to-be-detected image into N*N image blocks, that is, the to-be-detected image is cut into N blocks in the vertical axis, and then cut in the horizontal axis, thereby obtaining N*N image blocks, where N is a positive integer.

[0035] In step 203, for each of the at least two image blocks, the target detection is performed on the image block to obtain a detection box corresponding to a target object in the image block.

[0036] In this embodiment, for each of the at least two divided image blocks, the execution body detects the target object in the image block to obtain a detection box corresponding to the target object. The target object can be any obstacle, such as a vehicle, a pedestrian, etc. For example, the execution body first obtains the depth range information of each image block, and inputs the image block into the detection model corresponding to the depth range, so that the detection model detects the target object in the image block to obtain a detection box corresponding to the target object. Here, a corresponding detection model is trained for each depth range, that is, for the depth range of 0-30 meters, training data of the depth range is collected, and the training data is used for training to obtain a corresponding detection model. Since the distance range covered by the to-be-detected image scene is large, it is difficult to directly use a single model to predict the 3D attributes of all objects in the scene, and the regression value is often not very accurate. Therefore, in this embodiment, the regression task is converted into a classification task, the to-be-detected image is classified and divided first, and then different detection models are used for 3D detection, thereby improving the accuracy of the detection result.

[0037] In step 204, in response to determining that at least one detection box corresponds to a same target object, a target detection box of the target object is determined based on a position of the at least one detection box.

[0038] In this embodiment, since there is an overlap between different image blocks, and each image block is detected using a corresponding detection model, the overlapping part in different image blocks will have multiple different detection boxes, that is, a same target object will correspond to multiple detection boxes. In addition, even for a same target object in a same image block, it can also correspond to multiple detection boxes. Based on this, the execution body determines a target detection box for the target object from the multiple detection boxes based on position information of the multiple detection boxes.

[0039] For example, if multiple detection boxes corresponding to the same target object are in the same image block, the target detection box can be determined based on the confidence value of the detection box. If multiple detection boxes corresponding to the same target object are in different image blocks, the execution subject will determine the target detection box of the target object from the multiple detection boxes based on a Non-Maximum Suppression (NMS) method. The NMS algorithm is widely used in target detection scenarios, and its purpose is to eliminate redundant candidate boxes and find the best detection box.

[0040] The target detection method provided by the embodiments of the present disclosure first acquires a to-be-detected image; then divides the to-be-detected image into at least two image blocks based on depth information; then performs target detection on each of the at least two image blocks to obtain a detection box corresponding to a target object in the image block; and finally, in response to determining that at least one detection box corresponds to the same target object, determines a target detection box of the target object based on a position of the at least one detection box. The target detection method in the embodiments first divides the to-be-detected image according to the depth range, and then performs detection in different distance ranges, thereby improving the accuracy of 3D target detection and the accuracy of the 3D target detection result.

[0041] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0042] With reference to the foregoing description of the first embodiment, the target detection method according to the second embodiment of the present disclosure will be described in detail. Figure 3 , Figure 3 Flow 300 shows another embodiment of the target detection method according to the present disclosure. The target detection method includes the following steps:

[0043] Step 301: Acquire a to-be-detected image.

[0044] Step 302: Divide the to-be-detected image into at least two image blocks based on depth information.

[0045] Step 303: Perform target detection on each of the at least two image blocks to obtain a detection box corresponding to a target object in the image block.

[0046] Steps 301-303 are basically the same as steps 201-203 of the foregoing embodiments, and the specific implementation manner can refer to the foregoing description of steps 201-203, which will not be described here again.

[0047] In step 304, in response to determining that the at least one detection box corresponding to the same target object is in the same image block, the confidence of each detection box is determined, and the detection box with the highest confidence value is taken as the target detection box of the target object.

[0048] In this embodiment, if it is determined that the at least one detection box corresponding to the same target object is in the same image block, the execution subject of the target detection method (for example, the server 105 shown in the figure) determines the target detection box based on the confidence value of each detection box. That is, the execution subject obtains the confidence value of each detection box, and then takes the detection box with the highest confidence value as the target detection box of the target object. When the image block is detected using the detection model, the detection model also outputs the confidence value of each detection box, and the execution subject can directly obtain the confidence value of each detection box, so as to determine the target detection box based on the confidence value. Thus, the accuracy of the target detection box is ensured. Figure 1

[0049] In step 305, in response to determining that the at least one detection box corresponding to the same target object is in different image blocks with overlap, the target detection box of the target object is determined from the at least one detection box based on the non-maximum suppression method.

[0050] In this embodiment, if it is determined that the at least one detection box corresponding to the same target object is in different image blocks with overlap, the execution subject determines the target detection box of the target object from the multiple detection boxes based on the non-maximum suppression method. The NMS algorithm is widely used in the target detection scene, and the purpose is to eliminate redundant candidate boxes and find the best detection box. Thus, the accuracy of the target detection box is ensured.

[0051] In some optional implementations of this embodiment, step 305 includes: for each detection box in the at least one detection box, determining the distance weight of the detection box based on the depth information of the image block where the detection box is located and the position information of the detection box in the image block; determining the score of the detection box based on the confidence of the detection box and the distance weight; and taking the detection box with the highest score in the at least one detection box as the target detection box of the target object.

[0052] ​In the implementation, since the multiple bounding boxes corresponding to the same target object are in multiple image blocks with overlap, the execution subject determines the depth information of the image block where each bounding box is located, and determines the position information of the bounding box of the target object in the image block, and sets a distance weight for the bounding box based on the depth information and the position information. Generally, since the perception ability of the close distance is relatively higher, a higher distance weight is set for the bounding box of the close distance and with a larger proportion. Then the distance weight is multiplied by the confidence value of the bounding box, so as to obtain the final score of the bounding box. Finally, the bounding box with the highest score in at least one bounding box is taken as the target bounding box of the target object. Thus, it can be ensured that the determined target bounding box has stronger detection ability.

[0053] As an example, it is assumed that the image Y is divided according to the depth information to obtain a close distance image block A and a middle distance image block B, the close distance image block A and the middle distance image block B have overlap, the bounding box of the target object X in the close distance image block A is A1, the proportion of A1 in A is 60%, and the confidence of A1 is 0.9. The bounding box of the target object X in the middle distance image block B is B1, the proportion of B1 in B is 30%, and the confidence of B1 is 0.6. Since the image block where A1 is located is closer, and the proportion of A1 is larger, the execution subject sets the distance weight of A1 to 1.2, and sets the distance weight of B1 to 1.0. Therefore, the score of A1 can be calculated as 1.2*0.9=1.08, and the score of B1 can be calculated as 1.0*0.6=0.6. Therefore, the execution subject takes A1 as the target bounding box of the target object X.

[0054] From Figure 3 it can be seen that, compared with Figure 2 the embodiment corresponding to the target detection method in the embodiment, the process of determining the target bounding box is highlighted, so that the determined target bounding box has better detection ability, and the accuracy of the target detection result is improved.

[0055] With reference to Figure 4 , Figure 4 Fig. 4 shows a flow 400 of still another embodiment of the target detection method according to the present disclosure. The target detection method comprises the following steps:

[0056] In step 401, an image to be detected is acquired.

[0057] In step 402, the image to be detected is input into a division model, and at least two divided image blocks are output.

[0058] In the embodiment, the execution subject of the target detection method (for example, the target detection model) determines the depth information of the image block where each bounding box is located, and determines the position information of the bounding box of the target object in the image block, and sets a distance weight for the bounding box based on the depth information and the position information. Generally, since the perception ability of the close distance is relatively higher, a higher distance weight is set for the bounding box of the close distance and with a larger proportion. Then the distance weight is multiplied by the confidence value of the bounding box, so as to obtain the final score of the bounding box. Finally, the bounding box with the highest score in at least one bounding box is taken as the target bounding box of the target object. Thus, it can be ensured that the determined target bounding box has stronger detection ability. Figure 1The server 105 shown will input the image to be detected into the division model, and output at least two image blocks after division, wherein the division model is used to divide the image to be detected into at least two image blocks according to different depth information. That is, in this embodiment, the division model is pre-trained so that the division model can divide the image to be detected into multiple image blocks according to depth information. Since the division model has the ability to divide the image according to the depth information interval, using the division model for division can improve the division efficiency and also ensure the rationality of the division result.

[0059] In some optional implementations of this embodiment, the division model is obtained by training based on the following steps: obtaining a training sample set, wherein the training sample in the training sample set includes a sample image and a sub-image corresponding to the sample image, and the sub-image is obtained by dividing the sample image according to depth information; training an initial division model by taking the sample image as input and the sub-image corresponding to the sample image as output, to obtain a trained division model.

[0060] In this implementation, the above execution subject will first obtain a sample image and a plurality of sub-images corresponding to each sample image, and take the sample image and the sub-image corresponding to the sample image as a training sample, thereby obtaining a training sample set, which can be divided into multiple sub-images according to different depth information by using a manual division method. Then, the above execution subject will train an initial division model by taking the sample image as input and the sub-image corresponding to the sample image as output, thereby obtaining a trained division model. The division model trained in the above manner can quickly and accurately divide the image to be detected according to the depth range.

[0061] Step 403, for each image block in the at least two image blocks, determining the depth information of the image block.

[0062] In this embodiment, for each image block in the at least two image blocks, the above execution subject will first determine the depth information of the image block. Since the image to be detected is divided according to the depth information, the above execution subject will further obtain the depth information of each image block when it is divided.

[0063] Step 404, inputting the image block into the detection model corresponding to the depth information, and outputting a detection box corresponding to the target object in the image block.

[0064] In this embodiment, after determining the depth information of each image block, the execution subject determines the detection model corresponding to the depth information, and inputs the image block into the detection model, so as to obtain the detection box corresponding to the target object in the image block. Here, a plurality of detection models corresponding to the depth information are trained in advance, so that each image block can be accurately detected, and the accuracy of the detection result is improved.

[0065] In some optional implementation modes of this embodiment, the detection model is obtained by training based on the following steps: obtaining a training data set, wherein the training data in the training data set includes an original image and a detection box corresponding to the original image, the training data in the training data set has the same depth information, and the detection box is obtained by labeling a target object in the original image; training an initial detection model by taking the original image as input and taking the detection box corresponding to the original image as output, to obtain a trained detection model.

[0066] In this implementation mode, the execution subject first obtains an original image and a detection box corresponding to each original image, takes the original image and the detection box corresponding to the original image as training data, so as to obtain a training data set. Here, the target object in the original image can be labeled by using a manual labeling method, so as to obtain the corresponding detection box. Then, the execution subject trains an initial detection model by taking the original image as input and taking the detection box corresponding to the original image as output, so as to obtain a trained detection model. The detection model obtained by the above method can quickly and accurately detect the target object in the image block.

[0067] Step 405: In response to determining that the at least one detection box corresponding to the same target object is in the same image block, the confidence of each detection box is calculated respectively, and the detection box with the highest confidence value is taken as the target detection box of the target object.

[0068] Step 406: In response to determining that the at least one detection box corresponding to the same target object is in different image blocks with overlapping parts, the target detection box of the target object is determined from the at least one detection box based on a non-maximum suppression method.

[0069] Steps 405-406 are basically the same as steps 304-305 of the foregoing embodiment, and the specific implementation modes can refer to the foregoing description of steps 304-305, which will not be described here again.

[0070] From Figure 4 It can be seen that, compared with the embodiment corresponding to Figure 3 The target detection method in this embodiment highlights the steps of dividing the image to be detected and determining the detection box of the target object in the image block, so as to further improve the efficiency and accuracy of 3D target detection.

[0071] Further reference Figure 5 , Figure 5 An application scenario diagram of the target detection method according to the present disclosure is shown. In this application scenario, the execution subject inputs the to-be-detected image into the distance segmentation network, so that the distance segmentation network divides the to-be-detected image according to depth information to obtain divided 3*3 image blocks, the 3*3 image blocks include three image blocks of near distance, three image blocks of middle distance, and three image blocks of far distance, and there is overlap between the 3*3 image blocks, so as to ensure that the edge information of the image will not be lost when the image is divided, and it can be seen that the image blocks at the lower part will be larger because the obstacles at the near distance will occupy a larger pixel position in the image.

[0072] Then, the execution subject inputs each image block into the corresponding perception network to obtain the detection box corresponding to the target object in each image block. For example, the three image blocks of near distance are input into the near distance perception network, the three image blocks of middle distance are input into the middle distance perception network, and the three image blocks of far distance are input into the far distance perception network.

[0073] Finally, distance-aware weighting is performed based on the NMS strategy, that is, because the perception ability at the near distance is relatively higher, a higher weight value is set for the detection box of the near distance, and then the 3D detection boxes obtained by all distance networks are combined together for NMS, that is, based on the weight and confidence value of each 3D detection box, the final target detection box is determined.

[0074] Further reference Figure 6 As an implementation of the method shown in the above figures, the present disclosure provides an embodiment of a target detection device, which corresponds to the method embodiment shown in Figure 2 The device can be applied in various electronic devices.

[0075] As shown in Figure 6 , the target detection device 600 of the present embodiment includes an acquisition module 601, a division module 602, a detection module 603, and a determination module 604. The acquisition module 601 is configured to acquire a to-be-detected image; the division module 602 is configured to divide the to-be-detected image into at least two image blocks based on depth information, wherein there is overlap between the at least two image blocks; the detection module 603 is configured to perform target detection on each image block of the at least two image blocks to obtain a detection box corresponding to a target object in the image block; and the determination module 604 is configured to, in response to determining that at least one detection box corresponds to the same target object, determine a target detection box of the target object based on the position of the at least one detection box.

[0076] In the embodiment, the specific process of the acquisition module 601, the division module 602, the detection module 603 and the determination module 604 in the target detection apparatus 600 and the technical effects brought by the specific process can be referred to the corresponding description of the steps 101-104 respectively. Figure 2 The related description of the steps 201-204 in the corresponding embodiment will not be repeated here.

[0077] In some optional implementation of the embodiment, the determination module comprises: a first determination sub-module, configured to determine the confidence of each detection box in response to determining that the at least one detection box corresponding to the same target object is in the same image block, and determine the detection box of the target object of the target object as the detection box with the highest confidence value; and a second determination sub-module, configured to determine the target detection box of the target object from the at least one detection box based on a non-maximum suppression method in response to determining that the at least one detection box corresponding to the same target object is in different image blocks with overlap.

[0078] In some optional implementation of the embodiment, the second determination sub-module is further configured to: for each detection box in the at least one detection box, determine the distance weight of the detection box based on the depth information of the image block where the detection box is located and the position information of the detection box in the image block; determine the score of the detection box based on the confidence of the detection box and the distance weight; and determine the detection box with the highest score in the at least one detection box as the target detection box of the target object.

[0079] In some optional implementation of the embodiment, the division module comprises: a division sub-module, configured to input the to-be-detected image into a division model to output at least two divided image blocks, wherein the division model is used to divide the to-be-detected image into at least two image blocks according to different depth information.

[0080] In some optional implementation of the embodiment, the target detection apparatus 600 further comprises a first training module for training the division model, and the first training module is configured to: obtain a training sample set, wherein the training sample in the training sample set comprises a sample image and a sub-image corresponding to the sample image, and the sub-image is obtained by dividing the sample image according to depth information; train an initial division model by taking the sample image as input and the sub-image corresponding to the sample image as output to obtain the trained division model.

[0081] In some optional implementation of the embodiment, the detection module comprises: a third determination sub-module, configured to determine the depth information of the image block; and a detection sub-module, configured to input the image block into a detection model corresponding to the depth information to output the detection box corresponding to the target object in the image block.

[0082] In some optional implementation of the embodiment, the target detection apparatus 600 is configured to train a second training module of the detection model, and the second training module is configured to: obtain a training data set, wherein the training data in the training data set comprises an original image and a detection box corresponding to the original image, the training data in the training data set has the same depth information, and the detection box is obtained by labeling a target object in the original image; train an initial detection model by taking the original image as input and taking the detection box corresponding to the original image as output, to obtain the trained detection model.

[0083] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0084] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0085] As shown in Figure 7 The electronic device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. Various programs and data required for the operation of the electronic device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0086] Various components in the electronic device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, a speaker, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0087] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the object detection method. For example, in some embodiments, the object detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the object detection method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the object detection method by any other appropriate means, such as by means of firmware.

[0088] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0089] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0090] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0091] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0092] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0093] Cloud computer refers to a technology system that accesses elastic and scalable shared physical or virtual resource pools through a network, the resources can include servers, operating systems, networks, software, applications or storage devices, etc., and the resources can be deployed and managed in a demand-based and self-service manner. Through cloud computing technology, efficient and powerful data processing capabilities can be provided for artificial intelligence, blockchain and other technical applications and model training.

[0094] The computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, or a server of a distributed system, or a server combined with a blockchain.

[0095] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0096] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A target detection method, comprising: Acquire the image to be detected; The image to be detected is divided into at least two image blocks based on depth information, wherein there is overlap between the at least two image blocks; For each of the at least two image blocks, target detection is performed on the image block to obtain the detection box corresponding to the target object in the image block; In response to determining that the same target object corresponds to at least one detection box, determining the target detection box of the target object based on the position of the at least one detection box includes: in response to determining that the at least one detection box corresponding to the same target object is located in different image blocks with overlap, determining the target detection box of the target object from the at least one detection box based on a non-maximum suppression method; The step of determining the target detection box of the target object from the at least one detection box based on the nonmaximum suppression method includes: For each detection box in the at least one detection box, a distance weight is determined based on the depth information of the image block in which the detection box is located and the position information of the detection box in the image block; a score is determined based on the confidence level of the detection box and the distance weight. The detection box with the highest score among the at least one detection boxes is taken as the target detection box of the target object; The step of performing target detection on the image patch to obtain a detection box corresponding to the target object in the image patch includes: Determine the depth information of the image patch; The image patch is input into the detection model corresponding to the depth information, and the detection box corresponding to the target object in the image patch is output. Different depth ranges correspond to different detection models.

2. The method according to claim 1, wherein, The step of determining the target detection box of the target object based on the position of the at least one detection box in response to determining that the same target object corresponds to at least one detection box includes: In response to determining that at least one detection box corresponding to the same target object is in the same image block, the confidence of each detection box is determined, and the detection box with the highest confidence value is taken as the target detection box of the target object.

3. The method according to claim 1, wherein, The process of dividing the image to be detected into at least two image blocks based on depth information includes: The image to be detected is input into the segmentation model, and at least two image blocks are output after segmentation. The segmentation model is used to divide the image to be detected into at least two image blocks according to different depth information.

4. The method according to claim 3, wherein, The partitioning model is trained based on the following steps: Obtain a training sample set, wherein the training samples in the training sample set include sample images and sub-images corresponding to the sample images, and the sub-images are obtained by dividing the sample images according to depth information; The sample image is used as input, and the corresponding sub-image is used as output to train the initial segmentation model, thus obtaining the trained segmentation model.

5. The method according to claim 1, wherein, The detection model is trained based on the following steps: Obtain a training dataset, wherein the training data in the training dataset includes an original image and a detection box corresponding to the original image, the training data in the training dataset have the same depth information, and the detection box is obtained by annotating the target object in the original image; The original image is used as input, and the detection box corresponding to the original image is used as output to train the initial detection model, thus obtaining the trained detection model.

6. A target detection device, comprising: The acquisition module is configured to acquire the image to be detected; The segmentation module is configured to divide the image to be detected into at least two image blocks based on depth information, wherein there is overlap between the at least two image blocks; The detection module is configured to perform target detection on each of the at least two image blocks to obtain a detection box corresponding to the target object in the image block; The determination module is configured to determine the target detection box of the target object based on the position of the at least one detection box in response to determining that the same target object corresponds to at least one detection box; The determining module includes: The first determination submodule is configured to determine the target detection box of the target object from the at least one detection box based on a non-maximum suppression method in response to determining that at least one detection box corresponding to the same target object is located in different overlapping image blocks. The first determining submodule is further configured to: For each detection box in the at least one detection box, a distance weight is determined based on the depth information of the image block in which the detection box is located and the position information of the detection box in the image block; a score is determined based on the confidence level of the detection box and the distance weight. The detection box with the highest score among the at least one detection boxes is taken as the target detection box of the target object; The detection module is further configured as follows: Determine the depth information of the image patch; The image patch is input into the detection model corresponding to the depth information, and the detection box corresponding to the target object in the image patch is output. Different depth ranges correspond to different detection models.

7. The apparatus according to claim 6, wherein, The determining module includes: The second determination submodule is configured to, in response to determining that at least one detection box corresponding to the same target object is in the same image block, determine the confidence of each detection box and take the detection box with the highest confidence value as the target detection box of the target object.

8. The apparatus according to claim 6, wherein, The partitioning module includes: The segmentation submodule is configured to input the image to be detected into the segmentation model and output at least two segmented image blocks, wherein the segmentation model is used to divide the image to be detected into at least two image blocks according to different depth information.

9. The apparatus according to claim 8, wherein, The apparatus further includes a first training module for training a partitioning model, the first training module being configured to: Obtain a training sample set, wherein the training samples in the training sample set include sample images and sub-images corresponding to the sample images, and the sub-images are obtained by dividing the sample images according to depth information; The sample image is used as input, and the corresponding sub-image is used as output to train the initial segmentation model, thus obtaining the trained segmentation model.

10. The apparatus according to claim 6, wherein, The device further includes a second training module for training a detection model, the second training module being configured to: Obtain a training dataset, wherein the training data in the training dataset includes an original image and a detection box corresponding to the original image, the training data in the training dataset have the same depth information, and the detection box is obtained by annotating the target object in the original image; The original image is used as input, and the detection box corresponding to the original image is used as output to train the initial detection model, thus obtaining the trained detection model.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5.

Images