Object detection method, method for estimating distance of object by using image, host and system thereof

Abstract

The invention provides an object detection method. The object detection method comprises the following steps: receiving an image; and executing a deep neural network architecture according to the image to obtain more than one object bounding box, wherein the deep neural network architecture comprises a two-dimensional discrete wavelet transform.

Description

[0001] priority

[0002] This Chinese invention patent application claims the priority of a Taiwan patent with an application date of September 28, 2023 and application number 112137553. This patent application combines the technical solution of the above-mentioned Taiwan patent. Technical Field

[0003] The present application relates to distance estimation using images, and in particular to distance estimation using a single deep neural network model. Background Art

[0004] Smart vehicles need to estimate distances for navigation and driving. Current distance estimation usually uses active distance measuring instruments such as millimeter wave radar, lidar and laser radar. Although they have advantages such as high resolution, strong anti-interference ability, little weather influence, high-speed detection and non-contact performance, making them ideal for rangefinders. However, millimeter wave radar, lidar or laser radar are quite expensive, and there will be radiation and electromagnetic interference problems. Therefore, in addition to increasing costs, the deployment of the above-mentioned active rangefinders on smart vehicles will also cause major problems in electromagnetic compatibility certification and electromagnetic spectrum licensing.

[0005] Another passive distance measurement method is to estimate the distance of an object through the image taken by the camera. Due to the slow network speed, in order to achieve real-time performance, the computer on the vehicle must first detect the object through the deep neural network model architecture, and then use the image settings and the object's bounding box position to estimate the distance of the object. However, the computing resources of the computer on the vehicle are quite limited, and the computing power required by the deep neural network model architecture used must be lightweight.

[0006] However, due to the limitations of the existing deep neural network architecture, the object frame cannot adapt to the size of the object in each frame of the image, and the size of the object frame detected by the lightweight deep neural network model architecture is unstable. Since the distance estimation is based on the size of the object frame, if the size of the object in the previous frame is different from the size of the object in the next frame, the estimated distance will jump and change significantly, which will lead to a larger error in the distance estimation. When the vehicle has a large error in estimating the distance of the surrounding objects, the calculation of navigation and driving becomes more difficult.

[0007] Therefore, there is an urgent need for a deep neural network architecture that can be easily implemented on a vehicle computer without requiring a large amount of computing resources, and is used to stabilize the size of the object frame. Based on the stable object frame size, the distance of the object can be estimated more accurately and stably, which is beneficial for the navigation and driving of the vehicle. Summary of the invention

[0008] According to one embodiment of the present application, an object detection method is provided, comprising: receiving an image; and executing a deep neural network architecture according to the image to obtain one or more object bounding boxes, wherein the deep neural network architecture comprises a two-dimensional discrete wavelet transform.

[0009] Preferably, in order to fuse the information of the deep and shallow networks, the deep neural network architecture includes: a backbone architecture network, which includes the aforementioned two-dimensional discrete wavelet transform; a linked neck network, wherein the linked neck network includes a feature pyramid network, so as to extract features from the transformation results of the two-dimensional discrete wavelet transform in the backbone architecture network; and a detection head, wherein the detection head obtains an object bounding box of the type of object to be detected from the linked neck network.

[0010] Preferably, in order to detect objects of different scales, the detection head includes a large object detection head, a medium object detection head and a small object detection head, which are used to obtain the one or more object bounding boxes from multiple blocks of different sizes of the sequential neck network respectively.

[0011] Preferably, in order to enhance information about edges or features of objects, the transformation result of the two-dimensional discrete wavelet transform comprises a sum of at least two results among three results obtained by a high-pass filter of the two-dimensional discrete wavelet transform.

[0012] Preferably, in order to preserve the information of the original image, the transformation result of the two-dimensional discrete wavelet transform includes a result obtained by connecting the sum with a result obtained by a high-pass filter without the two-dimensional discrete wavelet transform.

[0013] Preferably, in order to extract features using a convolutional neural network, the backbone architecture network includes a convolutional neural network to obtain a convolution result, wherein the linked network further includes extracting features from a result obtained by connecting the transformation result in the backbone architecture network with the convolution result.

[0014] According to one embodiment of the present application, a method for estimating the distance of an object using an image is provided, comprising: executing the object detection method as described above; and estimating the distance between the object corresponding to the object bounding box and the camera device that captured the image using parameters corresponding to the object bounding box and the image.

[0015] Preferably, in order to simply perform a linear transformation to obtain the distance, the parameters corresponding to the image include a homography matrix for corresponding the position presented by the image to the position on the ground.

[0016] According to an embodiment of the present application, a host for object detection is provided, comprising: one or more processors for executing a plurality of computer instructions stored in a non-volatile memory to implement the object detection method as described above.

[0017] According to one embodiment of the present application, a system for estimating the distance of an object using an image is provided, comprising: a host, which includes one or more processors for executing multiple computer instructions stored in a non-volatile memory to implement the method for estimating the distance of an object using an image as described above; and the aforementioned camera device.

[0018] According to the object detection method, the method for estimating the distance of an object using an image, and the host and system thereof provided in the present application, an object can be detected based on an image, and the object frame can be stabilized, so that the distance can be estimated more accurately in post-processing. The feature of the present application is that the deep neural network architecture provided includes a discrete wavelet transform, which can enhance the boundaries and features of objects. Combined with a pyramid network as the architecture of a connected-order neck network, the features presented by the deep and shallow networks can be integrated to enhance the ability to locate and detect objects, and the complexity of the architecture can be improved so that it does not require too many layers to achieve stable and accurate detection of the object frame, and also provides a faster inference time. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a block diagram of a deep neural network architecture 100 according to an embodiment of the present application.

[0020] Figure 2 1 is a block diagram of a deep neural network architecture 100 according to an embodiment of the present application.

[0021] Figure 3 A block diagram of a two-dimensional discrete wavelet transform.

[0022] Figure 4 FIG. 4 is a flow chart of a two-dimensional discrete wavelet transform method 400 .

[0023] Figure 5 4 is a flow chart of a discrete wavelet transform convolutional neural network 124 according to an embodiment of the present application.

[0024] Figure 6 FIG. 1 is a schematic diagram of a continuous-stage neck network 130 according to an embodiment of the present application.

[0025] Fig. 7A and Figure 7B 1 and 2 are four object detection output results processed by the deep neural network architecture 100 according to an embodiment of the present application.

[0026] Figure 8FIG. 4 is a schematic diagram of a homography matrix H-Matrix according to an embodiment of the present application.

[0027] Fig. 9 FIG. 9 is a flow chart of a method 900 for estimating the distance of an object using an image according to an embodiment of the present application.

[0028] Fig.10 FIG. 1 is a block diagram of a system 1000 for distance estimation using images according to an embodiment of the present application.

[0029] Description of reference numerals:

[0030] 100: Deep Neural Network Architecture

[0031] 110: Input image

[0032] 120: Backbone Architecture Network

[0033] 122: Convolutional Neural Networks

[0034] 124: Discrete Wavelet Transform Convolutional Neural Network

[0035] 130: Continuous-order neck network

[0036] 140:Detection Head

[0037] 141: Large object detection head

[0038] 142: Medium object detection head

[0039] 143: Small object detection head

[0040] 150: Object bounding box of the object category to be detected

[0041] 400: Two-dimensional discrete wavelet transform method

[0042] 410~457: Steps

[0043] 510~560: Steps

[0044] 900: Method for estimating distance of an object using an image

[0045] 910~940: Steps

[0046] 1000: System for distance estimation using images

[0047] 1010:Host

[0048] 1011: Processor

[0049] 1012: Coprocessor

[0050] 1013: Peripheral device connection device

[0051] 1014: Storage device

[0052] 1020: Camera DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than comprehensive embodiments. Based on the embodiments in the present application, all other implementation methods obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0054] The terms "first", "second", "third", etc. (if any) in the specification and scope of the patent application of this application and the drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the objects described in the above descriptions can be interchanged where appropriate. In the description of the present application, "plurality" means two or more, unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. Such functional entities can be implemented in software form, or in one or more hardware circuits or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0055] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or mutual communication; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two components or the interaction relationship between two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0056] In order to make the purpose, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with drawings and specific implementation methods.

[0057] Please refer to Figure 1 As shown, it is a block diagram of a deep neural network architecture 100 according to an embodiment of the present application. Figure 2As shown, it is a block diagram of a deep neural network architecture 100 according to an embodiment of the present application. The deep neural network architecture 100 includes a three-layer deep neural network. The backbone architecture network 120 receives the input image. The chain neck network 130 extracts features from the backbone architecture network 120. Finally, there is a detection head 140, which is used to obtain the bounding box of the object of the desired detection category from the chain neck network 130. The output of the deep neural network architecture 100 is the bounding box of the object of the desired detection category.

[0058] The input image 110 may come from a camera lens or a camera device, and the camera lens may output a multi-band image. For example, the input image 110 may be encoded with the three primary colors red, blue, and green. However, those skilled in the art may understand that the camera lens may output a single-band image. For example, the input image 110 may be encoded with infrared, ultraviolet, or grayscale.

[0059] When the lens characteristics of the camera lens are known, the object captured by the input image 110 can be mapped to the distance between the camera lens according to the lens characteristics. In other words, the distance of the captured object is estimated using the input image 110. The function of the deep neural network architecture 100 is to stabilize the size of the bounding box of the detected object to facilitate distance estimation.

[0060] The backbone network 120 may include two layers of neural networks. The first layer may include a convolutional neural network (CNN) 122 or a derivative variation of a convolutional neural network, or may be referred to as a convolutional neural network. The second layer is a neural network that combines a discrete wavelet transform (DWT) and a convolutional neural network, which is also one of the inventive features of the present application. The present application will describe the second layer in detail later.

[0061] After the backbone network 120 performs multiple down-sampling, it is input to the neck network 130 for feature extraction, and finally output to the detection head 140. In one embodiment, the neck network 130 can include a feature pyramid network (FPN) plus a pyramid attention network (PAN) network, which can enable deep and shallow information in the architecture to communicate. In another embodiment, the neck network 130 can combine the spatial pyramid pooling layer (SPP) and the path aggregation network (PANet) two models.

[0062] like Figure 1 and Figure 2 As shown, the detection head 140 may include three different detection head branches, which perform detection according to different levels of the pyramid in the linked neck network 130, so as to detect large objects, medium objects and small objects. In each of the detection heads 141, 142 and 143, the image is divided into multiple blocks, each of which has multiple reference boxes (anchors). The detection heads 141, 142 and 143 can predict the offset between the target object and the reference box, and each reference box can predict one or more bounding boxes. Finally, the bounding boxes with low probability are eliminated by the trust index.

[0063] In order to make the bounding box of the object more stable, the present application adds discrete wavelet transform to the backbone network 120 to enhance the high-frequency domain of the input image 110. Because the high-frequency domain usually represents the boundary and features of the object, after enhancing the high-frequency domain, the subsequent sequential network 130 can reduce the probability of missing the edge or features of the object in the high-frequency domain when extracting features in the deep, middle and shallow layers. In other words, the detection head 140 can stably mark the bounding box corresponding to the object to be detected.

[0064] When the size of the bounding box corresponding to the same object in the continuous images does not change suddenly, the credibility of the measured distance will increase, which is beneficial to navigation and automatic driving. For example, when the size of the bounding box detected by the vehicle in front of it does not fluctuate, the distance of the front vehicle received by its automatic following driving system will not suddenly change greatly, so that the automatic following driving system must reflect the change in distance and accelerate or decelerate sharply. In this way, fuel consumption and brake wear can be reduced, and safety and passenger comfort can be improved.

[0065] Please refer to Figure 3 As shown, it is a block diagram of a two-dimensional discrete wavelet transform. The input signal x[m,n] is a two-dimensional array x, the first dimension is m, and the second dimension is n. g[n] is a low-pass filter that converts in the second dimension, which can filter out the high-frequency part and leave the low-frequency part. On the contrary, h[n] is a high-pass filter that converts in the second dimension, which can filter out the low-frequency part and leave the high-frequency part. After passing through g[n] and h[n], downsampling is performed respectively, for example, 2 times downsampling, and the second-dimensional low-frequency part v is obtained respectively. 1,L [m,n] and the second-dimensional high-frequency part v 1,H [m,n].

[0066] Then, for the above two parts, we filter them with the first-dimensional low-pass filter g[m] and the first-dimensional high-pass filter h[m] and then downsample them by 2 times to obtain four parts. Figure 3 From top to bottom, we can get the low-frequency parts x in the first and second dimensions. 1,LL , the part x of the first dimension high frequency and the second dimension low frequency 1,HL , the low frequency part of the first dimension and the high frequency part of the second dimension x 1,LH , and the first and second dimensions are high frequency parts x 1,HH .

[0067] Please refer to Figure 4 As shown, it is a flowchart of a two-dimensional discrete wavelet transform method 400. This flowchart may start from step 410. When any two steps have no direct or indirect causal relationship, the present application does not limit the order in which they are executed.

[0068] Step 410: Receive a two-dimensional image x[m,n].

[0069] Step 420: Perform a low-pass filtering step g[n] on the received image x[m,n] in the second dimension.

[0070] Step 425: Perform a high pass filtering step h[n] on the received image x[m,n] in the second dimension.

[0071] Step 430: Downsample the result obtained in step 420 by a factor of 2 to obtain v 1,L [m,n].

[0072] Step 435: Downsample the result obtained in step 425 by a factor of 2 to obtain v 1,H [m,n].

[0073] Step 440: Transform the received image v by the first dimension 1,L [m,n] is subjected to low-pass filtering step g[m].

[0074] Step 442: Construct the received image v using the first dimension 1,L [m,n] undergoes a high-pass filtering step h[m].

[0075] Step 445: Transform the received image v by the first dimension 1,H [m,n] is subjected to low-pass filtering step g[m].

[0076] Step 447: Construct the received image v using the first dimension 1,H [m,n] undergoes a high-pass filtering step h[m].

[0077] Step 450: Downsample the result obtained in step 440 by a factor of 2 to obtain x 1,LL [m,n]. That is, the first and second dimensions mentioned above are both low-frequency parts x 1,LL .

[0078] Step 452: Downsample the result obtained in step 442 by a factor of 2 to obtain x 1,HL [m,n]. That is, the first-dimensional high-frequency and second-dimensional low-frequency parts x 1,HL .

[0079] Step 455: Downsample the result obtained in step 445 by a factor of 2 to obtain x 1,LH [m,n]. That is, the first dimension low frequency and the second dimension high frequency part x 1,LH .

[0080] Step 457: Downsample the result obtained in step 447 by a factor of 2 to obtain x 1,HH [m,n]. That is, the first and second dimensions mentioned above are both high-frequency parts x 1,HH .

[0081] Figure 3 and Figure 4 The discrete wavelet transform shown can be used in the discrete wavelet transform convolutional neural network 124 to perform discrete wavelet transform on the image signal processed by the convolutional neural network 122. It can be understood by those skilled in the art that although Figure 3 and Figure 4 The discrete wavelet transform process shown is to first filter in the second dimension and then filter in the first dimension, but in another embodiment, filtering can also be performed in the first dimension and then in the second dimension.

[0082] Please refer to the discrete wavelet transform formula in Table 1, there are six equations in total.

[0083]

[0084]

[0085] Table 1 Discrete wavelet transform equation

[0086] Please refer to Figure 5 , which is a flow chart of a discrete wavelet transform convolutional neural network 124 according to an embodiment of the present application. The implementation of the discrete wavelet transform convolutional neural network 124 may include the following steps.

[0087] Step 510 : Receive input image. As mentioned above, the input image received in this step may come from the image signal processed by the convolutional neural network 122 .

[0088] Step 520: Chunk the received input image. Then, the process goes to step 400, which is Figure 3 or Figure 4 After the two-dimensional discrete wavelet transform is completed, four results are obtained. The process proceeds to step 530.

[0089] Step 530: Add the two or three results after high-pass filtering. As mentioned above, except for the first and second dimensions obtained in step 450 or equation (3), the low-frequency part x 1,LL In addition, the remaining three parts are filtered by at least one high-pass filter. In this step, two of the parts can be added, or the three parts can be added. For example, the three parts obtained in step 452, step 455, and step 457 can be added. It is also possible to add two of the three parts.

[0090] Step 540: Add the result of step 530 to the result of low-pass filtering (the first and second dimensions are both low-frequency parts x 1,LL ) are connected or connected as the transformation result. Then, the process proceeds to step 550.

[0091] Step 550: Perform convolutional neural network calculation on the block data obtained in step 520 to obtain a convolution result. Since step 550 has no causal relationship with the aforementioned two-dimensional discrete wavelet transform step 400, step 530 and step 540, the present application does not limit the execution order of step 550 and the aforementioned two-dimensional discrete wavelet transform step 400, step 530 and step 540.

[0092] Step 560: Connect or connect the transformation results obtained in step 540 and step 550 to the convolution result. Next, let the concatenated network 130 perform feature extraction on the processing result of the discrete wavelet transform convolution neural network 124.

[0093] The backbone network 120 provided in the present application includes a discrete wavelet transform. Since the discrete wavelet transform has a dimension reduction (downsampling) process and can separate different frequency domains of a two-dimensional image signal. Figure 5 The discrete wavelet transform convolutional neural network 124 shown has the function of dimensionality reduction, so that when performing feature extraction, information with enhanced object edges or features can be obtained, making the features read by the backbone architecture network 120 more diverse.

[0094] Please refer to Figure 6 As shown, it is a schematic diagram of a continuous-order network 130 according to an embodiment of the present application. When reading information from the backbone architecture network 120, images of different sizes can be obtained by downsampling. For example, if the information read is a 320×320 pixel array, a 160×160 pixel array, an 80×80 pixel array, a 40×40 pixel array, and a 20×20 pixel array can be obtained by multiple 2-fold downsampling. Feature extraction can be performed on pixel arrays of different sizes to obtain information of shallow and deep networks.

[0095] Then, through upsampling and downsampling steps, the information of shallow and deep networks is integrated to enhance the positioning ability at multiple scales. Figure 1 and Figure 2 As shown, the detection head 140 following the sequential neck network 130 may include a large object detection head 141, a medium object detection head 142, and a small object detection head 143. These detection heads may detect objects at multiple scales. A person skilled in the art may understand that the present application may use a variety of implementations of the sequential neck network 130. Figure 6 The figure is only one of the embodiments. As long as the linked-stage network 130 utilizes the architecture of the pyramid network, it can be used as an embodiment of the present application.

[0096] Please refer to Fig. 7A and Figure 7B, which are four object detection output results processed by the deep neural network architecture 100 according to an embodiment of the present application. In these four processed images, the bounding boxes of pedestrians, cars, and motorcycles can be seen. These three types of objects are the types of objects to be detected in this embodiment. Those of ordinary skill in the art can understand that the deep neural network architecture 100 can detect different types of objects.

[0097] As mentioned above, one of the purposes of this application is to use images to detect the distance of an object. In this application, a homography matrix can be used to transform and project an image from one plane to another plane. The parameters of the homography matrix can be set according to the characteristics of the aforementioned camera lens.

[0098] Please refer to Figure 8 , which is a schematic diagram of a homography matrix H-Matrix according to an embodiment of the present application. Figure 8 As shown in the figure, when the camera lens captures the B plane, it converts the B plane into the image of the A plane. When the coordinates (x, y) of point A on the A plane are converted into the coordinates (u, v) of point B on the B plane, the conversion can be performed through the homography matrix H-Matrix corresponding to the camera lens.

[0099] In one embodiment, the B plane can be regarded as the ground, and the A plane can be regarded as the imaging surface of the camera lens. Since the elements in the homography matrix H-Matrix of the camera lens are measured in advance, the coordinates of point A in the A plane image can be converted into the coordinates of point B in the B plane. Moreover, the relationship between the A plane and the B plane is known, so when the B plane is the ground, the distance between point B and the camera lens can be known. For example, in Fig. 7A and Figure 7B Among the bounding boxes of each object shown, the middle point of the lower edge of the bounding box can be regarded as point A. After conversion using the homography matrix H-Matrix, the distance between the object and the camera lens can be obtained.

[0100] Therefore, one of the goals of this application is to stabilize the size of the bounding box, because the bounding box is related to the estimation of the distance of the object. Please refer to Table 2, which is a method for calculating the homography matrix.

[0101]

[0102] Table 2 Homography matrix calculation equation

[0103] Those skilled in the art will appreciate that different homography matrices can be used to estimate distances according to different lens characteristics. A homography matrix is ​​a linear transformation, but the present application may use different types of linear transformation methods, or may use nonlinear transformation methods. For example, the aberrations of the edge and center of an image captured by a fisheye lens are different, and a nonlinear transformation method is required. In addition, a lookup table (LUT) may be set according to the characteristics of the lens. Therefore, the present application does not limit the method of converting image coordinates into ground coordinates.

[0104] Please refer to Fig. 9 , which is a flow chart of a method 900 for estimating the distance of an object using an image according to an embodiment of the present application. The method 900 for estimating the distance of an object using an image can be real-time, that is, the running time of the method 900 is limited. The method 900 starts with step 910.

[0105] Step 910: Receive an image. The image may come from a camera on the aforementioned vehicle.

[0106] Step 920: Utilize the deep neural network architecture provided by the present application to obtain the object of the type to be detected and its bounding box.

[0107] Step 930: Estimate the distance of the object based on the aforementioned bounding box and the parameters corresponding to the image. In one embodiment, the parameters corresponding to the image include a correspondence matrix. In another embodiment, the parameters corresponding to the image include a nonlinear function. In yet another embodiment, the parameters corresponding to the image include a lookup table.

[0108] Optional step 940: Output the estimated distance. In one embodiment, the distance can be output to the aforementioned navigation system, driving system, and safety system. The navigation system can plan an avoidance navigation route. If used in a missile system, an impact route can be planned. The driving system can perform corresponding driving behaviors to set the direction and speed of travel, such as an automatic following system. The security system can record or identify further features of an object. For example, when someone approaches a door, the security system can recognize the face to decide whether to open the door or sound an alarm.

[0109] Please refer to Fig.10 , which is a block diagram of a system 1000 for distance estimation using images according to an embodiment of the present application. The system 1000 for distance estimation using images can be implemented inside a vehicle, and the above-mentioned deep neural network architecture 100 can be implemented in a host 1010. In other words, the host 1010 can implement the method 900 for estimating the distance of an object using images or some steps thereof.

[0110] The host 1010 includes at least one processor 1011 for executing an operating system to control the host 1010 and / or the system 1000 for distance estimation using images. The processor 1011 may be a processor of x86, x64, ARM, RISC-V or other industrial standard instruction sets. The operating system may be an operating system of the UNIX series, Windows series, Android series, iOS series or other series, and may also be a real-time operating system.

[0111] The host 1010 may include one or more co-processors 1012 for accelerating the inference of the deep neural network architecture 100. The one or more co-processors 1012 may be a graphics processing unit (GPU), a neural network processor (NPU), an artificial intelligence processor (APU), or other processors with multiple vector logic and operation units for accelerating the inference of the deep neural network architecture 100. The present application does not limit the host 1010 to have a co-processor 1012 to achieve the inference of the deep neural network architecture 100.

[0112] The host 1010 also includes a peripheral device connection device 1013, which can be used to connect one or more camera devices 1020. The host 1010 may also include a storage device 1014 for storing the above-mentioned operating system and the program for implementing the deep neural network architecture 100. A person of ordinary skill in the art needs to have knowledge of computer organization and computer architecture, operating system, system program, artificial intelligence and deep neural network, and can make changes or derivatives to the above-mentioned host 1010 and the embodiment of the system 1000 for distance estimation using images, as long as it can implement the deep neural network architecture 100 provided in this application.

[0113] The camera device 1020 can be connected to the peripheral device connection device of the host 1010 through an industrial standard interface, such as common wired or wireless connection technology, such as UWB, WiFi, Bluetooth, USB, IEEE 1394, UART, iSCSI, PCI-E, SATA and other industrial standard technologies. This application does not limit the above industrial standard interface, but only requires that the speed of the camera device 1020 providing images can meet the real-time requirements of the system 1000 for distance estimation using images.

[0114] In one embodiment, the peripheral device connection device 1013 can output the above distance to other devices or systems. The result inferred by the above deep neural network architecture 100 includes more than one object bounding box to calculate the distance between the object and the camera device 1020. In another embodiment, the host 1010 can execute other procedures to receive the estimated distance. Other procedures are described in step 940.

[0115] In order to prove that the deep neural network architecture 100 provided by the present application can have good performance in a computing environment with a restricted vehicle environment, the applicant implements the deep neural network architecture 100 and two other deep neural network architectures to compare their performances.

[0116]

[0117] Table 3 Comparison table of the deep neural network architecture 100 provided in this application with other architectures

[0118] Table 3 provides a performance comparison of the deep neural network architecture 100 provided by the present application and two other architectures, Yolov3-tiny and Yolofastestv2. The floating point operations of architecture 100 are similar to those of Yolofastestv2, but much less than Yolov3-tiny. It can be seen that architecture 100 can be implemented in a lightweight computing environment.

[0119] In terms of object detection performance, architecture 100 detected 53 objects, more than the other two architectures. Regarding the error, the applicant takes the difference in pixels between the bottom edge of the bounding box and the bottom edge of the real box as the error. The sum of the errors corresponding to each object is the total error. The average error of each object can represent the average error of distance estimation. The architecture 100 provided in this application is the best in object detection rate and average pixel error. In terms of average error, architecture 100 can improve by 51.6% compared to Yolov3-tiny with a computational load of 5.54 GFlops lower. Compared with Yolofastestv2 with a similar computational load, it is improved by 61%.

[0120] According to one embodiment of the present application, an object detection method is provided, comprising: receiving an image; and executing a deep neural network architecture according to the image to obtain one or more object bounding boxes, wherein the deep neural network architecture comprises a two-dimensional discrete wavelet transform.

[0121] Preferably, in order to fuse the information of the deep and shallow networks, the deep neural network architecture includes: a backbone architecture network, which includes the aforementioned two-dimensional discrete wavelet transform; a linked neck network, wherein the linked neck network includes a feature pyramid network, so as to extract features from the transformation results of the two-dimensional discrete wavelet transform in the backbone architecture network; and a detection head, wherein the detection head obtains an object bounding box of the type of object to be detected from the linked neck network.

[0122] Preferably, in order to detect objects of different scales, the detection head includes a large object detection head, a medium object detection head and a small object detection head, which are used to obtain the one or more object bounding boxes from multiple blocks of different sizes of the sequential neck network respectively.

[0123] Preferably, in order to enhance information about edges or features of objects, the transformation result of the two-dimensional discrete wavelet transform comprises a sum of at least two results among three results obtained by a high-pass filter of the two-dimensional discrete wavelet transform.

[0124] Preferably, in order to preserve the information of the original image, the transformation result of the two-dimensional discrete wavelet transform includes a result obtained by connecting the sum with a result obtained by a high-pass filter without the two-dimensional discrete wavelet transform.

[0125] Preferably, in order to extract features using a convolutional neural network, the backbone architecture network includes a convolutional neural network to obtain a convolution result, wherein the linked network further includes extracting features from a result obtained by connecting the transformation result in the backbone architecture network with the convolution result.

[0126] According to one embodiment of the present application, a method for estimating the distance of an object using an image is provided, comprising: executing the object detection method as described above; and estimating the distance between the object corresponding to the object bounding box and the camera device that captured the image using parameters corresponding to the object bounding box and the image.

[0127] Preferably, in order to simply perform a linear transformation to obtain the distance, the parameters corresponding to the image include a homography matrix for corresponding the position presented by the image to the position on the ground.

[0128] According to an embodiment of the present application, a host for object detection is provided, comprising: one or more processors for executing a plurality of computer instructions stored in a non-volatile memory to implement the object detection method as described above.

[0129] According to one embodiment of the present application, a system for estimating the distance of an object using an image is provided, comprising: a host, which includes one or more processors for executing multiple computer instructions stored in a non-volatile memory to implement the method for estimating the distance of an object using an image as described above; and the aforementioned camera device.

[0130] According to the object detection method, the method for estimating the distance of an object using an image, and the host and system thereof provided in the present application, an object can be detected based on an image, and the object frame can be stabilized, so that the distance can be estimated more accurately in post-processing. The feature of the present application is that the deep neural network architecture provided includes a discrete wavelet transform, which can enhance the boundaries and features of objects. Combined with a pyramid network as the architecture of a connected-order neck network, the features presented by the deep and shallow networks can be integrated to enhance the ability to locate and detect objects, and the complexity of the architecture can be improved so that it does not require too many layers to achieve stable and accurate detection of the object frame, and also provides a faster inference time.

Claims

1. A method for detecting an object, characterized in that: Include: receiving an image; and A deep neural network architecture is run on the image to obtain one or more object bounding boxes. The deep neural network architecture includes a two-dimensional discrete wavelet transform.

2. The object detection method according to claim 1, characterized in that: The deep neural network architecture includes: A backbone architecture network, the backbone architecture network including the aforementioned two-dimensional discrete wavelet transform; A linked-order neck network, wherein the linked-order neck network includes a feature pyramid network, so as to extract features from the transformation result of the two-dimensional discrete wavelet transform in the backbone architecture network; as well as A detection head is provided, wherein the detection head obtains an object bounding box of a desired object type from the chained neck network.

3. The object detection method according to claim 2, characterized in that: The detection head includes a large object detection head, a medium object detection head and a small object detection head, which are used to obtain the one or more object boundary boxes from multiple blocks of different sizes of the linked neck network respectively.

4. The object detection method according to claim 2, characterized in that: The transformation result of the two-dimensional discrete wavelet transform includes the sum of at least two results among three results obtained by the high-pass filter of the two-dimensional discrete wavelet transform.

5. The object detection method according to claim 4, characterized in that: The transformation result of the two-dimensional discrete wavelet transform includes the transformation result obtained by connecting the sum with the result obtained by the high-pass filter without the two-dimensional discrete wavelet transform.

6. The object detection method according to claim 2, characterized in that: The backbone architecture network includes a convolutional neural network to obtain a convolution result, wherein the linked network further includes extracting features from a result obtained by connecting the transformation result in the backbone architecture network with the convolution result.

7. A method for estimating the distance of an object using an image, characterized in that: Include: An object detection method according to any one of claims 1 to 6; and The distance between the object corresponding to the object bounding box and the camera device that photographed the image is estimated using the parameters corresponding to the object bounding box and the image.

8. The method for estimating the distance of an object using an image according to claim 7, wherein: The parameters corresponding to the image include a homography matrix, which is used to correspond the position presented by the image to the position on the ground.

9. A host for object detection, characterized in that: It comprises: one or more processors for executing a plurality of computer instructions stored in a non-volatile memory to implement the object detection method according to any one of claims 1 to 6.

10. A system for estimating the distance of an object using an image, characterized in that: Include: a host computer comprising one or more processors for executing a plurality of computer instructions stored in a non-volatile memory to implement the method for estimating the distance of an object using an image as claimed in claim 7; and A photographic device as claimed in claim 7.

Images