A dual-camera-based vehicle target detection method and system

Abstract

The application discloses a kind of vehicle target detection method and system based on dual camera, it is related to intersection intelligent vehicle management field, including: according to the homographic transformation matrix the long-range image is mapped on the close-range image and obtains the coordinates of four vertexes of mapped long-range image, and according to the coordinates of four vertexes of mapped long-range image, the transformed detection result corresponding to the long-range image and the detection result corresponding to the close-range image are detected result fusion and output, to realize using long-range camera to improve the detection quantity of close-range camera far place, and then improve the detection quantity and detection sample of far small target, so as to improve the detection identification efficiency and accuracy of intersection vehicle.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle target detection at intersections, and in particular to a vehicle target detection method and system based on dual cameras. Background Technology

[0002] In the many technical fields of computer vision, object detection is a very fundamental task. Image segmentation, object tracking, and keypoint detection all typically rely on object detection. During object detection, the number, size, and pose of objects vary in each image, resulting in unstructured output—a significant difference from image classification. Furthermore, objects are often occluded or truncated, making object detection technology extremely challenging and one of the most focused areas of research since its inception.

[0003] Currently, in vehicle target detection, to improve the accuracy of camera detection at long distances and increase the camera's field of view, many manufacturers are constantly increasing the resolution of camera images. This has led to a sharp increase in the hardware computing power requirements of deep learning target detection algorithms. Furthermore, because the pixel area occupied by distant vehicle targets in close-up images captured by close-up cameras is still relatively small, the number of detected small distant targets and the number of detection samples have not significantly improved, resulting in low efficiency and accuracy in vehicle detection and recognition at intersections. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a vehicle target detection method and system based on dual cameras, which can solve the problem of low detection efficiency and accuracy of vehicles at intersections due to the limited number of detections and samples of small targets at a distance.

[0005] To achieve the above objectives, in one aspect, the present invention provides a vehicle target detection method based on dual cameras, the method comprising:

[0006] Select a preset number of mapping point pairs at different locations from the distant view image captured by the distant camera and the close-up image captured by the close-up camera;

[0007] Obtain the homography transformation matrix based on the mapping points;

[0008] The distant image is mapped onto the close-up image according to the homography transformation matrix, and the coordinates of the four vertices of the mapped distant image are obtained.

[0009] Vehicle target detection is performed based on the distant and close-up images respectively;

[0010] The detection results corresponding to the distant image are transformed according to the homography transformation matrix;

[0011] Based on the coordinates of the four vertices of the mapped distant image, the detection results corresponding to the transformed distant image and the detection results corresponding to the nearby image are fused and output.

[0012] Further, the step of obtaining the homography transformation matrix based on the mapping point pair includes:

[0013] The homography matrix is ​​calculated using the formula H = V(:, 9), where H is a homography matrix consisting of 9 floating-point numbers, [USV] = svd(A), U is the left singular matrix, S is a diagonal matrix, and V is the right singular matrix. A is the intermediate transformation matrix, with the mapping point of the distant image being (x, y) and the mapping point of the close-up image being (X, Y).

[0014] Furthermore, prior to the step of performing vehicle target detection based on the distant and close-up images respectively, the method includes:

[0015] Based on the coordinates of the four vertices of the mapped distant image, determine whether the superposition of the transformed distant image and the close-up image meets the preset conditions.

[0016] If it does not meet the requirements, then select a preset number of mapping point pairs at different locations from the distant view image captured by the distant camera and the close-up image captured by the close-up camera.

[0017] Furthermore, the step of performing vehicle target detection based on the distant view image and the close view image respectively includes:

[0018] Vehicle target detection is performed on the distant and close-up images using a pre-configured YOLOv5 algorithm.

[0019] Further, the step of fusing the transformed detection results corresponding to the distant image and the detection results corresponding to the near image based on the coordinates of the four vertices of the mapped distant image includes:

[0020] Delete the detection results corresponding to the near image located inside the quadrilateral formed by the four vertices of the mapped far image from the transformed detection results corresponding to the far image and the detection results corresponding to the near image.

[0021] Save the detection results and other detection results corresponding to the close-up image located at the boundary of the quadrilateral, and delete the detection results corresponding to the distant image located at the boundary of the quadrilateral.

[0022] On the other hand, the present invention provides a vehicle target detection system based on a dual-camera system, the system comprising:

[0023] The selection unit is used to select a preset number of mapping point pairs at different locations from the distant image captured by the distant camera and the close-up image captured by the close-up camera.

[0024] The acquisition unit is used to acquire the homography transformation matrix based on the mapping point pair;

[0025] The acquisition unit is further configured to map the distant image onto the near image according to the homography transformation matrix and acquire the coordinates of the four vertices of the mapped distant image;

[0026] The detection unit is used to perform vehicle target detection based on the distant view image and the close view image respectively;

[0027] The transformation unit is used to transform the detection result corresponding to the distant image according to the homography transformation matrix;

[0028] The fusion unit is used to fuse and output the detection results of the transformed detection results corresponding to the distant image and the detection results corresponding to the near image based on the coordinates of the four vertices of the mapped distant image.

[0029] Further, the acquisition unit is specifically used to calculate the homography matrix according to the formula H = V(:, 9), where H is a homography matrix composed of 9 floating-point numbers, [USV] = svd(A), U is the left singular matrix, S is the diagonal matrix, and V is the right singular matrix. A is the intermediate transformation matrix, with the mapping point of the distant image being (x, y) and the mapping point of the close-up image being (X, Y).

[0030] Furthermore, the selection unit is also used to determine whether the superposition of the transformed distant image and the close-up image meets the preset conditions based on the coordinates of the four vertices of the mapped distant image; if not, then a preset number of mapping point pairs at different positions are selected again from the distant image captured by the distant camera and the close-up image captured by the close-up camera.

[0031] Furthermore, the detection unit is specifically used to perform vehicle target detection on the distant and close-up images respectively using a preset YOLOv5 algorithm.

[0032] Furthermore, the fusion unit is specifically used to delete the detection results corresponding to the near image located inside the quadrilateral formed by the four vertices of the mapped far image from the transformed detection results corresponding to the far image and the detection results corresponding to the near image; save the detection results corresponding to the near image located at the boundary of the quadrilateral and other detection results, and delete the detection results corresponding to the far image at the boundary of the quadrilateral.

[0033] This invention provides a vehicle target detection method and system based on dual cameras. The method maps a distant image onto a near-field image using a homography transformation matrix and obtains the coordinates of the four vertices of the mapped distant image. Then, based on these coordinates, the transformed detection results from the distant image and the near-field image are fused and output. This method utilizes the distant camera to increase the number of distant targets detected by the near-field camera, thereby increasing the number of detected small targets and the number of detection samples, ultimately improving the efficiency and accuracy of vehicle detection and recognition at intersections. Attached Figure Description

[0034] Figure 1 This is a flowchart of a vehicle target detection method based on dual cameras provided by the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of a vehicle target detection system based on dual cameras provided by the present invention. Detailed Implementation

[0036] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0037] like Figure 1 As shown in the figure, an embodiment of the present invention provides a vehicle target detection method based on dual cameras, which includes the following steps:

[0038] 101. Select a preset number of mapping point pairs at different locations from the distant view image captured by the distant camera and the close-up image captured by the close-up camera.

[0039] Specifically, for example, select 4 pairs of mapping points for distant and close-up cameras. Each pair of distant and close-up cameras collects one distant image and one close-up image at the same time. On the distant image, try to select marker points that are not far from the 4 vertices. This can ensure that the selected points basically cover the entire image and make the calculation of the transformation matrix H more accurate. The distant point is denoted as (x,y) and the close-up point is denoted as (X,Y).

[0040] 102. Obtain the homography transformation matrix based on the mapping point pairs.

[0041] In this embodiment of the invention, step 102 may specifically include: calculating the homography matrix according to the formula H = V(:, 9), where H is a homography matrix composed of 9 floating-point numbers, [USV] = svd(A), U is a left singular matrix, S is a diagonal matrix, and V is a right singular matrix. A is the intermediate transformation matrix, with the mapping point of the distant image being (x, y) and the mapping point of the close-up image being (X, Y).

[0042] 103. Map the distant image onto the near image according to the homography transformation matrix and obtain the coordinates of the four vertices of the mapped distant image.

[0043] 104. Vehicle target detection is performed based on the distant and close-up images respectively.

[0044] In this embodiment of the invention, step 104 may specifically include: performing vehicle target detection on the distant image and the close-up image respectively using a preset YOLOv5 algorithm.

[0045] In this embodiment of the invention, to further improve the accuracy of target detection, before step 104, the method may further include: determining whether the superposition of the transformed distant image and the near image meets preset conditions based on the coordinates of the four vertices of the mapped distant image; if not, then selecting a preset number of mapping point pairs at different positions from the distant image captured by the distant camera and the near image captured by the near camera. For example, by observing the superposition effect, if there are areas with poor superposition or alignment, the selection of points at the relevant positions can be modified, and this modification can be repeated until the superposition effect meets the conditions.

[0046] 105. Transform the detection results corresponding to the distant image according to the homography transformation matrix.

[0047] 106. Based on the coordinates of the four vertices of the mapped distant image, fuse the transformed detection results corresponding to the distant image and the detection results corresponding to the near image, and output the results.

[0048] In this embodiment of the invention, step 106 may specifically include: deleting the detection results corresponding to the near-field image located inside the quadrilateral formed by the four vertices of the mapped far-field image from the transformed detection results corresponding to the far-field image and the detection results corresponding to the near-field image; saving the detection results corresponding to the near-field image located at the boundary of the quadrilateral and other detection results, and deleting the detection results corresponding to the far-field image at the boundary of the quadrilateral. For example, deleting the near-field image results located inside the quadrilateral, retaining the near-field image results located at the boundary of the quadrilateral, deleting the far-field image results corresponding to these detection results, and retaining other detection results.

[0049] This invention provides a vehicle target detection method based on dual cameras. The method maps a distant image onto a near-field image using a homography transformation matrix and obtains the coordinates of the four vertices of the mapped distant image. Then, based on these coordinates, the transformed detection results from the distant image and the detection results from the near-field image are fused and output. This method utilizes the distant camera to increase the number of distant targets detected by the near-field camera, thereby increasing the number of detected small targets and the number of detection samples, ultimately improving the efficiency and accuracy of vehicle detection and recognition at intersections.

[0050] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide a vehicle target detection system based on dual cameras, such as... Figure 2 As shown, the system includes: a selection unit 21, an acquisition unit 22, a detection unit 23, a transformation unit 24, and a fusion unit 25;

[0051] Selection unit 21 is used to select a preset number of mapping point pairs at different locations from the distant image captured by the distant camera and the close-up image captured by the close-up camera.

[0052] Acquisition unit 22 is used to acquire the homography transformation matrix based on the mapping point pair;

[0053] The acquisition unit 22 is also used to map the distant image onto the near image according to the homography transformation matrix and obtain the coordinates of the four vertices of the mapped distant image;

[0054] Detection unit 23 is used to perform vehicle target detection based on the distant view image and the close view image respectively;

[0055] Transformation unit 24 is used to transform the detection result corresponding to the distant image according to the homography transformation matrix;

[0056] The fusion unit 25 is used to fuse and output the detection results of the transformed detection results corresponding to the distant image and the detection results corresponding to the near image based on the coordinates of the four vertices of the mapped distant image.

[0057] Further, the acquisition unit 22 is specifically used to calculate the homography transformation matrix according to the formula H = V(:, 9), where H is a homography transformation matrix composed of 9 floating-point numbers, [USV] = svd(A), U is a left singular matrix, S is a diagonal matrix, and V is a right singular matrix. A is the intermediate transformation matrix, with the mapping point of the distant image being (x, y) and the mapping point of the close-up image being (X, Y).

[0058] Furthermore, the selection unit 21 is also used to determine whether the superposition of the transformed distant image and the close-up image meets the preset conditions based on the coordinates of the four vertices of the mapped distant image; if not, then select a preset number of mapping point pairs at different positions from the distant image captured by the distant camera and the close-up image captured by the close-up camera.

[0059] Furthermore, the detection unit 23 is specifically used to perform vehicle target detection on the distant and near images respectively using a preset YOLOv5 algorithm.

[0060] Furthermore, the fusion unit 25 is specifically used to delete the detection results corresponding to the near image located inside the quadrilateral formed by the four vertices of the mapped far image from the transformed detection results corresponding to the far image and the detection results corresponding to the near image; save the detection results corresponding to the near image located at the boundary of the quadrilateral and other detection results, and delete the detection results corresponding to the far image at the boundary of the quadrilateral.

[0061] This invention provides a vehicle target detection system based on a dual-camera setup. The system maps a distant image onto a near-field image using a homography transformation matrix, obtains the coordinates of the four vertices of the mapped distant image, and then fuses the transformed detection results from the distant image and the near-field image based on these coordinates, outputting the fusion results. This system utilizes the distant camera to increase the number of distant targets detected by the near-field camera, thereby increasing the number of small distant targets detected and the number of detection samples. Ultimately, this improves the efficiency and accuracy of vehicle detection and recognition at intersections.

[0062] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0063] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0064] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0065] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0066] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0067] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing systems, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0068] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0069] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0070] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A vehicle target detection method based on dual cameras, characterized in that, The method includes: Select a preset number of mapping point pairs at different locations from the distant view image captured by the distant camera and the close-up image captured by the close-up camera; Obtain the homography transformation matrix based on the mapping points; The distant image is mapped onto the close-up image according to the homography transformation matrix, and the coordinates of the four vertices of the mapped distant image are obtained. Vehicle target detection is performed based on the distant and close-up images respectively; The detection results corresponding to the distant image are transformed according to the homography transformation matrix; Based on the coordinates of the four vertices of the mapped distant image, the transformed detection results corresponding to the distant image and the detection results corresponding to the near image are fused and output; Before the step of performing vehicle target detection based on the distant and close-up images respectively, the method includes: Based on the coordinates of the four vertices of the mapped distant image, determine whether the superposition of the transformed distant image and the close-up image meets the preset conditions. If it does not meet the requirements, then select a preset number of mapping point pairs at different locations from the distant view image captured by the distant camera and the close-up image captured by the close-up camera.

2. The vehicle target detection method based on dual cameras according to claim 1, characterized in that, The step of obtaining the homography transformation matrix based on the mapping point pair includes: According to the formula Calculate the homography matrix, where H is a homography matrix consisting of 9 floating-point numbers. U is a left singular matrix, S is a diagonal matrix, and V is a right singular matrix. The mapping point of the image is (x, y), and the mapping point of the close-up image is (X, Y).

3. The vehicle target detection method based on dual cameras according to claim 1, characterized in that, The step of performing vehicle target detection based on the distant view image and the close view image respectively includes: Vehicle target detection is performed on the distant and close-up images using a pre-configured YOLOv5 algorithm.

4. The vehicle target detection method based on dual cameras according to claim 1, characterized in that, The step of fusing the detection results of the transformed detection result corresponding to the distant image and the detection result corresponding to the near image based on the coordinates of the four vertices of the mapped distant image includes: Delete the detection results corresponding to the near image located inside the quadrilateral formed by the four vertices of the mapped far image from the transformed detection results corresponding to the far image and the detection results corresponding to the near image. Save the detection results and other detection results corresponding to the close-up image located at the boundary of the quadrilateral, and delete the detection results corresponding to the distant image located at the boundary of the quadrilateral.

5. A vehicle target detection system based on dual cameras, characterized in that, The system includes: The selection unit is used to select a preset number of mapping point pairs at different locations from the distant image captured by the distant camera and the close-up image captured by the close-up camera. The acquisition unit is used to acquire the homography transformation matrix based on the mapping point pair; The acquisition unit is further configured to map the distant image onto the near image according to the homography transformation matrix and acquire the coordinates of the four vertices of the mapped distant image; The detection unit is used to perform vehicle target detection based on the distant view image and the close view image respectively; The transformation unit is used to transform the detection result corresponding to the distant image according to the homography transformation matrix; The fusion unit is used to fuse and output the detection results of the transformed detection results corresponding to the distant image and the detection results corresponding to the near image based on the coordinates of the four vertices of the mapped distant image. The selection unit is further configured to determine whether the superposition of the transformed distant image and the close-up image meets the preset conditions based on the coordinates of the four vertices of the mapped distant image; if not, it reselects a preset number of mapping point pairs at different positions from the distant image captured by the distant camera and the close-up image captured by the close-up camera.

6. A vehicle target detection system based on a dual-camera system according to claim 5, characterized in that, The acquisition unit is specifically used to obtain information according to the formula. Calculate the homography matrix, where H is a homography matrix consisting of 9 floating-point numbers. U is a left singular matrix, S is a diagonal matrix, and V is a right singular matrix. The mapping point is (x, y), and the mapping point of the close-up image is (X, Y).

7. A vehicle target detection system based on a dual-camera system according to claim 5, characterized in that, The detection unit is specifically used to perform vehicle target detection on the distant and close-up images respectively using a preset YOLOv5 algorithm.

8. A vehicle target detection system based on a dual-camera system according to claim 5, characterized in that, The fusion unit is specifically used to delete the detection results corresponding to the near image located inside the quadrilateral formed by the four vertices of the mapped far image from the transformed detection results corresponding to the far image and the detection results corresponding to the near image; save the detection results corresponding to the near image located at the boundary of the quadrilateral and other detection results, and delete the detection results corresponding to the far image at the boundary of the quadrilateral.

Images