An intersection target fusion method and system based on homography transformation matrix

Abstract

The application discloses a kind of intersection target fusion method and system based on homography transformation matrix, it is related to intelligent traffic management field, including: according to the pixel coordinates of pre-set feature points in the image collected by each intersection radar camera, two two pairing is matched with each other, according to the homography transformation matrix between opposite intersection and the corner point pixel coordinates of each target camera detection frame corresponding to each intersection radar camera, the camera target fusion data between opposite intersection radar camera is acquired, finally, according to the radar and visual target fusion data between opposite intersection radar camera and the spliced picture between opposite intersection camera in opposite intersection radar camera, intersection radar camera target fusion data is acquired;So high real-time, high-accuracy intersection target data fusion is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic management, and in particular to a method and system for crossroad target fusion based on homography transformation matrix. Background Technology

[0002] With the increasing number of cars in cities, road conditions are becoming increasingly complex, especially at intersections where vehicles, non-motorized vehicles, and pedestrians converge. Therefore, a combination of radar sensors and cameras is typically used to track and detect vehicles at multiple intersections. Thus, fusing radar and camera data from multiple intersections has become a key issue. Currently, radar-camera data fusion usually relies on a backpropagation (BP) neural network algorithm to fuse data collected from various sensors. However, the BP neural network algorithm is only suitable for fusing radar-camera data from a single intersection, and it suffers from high computational complexity, low real-time performance, and demanding hardware requirements. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a method and system for intersection target fusion based on homography transformation matrix, which can solve the problem that existing methods do not perform target fusion for intersections.

[0004] To achieve the above objectives, on the one hand, the present invention provides a method for intersection target fusion based on homography transformation matrix, the method comprising:

[0005] Obtain the pixel coordinates of preset feature points in the images captured by the radar cameras at each intersection;

[0006] Based on the pixel coordinates of preset feature points in the images captured by the radar cameras at opposing intersections, obtain the homography transformation matrix between opposing intersections;

[0007] Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection boxes of each target camera corresponding to each intersection's radar detector, camera target fusion data between opposing intersection radar detectors is obtained.

[0008] Based on the camera target fusion data between opposing intersection radar cameras and the pre-set matching and fusion relationship between the radar sensors and cameras of each radar camera, obtain the radar target fusion data between opposing intersection radar cameras.

[0009] Based on the target fusion data between opposing intersection radar cameras and the stitched images between opposing intersection cameras in the opposing intersection radar cameras, the target fusion data of the intersection radar cameras is obtained.

[0010] Furthermore, before the step of obtaining camera target fusion data between opposing intersection radar cameras based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the camera detection frames corresponding to each intersection radar camera, the method further includes:

[0011] Based on the homography transformation matrix corresponding to any intersection, the corner pixel coordinates of each camera target detection frame detected by the intersection radar camera are homography transformed to obtain the transformed corner pixel coordinates of the camera detection frame.

[0012] Further, the step of obtaining camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the camera detection frames corresponding to each intersection's radar detectors includes:

[0013] If the transformed pixel coordinates of the target point detection box captured by the intersection camera are all outside the field of view captured by the opposing intersection camera, then the camera pixel coordinate information of the target point is retained.

[0014] If the transformed pixel coordinates of the target point detection box captured by the intersection camera are all within the field of view captured by the opposing intersection camera, then the camera pixel coordinate information of the target point is deleted.

[0015] Further, the step of obtaining camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the camera detection frames corresponding to each intersection's radar detectors includes:

[0016] If at least one of the transformed pixel coordinates corresponding to the target point detection box captured by the intersection camera is within the field of view captured by the opposing intersection camera, then it is determined whether there is a camera detection box overlapping the target detection box in the target point detection box captured by the opposing intersection camera.

[0017] If it does not exist, the camera pixel coordinate information corresponding to the target point detection box collected by the intersection camera is retained;

[0018] If it exists, delete the camera pixel coordinate information corresponding to the target point detection box collected by the intersection camera with the highest overlap rate, and save the camera pixel coordinate information corresponding to the target point detection box collected by the opposite intersection camera.

[0019] Furthermore, the step of obtaining radar target fusion data between opposing intersection radar cameras based on camera target fusion data between opposing intersection radar cameras and the preset matching and fusion relationship between the radar sensors and cameras corresponding to each radar camera includes:

[0020] The target fusion data between the cameras at the opposing intersection is compared with the target collected by a camera in any direction at the opposing intersection to obtain the target collected by the camera at the intersection that was deleted.

[0021] Based on the matching and fusion relationship between the radar sensor and camera corresponding to the intersection's radar-sensing camera, determine whether the deleted targets collected by the intersection's cameras have matching radar targets:

[0022] If it exists, the radar target will be deleted.

[0023] On the other hand, the present invention provides an intersection target fusion system based on homography transformation matrix, the system comprising: an acquisition unit, used to acquire the pixel coordinates of preset feature points in the images collected by the radar cameras at each intersection;

[0024] The acquisition unit is also used to acquire the homography transformation matrix between opposing intersections based on the pixel coordinates of preset feature points in the image acquired by the opposing intersection radar camera;

[0025] The acquisition unit is also used to acquire camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each intersection's radar camera;

[0026] The acquisition unit is also used to acquire radar target fusion data between opposing intersection radar cameras based on camera target fusion data between opposing intersection radar cameras and the preset matching and fusion relationship between radar sensors and cameras corresponding to each radar camera.

[0027] The fusion unit is used to obtain the target fusion data of the crossroads radar cameras based on the fusion data of the radar targets between the radar cameras at opposite intersections and the stitched images between the cameras at opposite intersections in the radar cameras at opposite intersections.

[0028] Furthermore, the system also includes: a transformation unit;

[0029] The transformation unit is specifically used to perform homography transformation on the corner pixel coordinates of each camera target detection frame detected by the intersection radar detector according to the homography transformation matrix corresponding to any intersection, so as to obtain the transformed corner pixel coordinates of the camera detection frame.

[0030] Furthermore, the acquisition unit is specifically used to retain the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all outside the field of view acquired by the oncoming intersection camera; and to delete the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all within the field of view acquired by the oncoming intersection camera.

[0031] Furthermore, the acquisition unit is specifically configured to determine whether there is a camera detection box overlapping the target detection box if at least one of the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera is within the field of view acquired by the oncoming intersection camera; if not, retain the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera; if so, delete the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera with the highest overlap rate, and save the corresponding camera pixel coordinate information corresponding to the target point detection box acquired by the oncoming intersection camera.

[0032] Furthermore, the acquisition unit is specifically used to compare the camera target fusion data between the opposing intersection radar cameras with the target collected by any direction camera in the opposing intersection to obtain the deleted target collected by the intersection camera; based on the matching fusion relationship between the radar sensor corresponding to the intersection radar camera and the camera, it determines whether there is a matching radar target for the deleted target collected by the intersection camera; if so, the radar target is deleted.

[0033] This invention provides a method and system for crossroad target fusion based on homography transformation matrix. According to the pixel coordinates of preset feature points in the images collected by radar cameras at each intersection, opposing intersections are paired up. Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each radar camera at each intersection, camera target fusion data between opposing radar cameras is obtained. Finally, based on the radar target fusion data between opposing radar cameras and the stitched image between the opposing cameras in the radar cameras, crossroad target fusion data is obtained. This achieves a high real-time performance and high accuracy method for crossroad target fusion. Attached Figure Description

[0034] Figure 1 This is a flowchart of an intersection target fusion method based on homography transformation matrix provided by the present invention;

[0035] Figure 2 This is a schematic diagram of the structure of an intersection target fusion system based on homography transformation matrix provided by the present invention;

[0036] Figure 3 This is a schematic diagram of the definition of a coordinate system for an intersection provided by the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0038] like Figure 1As shown in the figure, an intersection target fusion method based on homography transformation matrix provided by an embodiment of the present invention includes the following steps:

[0039] 101. Obtain the pixel coordinates of preset feature points in the images captured by the radar cameras at each intersection.

[0040] Among them, there can be 4 preset feature points, and the range should be as large as possible. Generally, the 4 corner points are close to the overlapping area, such as the intersection of lane lines and road surface.

[0041] 102. Based on the pixel coordinates of preset feature points in the images captured by the radar cameras at opposite intersections, obtain the homography transformation matrix between opposite intersections.

[0042] Specifically, for example, such as Figure 3 The diagram defines a large coordinate system xoy for the intersections. The coordinate system for the radar camera at intersection 1 is x1o1y1, for intersection 2 it is x2o2y2, for intersection 3 it is x3o3y3, and for intersection 4 it is x4o4y4. Opposing intersections are stitched together. Taking the camera images from north-facing camera 1 and south-facing camera 3 as an example, the method for west-facing camera 4 and east-facing camera 2 is similar. This yields the homography matrix h24 for the camera field of view transformation from west to east and the homography matrix h13 for the camera field of view transformation from south to north. Because homography requires a large overlap of camera fields of view between different intersections for feature point selection, target fusion is first performed on the opposite intersections. The overlapping field of view is the entire area in the middle of the intersection. Four feature points are selected and the pixel coordinates of the four corner points in the camera images of the opposite traffic cameras are obtained. A1 and B1 represent the same corner point, and the same applies to other points. The pixel coordinates of the feature points of the north-facing traffic cameras are A1(x1_north, y1_north), A2(x2_north, y2_north), A3(x3_north, y3_north), A4(x4_north, y4_north), and the pixel coordinates of the feature points of the south-facing traffic cameras are B1(x1_south, y1_south), B2(x2_south, y2_south), B3(x3_south, y3_south), B4(x4_south, y4_south). Then, a transformation model for converting a south-facing camera image to a north-facing camera image is established by detecting the correspondence between feature point pairs. Next, the homography transformation matrix h13 between the two images is calculated, defined as follows: Then calculate the homography transformation matrix h13, which projects the south-facing camera plane onto the north-facing camera plane: Substituting the four pixel coordinates B1(x1_south, y1_south)-B4(x4_south, y4_south) of the south-facing camera feature point into (x1, y1)-(x4, y4), and substituting the four pixel coordinates A1(x1_north, y1_north)-A4(x4_north, y4_north) of the north-facing camera feature point into (X1, Y1)-(X4, Y4), we get: Next, perform eigenvalue decomposition on the large matrix on the left, take the last column of the decomposed unitary matrix, and rearrange it into a 3*3 matrix, which is the homography transformation matrix h13. Similarly, assign the pixel coordinates of the camera feature points of the west-facing camera to B1-B4, and the pixel coordinates of the camera feature points of the east-facing camera to A1-A4, to obtain the homography transformation matrix h24 of the west-facing camera plane projected onto the east-facing camera plane.

[0043] 103. Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection boxes of each target camera corresponding to each intersection's radar-guided camera, obtain the camera target fusion data between opposing intersection radar-guided cameras.

[0044] In this embodiment of the invention, before step 103, the method may further include: performing homography transformation on the corner pixel coordinates of each camera target detection frame detected by the intersection radar detector according to the homography transformation matrix corresponding to any intersection, so as to obtain the transformed corner pixel coordinates of the camera detection frame.

[0045] Specifically, for example, taking the corner pixel coordinates and h13 of the north-south facing radar camera detection frame as an example, the same applies to the east-west facing frame. The corner pixel coordinates frameSN = [SN1…SNm] of the south-facing radar camera detection frame, where m is the number of targets, are used to perform homography transformation on each target. SN1 represents the four corner pixel coordinates of the first target's camera detection frame, resulting in the transformed corner pixel coordinates Framesn = [sn1…snm]. Then, the four corner pixel coordinates of SN1 are substituted sequentially into (x, y) to obtain (X, Y), which are then assigned to the transformed four corner pixel coordinates sn1 of the detection frame.

[0046] in

[0047] At this point, step 103 may specifically include: if the transformed pixel coordinates corresponding to the target point detection box captured by the intersection camera are all outside the field of view captured by the opposing intersection camera, then retain the camera pixel coordinate information of the target point; if the transformed pixel coordinates corresponding to the target point detection box captured by the intersection camera are all within the field of view captured by the opposing intersection camera, then delete the camera pixel coordinate information of the target point.

[0048] Specifically, for example, if the transformed pixel coordinates of the south-facing target point's camera frame are *si*, where *si* represents the pixel coordinates of the four corner points of the *i*-th target's camera frame, and all of them are outside the north-facing camera's field of view, then the camera information for that target point is retained. Taking the top-left corner point (*si_leftup_x, *si_leftup_y*) as an example: *si_leftup_x* < 1 or *si_leftup_x* > *w_north*, where *w_north* is the width of the north-facing image; or *si_leftup_y* < 1 or *si_leftup_y* > *h_north*, where *h_north* is the height of the north-facing image. If any of these four conditions is met, then the top-left corner point is considered to be outside the north-facing camera's field of view. Then, the top-right, bottom-right, and bottom-left corner points are sequentially checked to see if they are outside the north-facing camera's field of view. If all four corner points are outside the north-facing camera's field of view, then the condition is satisfied. If the entire camera frame of a target point facing south is within the field of view of a camera facing north, then the camera information for that target point is deleted. Taking the top-left corner point (sn i_l eftup_x, sn i_l eftup_y) as an example: sn i_l eftup_x>=1 and sn i_l eftup_x<=w_north, and sn i_l eftup_y>=1 and sn i_l eftup_y<=h_north. All four conditions must be met for the top-left corner point to be considered within the field of view of a camera facing north. Then, the top-right corner point, bottom-right corner point, and bottom-left corner point are checked in turn to see if they are within the field of view of a camera facing north. If all four corner points are within the field of view of a camera facing north, then the condition is met.

[0049] Furthermore, if at least one of the transformed pixel coordinates corresponding to the target point detection box captured by the intersection camera is within the field of view captured by the oncoming intersection camera, then it is determined whether there is a camera detection box overlapping the target detection box in the target point detection box captured by the oncoming intersection camera; if not, the camera pixel coordinate information corresponding to the target point detection box captured by the intersection camera is retained; if so, the camera pixel coordinate information corresponding to the target point detection box captured by the intersection camera with the highest overlap rate is deleted, and the camera pixel coordinate information corresponding to the target point detection box captured by the oncoming intersection camera is saved.

[0050] Specifically, for example, if at least one corner of the camera frame of a target point facing south is within the field of view of a camera facing north, determine whether there is a camera frame facing north overlapping with it: frameSN_other = [SN1_other…SNn_other], where n is the number of targets, and SN i_other represents the pixel coordinates of the four camera frame corners of the i-th target. Taking the top left corner (sn i_l eftup_x, sn i_l eftup_y) of the camera frame of a target point facing south as an example: sn i_l eftup_x>=SN i_other_l eftup_x and sn i_l eftup_x<=SN i_other_r i ghtup_x and sn i_l eftup_y>=SN i_other_l eftup_y and sn i_l eftup_y<=SN i_other_r i ghtdown_y. All four conditions must be met for the top left corner to be considered to be within the camera frame facing north. Next, sequentially check if the top right, bottom right, and bottom left corners are within the north-facing camera frame. If at least one corner is within the north-facing camera frame, then there is an overlap with a north-facing camera frame. Continue searching for the (i+1)th north-facing target camera frame, checking for overlaps, until all north-facing target points have been searched. If not, retain the target point camera information facing south; if so, calculate the maximum overlap rate (rat io) between the south-facing target frame and the overlapping north-facing target frame: Area = (min(sn i_r i ghtup_x,SNj_other_r ghtup_x)-max(sn i_l eftup_x,SNj_other_l eftup_x))*(min(sn i_l eftdown_y,SNj_other_l eftdown_y)-max(sn i_l eftup_y,SNj_other_l eftup_y)); Area1 of the north-facing camera frame = (SNj_other_r i ghtup_x-SNj_other_l eftup_x)*(SNj_other_l eftdown_y-SNj_other_l eftup_y); Area2 of the south-facing camera frame = (sn i_r i ghtup_x-sn i_l eftup_y)*(SNj_other_l eftdown_y-SNj_other_l eftup_y); eftup_x)*(sn i_l eftdown_y-sn i_l eftup_y); overlap rate rat io=Area / (Area1+Area2-Area); mark the north-facing target point with the largest area as matched, and when processing the other south-facing targets, do not consider the matched north-facing camera points, delete the information of the south-facing target camera points, while retaining all the north-facing target point camera information.

[0051] 104. Based on the camera target fusion data between opposing intersection radar cameras and the pre-set matching and fusion relationship between the radar sensors and cameras of each radar camera, obtain the radar target fusion data between opposing intersection radar cameras.

[0052] In this embodiment of the invention, step 104 may specifically include: comparing the camera target fusion data between opposing intersection radar cameras with the target collected by any direction camera in the opposing intersection to obtain the deleted target collected by the intersection camera; determining whether there is a matching radar target for the deleted target collected by the intersection camera based on the matching fusion relationship between the radar sensor and the camera corresponding to the intersection radar camera; if so, deleting the radar target.

[0053] Specifically, for example, the target fusion data of the north-south camera is compared with the target of the south-facing camera to obtain the south-facing camera target to be deleted. Then, based on the radar-visual fusion matching relationship of the south-facing intersection, it is determined whether there is a matching radar target for the deleted camera target. If there is, the radar target is deleted. This yields the target fusion data of the north-south radar-visual cameras. The same applies to the east-west direction.

[0054] 105. Based on the target fusion data between opposing intersection radar cameras and the stitched images between opposing intersection cameras in the opposing intersection radar cameras, obtain the target fusion data of the intersection radar cameras.

[0055] In this embodiment of the invention, step 105 may specifically include: obtaining the total fused target data of the intersection based on the target fusion data of the east-west intersection radar camera and the north-south intersection radar camera, as well as the images stitched together by the north-south opposing intersection cameras and the east-west opposing intersection cameras. Specifically, the north-south homography transformation matrix h13 transforms the coordinates of each pixel in the south-facing image, assigning the corresponding RGB value of the south image to the RGB value at the north-facing transformed coordinates, thus completing the north-south image stitching c13; the east-west homography matrix h24 transforms the coordinates of each pixel in the west-facing image, assigning the corresponding RGB value of the west image to the RGB value at the east-facing transformed coordinates, thus completing the east-west image stitching c24, thereby obtaining the total fused target data of the intersection.

[0056] This invention provides a method for crossroads target fusion based on homography transformation matrix. According to the pixel coordinates of preset feature points in the images collected by radar cameras at each intersection, opposing intersections are paired up. Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each radar camera at each intersection, camera target fusion data between opposing radar cameras is obtained. Finally, based on the radar target fusion data between opposing radar cameras and the stitched image between the opposing cameras in the radar cameras, crossroads target fusion data is obtained. This achieves a high real-time performance and high accuracy method for crossroads target fusion.

[0057] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide an intersection target fusion system based on homography transformation matrix, such as... Figure 2 As shown, the system includes: an acquisition unit 21 and a fusion unit 22.

[0058] The acquisition unit 21 is used to acquire the pixel coordinates of preset feature points in the images collected by the radar cameras at each intersection.

[0059] The acquisition unit 21 is also used to acquire the homography transformation matrix between opposing intersections based on the pixel coordinates of preset feature points in the image acquired by the radar camera at the opposing intersection.

[0060] The acquisition unit 21 is further configured to acquire camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each intersection radar camera.

[0061] The acquisition unit 21 is further configured to acquire radar target fusion data between opposing intersection radar cameras based on camera target fusion data between opposing intersection radar cameras and the preset matching and fusion relationship between radar sensors and cameras corresponding to each radar camera.

[0062] The fusion unit 22 is used to obtain the target fusion data of the crossroads radar cameras based on the target fusion data between the radar cameras at opposite intersections and the stitched images between the cameras at opposite intersections in the radar cameras at opposite intersections.

[0063] Furthermore, the system also includes: a transformation unit 23;

[0064] The transformation unit 23 is specifically used to perform homography transformation on the corner pixel coordinates of each camera target detection frame detected by the intersection radar camera according to the homography transformation matrix corresponding to any intersection, so as to obtain the transformed corner pixel coordinates of the camera detection frame.

[0065] Furthermore, the acquisition unit 21 is specifically used to retain the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all outside the field of view acquired by the opposing intersection camera; and to delete the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all within the field of view acquired by the opposing intersection camera.

[0066] Furthermore, the acquisition unit 21 is specifically used to determine whether there is a camera detection box overlapping the target detection box in the target point detection box acquired by the intersection camera if at least one of the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera is within the field of view acquired by the oncoming intersection camera; if not, the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera is retained; if it exists, the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera with the largest overlap rate is deleted, and the camera pixel coordinate information corresponding to the target point detection box acquired by the oncoming intersection camera is saved.

[0067] Furthermore, the acquisition unit 21 is specifically used to compare the camera target fusion data between the opposing intersection radar cameras with the target collected by any direction camera in the opposing intersection to obtain the deleted target collected by the intersection camera; based on the matching fusion relationship between the radar sensor corresponding to the intersection radar camera and the camera, it is determined whether there is a matching radar target for the deleted target collected by the intersection camera; if so, the radar target is deleted.

[0068] This invention provides an intersection target fusion system based on homography transformation matrix. According to the pixel coordinates of preset feature points in the images collected by radar cameras at each intersection, opposing intersections are paired up. Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each radar camera at each intersection, camera target fusion data between opposing radar cameras is obtained. Finally, based on the radar target fusion data between opposing radar cameras and the stitched image between the opposing cameras in the radar cameras, intersection target fusion data is obtained. This achieves a high real-time performance and high accuracy intersection target fusion system.

[0069] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0070] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0071] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0072] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0073] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0074] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing systems, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0075] The steps of the methods or algorithms described in the embodiments of the present invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be disposed in an ASIC, which can be disposed in a user terminal. Optionally, the processor and the storage medium can also be disposed in different components of the user terminal.

[0076] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The discs and disks mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while discs typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for crossroad target fusion based on homography transformation matrix, characterized in that, The method includes: Obtain the pixel coordinates of preset feature points in the images captured by the radar cameras at each intersection; Based on the pixel coordinates of preset feature points in the images captured by the radar cameras at opposing intersections, obtain the homography transformation matrix between opposing intersections; Based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the target camera detection frames corresponding to each intersection's radar detectors, camera target fusion data between opposing intersection radar detectors is obtained. Before this step, the method further includes: Based on the homography transformation matrix corresponding to any intersection, the corner pixel coordinates of each camera target detection box detected by the intersection radar camera are homography transformed to obtain the transformed corner pixel coordinates of the camera detection box. The step of obtaining camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the camera detection frames corresponding to each intersection's radar detectors includes: If the transformed pixel coordinates of the target point detection box captured by the intersection camera are all outside the field of view captured by the opposing intersection camera, then the camera pixel coordinate information of the target point is retained. If the transformed pixel coordinates of the target point detection box captured by the intersection camera are all within the field of view captured by the opposing intersection camera, then the camera pixel coordinate information of the target point is deleted. If at least one of the transformed pixel coordinates corresponding to the target point detection box captured by the intersection camera is within the field of view captured by the opposing intersection camera, then it is determined whether there is a camera detection box overlapping the target detection box in the target point detection box captured by the opposing intersection camera. If it does not exist, the camera pixel coordinate information corresponding to the target point detection box collected by the intersection camera is retained; If it exists, delete the camera pixel coordinate information corresponding to the target point detection box collected by the intersection camera with the highest overlap rate, and save the camera pixel coordinate information corresponding to the target point detection box collected by the opposite intersection camera. Based on the camera target fusion data between opposing intersection radar cameras and the pre-set matching and fusion relationship between the radar sensors and cameras of each radar camera, obtain the radar target fusion data between opposing intersection radar cameras. Based on the target fusion data between opposing intersection radar cameras and the stitched images between opposing intersection cameras in the opposing intersection radar cameras, the target fusion data of the intersection radar cameras is obtained.

2. The intersection target fusion method based on homography transformation matrix according to claim 1, characterized in that, The step of obtaining radar target fusion data between opposing intersection radar cameras based on camera target fusion data between opposing intersection radar cameras and the pre-set matching and fusion relationship between the radar sensors and cameras of each radar camera includes: The target fusion data between the cameras at opposing intersections is compared with the target collected by a camera in any direction at the opposing intersection to obtain the target collected by the deleted intersection camera. Based on the matching and fusion relationship between the radar sensor and camera corresponding to the intersection's radar-sensing camera, determine whether the deleted targets collected by the intersection's cameras have matching radar targets: If it exists, the radar target will be deleted.

3. A target fusion system for intersections based on homography transformation matrix, characterized in that, The system includes: The acquisition unit is used to acquire the pixel coordinates of preset feature points in the images collected by the radar cameras at each intersection; The acquisition unit is also used to acquire the homography transformation matrix between opposing intersections based on the pixel coordinates of preset feature points in the image acquired by the opposing intersection radar camera; The acquisition unit is also used to acquire camera target fusion data between opposing intersections based on the homography transformation matrix between opposing intersections and the corner pixel coordinates of the detection frames of each target camera corresponding to each intersection's radar camera; The acquisition unit is also used to acquire radar target fusion data between opposing intersection radar cameras based on camera target fusion data between opposing intersection radar cameras and the preset matching and fusion relationship between radar sensors and cameras corresponding to each radar camera. The fusion unit is used to obtain the target fusion data of the crossroads radar cameras based on the fusion data of the radar targets between the radar cameras at opposite intersections and the stitched images between the cameras at opposite intersections in the radar cameras at opposite intersections. The system further includes: a transformation unit; The transformation unit is specifically used to perform homography transformation on the corner pixel coordinates of each camera target detection box detected by the intersection radar detector according to the homography transformation matrix corresponding to any intersection, so as to obtain the transformed corner pixel coordinates of the camera detection box. The acquisition unit is further configured to retain the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all outside the field of view acquired by the oncoming intersection camera; and delete the camera pixel coordinate information of the target point if the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera are all within the field of view acquired by the oncoming intersection camera. The acquisition unit is further configured to determine whether there is a camera detection box overlapping the target detection box if at least one of the transformed pixel coordinates corresponding to the target point detection box acquired by the intersection camera is within the field of view acquired by the oncoming intersection camera; if not, retain the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera; if so, delete the camera pixel coordinate information corresponding to the target point detection box acquired by the intersection camera with the highest overlap rate, and save the corresponding camera pixel coordinate information corresponding to the target point detection box acquired by the oncoming intersection camera.

4. The intersection target fusion system based on homography transformation matrix according to claim 3, characterized in that, The acquisition unit is further configured to compare the camera target fusion data between opposing intersection radar cameras with the target collected by a camera in any direction at the opposing intersection to obtain the deleted targets collected by the intersection camera; and to determine whether there is a matching radar target for the deleted targets collected by the intersection camera based on the matching fusion relationship between the radar sensor and the camera corresponding to the intersection radar camera. If there is, the radar target will be deleted.

Images