Multi-target matching optimization method and system for intelligent vehicle lamp self-closed loop system

Abstract

The invention provides a multi-target matching optimization method and system of an intelligent vehicle lamp self-closed loop system, and belongs to the technical field of vehicle lamp intelligent control. The method comprises the following steps: obtaining a view field overlapping area of a plurality of cameras based on the positions, view field angles and farthest detection distances of the cameras; judging whether the target object is located in the view field overlapping area or not; if yes, taking the target object as a to-be-matched target; calculating a shielding area of each to-be-matched target based on a preset shielding judgment mechanism; judging whether the to-be-matched target is located in the shielding area or not; if not, inputting the feature vector into a pre-trained neural network model to generate a feature vector corresponding to the feature vector; respectively calculating the feature similarity and the overlapping degree of the target to be matched so as to obtain an edge weight value; constructing a cost matrix based on the edge weight values; and adopting a Hungary algorithm to obtain an augmented path so as to obtain a multi-target matching result. The target matching process is optimized, consumption of computing resources is reduced, and matching accuracy and real-time performance are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle lamp control, and in particular to a multi-objective matching optimization method and system for an intelligent vehicle lamp self-closed loop system. Background Art

[0002] With the continuous advancement of automobile technology, the intelligent headlight self-closed loop system has gradually become one of the important components of modern automobiles. Traditional headlight systems mainly rely on manual operation or simple automatic switching mechanisms, and cannot respond flexibly to changes in the driving environment. In contrast, the intelligent headlight self-closed loop system integrates advanced sensor technology and computing platforms to automatically sense the surrounding environment and adjust the lighting mode accordingly, thereby improving the safety and comfort of night driving. The intelligent headlight self-closed loop system is usually equipped with multiple cameras to capture environmental information around the vehicle from different angles. By matching targets from multiple perspectives, it can ensure that the system can fully monitor the dynamics around the vehicle, reduce blind spots, and promptly detect potential dangerous factors such as pedestrians, other vehicles or obstacles. Multi-perspective information can verify each other, reduce false alarms and missed alarms caused by misjudgment of a single perspective, and improve the overall performance of the system.

[0003] In order to improve the accuracy and robustness of multi-target matching, researchers have developed a variety of matching algorithms and deep learning models based on visual features. At present, the mainstream matching methods include: feature point matching-based algorithms, deep learning-based re-identification (ReID) models, and geometric correction and camera calibration technologies. Among them, feature point matching algorithms rely on traditional image features, such as SIFT or ORB, but they are prone to failure under changes in lighting and perspective; while ReID-based models use convolutional neural networks to extract global and local features of the target, but they are highly dependent on the annotation of large-scale data.

[0004] In a multi-camera system, achieving high-precision multi-target matching faces many technical challenges. For example, perspective differences: the same target captured by different cameras may present different appearances due to factors such as angle and distance, resulting in inconsistent visual features; occlusion and target loss: the target may be partially occluded at certain perspectives, or even temporarily leave the field of view of a camera; high real-time requirements: in intelligent driving scenarios, the target matching algorithm needs to have low latency and high efficiency to support real-time decision-making.

[0005] Therefore, there is an urgent need for an efficient and robust multi-target matching technology that can achieve accurate target association under different camera perspectives. Summary of the invention

[0006] The purpose of the present invention is to overcome at least one technical problem existing in the prior art and to provide a multi-objective matching optimization method and system for an intelligent vehicle lamp self-closed loop system.

[0007] On the one hand, an embodiment of the present invention provides a multi-target matching optimization method for a smart vehicle lamp self-closed loop system, the method comprising: S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively; S2: obtaining the position of the camera through the coordinates of different cameras in the BEV coordinate system; S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and maximum detection distance of the camera; S4: judging whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; S5: in response to the target object being located in the overlapping area of ​​the field of view, taking at least one target object as a target to be matched, the target to be matched including the target object located in the overlapping area of ​​the field of view under different camera viewing angles; S6: calculating the relative position of each target to be matched in relation to the camera based on a preset occlusion judgment mechanism. The occlusion area under the corresponding camera perspective; S7: Use the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera perspective; S8: In response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, input it into the pre-trained neural network model to generate a corresponding feature vector; S9: Based on the feature vector, the cosine similarity formula is used to respectively calculate the feature similarity between two different targets to be matched; S10: In response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, the intersection and union ratio is used to respectively calculate the overlap degree of the target detection frames of the two different targets to be matched; S11: The edge weight value is calculated based on the feature similarity between the two different targets to be matched and the overlap degree of the target detection frames of the two different targets to be matched; S12: Construct a cost matrix based on the edge weight value; S13: Based on the cost matrix, the Hungarian algorithm is used to obtain an augmented path, and the matching results of multiple targets are obtained based on the augmented path.

[0008] Furthermore, the step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to respectively obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system includes: S101: converting the camera coordinates into the camera coordinates in the vehicle coordinate system through the external parameters of the camera; S102: converting the camera coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system; S103: converting the target object coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system.

[0009] Furthermore, the step S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and maximum detection distance of the camera includes: S301: calculating the boundary points of the camera field of view based on the position, field of view angle and maximum detection distance of the camera; S302: constructing the field of view polygon of each camera based on the boundary points of the camera field of view; S303: determining the overlapping part based on the field of view polygon of each camera, and the overlapping part is the field of view overlapping area.

[0010] Furthermore, the step S6: calculating the occlusion area of ​​each target to be matched under the corresponding camera perspective based on a preset occlusion judgment mechanism includes: S601: using the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system to calculate the angle between the camera and each vertex of the target to be matched; S602: selecting the two vertices with the largest angle to define the boundary of the occlusion area; S603: drawing a ray from the camera position to the two vertices located at the boundary of the occlusion area; S604: the intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched.

[0011] Furthermore, the step S7: using the iog formula to determine whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle includes:

[0012] The iog formula is:

[0013] When the iog(A, B) value is greater than or equal to a preset threshold, it is determined that the target to be matched is located in the occlusion area under the corresponding camera viewing angle;

[0014] Where A represents the target area, B represents the occluded area, and iog(A,B) represents the ratio of the intersection between the target area A and the occluded area B to the target area A.

[0015] Furthermore, the step S8: in response to the target to be matched not being located in the occluded area under the corresponding camera viewing angle, inputting it into a pre-trained neural network model to generate a corresponding feature vector includes: S801: extracting the target to be matched from the original image containing the target object captured by the camera to obtain a target sub-graph containing the target to be matched; S802: inputting the target sub-graph into the pre-trained neural network model for forward propagation; S803: outputting the feature vector of the target to be matched through the multi-layer convolution and activation function in the pre-trained neural network model; S804: storing the feature vector in a feature database.

[0016] Furthermore, after step S5, the following also includes: in response to the target object not being located in the overlapping area of ​​the field of view, the target is not included in the items to be matched; after step S8, the following also includes: in response to the target to be matched being located in the occlusion area under the corresponding camera viewing angle, the target is not included in the items to be matched.

[0017] Furthermore, the cosine similarity formula in step S9 is:

[0018] In the formula, C and D represent the feature vectors of two targets to be matched, and cos(C, D) represents the possibility that the two targets to be matched are the same target;

[0019] The intersection-over-union ratio in step S10 is:

[0020] In the formula, E and F represent the target detection frames of the two targets to be matched, G represents the circumscribed rectangle of E and F, GIOU is a measure of the overlap degree of the two target detection frames, and the GIOU value range is between -1 and 1;

[0021] The edge weight calculation formula in step S11 is: edg = ω1×cos(C,D)+ω2×GIOU;

[0022] Wherein, ω1 and ω2 are preset weight coefficients respectively.

[0023] Further, the step S13 also includes: S1301: when performing multi-frame target matching, comparing the target objects contained in the current frame with those contained in the previous frame to determine the target state, including: one or a combination of: a new target appears, a target disappears, the target remains unchanged, and the target changes; S1302: when the target state is in the target remains unchanged, using the matching information of the previous frame to directly reference its corresponding cost matrix, using the Hungarian algorithm based on the cost matrix to obtain an augmented path, and obtaining the matching results of multiple targets based on the augmented path;

[0024] S1303: When the target state is in the state of target change, update the feature vector of the changed target, repeat steps S9-S12 to obtain the corresponding cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path; S1304: When the target state is in the state of a new target appearing, repeat steps S9-S12 to calculate the edge weight value between the newly appeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path; S1305: When the target state is in the state of target disappearance, repeat steps S9-S12 to calculate the edge weight value between the disappeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path.

[0025] In a second aspect, an embodiment of the present invention provides a multi-objective matching optimization system for a self-closed-loop system of intelligent headlights, the system adopts the multi-objective matching optimization method for the self-closed-loop system of intelligent headlights, and the system includes: a coordinate conversion module, adapted to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively; a field of view overlapping area calculation module, adapted to obtain the position of the camera through the coordinates of different cameras in the BEV coordinate system, and obtain the field of view overlapping area of ​​multiple cameras based on the position, field of view angle and maximum detection distance of the camera; a first judgment module, adapted to judge whether the target object is located in the field of view overlapping area based on the coordinates of the different target objects in the BEV coordinate system; an occlusion area calculation module, adapted to respond to the target object being located in the field of view overlapping area as a target to be matched, the target to be matched includes the target of the target object located in the field of view overlapping area under different camera viewing angles; the occlusion area calculation module is adapted to calculate the occlusion area based on a preset occlusion judgment mechanism The occlusion area of ​​each target to be matched under the corresponding camera perspective; the second judgment module is suitable for using the iog formula to judge whether the target to be matched is located in the occlusion area under the corresponding camera perspective; the feature vector generation module is suitable for, in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, inputting it into a pre-trained neural network model to generate a corresponding feature vector; the edge weight calculation module is suitable for using the cosine similarity formula to calculate the feature similarity between two different targets to be matched based on the feature vector; in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, using the intersection and union ratio to calculate the overlap degree of the target detection frames of the two different targets to be matched; the edge weight value is calculated based on the feature similarity between the two different targets to be matched and the overlap degree of the target detection frames of the two different targets to be matched; the cost matrix construction module is suitable for constructing a cost matrix based on the edge weight value; the matching result generation module is suitable for obtaining an augmented path based on the cost matrix using the Hungarian algorithm, and obtaining the matching result of multiple targets based on the augmented path.

[0026] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the multi-objective matching optimization method of the above-mentioned intelligent vehicle light self-closed-loop system.

[0027] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned multi-objective matching optimization method of the intelligent vehicle light self-closed-loop system.

[0028] The beneficial effects of the present invention are:

[0029] The embodiment of the present application provides a multi-objective matching optimization method and system for an intelligent vehicle lamp self-closed loop system, which has the following advantages over the prior art:

[0030] (1) This application abandons the traditional image feature extraction method and adopts a deep neural network feature extraction method, which can more effectively capture the same features of the same target under different perspectives, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching.

[0031] (2) This application introduces an overlapping area extraction mechanism to ensure that targets are matched only in the shared area between cameras. In addition, the visibility of the target is evaluated in combination with the occlusion judgment mechanism to ensure that only unoccluded targets are matched. This strategy reduces unnecessary computational burden and improves matching efficiency and accuracy.

[0032] (3) This application adopts the Hungarian matching algorithm, which is a classic optimization method for solving the allocation problem. It constructs a cost matrix to find the matching solution with the lowest cost. In order to speed up the matching efficiency, in multi-frame target matching, the matching result of the previous frame is used as a priori information, and only the cost matrix of the changed part is updated to avoid repeated calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0034] Figure 1 This is a flow chart of a multi-objective matching optimization method for an intelligent vehicle light self-closed loop system provided in Example 1 of the present invention.

[0035] Figure 2 It is a flow chart of the method involved in step S1 provided in Example 1 of the present invention.

[0036] Figure 3 It is a flow chart of the method involved in step S3 provided in Example 1 of the present invention.

[0037] Figure 4 It is a schematic diagram of overlapping areas of fields of view of different cameras provided in Example 1 of the present invention.

[0038] Figure 5 It is a flow chart of the method involved in step S6 provided in Example 1 of the present invention.

[0039] Figure 6 It is a schematic diagram of an occlusion area under a camera field of view provided by Embodiment 1 of the present invention.

[0040] Figure 7 It is a flow chart of the method involved in step S8 provided in Example 1 of the present invention.

[0041] Figure 8 It is a flow chart of the method involved in step S13 provided in Example 1 of the present invention.

[0042] Fig. 9 This is a structural diagram of a multi-objective matching optimization system of an intelligent vehicle light self-closed loop system provided in Example 2 of the present invention.

[0043] Fig.10 This is a partial block diagram of an electronic device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0044] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0045] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0046] The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0047] Example 1

[0048] For ease of understanding, the inventive concept is generally described below before describing the embodiments of the present invention in detail:

[0049] The present application provides a multi-target matching optimization method and system for a self-closed loop system of an intelligent headlight, aiming to solve the target matching problems caused by differences in viewing angles, overlapping areas, occlusions, and inefficiency in intelligent headlights. By obtaining the internal and external parameters of different cameras, the coordinates of each camera are converted into coordinates in the vehicle coordinate system, and the vehicle coordinate points detected by each camera are further uniformly converted into coordinates under the bird's-eye view (BEV) to achieve alignment of different camera data. An overlapping area extraction mechanism is used to extract the intersection area of ​​different viewing angles, and the detection targets of the large viewing angle camera located in the intersection area are screened out, and the occlusion area of ​​each target under the small viewing angle camera under the BEV viewing angle is calculated. The large viewing angle camera targets that are not in the occlusion area are screened out using the occlusion judgment mechanism to ensure that only visible targets are matched and improve the accuracy of matching. In order to obtain the features of the same target under different viewing angles, a deep learning network structure is used to extract features of the screened targets and all targets under the small viewing angle, and the extracted features are used to construct a cost matrix. The optimal match is achieved by minimizing the total cost. When performing multi-frame matching, the cost matrix is ​​dynamically updated, and only the changed parts are updated to speed up the calculation efficiency.

[0050] The specific implementation is as follows:

[0051] like Figure 1 As shown, it is a flow chart of a multi-objective matching optimization method of an intelligent vehicle light self-closed loop system provided in this embodiment.

[0052] As an example, the method includes:

[0053] S1: Convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively.

[0054] S2: Get the position of the camera through the coordinates of different cameras in the BEV coordinate system.

[0055] S3: Obtaining overlapping areas of the fields of view of multiple cameras based on the positions, field of view angles, and maximum detection distances of the cameras.

[0056] S4: Determine whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system.

[0057] S5: In response to the target object being located in the overlapping area of ​​the fields of view, the at least one target object is used as a target to be matched, where the target to be matched includes targets of the target object located in the overlapping area of ​​the fields of view under different camera viewing angles.

[0058] S6: Calculate the occlusion area of ​​each to-be-matched target under the corresponding camera viewing angle based on a preset occlusion judgment mechanism.

[0059] S7: Use the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle.

[0060] S8: In response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting the target to be matched into a pre-trained neural network model to generate a corresponding feature vector.

[0061] S9: Based on the feature vector, the cosine similarity formula is used to calculate the feature similarity between two different targets to be matched.

[0062] S10: In response to the to-be-matched target not being located in the occlusion region corresponding to the to-be-matched target under the camera viewing angle, using an intersection-over-union ratio to respectively calculate the overlap degree of target detection frames of two different to-be-matched targets.

[0063] S11: Calculate an edge weight value based on the feature similarity between two different to-be-matched targets and the overlap degree of the target detection frames of the two different to-be-matched targets.

[0064] S12: Construct a cost matrix based on the edge weight values.

[0065] S13: Obtain an augmented path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmented path.

[0066] In some feasible embodiments, combined with Figure 2 As shown, the step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects to the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively includes: S101: converting the camera coordinates to the camera coordinates in the vehicle coordinate system through the camera's external parameters; S102: converting the camera coordinates in the vehicle coordinate system to the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system; S103: converting the target object coordinates in the vehicle coordinate system to the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system. Specifically, the camera coordinates are converted to the coordinates under the BEV (Bird's Eye View) perspective: the camera external parameters include the rotation matrix (R) and the translation vector (T), and the camera coordinates (X c ,Y c ,Z c ) is converted to the vehicle coordinate system (X cv ,Y cv ,Z cv), the conversion formula is (X cv ,Y cv ,Z cv )=R·(X c ,Y c ,Z c )+T, the camera coordinates in the vehicle coordinate system are converted to the BEV perspective, so (X cvb ,Y cvb ,Z cvb )=(X cv ,Y cv ,0). The coordinates of the target object are converted to the coordinates from the perspective of the BEV: the coordinates of the target object in the vehicle coordinate system (X ov ,Y ov ,Z ov ) is converted to the BEV coordinate system, the coordinates of the target object can be expressed as (X bev ,Y bev ,Z bev )=(X ov ,Y ov ,0). By obtaining the intrinsic and extrinsic parameters of different cameras, the coordinates of each camera are converted into the coordinates in the vehicle coordinate system, and the vehicle coordinate points detected by each camera are further uniformly converted into coordinates under the bird's-eye view (BEV) to achieve alignment of different camera data.

[0067] In some feasible embodiments, combined with Figure 3 and Figure 4 As shown, the step S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and farthest detection distance of the camera includes: S301: calculating the boundary points of the camera field of view based on the position, field of view angle and farthest detection distance of the camera; S302: constructing the field of view polygon of each camera based on the boundary points of the camera field of view; S303: determining the overlapping part based on the field of view polygon of each camera, and the overlapping part is the overlapping area of ​​the field of view. Specifically, according to the position, field of view angle and farthest detection distance of the camera, the boundary points of the field of view are calculated, and the field of view polygon of each camera is constructed. This polygon is composed of the camera position and boundary points to form a closed area. The overlapping part of multiple cameras is the overlapping area of ​​the field of view, such as Figure 4 The CEFD area shown is the overlapping area of ​​the fields of view of multiple cameras, side AB is the farthest detection distance of camera 2, and side CD is the farthest detection distance of camera 1. Through the overlapping area extraction mechanism, it is ensured that the target is matched only in the shared area, reducing the subsequent calculation amount.

[0068] In some feasible embodiments, combined with Figure 5 and Figure 6As shown, the step S6: based on the preset occlusion judgment mechanism, calculate the occlusion area of ​​each target to be matched under the corresponding camera perspective, including: S601: use the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system to calculate the angles from the camera to each vertex of the target to be matched; S602: select the two vertices with the largest angle to define the boundary of the occlusion area; S603: draw a ray from the camera position to the two vertices located at the boundary of the occlusion area; S604: the intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched. Specifically, as Figure 6 As shown, taking the vehicle target in the field of view of camera 1 as an example, the vector between the position of camera 1 and the vehicle coordinate point is used to calculate the angle from the camera to the four corner points of the vehicle, and the two vertices with the largest angle are selected. These vertices will be used to define the boundary of the occlusion area. By drawing rays from the camera position to the selected boundary vertices, the intersection of these rays and the horizontal line is calculated to obtain the four vertices of the occlusion area to constitute the occlusion area, as shown in Figure 6 The CMND area shown is the occluded area.

[0069] In some feasible implementations, the step S7: using the iog formula to determine whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle includes:

[0070] The iog formula is:

[0071] When the iog(A,B) value is greater than or equal to the preset threshold, it is judged that the target to be matched is located in the occlusion area under the corresponding camera viewing angle; wherein, the preset threshold can be set to 0.9. It should be noted that there is no restriction on the preset threshold here, and relevant technical personnel can change the threshold based on actual needs during actual application. In the formula, A represents the target area, B represents the occlusion area, and iog(A,B) represents the proportion of the intersection between the target area A and the occlusion area B to the target area A. By combining the occlusion judgment mechanism, the visibility of the target is evaluated to ensure that only unobstructed targets participate in the matching. This strategy reduces unnecessary computational burden while improving matching efficiency and accuracy.

[0072] In some feasible embodiments, combined with Figure 7As shown, step S8: in response to the target to be matched not being located in the occluded area under the corresponding camera perspective, inputting it into the pre-trained neural network model to generate the corresponding feature vector includes: S801: extracting the target to be matched from the original image containing the target object captured by the camera to obtain a target subgraph containing the target to be matched; S802: inputting the target subgraph into the pre-trained neural network model for forward propagation; S803: outputting the feature vector of the target to be matched through the multi-layer convolution and activation function in the pre-trained neural network model; S804: storing the feature vector in the feature database. Specifically, for each target after screening, extract the target to be matched to obtain the target subgraph containing the target to be matched, send it into the deep learning network to extract features, send the target subgraph into the neural network for forward propagation, and the network gradually extracts high-level feature representations through multi-layer convolution and activation functions. At the last layer of the network, the feature vector of each target is obtained. These feature vectors are high-dimensional, information-rich visual representations. The extracted feature values ​​are stored in the feature database together with the feature vectors as the attributes of each target. These feature values ​​and feature vectors will be used in the subsequent matching process. The use of deep neural networks to extract features can more effectively capture the same features of the same target under different perspectives, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching.

[0073] In some feasible implementations, the step S5 further includes: in response to the target object not being located in the overlapping area of ​​the field of view, the target is not included in the items to be matched; the step S8 further includes: in response to the target to be matched being located in the occlusion area under the corresponding camera viewing angle, the target is not included in the items to be matched. That is, targets that do not meet the requirements are not used in subsequent processes.

[0074] In some feasible implementations, the cosine similarity formula in step S9 is: In the formula, C and D represent the feature vectors of the two targets to be matched, respectively, and cos(C, D) represents the possibility that the two targets to be matched are the same target. The higher the similarity, the greater the weight of the edge, indicating that the two targets are more likely to be the same target;

[0075] The intersection-over-union ratio in step S10 is: In the formula, E and F represent the target detection frames of the two targets to be matched, G represents the circumscribed rectangle of E and F, and GIOU is a measure of the overlap degree of the two target detection frames. The GIOU value range is between -1 and 1. The higher the value, the more overlap there is and the greater the possibility of matching.

[0076] The edge weight calculation formula in step S11 is: edg = ω1 × cos (C, D) + ω2 × GIOU; wherein ω1 and ω2 are pre-set weight coefficients. Since the Hungarian algorithm aims to minimize the total cost, the negative value of the edge weight is used as the cost matrix. The Hungarian algorithm is a classic optimization method for solving allocation problems. By constructing a cost matrix, it finds the matching solution with the lowest cost. Among them, the cost matrix is ​​an N×N matrix, where N is the number of targets to be matched. Since the Hungarian algorithm is already very mature in the prior art, and this embodiment does not make any improvements to the Hungarian algorithm itself, the specific calculation process of the Hungarian algorithm will not be described here.

[0077] In some feasible embodiments, combined with Figure 8 As shown, the step S13 also includes: S1301: when performing multi-frame target matching, comparing the target objects contained in the current frame and the previous frame to determine the target state, including: one or a combination of: a new target appears, a target disappears, the target remains unchanged, and the target changes; S1302: when the target state is in the target remains unchanged, using the matching information of the previous frame to directly reference its corresponding cost matrix, based on the cost matrix, using the Hungarian algorithm to obtain an augmented path, and based on the augmented path, obtaining the matching results of multiple targets; S1303: when the target state is in the target change, updating the feature vector of the changed target, repeating steps S9-S12 to obtain the corresponding cost matrix, based on the cost matrix, using the Hungarian algorithm to obtain an augmented path, and based on the augmented path, obtaining the matching results of multiple targets;

[0078] S1304: When the target state is that a new target appears, repeat steps S9-S12 to calculate the edge weight values ​​between the newly appeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmented path, and obtain the matching results of multiple targets based on the augmented path; S1305: When the target state is that a target disappears, repeat steps S9-S12 to calculate the edge weight values ​​between the disappeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmented path, and obtain the matching results of multiple targets based on the augmented path. In order to speed up the matching efficiency, in multi-frame target matching, the matching results of the previous frame are used as prior information, and the cost matrix is ​​only updated for the changed part to avoid repeated calculations.

[0079] The above-mentioned embodiment can more effectively capture the same features of the same target under different perspectives by using a deep neural network to extract features, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching. By introducing an overlapping area extraction mechanism, it is ensured that targets are matched only in the shared area between cameras. In addition, combined with the occlusion judgment mechanism, the visibility of the target is evaluated to ensure that only unobstructed targets participate in the matching. This strategy reduces unnecessary computational burdens while improving matching efficiency and accuracy. By using edge weights to construct a cost matrix, the Hungarian algorithm is further combined to find the lowest-cost matching solution to obtain the optimal matching result for multiple targets.

[0080] Example 2

[0081] See also Fig. 9 , this embodiment provides a multi-objective matching optimization system structure diagram of an intelligent vehicle light self-closed loop system.

[0082] As an example, the system adopts the multi-objective matching optimization method of the intelligent vehicle light self-closed loop system described in Example 1, and the system includes:

[0083] The coordinate conversion module 910 is adapted to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively.

[0084] The field of view overlap area calculation module 920 is adapted to obtain the position of the camera through the coordinates of different cameras in the BEV coordinate system, and obtain the field of view overlap area of ​​multiple cameras based on the position, field of view angle and maximum detection distance of the camera.

[0085] The first judgment module 930 is adapted to judge whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system.

[0086] The occlusion area calculation module 940 is suitable for taking the target object as a target to be matched in response to the target object being located in the overlapping area of ​​the field of view, and the target to be matched includes the target object located in the overlapping area of ​​the field of view under different camera perspectives; based on a preset occlusion judgment mechanism, the occlusion area of ​​each target to be matched under the corresponding camera perspective is calculated.

[0087] The second judgment module 950 is adapted to use the iog formula to judge whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle.

[0088] The feature vector generation module 960 is adapted to input the target to be matched into a pre-trained neural network model to generate a corresponding feature vector in response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle.

[0089] The edge weight calculation module 970 is suitable for respectively calculating the feature similarity between two different to-be-matched targets based on the feature vector using the cosine similarity formula; in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, respectively calculating the overlap degree of the target detection frames of the two different to-be-matched targets using the intersection-and-union ratio; and calculating the edge weight value based on the feature similarity between the two different to-be-matched targets and the overlap degree of the target detection frames of the two different to-be-matched targets.

[0090] The cost matrix construction module 980 is adapted to construct a cost matrix based on the edge weight values.

[0091] The matching result generating module 990 is adapted to obtain an augmented path based on the cost matrix by using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmented path.

[0092] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the first embodiment.

[0093] It is worth mentioning that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0094] Example 3

[0095] See also Fig.10 An embodiment of the present invention further provides an electronic device, comprising: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the multi-objective matching optimization method of the intelligent vehicle light self-closed-loop system provided in Example 1.

[0096] The memory 702 and the processor 701 are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors 701 and the memory 702 together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor 701 is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor 701.

[0097] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management and other control functions. The memory 702 can be used to store data used by the processor 701 when performing operations.

[0098] Example 4

[0099] The embodiment of the present invention further proposes a storage medium, on which a multi-objective matching optimization method for an intelligent vehicle lamp self-closed loop system is stored, and when the multi-objective matching optimization program for the intelligent vehicle lamp self-closed loop system is executed, the steps of the multi-objective matching optimization method for an intelligent vehicle lamp self-closed loop system as described above are implemented. Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0100] The above is only an embodiment of the present invention. The common sense such as the known specific structure and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field of the invention before the application date or priority date, can know all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A multi-objective matching optimization method for an intelligent vehicle light self-closed loop system, characterized in that: The method comprises: S1: Convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively; S2: Get the position of the camera through the coordinates of different cameras in the BEV coordinate system; S3: Obtaining overlapping areas of the fields of view of multiple cameras based on the positions, field of view angles, and maximum detection distances of the cameras; S4: determining whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; S5: In response to the target object being located in the overlapping area of ​​the fields of view, taking the at least one target object as a target to be matched, wherein the target to be matched includes targets of the target object located in the overlapping area of ​​the fields of view under different camera viewing angles; S6: Calculating the occlusion area of ​​each to-be-matched target under the corresponding camera viewing angle based on a preset occlusion judgment mechanism; S7: using the iog formula to determine whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle; S8: In response to the to-be-matched target not being located in the occlusion region under the corresponding camera viewing angle, inputting the to-be-matched target into a pre-trained neural network model to generate a corresponding feature vector; S9: Calculate the feature similarity between two different matching targets using the cosine similarity formula based on the feature vector; S10: in response to the to-be-matched target not being located in the occlusion region under the corresponding camera viewing angle, using an intersection-over-union ratio to respectively calculate the overlap degree of target detection frames of two different to-be-matched targets; S11: Calculate an edge weight value based on the feature similarity between two different to-be-matched targets and the overlap degree of the target detection frames of the two different to-be-matched targets; S12: constructing a cost matrix based on the edge weight values; S13: Obtain an augmented path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmented path.

2. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively includes: S101: converting the camera coordinates into camera coordinates in a vehicle coordinate system by using the camera's extrinsic parameters; S102: Convert the camera coordinates in the vehicle coordinate system to the BEV coordinate system to obtain coordinates of different cameras in the BEV coordinate system; S103: Convert the coordinates of the target object in the vehicle coordinate system to the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system.

3. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S3: obtaining the overlapping area of ​​the fields of view of multiple cameras based on the positions, field of view angles and maximum detection distances of the cameras comprises: S301: Calculating the boundary points of the camera's field of view based on the camera's position, field of view angle, and maximum detection distance; S302: constructing a field of view polygon of each camera based on the boundary points of the camera field of view; S303: Determine an overlapping portion based on the field of view polygon of each camera, where the overlapping portion is a field of view overlapping area.

4. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S6: calculating the occlusion area of ​​each to-be-matched target under the corresponding camera viewing angle based on a preset occlusion judgment mechanism includes: S601: Calculate the angles between the camera and each vertex of the target to be matched using the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system; S602: Select two vertices with the largest angle to define the boundary of the occlusion area; S603: Draw a ray from the camera position to two vertices located at the boundary of the occlusion area; S604: The intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched.

5. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S7: using the iog formula to determine whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle includes: The iog formula is: When the iog(A, B) value is greater than or equal to a preset threshold, it is determined that the target to be matched is located in the occlusion area under the corresponding camera viewing angle; Where A represents the target area, B represents the occluded area, and iog(A,B) represents the ratio of the intersection between the target area A and the occluded area B to the target area A.

6. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S8: in response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting it into a pre-trained neural network model to generate a corresponding feature vector includes: S801: extracting a target to be matched from an original image containing a target object captured by a camera to obtain a target sub-image containing the target to be matched; S802: Input the target subgraph into the pre-trained neural network model for forward propagation; S803: Outputting a feature vector of the target to be matched through multiple layers of convolution and activation functions in a pre-trained neural network model; S804: Store the feature vector in a feature database.

7. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S5 further includes: in response to the target object not being located in the overlapping area of ​​the field of view, the target object is not included in the items to be matched; The step S8 further includes: in response to the target to be matched being located in the occlusion area under the corresponding camera viewing angle, the target is not included in the items to be matched.

8. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The cosine similarity formula in step S9 is: In the formula, C and D represent the feature vectors of two targets to be matched, and cos(C, D) represents the possibility that the two targets to be matched are the same target; The intersection-over-union ratio in step S10 is: In the formula, E and F represent the target detection frames of the two targets to be matched, G represents the circumscribed rectangle of E and F, GIOU is a measure of the overlap degree of the two target detection frames, and the GIOU value range is between -1 and 1; The edge weight calculation formula in step S11 is: edg = ω1×cos(C,D)+ω2×GIOU; Wherein, ω1 and ω2 are preset weight coefficients respectively.

9. The multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S13 further includes: S1301: when performing multi-frame target matching, comparing the target objects included in the current frame with those in the previous frame to determine the target state, including: a new target appears, a target disappears, the target remains unchanged, and the target changes, or a combination thereof; S1302: when the target state is unchanged, directly refer to the corresponding cost matrix using the matching information of the previous frame, obtain an augmented path based on the cost matrix using the Hungarian algorithm, and obtain matching results of multiple targets based on the augmented path; S1303: when the target state is in the target change state, update the feature vector of the changed target, repeat steps S9-S12 to obtain the corresponding cost matrix, obtain the augmenting path based on the cost matrix using the Hungarian algorithm, and obtain the matching results of multiple targets based on the augmenting path; S1304: when a new target appears in the target state, repeat steps S9-S12 to calculate the edge weight values ​​between the newly appeared target object and other target objects to update the cost matrix, obtain an augmenting path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmenting path; S1305: When the target state is that a target disappears, repeat steps S9-S12 to calculate the edge weight values ​​between the disappeared target object and other target objects to update the cost matrix, and use the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtain a matching result of multiple targets based on the augmenting path.

10. A multi-objective matching optimization system for an intelligent vehicle light self-closed loop system, characterized in that: The system adopts the multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to any one of claims 1 to 9, and the system comprises: A coordinate conversion module is used to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively; A field of view overlap area calculation module, adapted to obtain the position of a camera through the coordinates of different cameras in the BEV coordinate system, and obtain the field of view overlap area of ​​multiple cameras based on the position, field of view angle and maximum detection distance of the camera; A first judgment module, adapted to judge whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; The occlusion area calculation module is adapted to respond to the target object being located in the overlapping area of ​​the field of view as a target to be matched, wherein the target to be matched includes the target object located in the overlapping area of ​​the field of view under different camera viewing angles; and calculate the occlusion area of ​​each target to be matched under the corresponding camera viewing angle based on a preset occlusion judgment mechanism; The second judgment module is adapted to use the iog formula to judge whether the to-be-matched target is located in the occlusion area under the corresponding camera viewing angle; A feature vector generation module, adapted to, in response to the target to be matched not being located in the occlusion region under the corresponding camera viewing angle, input the target to be matched into a pre-trained neural network model to generate a feature vector corresponding thereto; The edge weight calculation module is adapted to respectively calculate the feature similarity between two different to-be-matched targets using the cosine similarity formula based on the feature vector; in response to the to-be-matched target not being located in the occlusion region under the corresponding camera viewing angle, respectively calculate the overlap degree of the target detection frames of the two different to-be-matched targets using the intersection-and-union ratio; and calculate the edge weight value based on the feature similarity between the two different to-be-matched targets and the overlap degree of the target detection frames of the two different to-be-matched targets; A cost matrix construction module is adapted to construct a cost matrix based on the edge weight values; The matching result generation module is adapted to obtain an augmented path based on the cost matrix using the Hungarian algorithm, and obtain matching results of multiple targets based on the augmented path.

Images