A multi-objective matching optimization method and system for an intelligent vehicle light self-closed loop system

Abstract

The present invention provides a multi-target matching optimization method and system for an intelligent vehicle light self-closed-loop system, belonging to the field of intelligent vehicle light control technology. The method includes: obtaining the overlapping field of view of multiple cameras based on the camera positions, field of view angles, and maximum detection distances; determining whether a target object is located within the overlapping field of view; if so, treating the target object as a target to be matched; calculating the occlusion area of ​​each target to be matched based on a preset occlusion judgment mechanism; determining whether the target to be matched is located within the occlusion area; if not, inputting the target object into a pre-trained neural network model to generate a corresponding feature vector; calculating the feature similarity and overlap of the target objects to be matched, thereby obtaining edge weights; constructing a cost matrix based on the edge weights; and using the Hungarian algorithm to obtain augmented paths to obtain multi-target matching results. This method optimizes the target matching process, reduces computing resource consumption, and improves matching accuracy and real-time performance.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent vehicle light control, and in particular to a multi-objective matching optimization method and system for an intelligent vehicle light self-closed loop system. Background Art

[0002] With the continuous advancement of automotive technology, intelligent headlight closed-loop systems have become an integral part of modern vehicles. Traditional headlight systems rely primarily on manual operation or simple automatic on / off mechanisms, failing to flexibly respond to changing driving conditions. In contrast, intelligent headlight closed-loop systems, integrating advanced sensor technology and computing platforms, can automatically perceive the surrounding environment and adjust lighting patterns accordingly, improving safety and comfort during nighttime driving. Intelligent headlight closed-loop systems typically utilize multiple cameras to capture information about the vehicle's surroundings from different angles. By matching objects from multiple perspectives, the system ensures comprehensive monitoring of the vehicle's surroundings, reducing blind spots and enabling timely detection of potential hazards such as pedestrians, other vehicles, and obstacles. This multi-perspective information can be cross-validated, reducing false positives and false negatives caused by misjudgment from a single perspective and improving overall system performance.

[0003] To improve the accuracy and robustness of multi-target matching, researchers have developed a variety of matching algorithms and deep learning models based on visual features. Currently, mainstream matching methods include: feature point matching-based algorithms, deep learning-based re-identification (ReID) models, and geometric correction and camera calibration techniques. Feature point matching algorithms rely on traditional image features such as SIFT or ORB, but are prone to failure under changes in lighting and perspective. ReID-based models use convolutional neural networks to extract global and local features of targets, but they are highly dependent on large-scale data annotation.

[0004] Achieving high-precision multi-target matching in multi-camera systems faces numerous technical challenges. For example, perspective differences: the same target captured by different cameras may appear different due to factors such as angle and distance, resulting in inconsistent visual features; occlusion and target loss: targets may be partially obscured from certain perspectives or even temporarily leave a camera's field of view; and high real-time requirements: in intelligent driving scenarios, target matching algorithms must have low latency and high efficiency to support real-time decision-making.

[0005] Therefore, there is an urgent need for an efficient and robust multi-target matching technology that can achieve accurate target association under different camera perspectives. Summary of the Invention

[0006] The purpose of the present invention is to overcome at least one technical problem existing in the prior art and to provide a multi-objective matching optimization method and system for an intelligent vehicle lamp self-closed loop system.

[0007] On the one hand, an embodiment of the present invention provides a multi-target matching optimization method for a self-closed-loop system of an intelligent vehicle lamp, the method comprising: S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively; S2: obtaining the position of the camera through the coordinates of different cameras in the BEV coordinate system; S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and maximum detection distance of the camera; S4: judging whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; S5: in response to the target object being located in the overlapping area of ​​the field of view, taking the at least one target object as a target to be matched, the target to be matched including the target object located in the overlapping area of ​​the field of view under different camera perspectives; S6: calculating the occlusion judgment mechanism of each target to be matched in relation to the camera. The occlusion area under the corresponding camera perspective; S7: Use the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera perspective; S8: In response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, input it into the pre-trained neural network model to generate a corresponding feature vector; S9: Based on the feature vector, the cosine similarity formula is used to respectively calculate the feature similarity between two different targets to be matched; S10: In response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, the intersection and union ratio is used to respectively calculate the degree of overlap of the target detection frames of the two different targets to be matched; S11: The edge weight value is calculated based on the feature similarity between the two different targets to be matched and the degree of overlap of the target detection frames of the two different targets to be matched; S12: Construct a cost matrix based on the edge weight value; S13: Based on the cost matrix, the Hungarian algorithm is used to obtain an augmenting path, and the matching results of multiple targets are obtained based on the augmenting path.

[0008] Furthermore, the step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively includes: S101: converting the camera coordinates into the camera coordinates in the vehicle coordinate system through the external parameters of the camera; S102: converting the camera coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system; S103: converting the target object coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system.

[0009] Furthermore, the step S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and maximum detection distance of the camera includes: S301: calculating the boundary points of the camera field of view based on the position, field of view angle and maximum detection distance of the camera; S302: constructing the field of view polygon of each camera based on the boundary points of the camera field of view; S303: determining the overlapping part based on the field of view polygon of each camera, and the overlapping part is the field of view overlapping area.

[0010] Furthermore, the step S6: calculating the occlusion area of ​​each target to be matched under the corresponding camera perspective based on a preset occlusion judgment mechanism includes: S601: using the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system to calculate the angles from the camera to the vertices of the target to be matched; S602: selecting the two vertices with the largest angle to define the boundary of the occlusion area; S603: drawing a ray through the camera position to the two vertices located at the boundary of the occlusion area; S604: the intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched.

[0011] Furthermore, the step S7: using the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle includes:

[0012] The iog formula is:

[0013] When the iog(A, B) value is greater than or equal to a preset threshold, it is determined that the target to be matched is located in the occlusion area under the corresponding camera perspective;

[0014] Where A represents the target area, B represents the occluded area, and iog(A,B) represents the ratio of the intersection between the target area A and the occluded area B to the target area A.

[0015] Furthermore, the step S8: in response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting it into a pre-trained neural network model to generate a corresponding feature vector includes: S801: extracting the target to be matched from the original image containing the target object captured by the camera to obtain a target subgraph containing the target to be matched; S802: inputting the target subgraph into the pre-trained neural network model for forward propagation; S803: outputting the feature vector of the target to be matched through the multi-layer convolution and activation function in the pre-trained neural network model; S804: storing the feature vector in a feature database.

[0016] Furthermore, after step S5, the following also includes: in response to the target object not being located in the overlapping area of ​​the field of view, the target is not included in the items to be matched; after step S8, the following also includes: in response to the target to be matched being located in the occlusion area under the corresponding camera perspective, the target is not included in the items to be matched.

[0017] Furthermore, the cosine similarity formula in step S9 is:

[0018] Where C and D represent the feature vectors of the two targets to be matched, and cos(C, D) represents the possibility that the two targets to be matched are the same target.

[0019] The intersection-over-union ratio in step S10 is:

[0020] Where E and F represent the target detection boxes of the two targets to be matched, G represents the circumscribed rectangle of E and F, and GIOU is a measure of the degree of overlap between the two target detection boxes. The GIOU value range is between -1 and 1.

[0021] The edge weight calculation formula in step S11 is: edg = ω1×cos(C, D)+ω2×GIOU;

[0022] Where ω1 and ω2 are preset weight coefficients.

[0023] Furthermore, the step S13 further includes: S1301: when performing multi-frame target matching, comparing the target objects included in the current frame with those included in the previous frame to determine the target state, including one or a combination of: a new target appears, a target disappears, the target remains unchanged, and the target changes; S1302: when the target state is in the target unchanged state, using the matching information of the previous frame to directly reference its corresponding cost matrix, using the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtaining a multi-target matching result based on the augmenting path;

[0024] S1303: When the target state is in a target change, update the feature vector of the changed target, repeat steps S9-S12 to obtain the corresponding cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path; S1304: When the target state is in a new target appearing, repeat steps S9-S12 to calculate the edge weight value between the newly appeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path; S1305: When the target state is in a target disappearance, repeat steps S9-S12 to calculate the edge weight value between the disappeared target object and other target objects to update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain the augmenting path, and obtain the matching results of multiple targets based on the augmenting path.

[0025] In a second aspect, an embodiment of the present invention provides a multi-objective matching optimization system for a self-closed-loop system of intelligent headlights, the system adopts the multi-objective matching optimization method for the self-closed-loop system of intelligent headlights, and the system includes: a coordinate conversion module, adapted to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively; a field of view overlapping area calculation module, adapted to obtain the position of the camera through the coordinates of different cameras in the BEV coordinate system, and obtain the field of view overlapping area of ​​multiple cameras based on the position, field of view angle and maximum detection distance of the camera; a first judgment module, adapted to judge whether the target object is located in the field of view overlapping area based on the coordinates of the different target objects in the BEV coordinate system; an occlusion area calculation module, adapted to respond to the target object being located in the field of view overlapping area as a target to be matched, the target to be matched including the target object located in the field of view overlapping area under different camera perspectives; the occlusion judgment mechanism is used to calculate the target object based on the preset occlusion judgment mechanism. The occlusion area of ​​each target to be matched under the corresponding camera perspective; the second judgment module is suitable for using the iog formula to judge whether the target to be matched is located in the occlusion area under the corresponding camera perspective; the feature vector generation module is suitable for inputting the target to be matched into a pre-trained neural network model to generate a corresponding feature vector in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective; the edge weight calculation module is suitable for using the cosine similarity formula to respectively calculate the feature similarity between two different targets to be matched based on the feature vector; in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective, the intersection and union ratio is used to respectively calculate the degree of overlap of the target detection frames of the two different targets to be matched; the edge weight value is calculated based on the feature similarity between the two different targets to be matched and the degree of overlap of the target detection frames of the two different targets to be matched; the cost matrix construction module is suitable for constructing a cost matrix based on the edge weight value; the matching result generation module is suitable for obtaining an augmenting path based on the cost matrix using the Hungarian algorithm, and obtaining the matching result of multiple targets based on the augmenting path.

[0026] In a third aspect, an embodiment of the present invention further provides an electronic device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the multi-objective matching optimization method of the above-mentioned intelligent vehicle light self-closed-loop system.

[0027] In a fourth aspect, an embodiment of the present invention further provides a readable storage medium, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the above-mentioned multi-objective matching optimization method of the intelligent vehicle light self-closed-loop system.

[0028] The beneficial effects of the present invention are:

[0029] The present invention provides a multi-objective matching optimization method and system for an intelligent vehicle lamp self-closed-loop system, which has the following advantages over the prior art:

[0030] (1) This application abandons the traditional image feature extraction method and adopts a deep neural network feature extraction method, which can more effectively capture the same features of the same target under different perspectives, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching.

[0031] (2) This application introduces an overlapping region extraction mechanism to ensure that targets are matched only within the shared region between cameras. Furthermore, an occlusion determination mechanism is incorporated to evaluate target visibility, ensuring that only unoccluded targets are matched. This strategy reduces unnecessary computational burden while improving matching efficiency and accuracy.

[0032] (3) This application uses the Hungarian matching algorithm, a classic optimization method for solving allocation problems. It constructs a cost matrix to find the matching solution with the lowest cost. To speed up matching efficiency, in multi-frame target matching, the matching results of the previous frame are used as prior information, and the cost matrix is ​​only updated for the changed parts to avoid repeated calculations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings and examples.

[0034] Figure 1 This is a flow chart of a multi-objective matching optimization method for an intelligent vehicle lamp self-closed-loop system provided in Example 1 of the present invention.

[0035] Figure 2 This is a flow chart of the method involved in step S1 provided in Example 1 of the present invention.

[0036] Figure 3 This is a flow chart of the method involved in step S3 provided in Example 1 of the present invention.

[0037] Figure 4 This is a schematic diagram of overlapping areas of fields of view of different cameras provided in Example 1 of the present invention.

[0038] Figure 5 This is a flow chart of the method involved in step S6 provided in Example 1 of the present invention.

[0039] Figure 6 This is a schematic diagram of an occlusion area under a camera field of view provided by Example 1 of the present invention.

[0040] Figure 7 This is a flow chart of the method involved in step S8 provided in Example 1 of the present invention.

[0041] Figure 8 This is a flow chart of the method involved in step S13 provided in Example 1 of the present invention.

[0042] Figure 9 This is a structural diagram of a multi-objective matching optimization system of an intelligent vehicle lamp self-closed loop system provided by Example 2 of the present invention.

[0043] Figure 10 This is a partial block diagram of an electronic device provided by Example 3 of the present invention. DETAILED DESCRIPTION

[0044] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the various operations as sequential processes, many of the operations therein can be implemented in parallel, concurrently, or simultaneously. In addition, the order of the various operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0045] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0046] The present invention will now be described in detail with reference to the accompanying drawings. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore only shows the components related to the present invention.

[0047] Example 1

[0048] For ease of understanding, the following is an overall description of the inventive concept before describing the embodiments of the present invention in detail:

[0049] This application provides a multi-target matching optimization method and system for a closed-loop intelligent headlight system. This method aims to address target matching challenges in intelligent headlights caused by perspective differences, overlapping areas, occlusions, and inefficiencies. By acquiring the intrinsic and extrinsic parameters of different cameras, the coordinates of each camera are converted to the vehicle coordinate system. The vehicle coordinate points detected by each camera are further uniformly converted to coordinates under the bird's-eye view (BEV) to achieve alignment of the different camera data. An overlapping region extraction mechanism is employed to extract the intersection of different perspectives. Targets detected by the wide-angle camera within the intersection are then filtered out. The occlusion region of each target detected by the narrow-angle camera under the BEV is calculated. An occlusion determination mechanism is then used to filter out targets detected by the wide-angle camera that are not within the occlusion region, ensuring that only visible targets are matched, thereby improving matching accuracy. To obtain features of the same target from different perspectives, a deep learning network structure is used to extract features from the filtered targets and all targets under the narrow-angle view. The extracted features are used to construct a cost matrix. Optimal matching is achieved by minimizing the total cost. During multi-frame matching, the cost matrix is ​​dynamically updated, only updating the changed parts, thereby improving computational efficiency.

[0050] The specific implementation is as follows:

[0051] like Figure 1 FIG. 1 is a flow chart of a multi-objective matching optimization method for an intelligent vehicle light self-closed-loop system provided in this embodiment.

[0052] As an example, the method includes:

[0053] S1: Convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively.

[0054] S2: Get the camera position through the coordinates of different cameras in the BEV coordinate system.

[0055] S3: Obtaining overlapping areas of the fields of view of multiple cameras based on the positions, field of view angles, and maximum detection distances of the cameras.

[0056] S4: Determine whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system.

[0057] S5: In response to the target object being located in the overlapping field of view area, the at least one target object is used as a target to be matched, where the target to be matched includes targets located in the overlapping field of view area under different camera viewing angles.

[0058] S6: Calculate the occlusion area of ​​each to-be-matched target under the corresponding camera viewing angle based on a preset occlusion judgment mechanism.

[0059] S7: Use the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle.

[0060] S8: In response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting the target to be matched into a pre-trained neural network model to generate a corresponding feature vector.

[0061] S9: Based on the feature vector, the cosine similarity formula is used to calculate the feature similarity between two different matching targets.

[0062] S10: In response to the target to be matched not being located in the occlusion region under the corresponding camera viewing angle, using intersection-over-union (IoU) to respectively calculate the degree of overlap of target detection frames of two different targets to be matched.

[0063] S11: Calculate an edge weight value based on the feature similarity between two different to-be-matched targets and the degree of overlap of the target detection frames of the two different to-be-matched targets.

[0064] S12: Constructing a cost matrix based on the edge weight values.

[0065] S13: Obtain an augmenting path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmenting path.

[0066] In some feasible embodiments, combined with Figure 2 As shown, the step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system and the coordinates of different target objects in the BEV coordinate system respectively includes: S101: converting the camera coordinates into the camera coordinates in the vehicle coordinate system through the camera's external parameters; S102: converting the camera coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of different cameras in the BEV coordinate system; S103: converting the target object coordinates in the vehicle coordinate system into the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system. Specifically, the camera coordinates are converted to the coordinates under the BEV (Bird's Eye View) perspective: the camera external parameters include the rotation matrix (R) and the translation vector (T), and the camera coordinates (X c ,Y c ,Z c ) is converted to the vehicle coordinate system (X cv ,Y cv ,Z cv), the conversion formula is (X cv ,Y cv ,Z cv )=R·(X c ,Y c ,Z c )+T, convert the camera coordinates in the vehicle coordinate system to the BEV perspective, so (X cvb ,Y cvb ,Z cvb )=(X cv ,Y cv ,0). The coordinates of the target object are converted to the coordinates under the BEV perspective: the coordinates of the target object in the vehicle coordinate system (X ov ,Y ov ,Z ov ) is converted to the BEV coordinate system, the coordinates of the target object can be expressed as (X bev ,Y bev ,Z bev )=(X ov ,Y ov ,0). By obtaining the intrinsic and extrinsic parameters of different cameras, the coordinates of each camera are converted into the coordinates in the vehicle coordinate system, and the vehicle coordinate points detected by each camera are further uniformly converted into coordinates under the bird's-eye view (BEV) to achieve alignment of different camera data.

[0067] In some feasible embodiments, combined with Figure 3 and Figure 4 As shown, the step S3: obtaining the overlapping area of ​​the field of view of multiple cameras based on the position, field of view angle and maximum detection distance of the camera includes: S301: calculating the boundary points of the camera field of view based on the position, field of view angle and maximum detection distance of the camera; S302: constructing the field of view polygon of each camera based on the boundary points of the camera field of view; S303: determining the overlapping part based on the field of view polygon of each camera, and the overlapping part is the field of view overlapping area. Specifically, according to the position, field of view angle and maximum detection distance of the camera, the boundary points of the field of view are calculated, and the field of view polygon of each camera is constructed. This polygon is composed of the camera position and boundary points to form a closed area. The overlapping part of multiple cameras is the area of ​​overlapping field of view, as shown in FIG. Figure 4 The CEFD region shown is the overlapping field of view of multiple cameras. Side AB is the maximum detection distance of camera 2, and side CD is the maximum detection distance of camera 1. The overlapping region extraction mechanism ensures that targets are matched only within the shared area, reducing subsequent computational effort.

[0068] In some feasible embodiments, combined with Figure 5 and Figure 6As shown, the step S6: calculating the occlusion area of ​​each target to be matched under the corresponding camera perspective based on the preset occlusion judgment mechanism includes: S601: using the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system to calculate the angles between the camera and each vertex of the target to be matched; S602: selecting the two vertices with the largest angle to define the boundary of the occlusion area; S603: drawing a ray from the camera position to the two vertices located at the boundary of the occlusion area; S604: the intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched. Specifically, as Figure 6 As shown, taking the vehicle target in the field of view of camera 1 as an example, the vector between the position of camera 1 and the vehicle coordinate point is used to calculate the angle from the camera to the four corner points of the vehicle, and the two vertices with the largest angle are selected. These vertices will be used to define the boundary of the occlusion area. By drawing rays from the camera position to the selected boundary vertices, the intersection of these rays and the horizontal line is calculated to obtain the four vertices of the occlusion area, which constitute the occlusion area, as shown in Figure 6 The CMND area shown is the occlusion area.

[0069] In some feasible implementations, the step S7: using the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle includes:

[0070] The iog formula is:

[0071] When the iog(A,B) value is greater than or equal to a preset threshold, it is determined that the target to be matched is located in the occlusion area under the corresponding camera perspective; wherein, the preset threshold can be set to 0.9. It should be noted that there is no restriction on the preset threshold here, and relevant technical personnel can change the threshold based on actual needs during actual application. In the formula, A represents the target area, B represents the occlusion area, and iog(A,B) represents the proportion of the intersection between the target area A and the occlusion area B in the target area A. By combining the occlusion judgment mechanism, the visibility of the target is evaluated to ensure that only unobstructed targets participate in the matching. This strategy reduces unnecessary computational burden while improving matching efficiency and accuracy.

[0072] In some feasible embodiments, combined with Figure 7As shown, step S8: in response to the target to be matched not being located in the occluded area under the corresponding camera perspective, inputting it into a pre-trained neural network model to generate a corresponding feature vector includes: S801: extracting the target to be matched from the original image containing the target object captured by the camera to obtain a target subgraph containing the target to be matched; S802: inputting the target subgraph into the pre-trained neural network model for forward propagation; S803: outputting a feature vector of the target to be matched through multiple layers of convolution and activation functions in the pre-trained neural network model; S804: storing the feature vector in a feature database. Specifically, for each target after screening, the target to be matched is extracted to obtain a target subgraph containing the target to be matched, which is then fed into a deep learning network for feature extraction. The target subgraph is then fed into the neural network for forward propagation. The network gradually extracts high-level feature representations through multiple layers of convolution and activation functions, and at the last layer of the network, a feature vector for each target is obtained. These feature vectors are high-dimensional, information-rich visual representations. The extracted feature values ​​are used as attributes of each target and are stored in the feature database together with the feature vectors. These feature values ​​and feature vectors will be used in the subsequent matching process. The use of deep neural networks to extract features can more effectively capture the same features of the same target under different perspectives, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching.

[0073] In some feasible implementations, after step S5, the following step further includes: in response to the target object not being located in the overlapping field of view area, excluding the target from the list of items to be matched; and after step S8, the following step further includes: in response to the target object being located in the occlusion area under the corresponding camera view angle, excluding the target from the list of items to be matched. In other words, targets that do not meet the requirements are not used in subsequent processes.

[0074] In some feasible implementations, the cosine similarity formula in step S9 is: In the formula, C and D represent the feature vectors of the two targets to be matched, respectively. Cos(C, D) represents the possibility that the two targets to be matched are the same target. The higher the similarity, the greater the weight of the edge, indicating that the two targets are more likely to be the same target.

[0075] The intersection-over-union ratio in step S10 is: Where E and F represent the target detection boxes of the two targets to be matched, G represents the circumscribed rectangle of E and F, and GIOU is a measure of the degree of overlap between the two target detection boxes. The GIOU value range is between -1 and 1. The higher the value, the more overlap there is and the greater the possibility of matching.

[0076] The edge weight calculation formula in step S11 is: edg = ω1 × cos (C, D) + ω2 × GIOU; wherein ω1 and ω2 are pre-set weight coefficients. Since the Hungarian algorithm aims to minimize the total cost, the negative value of the edge weight is used as the cost matrix. The Hungarian algorithm is a classic optimization method for solving allocation problems. It constructs a cost matrix to find the matching solution with the lowest cost. Among them, the cost matrix is ​​an N×N matrix, where N is the number of targets to be matched. Since the Hungarian algorithm is very mature in the prior art and this embodiment does not make any improvements to the Hungarian algorithm itself, the specific calculation process of the Hungarian algorithm will not be described here.

[0077] In some feasible embodiments, combined with Figure 8 As shown, the step S13 also includes: S1301: when performing multi-frame target matching, comparing the target objects contained in the current frame with those contained in the previous frame to determine the target state, including: one or a combination of: a new target appears, a target disappears, the target remains unchanged, and the target changes; S1302: when the target state is in the target unchanged state, using the matching information of the previous frame to directly reference its corresponding cost matrix, using the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtaining a matching result of multiple targets based on the augmenting path; S1303: when the target state is in the target change state, updating the feature vector of the changed target, repeating steps S9-S12 to obtain the corresponding cost matrix, using the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtaining a matching result of multiple targets based on the augmenting path;

[0078] S1304: When a new target appears, repeat steps S9-S12 to calculate the edge weights between the newly appeared target and other target objects, update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtain the multi-target matching results based on the augmenting path. S1305: When a target disappears, repeat steps S9-S12 to calculate the edge weights between the disappeared target and other target objects, update the cost matrix, use the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtain the multi-target matching results based on the augmenting path. To speed up matching efficiency, in multi-frame target matching, the matching results of the previous frame are used as prior information, and the cost matrix is ​​only updated for the changed parts to avoid repeated calculations.

[0079] The above-mentioned embodiment uses a deep neural network to extract features, which can more effectively capture the same features of the same target under different perspectives, significantly reduce the matching difficulty caused by perspective differences, and improve the accuracy of target matching. By introducing an overlapping area extraction mechanism, it is ensured that targets are matched only in the shared area between cameras. In addition, combined with the occlusion judgment mechanism, the visibility of the target is evaluated to ensure that only unobstructed targets participate in the matching. This strategy reduces unnecessary computational burden while improving matching efficiency and accuracy. By using edge weights to construct a cost matrix, the Hungarian algorithm is further combined to find the lowest-cost matching solution to obtain the optimal matching result for multiple targets.

[0080] Example 2

[0081] See also Figure 9 , this embodiment provides a multi-objective matching optimization system structure diagram of an intelligent vehicle light self-closed loop system.

[0082] As an example, the system adopts the multi-objective matching optimization method of the intelligent vehicle light self-closed loop system described in Example 1, and the system includes:

[0083] The coordinate conversion module 910 is adapted to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively.

[0084] The field of view overlap area calculation module 920 is adapted to obtain the position of the camera through the coordinates of different cameras in the BEV coordinate system, and obtain the field of view overlap area of ​​multiple cameras based on the position, field of view angle and maximum detection distance of the camera.

[0085] The first judgment module 930 is adapted to judge whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system.

[0086] The occlusion area calculation module 940 is adapted to respond to the target object being located within the overlapping area of ​​the field of view as a target to be matched, wherein the target to be matched includes the target object located within the overlapping area of ​​the field of view under different camera perspectives; and calculate the occlusion area of ​​each target to be matched under the corresponding camera perspective based on a preset occlusion judgment mechanism.

[0087] The second judgment module 950 is adapted to use the iog formula to judge whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle.

[0088] The feature vector generation module 960 is adapted to input the target to be matched into a pre-trained neural network model to generate a corresponding feature vector in response to the target to be matched not being located in the occlusion area under the corresponding camera perspective.

[0089] The edge weight calculation module 970 is adapted to calculate the feature similarity between two different to-be-matched targets respectively using the cosine similarity formula based on the feature vector; in response to the to-be-matched target not being located in the occlusion area under the corresponding camera perspective, the intersection-over-union ratio is used to calculate the degree of overlap of the target detection frames of the two different to-be-matched targets respectively; and the edge weight value is calculated based on the feature similarity between the two different to-be-matched targets and the degree of overlap of the target detection frames of the two different to-be-matched targets.

[0090] The cost matrix construction module 980 is adapted to construct a cost matrix based on the edge weight values.

[0091] The matching result generating module 990 is adapted to obtain an augmenting path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmenting path.

[0092] It is not difficult to find that this embodiment is a system embodiment corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0093] It is worth noting that all modules involved in this embodiment are logical units. In actual applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovations of this invention, this embodiment does not include units that are not closely related to solving the technical problems proposed by this invention. However, this does not mean that other units do not exist in this embodiment.

[0094] Example 3

[0095] See also Figure 10 An embodiment of the present invention further provides an electronic device, comprising: a memory and a processor; the memory stores at least one program instruction; the processor implements the multi-objective matching optimization method of the intelligent vehicle light self-closed-loop system provided in Example 1 by loading and executing the at least one program instruction.

[0096] The memory 702 and processor 701 are connected using a bus. The bus can include any number of interconnected buses and bridges, connecting various circuits of one or more processors 701 and memory 702. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and, therefore, are not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor 701 is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor 701.

[0097] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 702 can be used to store data used by the processor 701 when performing operations.

[0098] Example 4

[0099] An embodiment of the present invention further provides a storage medium storing a multi-objective matching optimization method for an intelligent vehicle light self-closed-loop system. When executed, the multi-objective matching optimization program for the intelligent vehicle light self-closed-loop system implements the steps of the multi-objective matching optimization method for the intelligent vehicle light self-closed-loop system described above. Because this storage medium incorporates all the technical solutions of all the aforementioned embodiments, it possesses at least all the beneficial effects of the technical solutions of the aforementioned embodiments, and therefore will not be further elaborated upon here.

[0100] The above is only an embodiment of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the field are aware of all common technical knowledge in the technical field of the invention before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A multi-objective matching optimization method for an intelligent vehicle light self-closed loop system, characterized in that: The method comprises: S1: Convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively; S2: Get the camera position through the coordinates of different cameras in the BEV coordinate system; S3: Obtaining overlapping areas of the fields of view of multiple cameras based on the positions, field of view angles, and maximum detection distances of the cameras; S4: determining whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; S5: In response to the target object being located in the overlapping field of view area, at least one target object is selected as a target to be matched, where the target to be matched includes targets of the target object located in the overlapping field of view area under different camera perspectives; S6: Calculate the occlusion area of ​​each target to be matched under the corresponding camera perspective based on the preset occlusion judgment mechanism; S7: Using the iog formula to determine whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle, the iog formula is: ; When the When the value is greater than or equal to the preset threshold, it is determined that the target to be matched is located in the occlusion area under the corresponding camera perspective; In the formula, A represents the target area, B represents the occluded area, Indicates the proportion of the intersection between the target area A and the occluded area B to the target area A; S8: In response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting the target to be matched into a pre-trained neural network model to generate a corresponding feature vector; S9: Calculate the feature similarity between two different matching targets using the cosine similarity formula based on the feature vector; S10: In response to the target to be matched not being located in the occlusion region under the corresponding camera perspective, using an intersection-over-union ratio to respectively calculate the degree of overlap of target detection frames of two different targets to be matched; S11: Calculating an edge weight value based on the feature similarity between two different to-be-matched targets and the degree of overlap of the target detection frames of the two different to-be-matched targets; S12: constructing a cost matrix based on the edge weight values; S13: Obtain an augmenting path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmenting path.

2. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S1: converting the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively includes: S101: converting the camera coordinates into camera coordinates in a vehicle coordinate system using the camera's extrinsic parameters; S102: Convert the camera coordinates in the vehicle coordinate system to the BEV coordinate system to obtain coordinates of different cameras in the BEV coordinate system; S103: Convert the coordinates of the target object in the vehicle coordinate system to the BEV coordinate system to obtain the coordinates of the target object in the BEV coordinate system.

3. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S3 of obtaining the overlapping area of ​​the fields of view of the multiple cameras based on the positions, field of view angles and maximum detection distances of the cameras includes: S301: Calculating the boundary points of the camera's field of view based on the camera's position, field of view angle, and maximum detection distance; S302: Constructing a field of view polygon for each camera based on the boundary points of the camera field of view; S303: Determine an overlapping portion based on the field of view polygons of each camera, where the overlapping portion is a field of view overlapping area.

4. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S6: calculating the occlusion area of ​​each to-be-matched target under the corresponding camera viewing angle based on a preset occlusion judgment mechanism includes: S601: Calculate the angles between the cameras and each vertex of the target to be matched using the vectors between the coordinates of different cameras in the BEV coordinate system and the coordinates of the target to be matched in the BEV coordinate system; S602: Select two vertices with the largest angle to define the boundary of the occlusion area; S603: Draw a ray from the camera position to two vertices located at the boundary of the occlusion area; S604: The intersection of the ray and the horizontal line is the occlusion area corresponding to the target to be matched.

5. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S8: in response to the target to be matched not being located in the occlusion area under the corresponding camera viewing angle, inputting the target to be matched into a pre-trained neural network model to generate a corresponding feature vector includes: S801: extracting a target to be matched from an original image containing a target object captured by a camera to obtain a target sub-image containing the target to be matched; S802: Inputting the target subgraph into the pre-trained neural network model for forward propagation; S803: Outputting a feature vector of the target to be matched through multiple layers of convolution and activation functions in a pre-trained neural network model; S804: Store the feature vector in a feature database.

6. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: After step S5, the further step includes: in response to the target object not being located in the overlapping area of ​​the field of view, not including the target object in the to-be-matched items; The step S8 further includes: in response to the target to be matched being located in the occlusion area under the corresponding camera viewing angle, the target is not included in the items to be matched.

7. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The cosine similarity formula in step S9 is: ; In the formula, C and D represent the feature vectors of two targets to be matched, It is expressed as the possibility that the two targets to be matched are the same target; The intersection-over-union ratio in step S10 is: ; Where E and F represent the target detection boxes of the two targets to be matched, and G represents the circumscribed rectangle of E and F. To measure the degree of overlap between two target detection frames, The value range is between -1 and 1; The edge weight calculation formula in step S11 is: ; Where, and are pre-set weight coefficients respectively.

8. The multi-objective matching optimization method for the intelligent vehicle light self-closed loop system according to claim 1, characterized in that: The step S13 further includes: S1301: When performing multi-frame target matching, comparing the target objects included in the current frame with those in the previous frame to determine the target state, including one or a combination of: a new target appears, a target disappears, the target remains unchanged, and the target changes; S1302: When the target state is unchanged, directly referencing the corresponding cost matrix using the matching information of the previous frame, obtaining an augmenting path based on the cost matrix using the Hungarian algorithm, and obtaining a matching result of multiple targets based on the augmenting path; S1303: When the target state is in target change, update the feature vector of the changed target, repeat steps S9-S12 to obtain the corresponding cost matrix, use the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtain a multi-target matching result based on the augmenting path; S1304: When a new target appears in the target state, repeat steps S9-S12 to calculate the edge weights between the newly appeared target object and other target objects to update the cost matrix, obtain an augmenting path based on the cost matrix using the Hungarian algorithm, and obtain a multi-target matching result based on the augmenting path; S1305: When the target state is that a target disappears, repeat steps S9-S12 to calculate the edge weight values ​​between the disappeared target object and other target objects to update the cost matrix, and use the Hungarian algorithm based on the cost matrix to obtain an augmenting path, and obtain a multi-target matching result based on the augmenting path.

9. A multi-objective matching optimization system for an intelligent vehicle light self-closed loop system, characterized in that: The system adopts the multi-objective matching optimization method of the intelligent vehicle light self-closed loop system according to any one of claims 1 to 8, and the system includes: A coordinate conversion module is used to convert the camera coordinate systems of different cameras and the target object coordinate systems of different target objects into the BEV coordinate system to obtain the coordinates of different cameras and the coordinates of different target objects in the BEV coordinate system respectively; A field of view overlap area calculation module is adapted to obtain the positions of the cameras through the coordinates of different cameras in the BEV coordinate system, and to obtain the field of view overlap area of ​​the multiple cameras based on the positions, field of view angles, and maximum detection distances of the cameras; A first judgment module is adapted to judge whether the target object is located in the overlapping area of ​​the field of view based on the coordinates of the different target objects in the BEV coordinate system; an occlusion region calculation module, adapted to, in response to the target object being located within the overlapping field of view region, use the target object as a target to be matched, the target object to be matched comprising targets of the target object located within the overlapping field of view region under different camera viewing angles; and calculate the occlusion region of each target to be matched under the corresponding camera viewing angle based on a preset occlusion judgment mechanism; The second judgment module is adapted to use the iog formula to judge whether the target to be matched is located in the occlusion area under the corresponding camera viewing angle; a feature vector generating module adapted to input the target to be matched into a pre-trained neural network model to generate a feature vector corresponding thereto in response to the target to be matched not being located within the occlusion area under the corresponding camera viewing angle; an edge weight calculation module adapted to calculate, based on feature vectors, a feature similarity between two different to-be-matched targets using a cosine similarity formula; in response to the to-be-matched targets not being located within an occlusion region under the corresponding camera viewing angle, respectively calculate a degree of overlap of target detection frames of the two different to-be-matched targets using an intersection-over-union ratio; and calculate an edge weight value based on the feature similarity between the two different to-be-matched targets and the degree of overlap of the target detection frames of the two different to-be-matched targets; A cost matrix construction module is adapted to construct a cost matrix based on the edge weight values; The matching result generation module is adapted to obtain an augmented path based on the cost matrix using the Hungarian algorithm, and obtain a matching result of multiple targets based on the augmented path.

Images