Target detection system and method based on lidar and vision-inspired learning

Abstract

The present invention relates to a target detection system using laser radar and visual heuristic learning, belonging to the field of environmental perception technology for autonomous unmanned systems. The system comprises a visual target detection module for real-time acquisition of visual data, target detection using a visual model 1, and sending the detection results as visual pseudo-label data to a heuristic learning module; a laser radar target detection module for real-time acquisition of laser radar data, target detection using a laser radar model 1, and sending the detection results as laser radar pseudo-label data to the heuristic learning module; and a heuristic learning module for evaluating pseudo-label information output by visual and laser radar targets using an evaluator. Based on the evaluation results, the system trains a visual model 2 and a laser radar model 2 using the pseudo-label information. The trained visual model 2 and laser radar model 2 are used to update the visual model 1 and the laser radar model 1, respectively. A method is also included.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous unmanned system environment perception, and relates to a target detection system and method using laser radar and vision-inspired learning. Background Art

[0002] Environmental perception is a key technology in autonomous unmanned systems (including driverless vehicles and intelligent robots), and forms the foundation for decision-making, planning, and control. LiDAR and cameras, as key environmental perception sensors, play a crucial role in target detection in these systems.

[0003] A Chinese patent application titled "3D Object Detection Method Based on Fusion of Point Cloud Data and Multi-view Image Data" (Application Number: CN202211250323.7) discloses a 3D object detection method based on the fusion of point cloud data and multi-view image data. The method comprises: acquiring point cloud data and multi-view image data from the Nuscenes dataset; generating point cloud BEV features and image BEV features; inputting the image and point cloud fusion network to generate image and point cloud fusion features; obtaining an initial predicted bounding box; and generating a final target bounding box. This method requires labeled data for learning.

[0004] The Chinese patent application, "3D Object Detection Model Training and 3D Object Detection Method and Apparatus" (Application Number: CN202211259102.6), discloses a 3D object detection model training and 3D object detection method. The method involves obtaining detection box annotation information for a data pair; inputting the point cloud data from the data pair into a teacher network and the binocular image into a student network; calculating a first loss function value; calculating a second loss function value; and adjusting the student network parameters based on the loss function values ​​to obtain a 3D object detection model. This method requires manual annotation of the detection box information. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to utilize the differences in detection capabilities of lidar and visual sensors in different scenarios or features to construct a heuristic learning method to improve the detection capabilities of the two sensor detection models, realize unsupervised learning of detection models, reduce the workload of manual data labeling, and improve the efficiency of detection model training.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In one aspect, the present invention provides a target detection system using laser radar and visual heuristic learning, comprising a visual target detection module, a laser radar target detection module, and a heuristic learning module;

[0008] The visual target detection module is used to collect visual data in real time, perform target detection through the visual model-1, and send the detection results as visual pseudo-label data to the heuristic learning module;

[0009] The LiDAR target detection module is used to collect LiDAR data in real time, perform target detection through LiDAR Model-1, and send the detection results as LiDAR pseudo-label data to the heuristic learning module;

[0010] The heuristic learning module uses an evaluator to evaluate the pseudo-label information output by the visual and lidar targets. Based on the evaluation results, the pseudo-label information is used to train the visual model-2 and the lidar model-2. The trained visual model-2 and lidar model-2 are used to update the visual model-1 and lidar model-1 respectively.

[0011] In another aspect, the present invention provides a target detection method using laser radar and vision-inspired learning, comprising the following steps:

[0012] S1: Real-time visual data acquisition: At time k, visual data is collected and target detection is performed using the visual model-1 to obtain visual target detection results and visual pseudo-label data;

[0013] S2: LiDAR real-time data acquisition: At time k, LiDAR data is collected and target detection is performed using LiDAR Model-1 to obtain LiDAR target detection results and LiDAR pseudo-label data.

[0014] S3: Use the evaluator to evaluate the pseudo-label data output by the visual and lidar targets, associate the pseudo-label data with the target data, and obtain the lidar and visual pseudo-label target association matrix. Then, make a decision based on the target association matrix to obtain the lidar and visual target detection consistency vector.

[0015] S4: Train based on the consistency vector of lidar and visual target detection to obtain a trained updated detection model, including visual model-2 and lidar model-2, which are used to update visual model-1 and lidar model-1 respectively.

[0016] Furthermore, step S3 uses an evaluator to evaluate the pseudo-label data output by the visual and lidar targets, specifically including the following steps:

[0017] S31: Target association: Output pseudo-label data T for visual and lidar targets K 、T L Establish the target feature vector x respectively i , calculate the target similarity s tar , calculate the target association matrix A based on the target similarity ij ;

[0018] S32: Calculate the target detection consistency vector: According to the association matrix A ij , T K 、T L The targets in are put into the set P Y K 、P N K 、P Y L 、P N L In the judgment of P N K 、P N L Is it Calculate the target detection consistency vector y i .

[0019] Furthermore, the step S31 specifically includes the following steps:

[0020] S311: Based on the laser radar and visual pseudo-label information, the feature vectors of the target are established respectively:

[0021] x i =[p,s] T

[0022] The pose feature p = [x, y, α] represents the x position, y position and target orientation of the target center point, and the shape feature s = [l, w] represents the length and width of the target detection box;

[0023] Establish target state model and measurement model:

[0024] x k =Bu k +w k

[0025] z k =Hx k +v k

[0026] where x k represents the target state at time k, z k represents the target measurement at time k, u k is the system input, B is the matrix that converts the input into the state, H is the measurement matrix, w k and v k are process noise and measurement noise, which are independent of each other;

[0027] S312: Calculate the target pose and shape feature similarity by weighted calculation to obtain target similarity, specifically including:

[0028] The feature similarity measurement method uses Euclidean distance, with two n-dimensional feature vectors, and by setting the distance threshold d max If the Euclidean distance between the feature vectors exceeds the distance threshold, the similarity s between the feature vectors is zero, achieving the screening effect on the measurement. The vector calculation method is as follows:

[0029]

[0030]

[0031] Where d(x,y) is the Euclidean distance between vector x and vector y, and n represents the dimension of the vector, so as to calculate the target pose and shape similarity s p 、s s ;

[0032] Then the target similarity s is obtained by weighted calculation tar :

[0033] s tar =l1s p +l2s s

[0034] Among them, l1 and l2 are the weights of each feature, and l1+l2=1.

[0035] (3) Use similarity to associate target data, specifically including:

[0036] Assuming that the number of target trajectories and the number of target measurements are p and q respectively, a p×q-dimensional similarity matrix S is obtained by the target similarity calculation method. p×q :

[0037]

[0038] where s tar,i,j Represents the similarity between target trajectory i and target measurement j, sets the threshold δ, when s tar,i,j <δ, let s tar,i,j = 0, and then filter out all-zero rows and all-zero columns, representing no measurement-related trajectories and no associated measurements, respectively, to reduce the subsequent calculation amount and obtain the similarity matrix S′ p×q , define an incidence matrix A ij :

[0039]

[0040] where a ij ∈{0,1},a ij =0 means no correlation, a ij=1 indicates mutual correlation. Under the set constraints, the values ​​of the elements of the correlation matrix are adjusted so that the sum of the products of the similarity matrix and the corresponding elements of the correlation matrix is ​​maximized. At this time, the correlation matrix is ​​the optimal solution.

[0041] Furthermore, the optimal allocation problem formula is as follows:

[0042] Objective function:

[0043] Constraints:

[0044] Furthermore, the step S32 specifically includes the following steps:

[0045] S321: Divide the visual pseudo-label target data into T K ={T1 K ,T K 2,T3 K ,...,T K n}, divide the LiDAR pseudo-label target data into T L ={T1 L ,T L 2,T3 L ,...,T L m};

[0046] S322: Set 4 types of target sets:

[0047] P Y K ={P Y1 K ,P Y2 K ,P Y3 K ,...,P Yn K}

[0048] P N K ={P N1 K ,P N2 K ,P N3 K ,...,P Nm K}

[0049] P Y L ={P Y1 L ,P Y2 L ,P Y3L ,...,P Yu L}

[0050] P N L ={P N1 L ,P N2 L ,P N3 L ,...,P Nv L}

[0051] Among them, P Y K is the visual pseudo-label target T K The set of successfully associated targets in P N K is the visual pseudo-label target T K The set of targets that are not successfully associated, P Y L is the LiDAR pseudo-label target T L The set of successfully associated targets in P N L is the LiDAR pseudo-label target T L The set of targets that were not successfully associated;

[0052] S323: Define an object detection consistency vector:

[0053] y i =[t1,t2,t3,t4] T

[0054] where t j ∈{0,1},j∈[1,4],when t k =1, t i =0,

[0055] S324: If P N K With P N L Both Let t1 = 1, the pseudo-label data of this group is consistent;

[0056] If P N K for P N L Not for Let t2 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model Ak ;

[0057] If P N K Not for P N L for Let t3 = 1, the pseudo-label data of this group is not consistent, and the visual network model A is used. k Training LiDAR network model A l ;

[0058] If P N K With P N L None Let t4 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model A k , while using the visual network model A k Training LiDAR network model A l .

[0059] Furthermore, the training and updating of the detection model in step S4 specifically includes:

[0060] (1) Case 1: y i =[t1,t2,t3,t4] T In t1=1, no data transmission is performed;

[0061] (2) Case 2: y i =[t1,t2,t3,t4] T In the case of t2=1, T L Transmit to A k Conduct further training;

[0062] (3) Case 3: y i =[t1,t2,t3,t4] T In t3=1, T K Transmit to A l Conduct further training;

[0063] (4) Case 4: y i =[t1,t2,t3,t4] T In t4=1, T L Transmit to A k , while T K Transmit to A l Conduct further training;

[0064] (5) After the number of samples accumulates to the training threshold ω, a new round of training is carried out to obtain the training updated detection model, including visual model-2 and lidar model-2, which are used to update visual model-1 and lidar model-1 respectively.

[0065] The beneficial effects of the present invention are as follows: The present invention proposes a target detection system and method using lidar and vision-inspired learning. This system utilizes the differences in the detection capabilities of lidar and vision sensors in different scenarios or features to interactively heuristically learn their respective detection samples to improve the detection capabilities of the two sensor detection models. An evaluator is also added to reduce the number of interactive samples, thereby lowering training costs. A data association method is used in the evaluator to improve evaluation accuracy. This system utilizes the differences in the detection capabilities of lidar and vision sensors in different scenarios or features to construct a heuristic learning method to improve the detection capabilities of the two sensor detection models, achieve unsupervised learning of the detection models, reduce the workload of manual data labeling, and improve the efficiency of detection model training.

[0066] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0068] Figure 1 This is a diagram showing the overall architecture of a target detection system and method using laser radar and visual heuristic learning, according to a preferred embodiment of the present invention;

[0069] Figure 2 This is the overall flow chart of the evaluator's work. DETAILED DESCRIPTION

[0070] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0071] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0072] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0073] like Figure 1 The figure shows the overall architecture of a target detection system and method based on laser radar and visual heuristic learning. The system includes three modules: a visual target detection module, a laser radar target detection module, and a heuristic learning module.

[0074] The visual target detection module is used to collect visual data in real time, perform target detection through the visual model-1, and send the detection results as visual pseudo-label data to the heuristic learning module;

[0075] The LiDAR target detection module is used to collect LiDAR data in real time, perform target detection through LiDAR Model-1, and send the detection results as LiDAR pseudo-label data to the heuristic learning module;

[0076] The heuristic learning module uses an evaluator to evaluate the pseudo-label information output by the visual and lidar targets, associates the pseudo-label information with the target data, and obtains the lidar and visual pseudo-label target association matrix; makes decisions based on the target association matrix to obtain the lidar and visual target detection consistency vector; and uses the pseudo-label information to train the visual model-2 and the lidar model-2 based on the lidar and visual target detection consistency vector. The trained visual model-2 and lidar model-2 are used to update the visual model-1 and the lidar model-1, respectively, to improve the performance of the visual target detection module and the lidar target detection module.

[0077] like Figure 2 Shown is the overall working flow chart of the evaluator.

[0078] (1) Target association. Establish the target’s feature vector:

[0079] x i =[p,s] T (1)

[0080] The pose feature p = [x, y, α] represents the x position, y position and target orientation of the target center point, and the shape feature s = [l, w] represents the length and width of the target detection box;

[0081] Establish target state model and measurement model:

[0082] x k =Bu k +w k (2)

[0083] z k =Hx k +v k (3)

[0084] where x k represents the target state at time k, z k represents the target measurement at time k, u k is the system input, B is the matrix that converts the input into the state, H is the measurement matrix, w k and v k are process noise and measurement noise, which are independent of each other.

[0085] (2) Calculate the target similarity. The feature similarity measurement method uses Euclidean distance. There are two n-dimensional feature vectors. By setting the distance threshold d max ,If the Euclidean distance between the feature vectors exceeds the distance threshold, the similarity s between the feature vectors is zero, achieving the screening effect on the measurement. The vector calculation method is as follows:

[0086]

[0087]

[0088] Where d(x,y) is the Euclidean distance between vector x and vector y, and n represents the dimension of the vector. This method can be used to calculate the target's pose and shape similarity s. p 、s s .

[0089] Then the target similarity s is obtained by weighted calculation tar :

[0090] s tar =l1s p +l2s s (6)

[0091] Among them, l1 and l2 are the weights of each feature, and l1+l2=1.

[0092] (3) Use similarity to associate target data. Assuming that the number of target trajectories and the number of target measurements are p and q respectively, a p×q-dimensional similarity matrix S is obtained by the target similarity calculation method. p×q :

[0093]

[0094] where s tar,i,j Represents the similarity between target trajectory i and target measurement j, sets the threshold δ, when s tar,i,j <δ, let s tar,i,j = 0, and then filter out all-zero rows and all-zero columns, representing no measurement-related trajectories and no associated measurements, respectively, to reduce the subsequent calculation amount and obtain the similarity matrix S′ p×q , define an incidence matrix A ij :

[0095]

[0096] where a ij ∈{0,1},a ij =0 means no correlation, a ij =1 indicates mutual correlation. Under the set constraints, the values ​​of the elements of the correlation matrix are adjusted so that the sum of the products of the similarity matrix and the corresponding elements of the correlation matrix is ​​maximized. At this time, the correlation matrix is ​​the optimal solution.

[0097] The optimal allocation problem is formulated as follows:

[0098] Objective function:

[0099] Constraints:

[0100] (4) Target detection consistency vector calculation. Divide the visual pseudo-label target information into T K ={T1 K ,T K 2,T3 K ,...,T K n}, divide the laser radar pseudo-label target information into T L ={T1 L ,T L 2,T3 L ,...,T L m};

[0101] (5) Set up 4 types of target sets:

[0102] P Y K ={P Y1 K ,P Y2 K ,P Y3 K ,...,P Yn K} (11)

[0103] P N K ={P N1 K ,P N2 K ,P N3 K ,...,P Nm K} (12)

[0104] P Y L ={P Y1 L ,P Y2 L ,P Y3 L ,...,P Yu L} (13)

[0105] P N L ={P N1 L ,P N2 L ,P N3 L ,...,P Nv L} (14)

[0106] Among them, P Y K is the visual pseudo-label target T K The set of successfully associated targets in P N K is the visual pseudo-label target T K The set of targets that are not successfully associated, P Y L is the LiDAR pseudo-label target T L The set of successfully associated targets in P N L is the LiDAR pseudo-label target T L The set of targets that were not successfully associated;

[0107] (6) Define an object detection consistency vector:

[0108] y i =[t1,t2,t3,t4] T (15)

[0109] where t j ∈{0,1},j∈[1,4],when t k =1, t i =0,

[0110] (7) If P N K With P N L Both Let t1 = 1, the pseudo-label data of this group is consistent;

[0111] (8) If P N K for P N L Not for Let t2 = 1. This set of pseudo-label data is not consistent and needs to use the lidar network model A. l Training visual network model A k ;

[0112] (9) If P N K Not for P N L for Let t3 = 1, this set of pseudo-label data is not consistent, and the visual network model A is required. k Training LiDAR network model A l ;

[0113] (10) If P N K With P N L None Let t4 = 1. This set of pseudo-label data is not consistent and needs to use the lidar network model A. l Training visual network model A k , while using the visual network model A k Training LiDAR network model A l .

[0114] (11) Training decision. According to the consistency vector y between the lidar and visual target detection i =[t1,t2,t3,t4] T , four processing situations are obtained, including:

[0115] Case 1: y i =[t1,t2,t3,t4] T In t1=1, no data transmission is required;

[0116] Case 2: y i =[t1,t2,t3,t4] T In the case of t2=1, T L Transmit to A k Conduct further training;

[0117] Case 3: y i =[t1,t2,t3,t4] T In t3=1, T K Transmit to A l Conduct further training;

[0118] Case 4: y i =[t1,t2,t3,t4] T In t4=1, T L Transmit to A k , while T K Transmit to A l Conduct further training;

[0119] (12) After the number of samples accumulates to the training threshold ω, a new round of training is carried out to obtain the training updated detection model, including visual model-2 and lidar model-2, which are used to update visual model-1 and lidar model-1 respectively.

[0120] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A laser radar and vision-inspired learning target detection system, characterized by: Includes visual target detection module, lidar target detection module and heuristic learning module; The visual target detection module is used to collect visual data in real time, perform target detection through the visual model-1, and send the detection results as visual pseudo-label data to the heuristic learning module; The LiDAR target detection module is used to collect LiDAR data in real time, perform target detection through LiDAR Model-1, and send the detection results as LiDAR pseudo-label data to the heuristic learning module; The heuristic learning module uses an evaluator to evaluate the pseudo-label information output by the visual and lidar targets, and according to the evaluation results, uses the pseudo-label information to train the visual model-2 and the lidar model-2, and uses the trained visual model-2 and lidar model-2 to update the visual model-1 and lidar model-1 respectively; The use of an evaluator to evaluate the pseudo-label information output by the visual and lidar targets specifically includes the following steps: S31: Target association: for visual pseudo-label data T K , LiDAR pseudo-label data T L Establish the target feature vector x respectively i , calculate the target similarity s tar , calculate the target association matrix A based on the target similarity ij ; S32: Calculate the target detection consistency vector: According to the association matrix A ij , T K 、T L The targets in the set are put into In, judge Is it Calculate the target detection consistency vector y i ; Step S32 specifically includes the following steps: S321: Divide the visual pseudo-label data into T K ={T1 K ,T K 2,T3 K ,...,T K n }, divide the laser radar pseudo-label data into T L ={T1 L ,T L 2,T3 L ,...,T L m }; S322: Set 4 types of target sets: in is the visual pseudo-label data T K The set of successfully associated targets in is the visual pseudo-label data T K The set of targets that were not successfully associated in is the laser radar pseudo label data T L The set of successfully associated targets in is the laser radar pseudo label data T L The set of targets that were not successfully associated; S323: Define an object detection consistency vector: <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =[t1,t2,t3,t4]<h2 style=";text-align:left;direction:ltr"> T where t j ∈{0,1},j∈[1,4],when t k =1, t i =0, S324: If and Both Let t1 = 1, the pseudo-label data of this group is consistent; like for Not for Let t2 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model A k ; like Not for for Let t3 = 1, the pseudo-label data of this group is not consistent, and the visual network model A is used. k Training LiDAR network model A l ; like and None Let t4 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model A k , while using the visual network model A k Training LiDAR network model A l .

2. A target detection method using laser radar and vision-inspired learning, characterized by: The following steps are involved: S1: Real-time visual data acquisition: At time k, visual data is collected and target detection is performed using the visual model-1 to obtain visual target detection results and visual pseudo-label data; S2: LiDAR real-time data acquisition: At time k, LiDAR data is collected and target detection is performed using LiDAR Model-1 to obtain LiDAR target detection results and LiDAR pseudo-label data. S3: Using an evaluator to evaluate the pseudo-label data output by the visual and lidar targets, associating the pseudo-label data with the target data to obtain a lidar and visual pseudo-label target association matrix, and then making a decision based on the target association matrix to obtain a lidar and visual target detection consistency vector; Step S3, using an evaluator to evaluate the pseudo-label data output by the visual and lidar targets, specifically includes the following steps: S31: Target association: Output pseudo-label data T for visual and lidar targets K 、T L Establish the target feature vector x respectively i , calculate the target similarity s tar , calculate the target association matrix A based on the target similarity ij ; S32: Calculate the target detection consistency vector: According to the association matrix A ij , T K 、T L The targets in the set are put into In, judge Is it Calculate the target detection consistency vector y i ; Step S32 specifically includes the following steps: S321: Divide the visual pseudo-label target data into T K ={T1 K ,T K 2,T3 K ,...,T K n }, divide the LiDAR pseudo-label target data into T L ={T1 L ,T L 2,T3 L ,...,T L m }; S322: Set 4 types of target sets: in is the visual pseudo-label target T K The set of successfully associated targets in is the visual pseudo-label target T K The set of targets that were not successfully associated in is the LiDAR pseudo-label target T L The set of successfully associated targets in is the LiDAR pseudo-label target T L The set of targets that were not successfully associated; S323: Define an object detection consistency vector: <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =[t1,t2,t3,t4]<h2 style=";text-align:left;direction:ltr"> T where t j ∈{0,1},j∈[1,4],when t k =1, t i =0, S324: If and Both Let t1 = 1, the pseudo-label data of this group is consistent; like for Not for Let t2 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model A k ; like Not for for Let t3 = 1, the pseudo-label data of this group is not consistent, and the visual network model A is used. k Training LiDAR network model A l ; like and None Let t4 = 1, this group of pseudo-label data is not consistent, and the laser radar network model A is used. l Training visual network model A k , while using the visual network model A k Training LiDAR network model A l ; S4: Train based on the consistency vector of lidar and visual target detection to obtain a trained updated detection model, including visual model-2 and lidar model-2, which are used to update visual model-1 and lidar model-1 respectively.

3. The target detection method based on laser radar and vision-inspired learning according to claim 2, characterized in that: The step S31 specifically includes the following steps: S311: Based on the laser radar and visual pseudo-label information, the feature vectors of the target are established respectively: x i =[p,s] T The pose feature p = [x, y, α] represents the x position, y position and target orientation of the target center point, and the shape feature s = [l, w] represents the length and width of the target detection box; Establish target state model and measurement model: <h2 style=";text-align:left;direction:ltr">x<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> =Bu<h2 style=";text-align:left;direction:ltr"> k <h2 style=";text-align:left;direction:ltr"> +w<h2 style=";text-align:left;direction:ltr"> k z k =Hx k +v k where x k represents the target state at time k, z k represents the target measurement at time k, u k is the system input, B is the matrix that converts the input into the state, H is the measurement matrix, w k and v k are process noise and measurement noise, which are independent of each other; S312: Calculate the target pose and shape feature similarity by weighted calculation to obtain target similarity, specifically including: The feature similarity measurement method uses Euclidean distance, with two n-dimensional feature vectors, and by setting the distance threshold d max If the Euclidean distance between the feature vectors exceeds the distance threshold, the similarity s between the feature vectors is zero, achieving the screening effect on the measurement. The vector calculation method is as follows: Where d(x,y) is the Euclidean distance between vector x and vector y, and n represents the dimension of the vector, so as to calculate the target pose and shape similarity s p 、s s ; Then the target similarity s is obtained by weighted calculation tar : s tar =l1s p +l2s s Where l1, l2 are the weights of each feature, and l1+l2=1; (3) Use similarity to associate target data, specifically including: Assuming that the number of target trajectories and the number of target measurements are p and q respectively, a p×q-dimensional similarity matrix S is obtained by the target similarity calculation method. p×q : where s tar,i,j Represents the similarity between target trajectory i and target measurement j, sets the threshold δ, when s tar,i,j <δ, let s tar,i,j = 0, and then filter out all-zero rows and all-zero columns, representing no measurement-related trajectories and no associated measurements, respectively, to reduce the subsequent calculation amount and obtain the similarity matrix S′ p×q , define an incidence matrix A ij : where a ij ∈{0,1},a ij =0 means no correlation, a ij =1 indicates mutual correlation. Under the set constraints, the values ​​of the elements of the correlation matrix are adjusted so that the sum of the products of the similarity matrix and the corresponding elements of the correlation matrix is ​​maximized. At this time, the correlation matrix is ​​the optimal solution.

4. The target detection method based on laser radar and vision-inspired learning according to claim 3, characterized in that: The optimal allocation problem formula is as follows: Objective function: Constraints:

5. The target detection method based on laser radar and vision-inspired learning according to claim 2, characterized in that: Step S4 obtains the training and updates the detection model, specifically including: (1) Case 1: y i =[t1,t2,t3,t4] T In t1=1, no data transmission is performed; (2) Case 2: y i =[t1,t2,t3,t4] T In the case of t2=1, T L Transmit to A k Conduct further training; (3) Case 3: y i =[t1,t2,t3,t4] T In t3=1, T K Transmit to A l Conduct further training; (4) Case 4: y i =[t1,t2,t3,t4] T In t4=1, T L Transmit to A k , while T K Transmit to A l Conduct further training; (5) After the number of samples accumulates to the training threshold ω, a new round of training is carried out to obtain the training updated detection model, including visual model-2 and lidar model-2, which are used to update visual model-1 and lidar model-1 respectively.

Images