Vehicle-mounted multi-camera target detection method, control device, storage medium and vehicle

Abstract

This invention relates to the field of autonomous driving technology, specifically providing an in-vehicle multi-camera target detection method, control device, storage medium, and vehicle. It aims to solve the problem of inconsistent depth predictions from multiple cameras on a vehicle for the same object, leading to fusion errors and further misjudgments. To this end, this invention constructs a joint loss function using a first loss function and a second loss function, and jointly trains the target detection models of multiple cameras. Then, image data collected by multiple cameras are input into the corresponding target detection models of each camera to obtain multiple target detection results. These results are then fused to obtain a fused target detection result. This invention improves the consistency of target detection results among multiple cameras without sacrificing the performance of each individual target detection model.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, specifically providing an in-vehicle multi-camera target detection method, control device, storage medium, and vehicle. Background Technology

[0002] In autonomous driving technology, it is necessary to fuse the outputs of all sensors to obtain a 360-degree omnidirectional fusion result to assist in autonomous driving. Due to the redundancy between sensors, there may be situations where multiple cameras can see the same object, resulting in multiple cameras outputting predictions for that object.

[0003] In the process of fusing prediction results from multiple sensors, the fusion is generally based on the distance between objects, the intersection-over-union (IoU) ratio of the object predicted by camera A projected onto camera B, and the similarity of the objects under different camera images. If the differences are too large, they will be judged as different objects. The fusion results obtained by applying multi-sensor fusion are generally much better than the prediction results of a single sensor. However, if there are inconsistencies in the depth predictions of the same object from multiple cameras, and when the degree of inconsistency is large, it will be judged as two objects, resulting in a fusion error and leading to serious misjudgment.

[0004] Accordingly, there is a need in this field for a new multi-camera prediction result fusion scheme to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned defects, this invention is proposed to provide a solution, or at least a partial solution, to the problem that when multiple cameras mounted on a vehicle predict the same object with inconsistent depths, fusion errors occur, which in turn lead to misjudgments.

[0006] In a first aspect, the present invention provides a vehicle-mounted multi-camera target detection method, the method comprising:

[0007] Image data acquired by multiple cameras are synchronized in time and then input into the corresponding target detection models of each camera to obtain multiple target detection results. The target detection models of the multiple cameras are jointly trained using a joint loss function constructed from a first loss function and a second loss function. The first loss function is a consistency loss function for the target detection results among the multiple target detection models, and the second loss function is a loss function between the target detection result of each target detection model and the ground truth value.

[0008] The multiple target detection results are fused to obtain a target detection fusion result.

[0009] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, the target detection model of the multiple cameras is obtained by joint training using a joint loss function constructed from a first loss function and a second loss function, including:

[0010] The image training data in the preset dataset is grouped according to the timestamp to obtain different groups of image training data, where each group of image training data has the same timestamp;

[0011] For each iteration of joint training, a set of image training data is input into the multiple object detection models respectively, so as to obtain the object detection results of the multiple object detection models for the same object respectively;

[0012] The consistency loss among the multiple target detection results is obtained according to the first loss function, and the original loss between the target detection result and the ground truth value of each target detection model is obtained according to the second loss function.

[0013] By applying the joint loss function, the joint loss for the current iteration is obtained based on the consistency loss and the original loss;

[0014] Based on the joint loss, the parameters of the multiple object detection models are updated by backpropagation to achieve joint training of the multiple object detection models.

[0015] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, obtaining the consistency loss among the multiple target detection results based on the first loss function includes:

[0016] Obtain multiple target detection results for the same target from multiple target detection models;

[0017] Based on the extrinsic parameters between the multiple cameras and the multiple target detection results, the first loss function is applied to obtain the consistency loss of the multiple target detection results in the same coordinate system.

[0018] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, the step of acquiring multiple target detection results of multiple target detection models for the same target includes:

[0019] For the same target, each target detection model obtains a matching 2D box that matches the true 2D box of the target from multiple predicted detection 2D boxes;

[0020] The average of the 3D coordinates of the matched 2D bounding box is used to obtain the target detection result of the target detection model for the target.

[0021] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, the step of obtaining a matching 2D box that matches the true 2D box of the target from multiple predicted detection 2D boxes for the same target includes:

[0022] Calculate the intersection-union ratio (IUU) between each detected 2D bounding box and the ground truth 2D bounding box;

[0023] The cross-union ratio is compared with a preset cross-union ratio threshold;

[0024] When the cross-union ratio is greater than the cross-union ratio threshold, the corresponding detection 2D box is taken as the matching 2D box.

[0025] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, the step of applying the first loss function to obtain the consistency loss of the multiple target detection results in the same coordinate system based on the extrinsic parameters between the multiple cameras and the multiple target detection results includes:

[0026] Based on the extrinsic parameters between different cameras, the target detection results of the multiple target detection models are transformed to the same camera coordinate system;

[0027] Based on the consistency loss function and the target detection results of different cameras in the same camera coordinate system, the consistency loss of the multiple target detection results in the same coordinate system is obtained.

[0028] In one technical solution of the above-mentioned vehicle-mounted multi-camera target detection method, the consistency loss function is a function of the offset between the center coordinates of the target detection results of the multiple target detection models for the same target in the same camera coordinate system; and / or,

[0029] The weight of the first loss function in the joint loss function is less than the weight of the second loss function in the joint loss function.

[0030] In a second aspect, a control device is provided, comprising a processor and a storage device, the storage device being adapted to store a plurality of program codes, the program codes being adapted to be loaded and run by the processor to perform the vehicle-mounted multi-camera target detection method described in any of the above-described technical solutions of the vehicle-mounted multi-camera target detection method.

[0031] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, the program codes being adapted to be loaded and run by a processor to perform the vehicle-mounted multi-camera target detection method described in any of the above-described technical solutions of the vehicle-mounted multi-camera target detection method.

[0032] In a fourth aspect, a vehicle is provided, the vehicle being equipped with a plurality of cameras, the vehicle including the control device described in the above-mentioned control device technical solution.

[0033] The present invention comprises one or more of the following technical solutions:

[0034] Beneficial effects:

[0035] In implementing the technical solution of this invention, a joint loss function is constructed using a first loss function and a second loss function to jointly train target detection models from multiple cameras. The first loss function is a consistency loss function between the target detection results of the multiple target detection models, and the second loss function is a loss function between the target detection result and the ground truth for each target detection model. Then, image data acquired by multiple cameras is time-synchronized and input into the corresponding target detection model for each camera to obtain multiple target detection results. These multiple target detection results are then fused to obtain a target detection fusion result. Through this configuration, since the target detection model for vehicle-mounted multi-camera systems in this invention is obtained through joint training using the first and second loss functions, the consistency of target detection results between multiple cameras can be further improved without sacrificing the performance of each target detection model. This ensures that when fusing target detection results from multiple cameras, the fusion error of classifying the same object as two objects can be effectively avoided, thereby improving the accuracy of the target detection fusion result. Attached Figure Description

[0036] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Wherein:

[0037] Figure 1 This is a schematic flowchart of the main steps of a vehicle-mounted multi-camera target detection method according to an embodiment of the present invention;

[0038] Figure 2 This is an example of image data captured by a vehicle-mounted forward-looking camera (FW).

[0039] Figure 3 This is image data captured by an example of a vehicle-mounted forward-looking FN camera;

[0040] Figure 4 This is image data captured by an example of a vehicle-mounted side-view FL camera;

[0041] Figure 5 This is a schematic diagram of the main architecture for jointly training multiple target detection models according to one embodiment of the present invention.

[0042] Figure 6 This is a schematic diagram comparing the target detection fusion results obtained by the vehicle-mounted multi-camera target detection method according to the present invention with those obtained by the detection method in the prior art. Detailed Implementation

[0043] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0044] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0045] In driver assistance technologies, the perception of the surrounding environment is often based on multiple sensors installed on the vehicle. For example... Figures 2 to 4 As shown, Figures 2 to 4 The images are collected simultaneously by the forward-looking FW camera, forward-looking FN camera, and side-looking FL camera installed on the vehicle. The box marks the same truck, which appears in the field of view of the three cameras. Therefore, each camera will have a prediction result for the truck, which can be the 3D coordinates of the truck. However, when these 3D coordinates are uniformly transformed into the vehicle coordinate system, the difference will be large (for example, the difference can be as high as 50m). This will cause the truck to be predicted as multiple objects during the multi-camera fusion process, which will further lead to misjudgment in the fusion process.

[0046] Accordingly, a new vehicle-mounted multi-camera target detection method is needed to solve the above problems.

[0047] See appendix Figure 1 , Figure 1This is a schematic flowchart illustrating the main steps of a vehicle-mounted multi-camera target detection method according to an embodiment of the present invention. Figure 1 As shown, the vehicle-mounted multi-camera target detection method in this embodiment of the invention mainly includes the following steps S101-S102.

[0048] Step S101: After synchronizing the image data acquired by multiple cameras in time, input them into the target detection model corresponding to each camera to obtain multiple target detection results. The target detection models of multiple cameras are jointly trained using a joint loss function constructed from a first loss function and a second loss function. The first loss function is the consistency loss function of the target detection results among multiple target detection models, and the second loss function is the loss function between the target detection result and the ground truth of each target detection model.

[0049] In this embodiment, a first loss function and a second loss function can be defined. The first loss function is a consistency loss function for the object detection results among multiple object detection models, and the second loss function is a loss function between the object detection result of each object detection model and the ground truth. The first and second loss functions can be combined to form a joint loss function, which is used to train object detection models from multiple cameras. The trained object detection models are then applied to perform object detection to obtain the object detection results for each camera. The consistency loss is a parameter used to evaluate the differences in 3D coordinates between the object detection results of different cameras for the same target, such as differences in depth or differences in the offset between center coordinates.

[0050] In one implementation, the consistency loss function is a function of the offset between the center coordinates of the target detection results of multiple target detection models for the same target in the same camera coordinate system.

[0051] In one implementation, the offset of the target detection results of any two cameras in the x, y, and z axes can be determined by the following formula (1) to further determine the consistency loss function (i.e., the first loss function):

[0052]

[0053] Among them, L con Let L be the consistency loss function, N be the number of target detection results, and L be the number of detection results. i,j,x (x i ,x j L is the offset of the x-coordinate between the center coordinates of the target detection results of the i-th camera and the j-th camera for the same target. j,j,y (y i ,y jL represents the offset of the y-coordinate between the center coordinates of the target detection results of the i-th camera and the j-th camera for the same target. i,j,z (z i ,z j ) represents the offset of the z-coordinate between the center coordinates of the target detection results of the i-th camera and the j-th camera for the same target.

[0054] In one implementation, the target detection result of one camera can be selected as a benchmark, and the offsets between the target detection results of other cameras and the benchmark can be obtained and summed to obtain the consistency loss function.

[0055] In one implementation, the loss function (i.e., the second loss function) between the target detection result and the true value of a target detection model can be determined by the following formula (2):

[0056] L k =L k (Δx k )+L k (Δy k )+L k (Δz k ),k=1…N (2)

[0057] Among them, L k Let L be the second loss function for the k-th object detection model, where N is the number of object detection results. k (Δx k L represents the offset of the x-coordinate between the center coordinates of the k-th target detection result and the true coordinates. k (Δy k L represents the offset of the y-coordinate between the center coordinates of the k-th target detection result and the true coordinates. k (Δz k ) represents the offset of the z-coordinate between the center coordinates of the k-th target detection result and the true coordinates.

[0058] In one implementation, the weight values ​​of the first loss function and the second loss function can be set in the joint loss function, as shown in the following formula (3). The joint loss function can be obtained by weighting the first loss function and the second loss function.

[0059]

[0060] Among them, L total Let λ1 be the weight of the first loss function and λ2 be the weight of the second loss function.

[0061] In one implementation, the weight λ1 of the first loss function is smaller than the weight λ2 of the second loss function. This ensures that the first loss function does not dominate during joint training, thereby minimizing the consistency loss between object detection models while maintaining the performance of each individual model. It also avoids obtaining trivial solutions during joint training.

[0062] In one implementation, the weight ratio between the first loss function and the second loss function can be 1:4.

[0063] Step S102: Fuse multiple target detection results to obtain a target detection fusion result.

[0064] In this embodiment, multiple target detection results can be fused to obtain a multi-camera-based target detection fusion result. The process of fusing multiple target detection results is high-level fusion, which involves obtaining target-level target detection results from the image data of each camera and then fusing these multiple target-level detection results.

[0065] Based on steps S101-S102 above, this embodiment of the invention applies a first loss function and a second loss function to construct a joint loss function for joint training of target detection models from multiple cameras. The first loss function is a consistency loss function between the target detection results of multiple target detection models, and the second loss function is a loss function between the target detection result and the ground truth for each target detection model. Then, image data acquired by multiple cameras is time-synchronized and input into the corresponding target detection model for each camera to obtain multiple target detection results. These multiple target detection results are then fused to obtain a target detection fusion result. Through this configuration, since the target detection model for vehicle-mounted multi-cameras in this embodiment of the invention is obtained through joint training using the first and second loss functions, it can further improve the consistency of target detection results between multiple cameras without sacrificing the performance of each target detection model. It can also effectively avoid the fusion error of classifying the same object as two objects when fusing target detection results from multiple cameras, thereby improving the accuracy of the target detection fusion result.

[0066] The following section further explains the process of jointly training object detection models from multiple cameras.

[0067] In one embodiment of the present invention, joint training of target detection models from multiple cameras can be performed through the following steps S201 to S205:

[0068] Step S201: Group the image training data in the preset dataset according to the timestamp to obtain different groups of image training data, wherein each group of image training data has the same timestamp.

[0069] In this embodiment, image training data in a preset dataset can be classified according to timestamps, and image training data with the same timestamp can be grouped together for iterative training of multiple object detection models. See the appendix for details. Figure 5 , Figure 5 This is a schematic diagram of the main architecture for jointly training multiple object detection models according to one embodiment of the present invention. Figure 5 As shown, taking the forward-looking FW camera, forward-looking FN camera, and side-looking FL camera as examples, there are corresponding 3 target detection models (it should be noted that, schematically, the models in the figure are shown in a box). The image training data collected by FW, FN, and FL can be used as a preset dataset, and grouped according to timestamps to obtain multiple groups such as "FW1, FN1, FL1", "FW2, FN2, FL2", and so on.

[0070] Step S202: For each iteration of joint training, a set of image training data is input into multiple object detection models to obtain the object detection results of the multiple object detection models for the same object.

[0071] In this embodiment, during each iteration of joint training, a set of image training data can be input into multiple object detection models, thereby obtaining the object detection results of multiple object detection models for the same object. For example... Figure 5 As shown, a set of image training data is input into the model to obtain target detection results for different targets (object 1, object 2, object 3, etc.) from different cameras.

[0072] In one embodiment, step S202 may further include steps S2021 and S2022:

[0073] Step S2021: For the same target, each target detection model obtains a matching 2D box that matches the true 2D box of the target from the multiple predicted detection 2D boxes.

[0074] In this embodiment, step S2021 may further include steps S20211 to S20213:

[0075] Step S20211: Calculate the cross-union ratio between each detected 2D bounding box and the ground truth 2D bounding box.

[0076] Step S20212: Compare the cross-union ratio with the preset cross-union ratio threshold.

[0077] Step S20213: When the cross-union ratio is greater than the cross-union ratio threshold, the corresponding detection 2D box is taken as the matching 2D box.

[0078] Step S2022: Calculate the average of the 3D coordinates of the matched 2D bounding boxes to obtain the target detection result of the target detection model.

[0079] Step S203: Obtain the consistency loss among multiple object detection results according to the first loss function, and obtain the original loss between the object detection result and the ground truth value of each object detection model according to the second loss function.

[0080] In this embodiment, the consistency loss can be obtained according to the following steps S2031 and S2032:

[0081] Step S2031: Obtain multiple target detection results for the same target from multiple target detection models.

[0082] Step S2032: Based on the extrinsic parameters between multiple cameras and the multiple target detection results, apply the first loss function to obtain the consistency loss of the multiple target detection results in the same coordinate system. For example, the consistency loss can be calculated according to the consistency loss function shown in formula (1).

[0083] In this embodiment, such as Figure 5 As shown, if the target is object 2, then the 3D coordinates of object 2 obtained from different object detection models can be transformed to a single camera coordinate system (FW camera coordinate system). This allows us to obtain loss@(FW, FL) (the offset of object 2 in the object detection results of FW camera and FL camera) and loss@(FW, FN) (the offset of object 2 in the object detection results of FW camera and FN camera). Based on loss@(FW, FL) and loss@(FW, FN), we can then obtain the consistency loss.

[0084] Step S204: Apply the joint loss function and obtain the joint loss for the current iteration based on the consistency loss and the original loss.

[0085] In this implementation, a joint loss function can be applied to obtain the joint loss of the current iteration based on the consistency loss and the original loss.

[0086] Step S205: Update the parameters of multiple object detection models by backpropagation based on the joint loss to achieve joint training of multiple object detection models.

[0087] In this embodiment, the parameters of multiple object detection models can be updated by backpropagation based on the joint loss, so as to achieve joint training of multiple object detection models.

[0088] In one implementation, gradient descent can be used to backpropagate and update the parameters of the object detection model, so as to minimize the consistency loss of the object detection results obtained by the object detection model, thereby achieving joint training of multiple object detection models.

[0089] In one implementation, the training cutoff condition for joint training is traversing the preset dataset a preset number of times. Those skilled in the art can also set the training cutoff condition according to the needs of the actual application, such as the target prediction model's prediction accuracy and consistency both reaching preset conditions.

[0090] In one implementation, see Appendix Figure 6 , Figure 6 This is a schematic diagram comparing the target detection fusion results obtained by the vehicle-mounted multi-camera target detection method according to the present invention with those obtained by existing detection methods. Figure 6 As shown, Figure 6 The top center shows the target detection results of existing technologies, while the bottom center shows the target fusion detection results obtained by applying the embodiments of the present invention. The target fusion detection results obtained by the vehicle-mounted multi-camera target detection method of the present invention have fewer fusion errors than those obtained by existing technologies, effectively improving the accuracy of multi-camera target detection.

[0091] It should be noted that although the steps in the above embodiments are described in a specific order, those skilled in the art will understand that in order to achieve the effects of the present invention, different steps do not necessarily have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders, and these variations are all within the scope of protection of the present invention.

[0092] Those skilled in the art will understand that all or part of the processes in the method of the above embodiment of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable storage medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable storage medium does not include electrical carrier signals and telecommunication signals.

[0093] Furthermore, the present invention also provides a control device. In one embodiment of the control device according to the present invention, the control device includes a processor and a storage device. The storage device can be configured to store a program for executing the vehicle-mounted multi-camera target detection method of the above-described method embodiments. The processor can be configured to execute the program in the storage device, which includes, but is not limited to, a program for executing the vehicle-mounted multi-camera target detection method of the above-described method embodiments. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This control device can be a control device device comprising various electronic devices.

[0094] Furthermore, the present invention also provides a computer-readable storage medium. In one embodiment of the computer-readable storage medium according to the present invention, the computer-readable storage medium can be configured to store a program for performing the vehicle-mounted multi-camera target detection method of the above-described method embodiments. This program can be loaded and run by a processor to implement the above-described vehicle-mounted multi-camera target detection method. For ease of explanation, only the parts related to the embodiments of the present invention are shown; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. The computer-readable storage medium can be a storage device comprising various electronic devices. Optionally, in the embodiments of the present invention, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0095] Furthermore, the present invention also provides a vehicle, in one embodiment of the present invention, wherein the vehicle is provided with a plurality of cameras, and the vehicle includes the control device described in the above-described control device embodiment.

[0096] Furthermore, it should be understood that since the various modules are only provided to illustrate the functional units of the device of the present invention, the physical devices corresponding to these modules may be the processor itself, or a part of the processor's software, hardware, or a combination of software and hardware. Therefore, the number of modules shown in the figures is merely illustrative.

[0097] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0098] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A vehicle-mounted multi-camera target detection method, characterized in that, The method includes: Image data acquired by multiple cameras are synchronized in time and then input into the corresponding target detection models of each camera to obtain multiple target detection results. The target detection models of the multiple cameras are jointly trained using a joint loss function constructed from a first loss function and a second loss function. The first loss function is a consistency loss function for the target detection results among the multiple target detection models, and the second loss function is a loss function between the target detection result of each target detection model and the ground truth value. The multiple target detection results are fused to obtain a target detection fusion result; The target detection models of the multiple cameras are obtained by joint training using a joint loss function constructed from the first loss function and the second loss function, including: The image training data in the preset dataset is grouped according to the timestamp to obtain different groups of image training data, where each group of image training data has the same timestamp; For each iteration of joint training, a set of image training data is input into the multiple object detection models respectively, so as to obtain the object detection results of the multiple object detection models for the same object respectively; The consistency loss among the multiple target detection results is obtained according to the first loss function, and the original loss between the target detection result and the ground truth value of each target detection model is obtained according to the second loss function. By applying the joint loss function, the joint loss for the current iteration is obtained based on the consistency loss and the original loss; Based on the joint loss, the parameters of the multiple object detection models are updated by backpropagation to achieve joint training of the multiple object detection models.

2. The method according to claim 1, characterized in that, The consistency loss among the multiple target detection results is obtained based on the first loss function, including: Obtain multiple target detection results for the same target from multiple target detection models; Based on the extrinsic parameters between the multiple cameras and the multiple target detection results, the first loss function is applied to obtain the consistency loss of the multiple target detection results in the same coordinate system.

3. The method according to claim 2, characterized in that, The step of obtaining multiple target detection results for the same target from multiple target detection models includes: For the same target, each target detection model obtains a matching 2D box that matches the true 2D box of the target from multiple predicted detection 2D boxes; The average of the 3D coordinates of the matched 2D bounding box is used to obtain the target detection result of the target detection model for the target.

4. The method according to claim 3, characterized in that, For the same target, each target detection model obtains a matching 2D box that matches the ground truth 2D box of the target from multiple predicted detection 2D boxes, including: Calculate the intersection-union ratio (IUU) between each detected 2D bounding box and the ground truth 2D bounding box; The cross-union ratio is compared with a preset cross-union ratio threshold; When the cross-union ratio is greater than the cross-union ratio threshold, the corresponding detection 2D box is taken as the matching 2D box.

5. The method according to claim 2, characterized in that, The step of applying the first loss function to obtain the consistency loss of the multiple target detection results in the same coordinate system based on the extrinsic parameters between the multiple cameras and the multiple target detection results includes: Based on the extrinsic parameters between different cameras, the target detection results of the multiple target detection models are transformed to the same camera coordinate system; Based on the consistency loss function and the target detection results of different cameras in the same camera coordinate system, the consistency loss of the multiple target detection results in the same coordinate system is obtained.

6. The method according to any one of claims 1 to 5, characterized in that, The consistency loss function is a function of the offset between the center coordinates of the target detection results of the multiple target detection models for the same target in the same camera coordinate system; and / or, The weight of the first loss function in the joint loss function is less than the weight of the second loss function in the joint loss function.

7. A control device, comprising a processor and a storage device, said storage device being adapted to store a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by the processor to perform the method of any one of claims 1 to 6.

8. A computer-readable storage medium storing a plurality of program codes, characterized in that, The program code is adapted to be loaded and run by a processor to perform the method of any one of claims 1 to 6.

9. A vehicle, characterized in that, The vehicle is equipped with multiple cameras, and the vehicle includes the control device as described in claim 7.

Images