Cross-mirror target tracking method, device, equipment and medium applied to vehicles

Abstract

The present application relates to a cross-mirror target tracking method, apparatus, equipment and medium for vehicle use, wherein the method comprises: obtaining video frames captured by each camera, and performing target detection on the video frames to determine the detection target; using the same tracker to track the detection target in the video frames captured by the spaced cameras to generate a plurality of tracking units; inputting the tracking results corresponding to the two trackers into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-mirror targets. In this way, the accuracy of different cameras in recognizing the same target can be improved, thereby ensuring the environmental perception effect.

Description

Technical Field

[0001] The present application relates to the field of automotive environment perception technology, and in particular to a cross-mirror target tracking method, device, equipment and medium applied to a vehicle. Background Art

[0002] With the development of technology, target tracking has been increasingly widely used in human-computer interaction, automatic monitoring, video retrieval, traffic detection, and vehicle navigation. Among them, the task of target tracking is to determine the geometric state of the target in the video stream, including position, shape, size, etc. In the current technical solution, a camera can be set on the vehicle to obtain video stream information around the vehicle and identify and track surrounding targets (such as pedestrians or vehicles) to achieve environmental perception. However, if multiple cameras are set on the vehicle for environmental perception, it is common for two adjacent cameras to capture the same target, and it is easy to identify the same target captured by different cameras as different targets, thereby affecting the accuracy of environmental perception. Therefore, how to improve the accuracy of different cameras in recognizing the same target to ensure the environmental perception effect has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] This application provides a method, device, equipment, and medium for cross-camera target tracking in vehicles, addressing the problem in related technologies whereby the same target captured by different cameras is easily identified as different targets, affecting the accuracy of environmental perception. This method can improve the accuracy of different cameras in identifying the same target, thereby ensuring effective environmental perception.

[0004] A first aspect of the present application provides a cross-mirror target tracking method for a vehicle, wherein a plurality of cameras are arranged along the circumference of the vehicle, the fields of view of adjacent cameras partially overlap, and the fields of view of spaced cameras are independent of each other, and the method comprises the following steps: obtaining video frames captured by each camera, and performing target detection on the video frames to determine the detection targets; using the same tracker to track the detection targets in the video frames captured by the spaced cameras to generate a plurality of tracking units; inputting the tracking results corresponding to the two trackers into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-mirror targets.

[0005] According to the above technical means, in the embodiment of the present application, multiple cameras are set along the circumference of the vehicle, the fields of view of adjacent cameras are partially overlapped, and the fields of view of cameras set at intervals are independent of each other. The video frames captured by each camera are obtained, and target detection is performed on the video frames to determine the detection target. In addition, the detection target in the video captured by the cameras set at intervals is tracked using the same tracker to generate a number of tracking units, thereby improving the accuracy of subsequent recognition. The tracking results corresponding to the two trackers are then input into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target. The target recognition model is retrained by training data with cross-lens targets, thereby improving the accuracy of recognition of the same detection target and ensuring the environmental perception effect.

[0006] Optionally, after determining the tracking units belonging to the same target, the method further includes: allocating the same identification information to the tracking units belonging to the same detection target for association.

[0007] According to the above technical means, the embodiment of the present application can realize the association of tracking units belonging to the same detection target.

[0008] Optionally, the target recognition model is retrained, including: obtaining training data with cross-camera targets and a pre-trained basic recognition model; using the training data with cross-camera targets to retrain the basic recognition model, so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras.

[0009] According to the above technical means, the accuracy of the target recognition model in identifying cross-lens targets can be improved.

[0010] Optionally, obtaining training data with cross-lens targets includes: detecting whether a detection target enters or leaves the overlapping field of view between a camera and an adjacent camera; if so, storing the detection data corresponding to the current camera as training data with cross-lens targets.

[0011] According to the above technical means, training data with cross-camera targets can be automatically obtained without manual labeling.

[0012] Optionally, the detection data corresponding to the current camera is stored as training data with cross-lens targets, including: obtaining the detection data corresponding to the current camera; and storing the detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold as training data with cross-lens targets.

[0013] According to the above technical means, the validity of the training data can be guaranteed by screening the training data with cross-lens targets.

[0014] A second aspect of the present application provides a cross-mirror target tracking device for a vehicle, wherein a plurality of cameras are arranged along the circumference of the vehicle, the fields of view of adjacent cameras partially overlap, and the fields of view of spaced cameras are independent of each other. The device includes: a target detection module for acquiring video frames captured by each camera, and performing target detection on the video frames to determine the detection targets; a target tracking module for using the same tracker to track the detection targets in the video frames captured by the spaced cameras to generate a plurality of tracking units; a target matching module for inputting the tracking results corresponding to the two trackers into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-mirror targets.

[0015] Optionally, after determining the tracking units belonging to the same target, the target matching module is further configured to: assign the same identification information to the tracking units belonging to the same detected target for association.

[0016] Optionally, it also includes a model training module, which is used to: obtain training data with cross-camera targets and a pre-trained basic recognition model; use the training data with cross-camera targets to retrain the basic recognition model so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras.

[0017] Optionally, the model training module is used to: detect whether a detection target enters or leaves the overlapping field of view between a camera and an adjacent camera; if so, store the detection data corresponding to the current camera as training data with cross-camera targets.

[0018] Optionally, the model training module is used to: obtain detection data corresponding to the current camera; and store detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold as training data with cross-camera targets.

[0019] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the cross-mirror target tracking method for a vehicle as described in the above embodiment.

[0020] A fourth aspect of the present application provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the cross-mirror target tracking method applied to a vehicle as described in the above embodiment.

[0021] In the embodiment of the present application, multiple cameras can be set along the circumference of the vehicle. The fields of view of adjacent cameras partially overlap, and the fields of view of cameras set at intervals are independent of each other. The video frames captured by each camera are obtained, and target detection is performed on the video frames to determine the detection target. In addition, the detection target in the video captured by the cameras set at intervals is tracked using the same tracker to generate a number of tracking units, thereby improving the accuracy of subsequent recognition. The tracking results corresponding to the two trackers are then input into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target. The target recognition model is retrained by training data with cross-camera targets, thereby improving the accuracy of recognition of the same detection target and ensuring the environmental perception effect.

[0022] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 Schematic diagram of a flow chart of a cross-mirror target tracking method for a vehicle provided in accordance with an embodiment of the present application;

[0025] Figure 2 A schematic diagram of camera installation and corresponding field of view of a vehicle according to one embodiment of the present application;

[0026] Figure 3 A schematic diagram of a camera perspective when a target enters an overlapping field of view according to one embodiment of the present application;

[0027] Figure 4 A schematic diagram of tracker allocation according to one embodiment of the present application;

[0028] Figure 5 A schematic diagram of tracking a dynamic target according to an embodiment of the present application;

[0029] Figure 6 A schematic diagram of calling a main functional interface according to an embodiment of the present application;

[0030] Figure 7Schematic diagram of a video frame with a cross-lens target according to one embodiment of the present application;

[0031] Figure 8 Schematic diagram of a cross-mirror target tracking device applied to a vehicle according to an embodiment of the present application;

[0032] Figure 9 is an example diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the cross-mirror target tracking method, device, equipment and medium applied to a vehicle according to an embodiment of the present application with reference to the accompanying drawings. In response to the problem mentioned in the above background technology that the same target captured by different cameras is easily identified as different targets, which affects the accuracy of environmental perception, the present application provides a cross-mirror target tracking method applied to a vehicle. In this method, multiple cameras can be set along the circumference of the vehicle, and the fields of view of adjacent cameras are partially overlapped, and the fields of view of cameras set at intervals are independent of each other. The video frames captured by each camera are obtained, and target detection is performed on the video frames to determine the detection target. In addition, the detection target in the video captured by the cameras set at intervals is tracked using the same tracker to generate a number of tracking units, thereby improving the accuracy of subsequent recognition. The tracking results corresponding to the two trackers are then input into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target. The target recognition model is retrained by training data with cross-mirror targets, thereby improving the accuracy of recognition of the same detection target, thereby ensuring the environmental perception effect.

[0035] Specifically, Figure 1 This is a flow chart of a cross-mirror target tracking method for a vehicle provided in an embodiment of the present application. In this embodiment, multiple cameras are arranged along the circumference of the vehicle, with the fields of view of adjacent cameras partially overlapping, and the fields of view of spaced cameras being independent of each other.

[0036] Among them, the field of view can be the horizontal shooting range of the camera (for example, the horizontal field of view angle FOV is 90°, etc.). Through adjustment, the fields of view of adjacent cameras can partially overlap, that is, the camera has an independent field of view and also has an overlapping field of view with other adjacent cameras (i.e. overlap-view). The fields of view of cameras set at intervals are independent of each other and there is no overlap.

[0037] by Figure 2 As shown in the example, a vehicle is equipped with six cameras along its circumference: camera1-front, camera2-front-right, camera3-rear-right, camera4-rear, camera5-rear-left, and camera6-front-left. Each camera has an independent field of view, with overlapping fields of view between adjacent cameras, such as between camera1-front and camera6-front-left, between camera6-front-left and camera5-rear-left, and so on. Furthermore, the fields of view of cameras spaced apart are independent of each other, such as between camera1-front and camera5-rear-left, between camera6-front-left and camera4-rear, and so on, with no overlapping fields of view between them.

[0038] Please continue to refer to Figure 1 The cross-mirror target tracking method for a vehicle includes at least steps S110 to S130, which are described in detail as follows:

[0039] In step S110 , video frames captured by each camera are acquired, and target detection is performed on the video frames to determine a detection target.

[0040] In this embodiment, each camera can obtain video stream information within its own shooting range and perform frame processing on the video stream information (for example, 30 frames per second) to obtain a corresponding set of video frames. Based on the video frames obtained by each camera, target detection can be performed to identify detection targets contained in the video frames. The detection targets can include vehicles, pedestrians, cyclists, etc.

[0041] It should be understood that when a target enters an overlapping field of view, both cameras corresponding to the overlapping field of view can capture the target, so the target will be recognized in the video frames captured by each camera. Figure 3As shown in the figure, when the target enters the overlapping field of view between cameras camera3-rear-right and camera4-rear, both camera3-rear-right and camera4-rear will capture the target.

[0042] In step S120, the detection targets in the video frames captured by the spaced cameras are tracked using the same tracker to generate a plurality of tracking units.

[0043] In this embodiment, the same tracker can be used for the cameras set at intervals. Figure 2 As shown in the figure, the target detection results of camera1, camera3 and camera5 are tracked by one group of trackers, and the remaining cameras, namely camera2, camera4 and camera6, are tracked by another group of trackers (such as Figure 4 As shown in the figure, camera1, camera3, and camera5 use tracker A for target tracking, while camera2, camera4, and camera6 use tracker B for tracking. In other words, the three video frames captured by the spaced cameras form a large image, and a single tracker is used to track the target. Thus, each tracker can track each detected target captured by the camera and generate a corresponding tracking unit.

[0044] In one example, the tracker's tracking algorithm can use the Deep Sort algorithm, which is commonly used in multi-target tracking. Its general process involves using a Kalman filter to predict tracks, using the Hungarian algorithm to match the predicted tracks with detections in the current frame (including cascade matching and IOU matching), and then using the Kalman filter to update them.

[0045] Since target tracking uses the tracking by detection method, when the detection results are passed through the Kalman filter and Hungarian matching is performed, if only one tracker is used for six cameras, matching errors may easily occur when the targets appear in overlapping fields of view, and the IDs of the same detected targets cannot be unified. Therefore, using two trackers can solve the problem of Hungarian matching errors.

[0046] In step S130, the tracking results corresponding to the two trackers are input into a pre-trained target recognition model for identification to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-lens targets.

[0047] The target recognition model can be a recognition model used to identify the tracking results of different trackers to determine the tracking units belonging to the same detection target. It is worth noting that the target recognition model can be retrained with training data with cross-lens targets to improve the accuracy of the target recognition model in detecting cross-lens targets. The cross-lens target is a target that enters or leaves the overlapping field of view, such as Figure 3 As shown in , a target is a cross-lens target when it is in the overlapping field of view between camera3-rear-right and camera4-rear; or as Figure 5 As shown in the figure, when the target leaves the overlapping field of view and enters the independent field of view, it is also called a cross-mirror target.

[0048] In this embodiment, the tracking results corresponding to the two trackers can be input into a pre-trained target recognition model for recognition. The tracking results may include, but are not limited to, video frame sequences from six cameras, corresponding target detection results, and camera calibration parameters. The target recognition model can output corresponding human tracking results, vehicle tracking results, and corresponding attributes. It should be noted that after training, the target recognition model can associate tracking units belonging to the same detected target to avoid duplicate recognition.

[0049] In one embodiment, after determining the tracking units belonging to the same target, the method further includes:

[0050] Tracking units belonging to the same detection target are assigned the same identification information for association.

[0051] In this embodiment, tracking units belonging to the same target determined by the target recognition model can be assigned the same identification information to be associated. Figure 3 The pedestrian target shown can be assigned the same ID number to determine the association between different tracking units. In this way, by associating tracking units belonging to the same detection target through the same identification information, it is easy to identify the same detection target, thereby ensuring the effect of environmental perception.

[0052] Based on the foregoing embodiment, in one embodiment of the present application, retraining the target recognition model includes:

[0053] Obtain training data with cross-camera targets and a pre-trained basic recognition model;

[0054] The basic recognition model is retrained using the training data with cross-camera targets, so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras, thereby obtaining a target recognition model.

[0055] In this embodiment, the basic recognition model can be a recognition model trained using standard training data. In one example, the basic recognition model can be built and trained using the Yolov5 algorithm, supplemented by pre-labeled training samples and the KITTI dataset for basic training. Furthermore, by statistically analyzing and setting dynamic target sizes, the basic recognition model focuses only on pedestrians, cyclists, and vehicles.

[0056] After the basic recognition model training is completed, training data with cross-camera targets can be obtained to retrain the trained basic recognition model to improve the accuracy of the basic recognition model in recognizing and matching cross-camera targets. That is, after retraining, the target recognition model can accurately identify the same detection target in video frames taken by different cameras.

[0057] In one embodiment of the present application, a heterogeneous computing platform consisting of NVIDIA's JETSON Xavier dual-module processor and Infineon's TC297 functional safety processor can be used to build a hardware inference platform. The frame rate of each camera is 20FPS, and its main functional function interface is as follows: Figure 6 shown.

[0058] Based on the above embodiment, in one embodiment of the present application, obtaining training data with cross-camera targets includes:

[0059] Detect whether a detection target enters or leaves the overlapping field of view between the camera and the adjacent camera;

[0060] If so, the detection data corresponding to the current camera is stored as training data with cross-lens targets.

[0061] In this embodiment, a preliminary tracker can be pre-set to detect whether the detection target enters or leaves the overlapping field of view between the camera and the adjacent camera. If so, the detection data corresponding to the current camera can be automatically saved as training data with cross-camera targets. Specifically, for cross-camera targets, only the data within the camera's shooting range can be retained, and the out-of-bounds data can be discarded (e.g., Figure 7 As shown in FIG, there is no cross-mirror target in the rearview camera (i.e., the rear view angle in the figure). Therefore, training data with cross-mirror targets can be obtained without manual labeling, which improves the efficiency of training data acquisition.

[0062] In one embodiment of the present application, the detection data corresponding to the current camera is stored as training data with cross-camera targets, including:

[0063] Get the detection data corresponding to the current camera;

[0064] Detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold is stored as training data with cross-lens targets.

[0065] In this embodiment, when a cross-camera target is detected, the detection data corresponding to the current camera can be obtained and filtered. Specifically, the detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold can be stored as training data with a cross-camera target.

[0066] It should be understood that through the above screening, it can be ensured that the cross-lens target occupies a certain area in the video frame, thereby improving the effectiveness of the training data and avoiding the inability to identify and track the target because the cross-lens target is too small in the video frame, affecting the subsequent retraining effect.

[0067] It should be noted that the predetermined threshold value may be determined by those skilled in the art based on prior experience, and may be, for example, 0.5 or 0.6, etc. The above figures are merely illustrative examples and are not particularly limited thereto.

[0068] According to the cross-camera target tracking method for vehicles proposed in an embodiment of the present application, the video frames captured by each camera are obtained, and target detection is performed on the video frames to determine the detection target. In addition, the detection target in the video captured by the spaced cameras is tracked using the same tracker to generate multiple tracking units, thereby improving the accuracy of subsequent recognition. The tracking results corresponding to the two trackers are then input into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target. The target recognition model is retrained using training data with cross-camera targets, thereby improving the accuracy of recognition of the same detection target and ensuring the environmental perception effect.

[0069] Next, a device for quantitatively evaluating the expected functional safety of a vehicle according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0070] Figure 8 It is a block diagram of a cross-mirror target tracking device applied to a vehicle according to an embodiment of the present application.

[0071] Multiple cameras are set along the circumference of the vehicle, with the fields of view of adjacent cameras partially overlapping, and the fields of view of cameras set at intervals are independent of each other. Figure 8 As shown, the cross-mirror target tracking device applied to a vehicle includes:

[0072] The target detection module 810 is used to obtain the video frames captured by each camera and perform target detection on the video frames to determine the detection target;

[0073] The target tracking module 820 is configured to track the detected targets in the video frames captured by the spaced cameras using the same tracker to generate a plurality of tracking units;

[0074] The target matching module 830 is used to input the tracking results corresponding to the two trackers into a pre-trained target recognition model for identification to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-lens targets.

[0075] Optionally, after determining the tracking units belonging to the same target, the target matching module 830 is further configured to: assign the same identification information to the tracking units belonging to the same detected target for association.

[0076] Optionally, a model training module 840 is also included, and the model training module 840 is used to: obtain training data with cross-camera targets and a pre-trained basic recognition model; use the training data with cross-camera targets to retrain the basic recognition model so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras.

[0077] Optionally, the model training module 840 is used to: detect whether a detection target enters or leaves the overlapping field of view between a camera and an adjacent camera; if so, store the detection data corresponding to the current camera as training data with cross-camera targets.

[0078] Optionally, the model training module 840 is used to: obtain detection data corresponding to the current camera; and store detection data whose ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold as training data with cross-camera targets.

[0079] It should be noted that the above explanation of the embodiment of the cross-mirror target tracking method applied to a vehicle is also applicable to the cross-mirror target tracking device applied to a vehicle in this embodiment, and will not be repeated here.

[0080] According to the cross-camera target tracking device for vehicles proposed in an embodiment of the present application, the video frames captured by each camera are obtained, and target detection is performed on the video frames to determine the detection target. In addition, the detection target in the video captured by the spaced cameras is tracked using the same tracker to generate multiple tracking units, thereby improving the accuracy of subsequent recognition. The tracking results corresponding to the two trackers are then input into a pre-trained target recognition model for recognition to determine the tracking units belonging to the same detection target. The target recognition model is retrained using training data with cross-camera targets, thereby improving the accuracy of recognition of the same detection target and ensuring the environmental perception effect.

[0081] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0082] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0083] When the processor 902 executes the program, the cross-mirror target tracking method applied to a vehicle provided in the above embodiment is implemented.

[0084] Furthermore, the electronic device further includes:

[0085] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0086] The memory 901 is used to store computer programs that can be run on the processor 902 .

[0087] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0088] If the memory 901, processor 902, and communication interface 903 are implemented independently, the communication interface 903, memory 901, and processor 902 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0089] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0090] The processor 902 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0091] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned cross-mirror target tracking method applied to a vehicle.

[0092] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0093] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0094] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0095] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0096] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0097] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0098] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0099] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A cross-mirror target tracking method for a vehicle, characterized in that: Multiple cameras are arranged along the circumference of the vehicle, with the fields of view of adjacent cameras partially overlapping, and the fields of view of cameras arranged at intervals being independent of each other; The method comprises: Obtaining video frames captured by each camera, and performing target detection on the video frames to determine the detection target; Tracking the detection targets in the video frames captured by the spaced cameras using the same tracker to generate a plurality of tracking units; The tracking results corresponding to the two trackers are input into a pre-trained target recognition model for identification to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-lens targets.

2. The method according to claim 1, characterized in that After determining the tracking units belonging to the same target, the method further includes: Tracking units belonging to the same detection target are assigned the same identification information for association.

3. The method according to claim 1 or 2, characterized in that Retraining the object recognition model includes: Obtain training data with cross-camera targets and a pre-trained basic recognition model; The basic recognition model is retrained using the training data with cross-camera targets, so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras.

4. The method according to claim 3, characterized in that Obtain training data with cross-camera targets, including: Detect whether a detection target enters or leaves the overlapping field of view between the camera and the adjacent camera; If so, the detection data corresponding to the current camera is stored as training data with cross-lens targets.

5. The method according to claim 4, characterized in that Store the detection data corresponding to the current camera as training data with cross-camera targets, including: Get the detection data corresponding to the current camera; Detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold is stored as training data with cross-lens targets.

6. A cross-mirror target tracking device for a vehicle, characterized in that: Multiple cameras are arranged along the circumference of the vehicle, with the fields of view of adjacent cameras partially overlapping, and the fields of view of cameras arranged at intervals being independent of each other; The device comprises: The target detection module is used to obtain the video frames captured by each camera and perform target detection on the video frames to determine the detection target; A target tracking module is used to track the detection targets in the video frames captured by the spaced cameras using the same tracker to generate a plurality of tracking units; A target matching module is used to input the tracking results corresponding to the two trackers into a pre-trained target recognition model for identification to determine the tracking units belonging to the same detection target, wherein the target recognition model is retrained by training data with cross-lens targets.

7. The device according to claim 6, characterized in that After determining the tracking units belonging to the same target, the target matching module is further configured to: Tracking units belonging to the same detection target are assigned the same identification information for association.

8. The device according to claim 6 or 7, characterized in that It also includes a model training module, which is used to: Obtain training data with cross-camera targets and a pre-trained basic recognition model; The basic recognition model is retrained using the training data with cross-camera targets, so that the basic recognition model can accurately identify the same detection target in video frames taken by different cameras.

9. The device according to claim 8, characterized in that The model training module is used to: Detect whether a detection target enters or leaves the overlapping field of view between the camera and the adjacent camera; If so, the detection data corresponding to the current camera is stored as training data with cross-lens targets.

10. The device according to claim 9, characterized in that The model training module is used to: Get the detection data corresponding to the current camera; Detection data in which the ratio of the size of the actual detection frame corresponding to the detection target to the size of the expected detection frame reaches a predetermined threshold is stored as training data with cross-lens targets.

11. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cross-mirror target tracking method for a vehicle as described in any one of claims 1 to 5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the cross-mirror target tracking method applied to a vehicle as described in any one of claims 1 to 5.

Images