Target tracking method and abnormal behavior early warning method based on multi-camera cooperation

Abstract

The invention discloses a target tracking method and an abnormal behavior early warning method based on multi-camera cooperation, and relates to the technical field of video monitoring. A plurality of cameras and a laser radar are arranged, and the laser radar is used as a coordinate system to establish a target database; the plurality of cameras are used for identifying building feature data and targets in shot image data according to the target database, and the plurality of cameras are used for transmitting the identified building feature data to the image processing module. The data of the binocular camera and the laser radar can be temporarily stored by using the temporary storage unit, and when the main acquisition unit and the auxiliary acquisition unit are switched, the data in the temporary storage unit can be conveniently retrieved to supplement the target data in the ID tracking area, so that the target data is perfected, and the tracking efficiency is improved. And the data accuracy in the transition time of target tracking and camera switching is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and specifically to a target tracking method and an abnormal behavior warning method based on multi-camera collaboration. Background Art

[0002] With the rapid development of intelligent surveillance technology, target tracking and abnormal behavior warning based on multi-camera collaboration have become research hotspots in the fields of computer vision and security. Traditional single-camera systems have deficiencies in terms of perspective coverage, occlusion handling, and device switching efficiency. In the existing technology, multi-camera collaboration often leads to tracking interruption due to inaccurate data synchronization, and lacks a dynamic complementary mechanism between lidar and cameras, resulting in low reliability of abnormal behavior analysis. Therefore, we propose a target tracking method and an abnormal behavior warning method based on multi-camera collaboration. Summary of the Invention

[0003] The purpose of the present invention is to provide a target tracking method and an abnormal behavior warning method based on multi-camera collaboration.

[0004] To achieve the above purpose, the present invention provides the following technical solutions: A target tracking method based on multi-camera collaboration, the specific steps are as follows:

[0005] Step 1: Set multiple binocular cameras and lidar, initialize the lidar coordinate system, start the lidar to perform a global scan of the scene, obtain initial building coordinate data and feature points, and establish a target database based on the initial scan data as the reference coordinate system for multi-camera collaboration;

[0006] Step 2: Multiple cameras transmit the recognized building feature data to the image processing module, and the image processing module matches the distance between the building feature data in the image data and the cameras according to the monocular ranging method;

[0007] Step 3: After Step 2 is completed, start the lidar to scan the surrounding targets to obtain target data, building feature data, and building coordinate data;

[0008] Step 4: Use the SIFT feature point extraction and RANSAC algorithm for robust matching, match the building feature data scanned by the lidar with the data recognized by the cameras, add time stamps to the lidar and camera data, calculate the relative pose between the camera and the lidar by the monocular ranging method, and establish a transformation matrix;

[0009] Step 5: Set an ID data processing module. After the camera recognizes the target data, it transmits the target data to the ID data processing module. The ID data processing module establishes an ID tracking area based on the target data and sets the camera as the main acquisition unit for the target data;

[0010] Step 6: A matching unit is set in the ID data processing module. When the data is matched, the lidar and camera data with a timestamp difference less than the threshold are selected. When the timestamp differences between the camera data and the lidar data are both less than the threshold, the camera data is preferentially selected as the main acquisition unit, and the lidar is set as the secondary acquisition unit. The unmatched data is marked as to be verified and stored in the temporary storage unit for processing by the dynamic adjustment module.

[0011] Step 7: The temporary storage unit adopts a FIFO queue with a storage capacity upper limit of 1000 data items. When it is exceeded, the earliest data is automatically overwritten. When switching between the main acquisition unit and the secondary acquisition unit, the temporary data within the most recent 5 seconds is preferentially called, and the Kalman filter is combined to predict the target position.

[0012] As a further solution of the present invention: The multiple cameras are all binocular cameras, and the specific working steps of the binocular camera are as follows:

[0013] S1.1: The binocular camera acquires image data through the first monocular camera. The binocular camera consists of two coaxial 2-million-pixel CMOS sensors with a baseline distance of 12 cm and a frame rate synchronization error ≤ 1 ms. The first monocular camera real-time collects a wide-angle image, and the field of view angle of the first monocular camera is 120° FOV for rapid target detection. When a target is detected, the second monocular camera switches to a telephoto lens, and the field of view angle of the second monocular camera is 30° FOV for high-precision feature extraction.

[0014] S1.2: The image processing module obtains the target image and the building feature image according to the target database, and identifies the target image and the building feature image based on the image data obtained by the first monocular camera in the image processing module.

[0015] S1.3: After the image processing module identifies the building feature image, the distance and angle of the building feature data are calculated by the monocular ranging method.

[0016] S1.4: After the image processing module identifies the target image, the second monocular camera conducts a detailed identification of the target, obtains the detailed data of the target in the image data, and transmits the detailed data of the target to the ID data processing module.

[0017] As a further solution of the present invention, the specific steps of Step 4 are as follows:

[0018] S4.1: A pattern recognition unit is set, and the pattern recognition unit is used to recognize the transmission data of the lidar.

[0019] S4.2. When the pattern recognition unit determines that the building feature data of the lidar and the camera do not match, the lidar sends the scan data to the robustness processing algorithm. The robustness processing algorithm matches the identified building feature data of the lidar with the building feature data identified by the camera, and then the robustness processing algorithm analyzes the binocular camera coordinate system based on the building coordinate data scanned by the lidar and the monocular ranging method, calculates the relative pose between the camera and the lidar through the monocular ranging method, establishes a transformation matrix, and further analyzes the tree diagram of the lidar and the camera according to the transformation matrix;

[0020] S4.3. After the pattern recognition unit recognizes that the building feature data of the lidar matches the building feature data identified by the camera, the lidar transmits the scanned target data to Step 5.

[0021] As a further solution of the present invention, the specific steps of Step 5 are as follows:

[0022] S5.1. After the building feature data of the lidar matches the building feature data identified by the camera, the second monocular camera transmits the identified target detailed data to the ID tracking area;

[0023] S5.2. The ID tracking area sets the current binocular camera as the main acquisition unit.

[0024] As a further solution of the present invention, the specific steps of Step 6 are as follows:

[0025] S6.1. Set up a temporary storage unit, which is used to store the target data of the lidar and the target data of the binocular camera;

[0026] S6.2. When the ID tracking area receives the target data transmitted by the binocular camera as the main acquisition unit, the lidar stores the data that duplicates the target data into the temporary storage unit.

[0027] As a further solution of the present invention, the specific steps of Step 7 are as follows:

[0028] S7.1. Set up a path analysis unit, and use the path analysis unit to identify the target movement path to predict the target movement path;

[0029] S7.2. When it is predicted that the target movement path moves out of the monitoring range of the current binocular camera, the relationship between the main acquisition unit and the secondary acquisition unit between the binocular camera and the lidar is adjusted. When switching the relationship between the main acquisition unit and the secondary acquisition unit, the stored data in the temporary storage unit is retrieved to supplement the target data in the ID tracking area;

[0030] S7.3. Set up a dynamic adjustment module. Use the dynamic adjustment module to retrieve the tree diagram and analyze the next monitoring range that the target movement path moves to. Analyze the time value generated when reaching the next monitoring range. The time value is obtained through a formula. The specific formula is:

[0031]

[0032] Among them, T represents the estimated arrival value, d represents the remaining path distance obtained according to historical records, and u represents the current speed of the target.

[0033] S7.4. The dynamic adjustment module switches the identities of the main acquisition unit and the secondary acquisition unit of the binocular camera and the lidar according to the time value. Then, when the lidar identifies that the target reaches the corresponding time, it switches to the binocular camera in the next monitoring range as the main acquisition unit, and then rematches the target data scanned by the lidar with the target data captured by the binocular camera.

[0034] In addition, the present invention also provides an abnormal behavior warning method based on multi-camera collaboration, which is characterized by including the following steps:

[0035] S100. Build an action state recognition library and use the action state recognition library to obtain target data in the ID tracking area.

[0036] S200. Set up an abnormal behavior judgment module and a depth analysis module. When the abnormal behavior judgment module obtains an abnormal behavior of the target data, it classifies the abnormal behavior. The specific abnormal classifications are speed greater than the normal threshold, trajectory deviating from the normal path, and morphological mutation. Then, the depth analysis module retrieves the target feature data in the ID tracking area and the data in the temporary storage unit for analysis. The depth analysis module uses an LSTM network to predict the target trajectory, calculates the Hausdorff distance between the actual trajectory and the predicted trajectory, and triggers an alarm if the deviation is greater than the abnormal threshold. Analyze the target posture through the OpenPose skeleton model to detect abnormal actions.

[0037] S300. Set up an alarm module, and the alarm module issues an alarm according to the analysis of abnormal actions and abnormal thresholds by the depth analysis module.

[0038] As a further solution of the present invention, the specific steps in S200 are as follows:

[0039] S210. The depth analysis module retrieves the target data scanned by the lidar and the target data of the binocular camera for analyzing the differential features.

[0040] S220. The depth analysis module analyzes the target data missing from the main acquisition unit, and the secondary acquisition unit supplements the missing target features to improve the target data.

[0041] With the above technical solution, compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. By setting up a temporary storage unit, the present invention facilitates the temporary storage of data from the binocular camera and the lidar using the temporary storage unit. When switching between the main acquisition unit and the secondary acquisition unit, it is convenient to retrieve the data in the temporary storage unit to supplement the target data in the ID tracking area, thereby improving the target data and enhancing the data accuracy during the camera switching period for target tracking.

[0043] 2. The present invention enables the binocular camera and the lidar to cooperate with each other, and then analyzes the positions between the binocular camera and the lidar based on the building feature data and the building coordinate data. Subsequently, it is convenient for the binocular camera and the lidar to cooperate to obtain target data, thereby obtaining more accurate target data and improving the accuracy of the subsequent predicted target tracking path.

[0044] 3. The present invention improves the accuracy of the depth analysis module and reduces the possibility of subsequent abnormal target behaviors by completing the missing parts of the target according to the data in the temporary storage unit and filling in the occluded parts collected by the main acquisition unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Schematic diagram of the specific steps of target tracking in the embodiment of the present invention;

[0046] Figure 2 Schematic diagram of the steps of the temporary storage unit in the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0047] The following further describes the specific embodiments of the present invention with reference to the accompanying drawings. It should be noted here that the description of these embodiments is for helping to understand the present invention, but does not limit the present invention.

[0048] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Please refer to the attached Figure 1 - attached Figure 2 , for the target tracking method and abnormal behavior warning method based on multi-camera collaboration of the present invention, the specific steps are as follows:

[0050] Step 1: Set up multiple binocular cameras and lidars, initialize the lidar coordinate system, start the lidar to perform a global scan of the scene, obtain the initial building coordinate data and feature points, and establish a target database based on the initial scan data as the reference coordinate system for multi-camera collaboration;

[0051] Step 2: Multiple cameras transmit the recognized building feature data to the image processing module, and the image processing module matches the distance between the building feature data in the image data and the cameras according to the monocular ranging method;

[0052] Step 3: After Step 2 is completed, start the lidar to scan the surrounding targets to obtain target data, building feature data, and building coordinate data;

[0053] Step 4: Use the SIFT feature point extraction and RANSAC algorithms for robust matching. Match the building feature data scanned by the lidar with the camera recognition data. Add timestamps to the lidar and camera data, calculate the relative pose between the camera and the lidar through the monocular ranging method, and establish a transformation matrix;

[0054] Step 5: Set up an ID data processing module. After the camera recognizes the target data, it transmits the target data to the ID data processing module. The ID data processing module establishes an ID tracking area based on the target data and sets this camera as the main acquisition unit for the target data;

[0055] Step 6: Set up a matching unit in the ID data processing module. When the matching unit performs data matching, it selects the lidar and camera data with a timestamp difference less than the threshold. When the timestamp differences between the camera data and the lidar data are both less than the threshold, the camera data is preferentially selected as the main acquisition unit, and the lidar is set as the secondary acquisition unit. The unmatched data is marked as to be verified and stored in the temporary storage unit for processing by the dynamic adjustment module;

[0056] Step 7: The temporary storage unit uses a FIFO queue with a storage capacity upper limit of 1000 data items. When it is exceeded, the earliest data is automatically overwritten. When switching the main acquisition unit and the secondary acquisition unit, the temporary data within the last 5 seconds is preferentially called, and the Kalman filter is combined to predict the target position.

[0057] In an embodiment of the present invention: Multiple cameras are all binocular cameras, and the specific working steps of the binocular cameras are as follows:

[0058] S1.1: The binocular camera obtains image data through the first monocular camera. The binocular camera consists of two coaxial 2-million-pixel CMOS sensors with a baseline distance of 12 cm and a frame rate synchronization error ≤ 1 ms. The first monocular camera real-time collects a wide-angle image, and the field of view angle of the first monocular camera is 120° FOV for rapid target detection. When a target is detected, the second monocular camera switches to a telephoto lens, and the field of view angle of the second monocular camera is 30° FOV for high-precision feature extraction;

[0059] S1.2. The image processing module obtains the target image and the building feature image according to the target database. When the image processing module identifies the target image and the building feature image based on the image data obtained by the first monocular camera;

[0060] S1.3. After the image processing module identifies the building feature image, calculate the distance and angle of the building feature data by the monocular ranging method;

[0061] S1.4. After the image processing module identifies the target image, the second monocular camera performs detailed identification on the target, obtains the detailed data of the target in the image data, and transmits the detailed data of the target to the ID data processing module.

[0062] In an embodiment of the present invention, the specific steps of step four are as follows:

[0063] S4.1. Set up a pattern recognition unit, and the pattern recognition unit is used to identify the transmission data of the lidar;

[0064] S4.2. When the pattern recognition unit determines that the building feature data of the lidar and the camera do not match, the lidar sends the scan data to the robustness processing algorithm. The robustness processing algorithm matches the identified building feature data of the lidar with the building feature data identified by the camera, and then the robustness processing algorithm analyzes the binocular camera coordinate system according to the building coordinate data scanned by the lidar and the monocular ranging method, calculates the relative pose between the camera and the lidar by the monocular ranging method, establishes a transformation matrix, and further analyzes the tree diagram of the lidar and the camera according to the transformation matrix;

[0065] S4.3. After the pattern recognition unit identifies that the building feature data of the lidar matches the building feature data identified by the camera, the lidar transmits the scanned target data to step five.

[0066] In an embodiment of the present invention, the specific steps of step five are as follows:

[0067] S5.1. After the building feature data of the lidar matches the building feature data identified by the camera, the second monocular camera transmits the identified detailed target data to the ID tracking area;

[0068] S5.2. The ID tracking area sets the current binocular camera as the main acquisition unit.

[0069] In an embodiment of the present invention, the specific steps of step six are as follows:

[0070] S6.1. Set up a temporary storage unit, and the temporary storage unit is used to store the target data of the lidar and the target data of the binocular camera;

[0071] S6.2. When the ID tracking area receives the target data transmitted by the binocular camera as the main acquisition unit, the lidar stores the data that duplicates the target data in the temporary storage unit.

[0072] In an embodiment of the present invention, the specific steps of step seven are as follows:

[0073] S7.1. Set up a path analysis unit, and use the path analysis unit to identify the target movement path to predict the target movement path;

[0074] S7.2. When it is predicted that the target movement path moves out of the monitoring range of the current binocular camera, the relationship between the main acquisition unit and the secondary acquisition unit between the binocular camera and the lidar is adjusted. When switching the relationship between the main acquisition unit and the secondary acquisition unit, the stored data in the temporary storage unit is retrieved to supplement the target data in the ID tracking area;

[0075] S7.3. Set up a dynamic adjustment module, use the dynamic adjustment module to retrieve the tree diagram, and analyze the next monitoring range that the target movement path moves to, and analyze the time value generated when reaching the next monitoring range. The time value is obtained through a formula. The specific formula is:

[0076]

[0077] Among them, T represents the estimated arrival value, d represents the remaining path distance obtained according to historical records, and u represents the current speed of the target;

[0078] S7.4. The dynamic adjustment module switches the identities of the main acquisition unit and the secondary acquisition unit of the binocular camera and the lidar according to the time value. Then, when the lidar recognizes that the target arrives at the corresponding time, it switches to the binocular camera in the next monitoring range as the main acquisition unit, and then rematches the target data scanned by the lidar with the target data captured by the binocular camera.

[0079] In addition, the present invention also provides an abnormal behavior warning method based on multi-camera collaboration, which is characterized by including the following steps:

[0080] S100. Build an action state recognition library, and use the action state recognition library to obtain the target data in the ID tracking area;

[0081] S200. Set up an abnormal behavior judgment module and a deep analysis module. When the abnormal behavior judgment module detects an abnormal behavior in the target data, it classifies the abnormal behavior. The specific abnormal classifications are speed greater than the normal threshold, trajectory deviation from the normal path, and morphological mutation. Then, the deep analysis module retrieves the target feature data in the ID tracking area and the data in the temporary storage unit for analysis. The deep analysis module uses an LSTM network to predict the target trajectory, calculates the Hausdorff distance between the actual trajectory and the predicted trajectory, and triggers an alarm if the deviation is greater than the abnormal threshold. It analyzes the target posture through the OpenPose skeleton model to detect abnormal actions;

[0082] S300. Set up an alarm module. The alarm module issues an alarm based on the analysis of abnormal actions by the deep analysis module and the abnormal threshold.

[0083] In an embodiment of the present invention, the specific steps in S200 are as follows:

[0084] S210. The deep analysis module retrieves the target data scanned by the lidar and the target data of the binocular camera to analyze the differential features;

[0085] S220. The deep analysis module analyzes the target data missing from the main acquisition unit, and the auxiliary acquisition unit supplements the missing target features to improve the target data.

[0086] Example 1. Please refer to the appendix Figure 1 , the deep analysis module obtains the target data through a formula. The specific formula is as follows:

[0087] S = α·T + β·R + W1 + W2

[0088] Where T and R are the target data (such as 3D coordinates, RGB features) in the overlapping area of the lidar and the binocular camera, α and β are dynamic weight coefficients, adaptively adjusted through the covariance matrix, and W1 and W2 are supplementary data in the non-overlapping area (such as the reflection intensity of the lidar, the texture information of the camera).

[0089] Example 2. Please refer to the appendix Figure 2 , the monocular ranging method calculates the distance through ORB feature point matching and triangulation;

[0090] The robustness processing algorithm uses Kalman filtering to fuse lidar and camera data to update the target state matrix.

[0091] Example 3. Please refer to the appendix Figure 2 , when the ID tracking area obtains the data in the temporary storage unit, it obtains the data in the temporary storage unit through the timestamp and compensates for the data during the switching process of the temporary storage unit.

[0092] Specifically, through the cooperation of the binocular camera and the lidar, the position between the binocular camera and the lidar is analyzed based on the building feature data and the building coordinate data, and then it is convenient for the binocular camera and the lidar to cooperate with each other to obtain the target data, so as to obtain more accurate target data, and improve the accuracy of the target tracking path for subsequent prediction.

[0093] Specifically, by setting up a temporary storage unit, it is convenient to temporarily store the data of the binocular camera and the lidar using the temporary storage unit. When switching between the main acquisition unit and the secondary acquisition unit, it is convenient to retrieve the data in the temporary storage unit to supplement the target data in the ID tracking area, thereby improving the target data and enhancing the data accuracy during the camera switching period for target tracking.

[0094] Specifically, by improving the missing parts of the target according to the data in the temporary storage unit and filling in the occluded parts collected by the main acquisition unit, the accuracy of the depth analysis module is improved, and the possibility of subsequent abnormal target behavior is reduced.

[0095] Although the present invention is disclosed above in a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modification, equivalent change and decoration made to the above embodiment based on the technical essence of the present invention without departing from the technical solution of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. A target tracking method based on multi-camera collaboration, characterized in that The specific steps are as follows: Step 1: Set up multiple binocular cameras and lidar, initialize the lidar coordinate system, start the lidar to perform a global scan of the scene, obtain initial building coordinate data and feature points, and establish a target database based on the initial scan data as the reference coordinate system for multi-camera collaboration; Step 2: Multiple cameras transmit the recognized building feature data to the image processing module, and the image processing module matches the distance between the building feature data in the image data and the camera according to the monocular ranging method; Step 3: After Step 2 is completed, start the lidar to scan the surrounding targets to obtain target data, building feature data, and building coordinate data; Step 4: Use the SIFT feature point extraction and RANSAC algorithms for robust matching, match the building feature data scanned by the lidar with the camera recognition data, add time stamps to the lidar and camera data, calculate the relative pose between the camera and the lidar through the monocular ranging method, and establish a transformation matrix; Step 5: Set up an ID data processing module. After the camera recognizes the target data, it transmits the target data to the ID data processing module. The ID data processing module establishes an ID tracking area based on the target data and sets the camera as the main acquisition unit for the target data; Step 6: Set up a matching unit in the ID data processing module. When the matching unit performs data matching, it selects the lidar and camera data with a time stamp difference less than the threshold. When the time stamp differences between the camera data and the lidar data are both less than the threshold, the camera data is preferentially selected as the main acquisition unit, and the lidar is set as the secondary acquisition unit. The unmatched data is marked as to be verified and stored in the temporary storage unit for processing by the dynamic adjustment module; Step 7: The temporary storage unit uses a FIFO queue with a storage capacity upper limit of 1000 pieces of data. When it is exceeded, the earliest data is automatically overwritten. When switching between the main acquisition unit and the secondary acquisition unit, the temporary data within the last 5 seconds is preferentially called, and the Kalman filter is combined to predict the target position.

2. The object tracking method based on multi-camera collaboration according to claim 1, wherein: The multiple cameras are all binocular cameras, and the specific working steps of the binocular camera are as follows: S1.1: The binocular camera obtains image data through the first monocular camera. The binocular camera consists of two coaxial 2 million pixel CMOS sensors with a baseline distance of 12 cm and a frame rate synchronization error ≤ 1 ms. The first monocular camera real-time collects a wide-angle image, and the field of view angle of the first monocular camera is 120° FOV for rapid target detection. When a target is detected, the second monocular camera switches to a telephoto lens, and the field of view angle of the second monocular camera is 30° FOV for high-precision feature extraction; S1.2: The image processing module obtains the target image and the building feature image according to the target database, and the image processing module identifies the target image and the building feature image according to the image data obtained by the first monocular camera; S1.3: After the image processing module recognizes the building feature image, it calculates the distance and angle of the building feature data through the monocular ranging method; S1.

4. After the image processing module recognizes the target image, the second monocular camera performs a detailed recognition of the target, obtains the detailed data of the target in the image data, and transmits the detailed data of the target to the ID data processing module.

3. The object tracking method based on multi-camera collaboration according to claim 2, characterized in that, The specific steps of Step 4 are as follows: S4.

1. Set up a pattern recognition unit, which is used to recognize the transmission data of the lidar. S4.

2. When the pattern recognition unit determines that the building feature data of the lidar and the camera do not match, the lidar sends the scan data to the robustness processing algorithm. The robustness processing algorithm matches the recognized building feature data of the lidar with the building feature data recognized by the camera, and then the robustness processing algorithm analyzes the binocular camera coordinate system based on the building coordinate data scanned by the lidar and the monocular ranging method, calculates the relative pose between the camera and the lidar through the monocular ranging method, establishes a transformation matrix, and then analyzes the tree diagram of the lidar and the camera according to the transformation matrix. S4.

3. When the pattern recognition unit recognizes that the building feature data of the lidar matches the building feature data recognized by the camera, the lidar transmits the scanned target data to Step 5.

4. A target tracking method based on multi-camera collaboration according to claim 3, characterized in that The specific steps of Step 5 are as follows: S5.

1. After the building feature data of the lidar matches the building feature data recognized by the camera, the second monocular camera transmits the recognized detailed target data to the ID tracking area. S5.

2. The ID tracking area sets the current binocular camera as the main acquisition unit.

5. The object tracking method based on multi-camera collaboration according to claim 4, characterized in that, The specific steps of Step 6 are as follows: S6.

1. Set up a temporary storage unit, which is used to store the target data of the lidar and the target data of the binocular camera. S6.

2. When the ID tracking area receives the target data transmitted by the binocular camera as the main acquisition unit, the lidar stores the data that duplicates the target data into the temporary storage unit.

6. A target tracking method based on multi-camera collaboration according to claim 5, characterized in that, The specific steps of Step 7 are as follows: S7.

1. Set up a path analysis unit to identify the target movement path using the path analysis unit to predict the target movement path. S7.

2. When it is predicted that the target movement path moves out of the monitoring range of the current binocular camera, the relationship between the main acquisition unit and the secondary acquisition unit between the binocular camera and the lidar is adjusted. When switching the relationship between the main acquisition unit and the secondary acquisition unit, the stored data in the temporary storage unit is retrieved to supplement the target data in the ID tracking area. S7.

3. Set up a dynamic adjustment module to retrieve the tree diagram using the dynamic adjustment module, analyze the next monitoring range that the target movement path moves to, and analyze the time value generated when reaching the next monitoring range. The time value is obtained through a formula. The specific formula is: Where T represents the estimated arrival value, d represents the remaining path distance obtained based on historical records, and u represents the current speed of the target. S7.

4. The dynamic adjustment module switches the identities of the main and secondary acquisition units of the binocular camera and the lidar according to the time value, and then, after the lidar identifies the target and reaches the corresponding time, switches to the binocular camera in the next monitoring range as the main acquisition unit, and then rematches the target data scanned by the lidar with the target data captured by the binocular camera.

7. An abnormal behavior early warning method based on multi-camera collaboration, characterized in that, Applicable to the target tracking method described in any one of claims 1-6, comprising the following steps: S100. Construct an action state recognition library, and use the action state recognition library to obtain target data in the ID tracking area; S200. Set an abnormal behavior judgment module and a depth analysis module. When the abnormal behavior judgment module obtains an abnormal behavior of the target data, it classifies the abnormal behavior. The specific abnormal classifications are speed greater than the normal threshold, trajectory deviation from the normal path, and morphological mutation. Then the depth analysis module retrieves the target feature data in the ID tracking area and the data in the temporary storage unit for analysis. The depth analysis module uses the LSTM network to predict the target trajectory, calculates the Hausdorff distance between the actual trajectory and the predicted trajectory, and triggers an alarm if the deviation is greater than the abnormal threshold. Analyze the target pose through the OpenPose skeleton model to detect abnormal actions; S300. Set an alarm module, and the alarm module issues an alarm according to the analysis of the abnormal actions and the abnormal threshold by the depth analysis module.

8. The abnormal behavior early warning method based on multi-camera collaboration according to claim 7, characterized in that, The specific steps in S200 are as follows: S210. The depth analysis module retrieves the target data scanned by the lidar and the target data of the binocular camera for analyzing the differential features; S220. The depth analysis module analyzes the target data missing from the main acquisition unit, and the secondary acquisition unit supplements the missing target features to improve the target data.

Images