Human shape tracking system and method based on dual-channel target detection

Abstract

The invention relates to a human shape tracking system and method based on dual-channel target detection, and the method achieves the relay tracking on a single-chip dual-sensor (one path of gun camera and one path of dome camera) system, optimizes a human shape tracking algorithm, and enables the gun camera to continuously pay attention to an area in which a user is interested. A ball machine pays attention to a point target, two cameras are used for covering a monitoring area from different angles and positions, and missed tracking caused by view angle limitation of a single camera is reduced. When the target disappears from the visual field of one camera, the system can quickly switch to the other camera for tracking, and it is ensured that the target is not missed. Meanwhile, the false alarm rate is optimized through the human shape tracking module. The problem that a traditional camera easily recognizes a vehicle as a person and carries out false alarm is solved. And the false drop rate is reduced through filtering of the comprehensive mass score. Meanwhile, a framework is built, and a more convenient scheme is provided for filtering misinformation targets which are not interested in by the user. According to the method, the difficulty is low, the calculation amount is small, the requirement for hardware resources is reduced, and then the equipment cost, the operation cost and the maintenance cost are remarkably reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of human figure tracking, and in particular to a human figure tracking system and method based on dual-channel target detection. Background Art

[0002] With the development and progress of science and technology, the security industry has developed rapidly. Smart surveillance cameras have fully penetrated into human life. Surveillance cameras can be seen everywhere in people's clothing, food, housing, transportation, entertainment, health, education and other aspects. With the development of computer vision and artificial intelligence technology, human tracking technology, as one of the core technologies, has received widespread attention in the smart surveillance industry. Although this technology has made significant progress, there are still many defects and shortcomings.

[0003] Traditional "human tracking technology" has the following defects and deficiencies in security equipment:

[0004] 1. Mistracking

[0005] Mistracking means that the monitoring system mistakenly identifies a non-target object as a target object, or mistakenly tracks the wrong target object. Specifically, it identifies the wrong target such as a moving bird or a moving car in the video as a human figure and tracks it. Frequent mistracking leads to a decrease in user trust in the system, affecting the user experience.

[0006] 2. Missed tracking

[0007] Missed tracking refers to the monitoring system failing to successfully track the target object, resulting in the target not being detected and tracked within a certain period of time in the monitoring area. In the process of human tracking, the system frequently fails to successfully track the target object. The main reasons are computing resource limitations and camera angle and position issues. Missed tracking may lead to the loss of important monitoring information and reduce the monitoring effect of the system.

[0008] 3. High equipment cost

[0009] The technology required for a humanoid tracking system is difficult and complex, and has high hardware requirements. The tracking algorithm requires high performance and high resolution; it consumes a lot of computing resources; and the maintenance personnel must have professional technical capabilities. This increases the equipment cost, maintenance cost, and operating cost. The high cost makes it difficult for small and medium-sized enterprises and individuals to afford it, which affects the promotion and application of the technology. Summary of the invention

[0010] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a humanoid tracking system and method based on dual-channel target detection, which realizes relay tracking on a single-chip dual-sensor system (one gun camera and one ball camera) and optimizes the humanoid tracking algorithm. The gun camera can continuously focus on the area of ​​interest to the user; the ball camera can focus on key targets and continuously track. At the same time, the present invention optimizes the humanoid tracking module so that it does not increase the computing power resources occupied, and optimizes the humanoid tracking service.

[0011] The present invention solves the technical problem by adopting the following technical solutions:

[0012] A humanoid tracking system based on dual-channel target detection includes a gun camera, a ball camera, an algorithm module, a humanoid tracking module, an alarm module, a tracking module and a ball control module, wherein the output ends of the gun camera and the ball camera are respectively connected to the input end of the algorithm module, the output end of the algorithm module is connected to the input end of the humanoid tracking module, the output end of the humanoid tracking module is respectively connected to the input end of the alarm module and the tracking module, the output end of the alarm module is respectively connected to the input end of the humanoid tracking module and the tracking module, the output end of the tracking module is respectively connected to the ball control module and the humanoid tracking module, wherein:

[0013] The algorithm module is used to process each frame of image captured by the ball camera and the gun camera and output the target trajectory information;

[0014] The human tracking module is used to receive the target trajectory information output by the algorithm and the parameter configuration of the user; through internal processing, it decides the alarm to be reported and the corresponding detection target information;

[0015] The alarm module is used to obtain the alarm information from the human tracking module and report it; at the same time, it notifies the tracking module to start tracking;

[0016] The tracking module is used for relay tracking, which performs PTZ coordinate conversion and dispatches the ball control module to control the movement of the ball machine;

[0017] The ball control module is used to receive instructions from the tracking module and control the movement of the ball camera.

[0018] Moreover, the target trajectory information output by the algorithm module includes target motion state, quality score, target ID, target position and target characteristic value; the user's parameter configuration includes area and sensitivity, and the ball control module controls the movement of the ball camera including starting ball control, ball camera rotation, ball camera stationary and stopping ball control.

[0019] A tracking method of a humanoid tracking system based on dual-channel target detection comprises the following steps:

[0020] Step 1: The gun camera and ball camera collect image information in real time;

[0021] Step 2: The algorithm module processes each frame of the image collected by the ball camera and the gun camera, and outputs the target trajectory information;

[0022] Step 3: The humanoid tracking module receives the target trajectory information output by the algorithm and receives the user's parameter configuration; through internal processing, it decides the alarm to be reported and the corresponding detection target information;

[0023] Step 4: The alarm module obtains the alarm information from the human tracking module and reports it; at the same time, it notifies the tracking module to start tracking;

[0024] Step 5: The tracking module performs relay tracking by performing PTZ coordinate conversion and dispatching the ball control module to control the movement of the ball camera;

[0025] Step 6: The ball control module receives the instruction from the tracking module and controls the movement of the ball camera.

[0026] Furthermore, the step 3 comprises the following steps:

[0027] Step 3.1, the humanoid tracking module receives the target trajectory information and obtains the target trajectory detection result according to the user's parameter configuration;

[0028] Step 3.2, the human tracking module updates the alarm status according to the target trajectory detection result;

[0029] Step 3.3, the humanoid tracking module selects the tracking target and searches according to the user's setting information;

[0030] Step 3.4: Update the tracking target trajectory of the tracking target selected in step 3.3.

[0031] Moreover, the specific implementation method of step 3.1 is: according to the user preference information, the targets are filtered hierarchically in turn, people, vehicles are selected, areas are selected and motion status is updated, and the target trajectory detection results are output. The target trajectory detection results include target type, target ID, target rectangular frame, target quality score, target motion, target stillness, current detection type and whether it is in the area.

[0032] Moreover, the specific implementation method of step 3.2 is: filtering targets that do not meet the alarm requirements, including removing targets outside the area, removing targets with poor quality scores, and writing the alarm type into the global variable. At the same time, if tracking is turned on, the alarm condition and tracking type remain unchanged.

[0033] Furthermore, the step 4 comprises the following steps:

[0034] Step 4.1, polling two channels and writing the sorted tracking status information into the shared memory, wherein the tracking status information includes the start of tracking, tracking in progress, end of tracking, tracking channel and coordinates of the tracking target;

[0035] Step 4.2, the alarm module receives the alarm information and notifies the tracking module to enable tracking; if the alarm module fails to receive the alarm information for about 4-5 seconds, it notifies the tracking module to disable;

[0036] Step 4.3: When the tracking module is enabled, the tracking information is obtained from the human tracking optimization module of the media; if the target is found, the tracking is considered to have started; if the target is not found or the object remains stationary for about 15 seconds, the tracking is considered to have ended; in other cases, the tracking is considered to be in progress.

[0037] Furthermore, the step 5 comprises the following steps:

[0038] Step 5.1, the tracking module reads the tracking status from the shared memory;

[0039] Step 5.2, the tracking module controls the ball according to the ball control strategy;

[0040] Step 5.3: After the ball control operation is completed, the ball control information, including tracking channel information and tracking coordinate information, is recorded, and the process returns to step 5.1.

[0041] The advantages and positive effects of the present invention are:

[0042] The present invention realizes relay tracking on a single-chip dual-sensor system (one gun camera and one ball camera), optimizes the humanoid tracking algorithm, enables the gun camera to continuously focus on the area of ​​interest to the user; enables the ball camera to focus on key targets, and uses two cameras to cover the monitoring area from different angles and positions, thereby reducing missed tracking caused by a single camera due to the limitation of the viewing angle. When the target disappears from the field of view of one camera, the system can quickly switch to another camera for tracking to ensure that the target is not missed. At the same time, the present invention optimizes the false alarm rate through a humanoid tracking module. It solves the problem that traditional cameras easily identify cars as people and make false alarms. By filtering the comprehensive quality score, the false detection rate is reduced. At the same time, a framework is built to provide a more convenient solution for filtering false alarm targets that the user is not interested in. The present invention has low difficulty and small calculation amount, reduces the requirements for hardware resources, and thus significantly reduces equipment costs, operating costs, and maintenance costs. The system proposes a separate humanoid tracking module without increasing the occupation of algorithm resources. The humanoid tracking module decides whether to track, does not occupy computing resources, and improves the processing efficiency of the system. In business, we take advantage of the humanoid tracking module and combine it with the business layout to cleverly implement humanoid relay tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a system structure diagram of the present invention;

[0044] Figure 2 The present invention is a flow chart of the tracking state of the maintenance gun camera and ball camera;

[0045] Figure 3 It is a flow chart of the ball control strategy of the present invention;

[0046] Figure 4 This is a flow chart of the humanoid tracking optimization solution of the present invention;

[0047] Figure 5 The flow chart of obtaining target trajectory results of the present invention;

[0048] Figure 6 Determine the alarm situation flow chart for the present invention;

[0049] Figure 7 A flow chart for selecting a tracking target for the present invention;

[0050] Figure 8 Update flow chart for tracking trajectory of the present invention; DETAILED DESCRIPTION

[0051] The present invention is further described in detail below with reference to the accompanying drawings.

[0052] A human tracking system based on dual-channel target detection, such as Figure 1 As shown, it includes a gun camera, a ball camera, an algorithm module, a humanoid tracking module, an alarm module, a tracking module and a ball control module, wherein the output ends of the gun camera and the ball camera are respectively connected to the input end of the algorithm module, the output end of the algorithm module is connected to the input end of the humanoid tracking module, the output end of the humanoid tracking module is respectively connected to the input end of the alarm module and the tracking module, the output end of the alarm module is respectively connected to the input end of the humanoid tracking module and the tracking module, the output end of the tracking module is respectively connected to the ball control module and the humanoid tracking module,

[0053] Among them, the algorithm module is used to process each frame of image collected by the ball camera and the gun camera, and output the target trajectory information; including target motion status, quality score, target ID, target position, target characteristic value and other information.

[0054] The human tracking module is used to receive the target trajectory information output by the algorithm and receive the user's parameter configuration; such as area and sensitivity information, through internal processing, decide the alarm to be reported and the corresponding detection target information; this module processes the detection target to achieve the goal of outputting a stable tracking target.

[0055] The alarm module is used to obtain the alarm information from the human tracking module and report it; at the same time, it notifies the tracking module to start tracking; if the alarm information cannot be obtained continuously, it notifies the tracking module to stop tracking.

[0056] The tracking module is used for relay tracking, which performs PTZ coordinate conversion and dispatches the ball control module to control the movement of the ball machine;

[0057] The ball control module is used to receive the instructions from the tracking module and control the movement of the ball camera, including starting ball control, ball camera rotation, ball camera stillness, and stopping ball control.

[0058] A tracking method of a humanoid tracking system based on dual-channel target detection comprises the following steps:

[0059] Step 1: The gun camera and ball camera collect image information in real time.

[0060] Step 2: The algorithm module processes each frame of the image captured by the ball camera and the gun camera, and outputs the target trajectory information.

[0061] Step 3: The humanoid tracking module receives the target trajectory information output by the algorithm and receives the user's parameter configuration; through internal processing, it decides the alarm to be reported and the corresponding detection target information.

[0062] Based on the target trajectory supported by the algorithm, a human tracking module is added to the media layer to optimize human tracking. When detecting targets, human information is filtered based on the area, target type, confidence, etc., and the alarm situation in the area is reported. The tracking target is obtained from the face tracking module instead of the algorithm. The face tracking module selects the tracking target based on user preferences. And the trajectory information of the tracking target is maintained. Figure 4 As shown, step 3 includes the following steps:

[0063] Step 3.1: The humanoid tracking module receives the target trajectory information and obtains the target trajectory detection result according to the user's parameter configuration.

[0064] like Figure 5 As shown, according to the user preference information (the user's selection method in the IE interface, such as area and sensitivity), the targets are filtered hierarchically, and people, vehicles, areas and motion states are selected and updated, and the target trajectory detection results are output. The target trajectory detection results include target type, target ID, target rectangle, target quality score, target motion, target stillness, current detection type and whether it is in the area.

[0065] Step 3.2: The humanoid tracking module updates the alarm status according to the target trajectory detection result.

[0066] like Figure 6As shown in the figure, targets that do not meet the alarm requirements are filtered, such as removing targets outside the area and targets with poor quality scores, and the alarm type is written into the global variable. Determine whether to enable tracking. If tracking is enabled, the alarm situation remains unchanged with the tracking type. If tracking is not enabled, the alarm situation is alarmed according to the actual reported target type, and no tracking is performed.

[0067] Step 3.3: The humanoid tracking module selects and searches for a tracking target based on the user's setting information.

[0068] like Figure 7 As shown in the figure, according to the user's preference, the tracking target of interest is selected. The static targets and the target types of interest are filtered out first, and then the target is selected by the comprehensive quality score.

[0069] Step 3.4: Update the tracking target trajectory of the tracking target selected in step 3.3.

[0070] like Figure 8 As shown, the target track is updated, and the current ID is searched in the algorithm full screen target. If the current ID is found, the target frame, score, type and other information of the ID are updated. If the tracking type does not match or the target is not found for less than 1s, the target is de-shaked; if it is not found for 1s, it is considered to have disappeared. Clear the information.

[0071] Step 4: The alarm module obtains the alarm information from the human tracking module and reports it; at the same time, it notifies the tracking module to start tracking.

[0072] like Figure 2 As shown, step 4 includes the following steps:

[0073] Step 4.1, set the current channel = 0; where channel 0 is a gun camera and channel 1 is a ball camera;

[0074] Step 4.2: Determine whether the channel is less than 2. If so, proceed to step 4.3; otherwise, return to step 4.1.

[0075] Step 4.3, determine whether tracking is enabled, if so, proceed to step 4.4, otherwise proceed to step 4.5; at the same time, the alarm module receives the alarm information, starts linkage tracking, and automatically modifies the tracking enable state;

[0076] Step 4.4, obtain tracking information, determine the target status, and proceed to step 4.6;

[0077] If the target is found, tracking is considered to have started; if the target is not found or the object remains stationary for about 15 seconds, tracking is considered to have ended; in other cases, tracking is considered to be in progress;

[0078] Step 4.5, write the tracking status into the shared memory;

[0079] Tracking status includes starting tracking, tracking in progress, ending tracking, tracking channel and coordinates of the tracking target;

[0080] Step 4.6, make the channel +1 and return to step 4.2.

[0081] Step 5: The tracking module performs relay tracking by performing PTZ coordinate conversion and dispatching the ball control module to control the movement of the ball camera.

[0082] like Figure 3 As shown, step 5 includes the following steps:

[0083] Step 5.1, set startTrack = FALSE, where FALSE means not in the process of ball possession;

[0084] Step 5.2, the tracking module reads the tracking status from the shared memory;

[0085] Step 5.3, the tracking module controls the ball according to the ball control strategy;

[0086] Step 5.3.1, determine whether startTrack = FALSE, if it is equal, determine the state of receiving the start tracking, if tracking is started, then the ball control starts StartTrack = TRUE, otherwise end this loop and proceed to step 5.2. Otherwise proceed to step 5.3.2;

[0087] Step 5.3.2, determine whether the tracking channel = Trackchn is received, if so, proceed to step 5.3.3, otherwise proceed to step 5.3.5;

[0088] Step 5.3.3, determine whether the status of ending tracking is received, if so, proceed to step 5.3.4, otherwise, ball tracking;

[0089] Step 5.3.4, determine whether there is a low priority channel, if there is a low priority channel, then the tracking ends StartTrack = FALSE; otherwise Trackchn = low priority channel, and this cycle ends;

[0090] Step 5.3.5, determine whether the high priority channel is being tracked (high priority refers to the gun camera tracking channel, low priority refers to the ball camera tracking channel). If it is being tracked, track the original target and perform ball control tracking. Otherwise, track the newly received target and perform ball control tracking.

[0091] Step 5.4: After the ball control operation is completed, the ball control information, including tracking channel information and tracking coordinate information, is recorded, and the process returns to step 5.1.

[0092] Step 6: The ball control module receives the instruction from the tracking module and controls the movement of the ball camera.

[0093] The present invention realizes relay tracking on a single-chip dual-sensor system (one gun camera and one ball camera), optimizes the humanoid tracking algorithm, enables the gun camera to continuously focus on the area of ​​interest to the user; enables the ball camera to focus on key targets, and uses two cameras to cover the monitoring area from different angles and positions, thereby reducing missed tracking caused by a single camera due to the limitation of the viewing angle. When the target disappears from the field of view of one camera, the system can quickly switch to another camera for tracking to ensure that the target is not missed. At the same time, the present invention optimizes the false alarm rate through a humanoid tracking module. It solves the problem that traditional cameras easily identify cars as people and make false alarms. By filtering the comprehensive quality score, the false detection rate is reduced. At the same time, a framework is built to provide a more convenient solution for filtering false alarm targets that the user is not interested in. The present invention has low difficulty and small calculation amount, reduces the requirements for hardware resources, and thus significantly reduces equipment costs, operating costs, and maintenance costs. The system proposes a separate humanoid tracking module without increasing the occupation of algorithm resources. The humanoid tracking module decides whether to track, does not occupy computing resources, and improves the processing efficiency of the system. In business, we take advantage of the humanoid tracking module and combine it with the business layout to cleverly implement humanoid relay tracking.

[0094] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A humanoid tracking system based on dual-channel target detection, characterized in that: It includes a gun camera, a ball camera, an algorithm module, a humanoid tracking module, an alarm module, a tracking module and a ball control module, wherein the output ends of the gun camera and the ball camera are respectively connected to the input end of the algorithm module, the output end of the algorithm module is connected to the input end of the humanoid tracking module, the output end of the humanoid tracking module is respectively connected to the input end of the alarm module and the tracking module, the output end of the alarm module is respectively connected to the input end of the humanoid tracking module and the tracking module, the output end of the tracking module is respectively connected to the ball control module and the humanoid tracking module, wherein, The algorithm module is used to process each frame of image captured by the ball camera and the gun camera and output the target trajectory information; The human tracking module is used to receive the target trajectory information output by the algorithm and the parameter configuration of the user; through internal processing, it decides the alarm to be reported and the corresponding detection target information; The alarm module is used to obtain the alarm information from the human tracking module and report it; at the same time, it notifies the tracking module to start tracking; The tracking module is used for relay tracking, which performs PTZ coordinate conversion and dispatches the ball control module to control the movement of the ball machine; The ball control module is used to receive instructions from the tracking module and control the movement of the ball camera.

2. The humanoid tracking system based on dual-channel target detection according to claim 1, characterized in that: The target trajectory information output by the algorithm module includes target motion state, quality score, target ID, target position and target characteristic value; the user's parameter configuration includes area and sensitivity, and the ball control module controls the movement of the ball camera including starting ball control, ball camera rotation, ball camera stillness and stopping ball control.

3. A tracking method of a humanoid tracking system based on dual-channel target detection according to any one of claims 1 to 2, characterized in that: The following steps are involved: Step 1: The gun camera and ball camera collect image information in real time; Step 2: The algorithm module processes each frame of the image collected by the ball camera and the gun camera, and outputs the target trajectory information; Step 3: The humanoid tracking module receives the target trajectory information output by the algorithm and receives the user's parameter configuration; through internal processing, it decides the alarm to be reported and the corresponding detection target information; Step 4: The alarm module obtains the alarm information from the human tracking module and reports it; at the same time, it notifies the tracking module to start tracking; Step 5: The tracking module performs relay tracking by performing PTZ coordinate conversion and dispatching the ball control module to control the movement of the ball camera; Step 6: The ball control module receives the instruction from the tracking module and controls the movement of the ball camera.

4. The tracking method of a humanoid tracking system based on dual-channel target detection according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1, the humanoid tracking module receives the target trajectory information and obtains the target trajectory detection result according to the user's parameter configuration; Step 3.2, the human tracking module updates the alarm status according to the target trajectory detection result; Step 3.3, the humanoid tracking module selects the tracking target and searches according to the user's setting information; Step 3.4: Update the tracking target trajectory of the tracking target selected in step 3.

3.

5. The tracking method of a humanoid tracking system based on dual-channel target detection according to claim 4, characterized in that: The specific implementation method of step 3.1 is: according to the user preference information, the targets are filtered hierarchically in turn, people, vehicles are selected, areas are selected and motion status is updated, and the target trajectory detection results are output. The target trajectory detection results include target type, target ID, target rectangular frame, target quality score, target motion, target stillness, current detection type and whether it is in the area.

6. The tracking method of a humanoid tracking system based on dual-channel target detection according to claim 4, characterized in that: The specific implementation method of step 3.2 is: filter the targets that do not meet the alarm requirements, including removing targets outside the area, removing targets with poor quality scores, and writing the alarm type into the global variable. At the same time, if tracking is turned on, the alarm condition and tracking type remain unchanged.

7. The tracking method of a humanoid tracking system based on dual-channel target detection according to claim 3, characterized in that: The step 4 comprises the following steps: Step 4.1, polling two channels and writing the sorted tracking status information into the shared memory, wherein the tracking status information includes the start of tracking, tracking in progress, end of tracking, tracking channel and coordinates of the tracking target; Step 4.2, the alarm module receives the alarm information and notifies the tracking module to enable tracking; Step 4.3: When the tracking module is enabled, the tracking information is obtained from the human tracking optimization module of the media; if the target is found, the tracking is considered to have started; if the target is not found or the object remains stationary for about 15 seconds, the tracking is considered to have ended; in other cases, the tracking is considered to be in progress.

8. The tracking method of a humanoid tracking system based on dual-channel target detection according to claim 3, characterized in that: The step 5 comprises the following steps: Step 5.1, the tracking module reads the tracking status from the shared memory; Step 5.2, the tracking module controls the ball according to the ball control strategy; Step 5.3: After the ball control operation is completed, the ball control information, including tracking channel information and tracking coordinate information, is recorded, and the process returns to step 5.1.

Images