Human body tracking method, device, electronic device and storage medium

Abstract

The present invention provides a human body tracking method, device, electronic device and storage medium, wherein the method comprises: determining a target person and performing human body tracking on the target person; if tracking fails and the reason for the tracking failure is crossing a monitoring area, determining a target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, the movement direction is obtained based on historical movement data of the target person, and the multiple candidate monitoring areas are obtained by grid division; based on the monitoring video within the target monitoring area, performing human body tracking on the target person, solving the problem that the target person cannot be tracked when crossing monitoring areas, realizing human body tracking in complex scenarios, and improving the success rate of human body tracking across monitoring areas.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a human body tracking method, device, electronic device and storage medium. Background Art

[0002] Human motion detection and tracking remain crucial and challenging topics in computer vision and image processing. Human tracking technology, widely used in applications such as video surveillance, security, indoor and outdoor robotics, abnormal behavior analysis, and crowd counting, holds great research value and broad application prospects.

[0003] Currently, the commonly used tracking technologies are divided into region-based tracking, feature-based tracking, deformable template-based tracking and model-based tracking. Current tracking technologies have achieved good tracking results in some simple scenarios.

[0004] However, in scenarios involving multiple monitoring devices, existing human body tracking technologies are unable to continue tracking the target object and are prone to tracking failures. Summary of the Invention

[0005] The present invention provides a human body tracking method, device, electronic device and storage medium, which are used to solve the defect in the prior art that target tracking cannot be performed in a scenario where there are multiple monitoring devices.

[0006] The present invention provides a human body tracking method, comprising:

[0007] Identify a target person and perform body tracking on the target person;

[0008] If tracking fails and the reason for the tracking failure is crossing the monitoring area, then determining the target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, wherein the movement direction is obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas are obtained by grid division;

[0009] Based on the surveillance video in the target monitoring area, the target person is tracked.

[0010] According to a human body tracking method provided by the present invention, the human body tracking of the target person further includes:

[0011] If tracking fails and the reason for the failure is that the target is in disguise, then new persons in the target monitoring area are identified based on the portraits of the residents in the target monitoring area and the portraits of the detection persons in the target monitoring area, where the detection persons are obtained by performing person detection on the surveillance video in the target monitoring area;

[0012] The identities of the newly added persons are matched with the target persons, and the newly added persons who are successfully matched are used as the target persons for body tracking.

[0013] According to a human tracking method provided by the present invention, determining a new person in the target monitoring area based on the person portraits of each resident in the target monitoring area and the person portraits of the detection personnel in the target monitoring area includes:

[0014] Match the portraits of each resident and each inspector. If the portrait of any inspector does not match the portraits of all residents, the inspector will be added as a new person.

[0015] According to a human tracking method provided by the present invention, the identity matching of each newly added person with the target person includes:

[0016] Based on the motion data of each newly added person and the target person, and / or the facial image of each newly added person and the disguised facial image of the target person, identity matching is performed between each newly added person and the target person.

[0017] According to a human tracking method provided by the present invention, the identity matching of each newly added person and the target person based on the motion data of each newly added person and the target person includes:

[0018] Inputting the motion data of each newly added person in the corresponding surveillance video into the visibility feature extraction network, obtaining the visibility data of each newly added person output by the visibility feature extraction network, and the hardware consumption for obtaining the visibility data of each newly added person;

[0019] Calculate the difference between the visibility data of each newly added person and the hardware consumption for obtaining the visibility data of each newly added person and the visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person;

[0020] Based on the calculated difference, each newly added person is matched with the target person.

[0021] According to a human tracking method provided by the present invention, the disguised facial image of the target person is determined based on the following steps:

[0022] Inputting the target person's facial image into a disguised image generator to obtain a disguised facial image output by the disguised image generator;

[0023] The camouflaged image generator is constructed based on a generative adversarial network.

[0024] According to a human body tracking method provided by the present invention, tracking the target person includes:

[0025] Performing face detection on the current image frame of the surveillance video to determine the face area of ​​the target person in the current image frame;

[0026] Determining a portrait region of the target person in the current image frame based on the face region;

[0027] Based on the portrait area, the visibility data of the motion data of the target person is updated, and the portrait area in the next image frame is tracked based on the visibility data.

[0028] The present invention also provides a human body tracking device, comprising:

[0029] A target determination unit, configured to determine a target person and perform body tracking on the target person;

[0030] an area determination unit, configured to, if tracking fails and the reason for the tracking failure is crossing a monitoring area, determine a target monitoring area from a plurality of candidate monitoring areas based on a movement direction of the target person, wherein the movement direction is obtained based on historical movement data of the target person, and the plurality of candidate monitoring areas are obtained by grid division;

[0031] The human body tracking unit is used to track the target person based on the monitoring video in the target monitoring area.

[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-described human body tracking methods when executing the computer program.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-described human body tracking methods.

[0034] The human body tracking method, device, electronic device and storage medium provided by the present invention track the target person after determining the target person. When the human body tracking fails due to tracking across monitoring areas, the target monitoring area is determined from multiple candidate monitoring areas according to the movement direction of the target person, and the target person is tracked again within the target monitoring area. This solves the problem of being unable to continue tracking the target person when crossing monitoring areas, realizes human body tracking in complex scenarios, and improves the success rate of human body tracking across monitoring areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 This is one of the flow charts of the human body tracking method provided by the present invention;

[0037] Figure 2 This is the second flow chart of the human body tracking method provided by the present invention;

[0038] Figure 3 is an overall flow chart of the human body tracking method provided by the present invention;

[0039] Figure 4 is a schematic structural diagram of the human body tracking device provided by the present invention;

[0040] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0041] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0042] Human motion detection and tracking are hot topics in computer vision, with broad application prospects and significant research value across many fields. Human motion detection is the process of determining the position, scale, and posture of a moving person within a sequence of video frames, while human tracking is the process of determining the correspondence between frames within a sequence of video frames.

[0043] Currently commonly used human tracking systems primarily consist of an identification module, a first determination module, a second determination module, and a tracking module. The identification module is used to identify the person being tracked and identify the facial region based on the current frame's main image. The first determination module is used to determine the portrait region based on the facial region. The second determination module is used to determine the human feature region based on the portrait region. Human feature data is then obtained from the human feature region, and the portrait region in the next frame's main image is tracked based on this human feature data, achieving tracking of the person being tracked based on the human feature data.

[0044] Although the above solution can track the tracked person in some simple scenarios, when the tracked person appears across monitoring devices, the existing human body tracking technology cannot continue to track the tracked person, resulting in the failure of tracking the tracked person.

[0045] In view of the above situation, the present invention provides a human body tracking method. Figure 1 This is one of the flow charts of the human body tracking method provided by the present invention, such as Figure 1 As shown, the method includes:

[0046] Step 110: determine the target person and perform body tracking on the target person.

[0047] Here, the target person is the person whose body needs to be tracked. The target person can be of various types, including ordinary citizens or special personnel, and this is not specifically limited in the embodiments of the present invention. The target person may be one or multiple. If there are multiple target persons, body tracking is required for each target person. Body tracking is the process of tracking a person based on their relevant characteristics. The relevant characteristics of a person represent their characteristics, such as their movement posture and facial features.

[0048] After determining the target person who needs to be tracked, the current image frame of the target person in the current monitoring video is obtained, the current image frame is detected, the relevant features of the target person in the current image frame are obtained, and the target person is tracked based on the relevant features.

[0049] In step 120, if tracking fails and the reason for the tracking failure is crossing the monitoring area, the target monitoring area is determined from multiple candidate monitoring areas based on the movement direction of the target person. The movement direction is obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas are obtained by grid division.

[0050] Here, crossing the monitoring area means crossing the area that the current monitoring device can monitor. The target person's movement direction is the target person's direction of travel during the body tracking process. The target person's movement direction is obtained based on the target person's historical movement data. The historical movement data here refers to the target person's previous movement data. The historical movement data is obtained during the body tracking process of the target person. The historical movement data can be the target person's movement posture or facial expression, which is not specifically limited in the embodiments of the present invention.

[0051] The candidate monitoring area is the area obtained by dividing the community into grids. The target monitoring area is the monitoring area where the target person may appear, selected from multiple candidate monitoring areas based on the target person's movement direction.

[0052] Specifically, in step 110, after tracking the target person, if the tracking result is a tracking failure, and the reason for the tracking failure is that the target person has left the area that the current monitoring device can monitor and moved to another monitoring area. In this case, the target person's movement direction must be obtained from the target person's historical movement data. Based on the target person's movement direction, the target monitoring area is determined from the multiple candidate monitoring areas that have been divided.

[0053] Correspondingly, if the tracking result of the human body tracking is successful, the target person is continued to be tracked in the next image frame according to the relevant features of the target person.

[0054] Step 130: Track the target person based on the surveillance video in the target surveillance area.

[0055] Specifically, after step 120, the target monitoring area is obtained, the monitoring video in the target monitoring area is turned on, the current image frame of the monitoring video in the target monitoring area is obtained, the current image frame is detected, the relevant features of the target person in the current image frame are obtained, and the target person is tracked according to the relevant features.

[0056] The human body tracking method provided by the present invention tracks the target person after determining the target person. When the human body tracking fails due to tracking across monitoring areas, the target monitoring area is determined from multiple candidate monitoring areas according to the movement direction of the target person, and the target person is tracked again within the target monitoring area. This solves the problem of being unable to continue tracking the target person when crossing monitoring areas, realizes human body tracking in complex scenarios, and improves the success rate of human body tracking across monitoring areas.

[0057] In addition, existing human tracking technologies cannot identify the person being tracked in scenarios where the person's similarity is high, the person is obscured, or the person is disguised, resulting in tracking failure. Furthermore, if a person is obscured or tracking fails for other reasons, and then reappears within the monitoring area, existing human tracking technologies will be unable to continue tracking the person.

[0058] In view of the above situation, based on the above embodiment, in step 110 and step 130, the target person is tracked, and then the following steps are further included:

[0059] If tracking fails and the reason for the failure is that the target is in disguise, the newly added persons in the target monitoring area are identified based on the portraits of the residents in the target monitoring area and the portraits of the detection personnel in the target monitoring area. The detection personnel are obtained by performing person detection on the surveillance videos in the target monitoring area.

[0060] The identities of the newly added persons are matched with the target persons, and the newly added persons who are successfully matched are used as the target persons for body tracking.

[0061] Here, "disguised target" refers to a target person who has been disguised. "Detected persons" refers to persons detected within a preset time period within the target surveillance area through surveillance video. "Person detection" refers to the detection of all persons appearing in the surveillance video within the target surveillance area. The preset time period is pre-set and can be 30 minutes or 1 hour, and is not specifically limited in this embodiment of the present invention. "Newly added persons" refers to persons newly added to the target surveillance area within the preset time period.

[0062] It should be noted that the embodiment of the present invention can be executed after determining the target person in step 110 and tracking the target person, or it can be executed after tracking the target person based on the surveillance video in the target monitoring area in step 130. The embodiment of the present invention does not make specific limitations on this.

[0063] Specifically, after tracking the target person, if the tracking result is a tracking failure, and the reason for the tracking failure is that the target person is in disguise, it indicates that the target person is in disguise. In this case, it is necessary to obtain the person portraits of each resident in the target monitoring area and perform person detection on the surveillance video within the target monitoring area. All persons appearing in the surveillance video within the target monitoring area within a preset time period are detected and the detected persons are regarded as the detected persons in the target monitoring area. Then, based on the person portraits of each resident in the target monitoring area and the person portraits of each detected person in the target monitoring area, the new persons in the target monitoring area are determined.

[0064] It should be noted that the profiles of residents within the target monitoring area are derived from the data resources of the city where the target monitoring area is located. This data resource enables real-time analysis of the city's overall situation and the effective allocation of public resources.

[0065] Furthermore, the identities of the newly added persons in the target monitoring area are matched with the target persons one by one to determine whether the identities of the newly added persons and the target persons are matched successfully, that is, whether the newly added persons are the target persons; if the identities of any newly added persons and the target persons are matched successfully, the newly added persons with the matched identities are taken as the target persons, and the target persons are tracked; if the identities of the newly added persons and the target persons are not matched successfully, it indicates that the newly added persons in the target monitoring area are not the target persons, that is, the target persons do not appear in the target monitoring area, and the human tracking is ended.

[0066] Correspondingly, if the tracking result of the human body tracking of the target person is successful, the target person is continued to be tracked in the next image frame according to the relevant features of the target person.

[0067] If the tracking result of the human body tracking is tracking failure, and the reason for the tracking failure is not the target disguise, for example, the reason for the tracking failure is that the monitoring equipment in the target monitoring area is not fully covered, then the human body tracking is terminated.

[0068] Based on the above embodiment, determining new personnel in the target monitoring area based on the personnel portraits of the residents in the target monitoring area and the personnel portraits of the detection personnel in the target monitoring area includes:

[0069] Match the profiles of each resident and each inspector. If the profile of any inspector does not match the profiles of all residents, the inspector will be added as a new person.

[0070] Specifically, after obtaining the personal portraits of each resident in the target monitoring area and the personal portraits of each detection personnel in the target monitoring area, the personal portraits of each resident in the target monitoring area can be matched one by one with the personal portraits of each detection personnel. If the personal portrait of any detection personnel matches the personal portrait of any resident, it indicates that the detection personnel is a resident in the target monitoring area.

[0071] It should be noted that the algorithm for performing personnel portrait matching may be a facial portrait comparison algorithm, which may be a RetinaFace algorithm or other facial portrait comparison algorithms, and the embodiment of the present invention does not specifically limit this.

[0072] Accordingly, if the personnel portrait of any detection personnel does not match the personnel portraits of all residents in the target monitoring area, it means that the detection personnel is not a resident of the target monitoring area, and the detection personnel will be regarded as a new person in the target monitoring area.

[0073] Based on the above embodiment, identity matching of each newly added person with the target person is performed, including:

[0074] Based on the motion data of each newly added person and the target person, and / or the facial image of each newly added person and the disguised facial image of the target person, identity matching is performed on each newly added person and the target person.

[0075] Here, motion data refers to data related to a person's movement captured from surveillance video. Motion data can include movement posture, movement frequency, or facial expression during movement, but this is not specifically limited in the present embodiment. A facial image is an image obtained by performing facial recognition on a person's face. A disguised facial image is an image obtained by disguising a facial image. For example, a facial image can be disguised by occluding, falsifying age, falsifying expression, swapping, beautifying, or characterizing the face to generate a disguised facial image.

[0076] Specifically, after determining the new personnel in the target monitoring area, the motion data of each new personnel can be obtained from the monitoring video in the target monitoring area, and the identities of each new personnel can be matched with the target personnel based on the motion data of each new personnel and the motion data of the target personnel obtained when tracking the target personnel; face detection can also be performed on the new personnel to obtain the face image of the new personnel, and the face image of the target personnel can be disguised to obtain the disguised face image of the target personnel, and the identities of each new personnel can be matched with the target personnel based on the face image of each new personnel and the disguised face image of the target personnel; identity matching can also be performed on each new personnel and the target personnel based on the motion data of each new personnel and the motion data of the target personnel, as well as the face image of each new personnel and the disguised face image of the target personnel.

[0077] It should be noted that the facial images of the target person are facial images obtained from multiple angles when tracking the target person.

[0078] Furthermore, it is determined whether the identities of the newly added persons match the identities of the target persons successfully; if any newly added person matches the identities of the target persons successfully, it indicates that the newly added person in the target monitoring area is the target person, and the newly added person who has been successfully matched is used as the target person, and the target person is tracked.

[0079] Correspondingly, if the identities of the newly added persons and the target person are not matched successfully, it means that the newly added persons in the target monitoring area are not the target person, that is, the target person does not appear in the target monitoring area, and the human tracking is ended.

[0080] Based on the above embodiment, identity matching is performed on each newly added person and the target person based on their movement data, including:

[0081] Input the motion data of each newly added person in the corresponding surveillance video into the visibility feature extraction network, obtain the visibility data of each newly added person output by the visibility feature extraction network, and the hardware consumption of obtaining the visibility data of each newly added person;

[0082] Calculate the difference between the visibility data of each newly added person and the hardware consumption for obtaining the visibility data of each newly added person, and the visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person;

[0083] Based on the calculated difference, the identity of each new person is matched with the target person.

[0084] Here, the visibility feature extraction network refers to the network used to extract visibility data from motion data, and its hardware consumption. Hardware consumption refers to the hardware consumption required to process the visibility data, including CPU consumption and memory consumption. Visibility data can be pre-set and include information such as a person's arms, legs, and motion posture.

[0085] Specifically, the motion data of each newly added person in the surveillance video within the target monitoring area is input into the visibility feature extraction network, and the visibility feature extraction network performs visibility feature extraction on the input motion data of each newly added person, and the visibility feature extraction network outputs the visibility data of each newly added person, as well as the hardware consumption for obtaining the visibility data of each newly added person.

[0086] It should be noted that the visual feature extraction network is built based on the dual attention VLAD network, which includes a convolutional layer, a spatial attention VLAD layer, a channel attention VLAD layer, a mixed error function and a fully connected layer.

[0087] The visibility data of each newly added person output by the visibility feature extraction network and the hardware consumption for obtaining the visibility data of each newly added person are differentially calculated with the visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person. There are multiple methods for performing the differential calculation. The visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person can be used as the first data; the visibility data of each newly added person and the hardware consumption for obtaining the visibility data of each newly added person can be used as the second data; the first data and the second data are input into a long short-term memory (LSTM) network, and the LSTM performs a differential calculation on the first data and the second data to obtain the difference between the first data and the second data output by the LSTM; the first data and the second data can also be interpolated using a differential algorithm to obtain the difference between the first data and the second data; this is not limited in the embodiment of the present invention.

[0088] Furthermore, based on the difference between each newly added person's visibility data and the target person's visibility data, as well as the difference between the hardware consumption of obtaining each newly added person's visibility data and the hardware consumption of obtaining the target person's visibility data, each newly added person is matched with the target person. If the calculated difference between any newly added person and the target person is within a preset range, it indicates that the identity of the newly added person and the target person are successfully matched, and the newly added person is designated as the target person and body tracking is performed on the target person. The preset range here can be pre-set according to actual needs.

[0089] Correspondingly, if the calculated differences between each newly added person and the target person are not within the preset range, it indicates that the identity matching of each newly added person and the target person is unsuccessful, and each newly added person in the target monitoring area is not the target person. The target person does not appear in the target monitoring area, and this human tracking ends.

[0090] The method provided in an embodiment of the present invention extracts visibility data and the hardware consumption of obtaining visibility data through a visibility feature extraction network. By using the difference between the visibility data and the hardware consumption of obtaining visibility data, the tracking range of the target person for human body tracking is narrowed, thereby improving tracking efficiency.

[0091] Based on the above embodiment, the disguised facial image of the target person is determined based on the following steps:

[0092] Inputting the target person's face image into the disguised image generator to obtain a disguised face image output by the disguised image generator;

[0093] The disguised image generator is built based on a generative adversarial network.

[0094] Here, after obtaining the target person's facial image, the target person's facial image is input into the disguised image generator, which disguises the input target person's facial image and outputs the disguised facial image of the target person.

[0095] Before inputting the target person's facial image into the disguised image generator, the disguised image generator must be constructed. This is built using a generative adversarial network (GAN). A GAN is a deep learning model that consists of a generative model and a discriminative model. The discriminative model is used to identify objects in an image, while the generative model generates new images based on the input image.

[0096] Based on the above embodiments, Figure 2 This is a second flow chart of the human body tracking method provided by the present invention, as shown in FIG. Figure 2 As shown, in step 110 and step 130, the target person is tracked, including:

[0097] Step 210: performing face detection on the current image frame of the surveillance video to determine the face area of ​​the target person in the current image frame;

[0098] Step 220: determining a portrait region of the target person in the current image frame based on the face region;

[0099] Step 230 : Based on the portrait area, update the visibility data of the motion data of the target person, and track the portrait area in the next image frame based on the visibility data.

[0100] Specifically, in step 210, after the target person is determined, face detection is performed on the current image frame of the current surveillance video to detect the target person's face region in the current image frame. After obtaining the target person's face region in the current image frame, step 220 is executed to determine the target person's portrait region in the current image frame based on the target person's face region. Then, step 230 is executed to update the visibility data of the target person's motion data based on the target person's portrait region in the current image frame. Based on the updated target person's visibility data, the target person's portrait region in the next image frame is tracked.

[0101] The human body tracking method provided by an embodiment of the present invention performs face detection on the current image frame to obtain the face area of ​​the target person; determines the portrait area of ​​the target person based on the face area; updates the visibility data of the target person's motion data based on the portrait area, and tracks the portrait area in the next image frame based on the visibility data. The target person is tracked by combining the face area, portrait area, visibility data and hardware consumption of obtaining the visibility data of the target person.

[0102] Figure 3 The overall flow chart of the human body tracking method provided by the embodiment of the present invention is as follows: Figure 3 As shown, the method includes:

[0103] Step 310, determining the target person;

[0104] Step 311: performing face detection on the current image frame of the surveillance video to determine the face area of ​​the target person in the current image frame;

[0105] Step 312: determining a portrait region of the target person in the current image frame based on the face region;

[0106] Step 313: updating the visibility features of the target person's motion data based on the portrait area, and tracking the portrait area in the next image frame based on the visibility features;

[0107] Step 320, determine whether tracking fails; if so, execute step 330; otherwise, execute step 313;

[0108] Step 330: determine whether the reason for the tracking failure is crossing the monitoring area; if so, execute step 340; otherwise, execute step 360;

[0109] Step 340 , based on the movement direction of the target person, determine a target monitoring area from multiple candidate monitoring areas, and perform body tracking on the target person based on the surveillance video within the target monitoring area;

[0110] Step 350, determine whether the tracking fails; if so, execute step 360; otherwise, end the human body tracking;

[0111] Step 360: Determine whether the tracking failure is due to the target being in disguise; if so, proceed to step 370; otherwise, terminate the current human tracking.

[0112] Step 370: Perform personnel detection on the surveillance video within the target surveillance area to identify each detected person;

[0113] Step 371: Determine new personnel in the target monitoring area based on the personnel portraits of each resident in the target monitoring area and the personnel portraits of each detection personnel in the target monitoring area;

[0114] Step 372: performing identity matching between each newly added person and the target person based on their movement data;

[0115] Step 373 , performing identity matching between each newly added person and the target person based on the facial image of each newly added person and the disguised facial image of the target person;

[0116] Step 374 , performing identity matching between each newly added person and the target person based on the motion data of each newly added person and the target person, as well as the facial image of each newly added person and the disguised facial image of the target person;

[0117] Step 380, determine whether the match is successful; if so, execute step 390; otherwise, end the human body tracking;

[0118] In step 390, the newly added person who has been successfully matched is used as the target person for body tracking.

[0119] The human body tracking device provided by the present invention is described below. The human body tracking device described below and the human body tracking method described above can be referenced to each other.

[0120] Figure 4 Schematic diagram of the structure of the human body tracking device provided by the present invention. Figure 4 As shown, the device includes:

[0121] The target determination unit 410 is used to determine a target person and perform body tracking on the target person;

[0122] an area determination unit 420 configured to, if tracking fails and the reason for the tracking failure is crossing a monitoring area, determine a target monitoring area from a plurality of candidate monitoring areas based on a movement direction of the target person, wherein the movement direction is obtained based on historical movement data of the target person, and the plurality of candidate monitoring areas are obtained by grid division;

[0123] The human body tracking unit 430 is used to track the target person based on the surveillance video in the target surveillance area.

[0124] The human body tracking device provided by the embodiment of the present invention tracks the target person after determining the target person. When the human body tracking fails due to tracking across monitoring areas, the target monitoring area is determined from multiple candidate monitoring areas based on the target person's movement direction, and the target person is tracked again within the target monitoring area. This solves the problem of being unable to continue tracking the target person when crossing monitoring areas, realizes human body tracking in complex scenarios, and improves the success rate of human body tracking across monitoring areas.

[0125] Based on the above embodiment, the device further includes a new person determining unit, which is configured to:

[0126] If tracking fails and the reason for the failure is that the target is in disguise, then new persons in the target monitoring area are identified based on the portraits of the residents in the target monitoring area and the portraits of the detection persons in the target monitoring area, where the detection persons are obtained by performing person detection on the surveillance video in the target monitoring area;

[0127] The identities of the newly added persons are matched with the target persons, and the newly added persons who are successfully matched are used as the target persons for body tracking.

[0128] Based on the above embodiment, the newly added personnel determination unit is configured to:

[0129] Match the portraits of each resident and each inspector. If the portrait of any inspector does not match the portraits of all residents, the inspector will be added as a new person.

[0130] Based on the above embodiment, the apparatus further includes an identity matching unit, configured to:

[0131] Based on the motion data of each newly added person and the target person, and / or the facial image of each newly added person and the disguised facial image of the target person, identity matching is performed between each newly added person and the target person.

[0132] Based on the above embodiment, the identity matching unit is used to:

[0133] Inputting the motion data of each newly added person in the corresponding surveillance video into the visibility feature extraction network, obtaining the visibility data of each newly added person output by the visibility feature extraction network, and the hardware consumption for obtaining the visibility data of each newly added person;

[0134] Calculate the difference between the visibility data of each newly added person and the hardware consumption for obtaining the visibility data of each newly added person and the visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person;

[0135] Based on the calculated difference, each newly added person is matched with the target person.

[0136] Based on the above embodiment, the apparatus further includes image determination units configured to:

[0137] Inputting the target person's facial image into a disguised image generator to obtain a disguised facial image output by the disguised image generator;

[0138] The camouflaged image generator is constructed based on a generative adversarial network.

[0139] Based on the above embodiment, the human body tracking unit 430 is used to:

[0140] Performing face detection on the current image frame of the surveillance video to determine the face area of ​​the target person in the current image frame;

[0141] Determining a portrait region of the target person in the current image frame based on the face region;

[0142] Based on the portrait area, the visibility data of the motion data of the target person is updated, and the portrait area in the next image frame is tracked based on the visibility data.

[0143] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute a human body tracking method, which includes: determining a target person and tracking the target person; if tracking fails and the reason for the tracking failure is that the target person crosses the monitoring area, determining a target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, the movement direction being obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas being obtained by grid division; and tracking the target person based on the monitoring video within the target monitoring area.

[0144] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the human body tracking method provided by the above methods, which method includes: determining a target person and tracking the target person; if the tracking fails and the reason for the tracking failure is across the monitoring area, determining the target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, the movement direction is obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas are obtained by grid division; based on the surveillance video within the target monitoring area, tracking the target person.

[0146] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned human body tracking methods, the method comprising: determining a target person and performing human body tracking on the target person; if the tracking fails and the reason for the tracking failure is across the monitoring area, determining a target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, the movement direction being obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas being obtained by grid division; and performing human body tracking on the target person based on the monitoring video within the target monitoring area.

[0147] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0148] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A human body tracking method, characterized in that: include: Identify a target person and perform body tracking on the target person; If tracking fails and the reason for the tracking failure is crossing the monitoring area, then determining the target monitoring area from multiple candidate monitoring areas based on the movement direction of the target person, wherein the movement direction is obtained based on the historical movement data of the target person, and the multiple candidate monitoring areas are obtained by grid division; Tracking the target person based on the surveillance video within the target surveillance area; The human body tracking of the target person further includes: If tracking fails and the reason for the failure is that the target is in disguise, then new persons in the target monitoring area are identified based on the portraits of the residents in the target monitoring area and the portraits of the detection persons in the target monitoring area, where the detection persons are obtained by performing person detection on the surveillance video in the target monitoring area; The identities of the newly added persons are matched with the target persons, and the newly added persons who are successfully matched are used as the target persons for body tracking.

2. The human body tracking method according to claim 1, characterized in that: The determining of new personnel in the target monitoring area based on the personnel portraits of the residents in the target monitoring area and the personnel portraits of the detection personnel in the target monitoring area includes: Match the portraits of each resident and each inspector. If the portrait of any inspector does not match the portraits of all residents, the inspector will be added as a new person.

3. The human body tracking method according to claim 1, wherein: The identity matching of each newly added person with the target person includes: Based on the motion data of each newly added person and the target person, and / or the facial image of each newly added person and the disguised facial image of the target person, identity matching is performed between each newly added person and the target person.

4. The human body tracking method according to claim 3, wherein: The identity matching of each newly added person with the target person based on the movement data of each newly added person and the target person includes: Inputting the motion data of each newly added person in the corresponding surveillance video into the visibility feature extraction network, obtaining the visibility data of each newly added person output by the visibility feature extraction network, and the hardware consumption for obtaining the visibility data of each newly added person; Calculate the difference between the visibility data of each newly added person and the hardware consumption for obtaining the visibility data of each newly added person and the visibility data of the target person and the hardware consumption for obtaining the visibility data of the target person; Based on the calculated difference, each newly added person is matched with the target person.

5. The human body tracking method according to claim 3, characterized in that: The disguised facial image of the target person is determined based on the following steps: Inputting the target person's facial image into a disguised image generator to obtain a disguised facial image output by the disguised image generator; The camouflaged image generator is constructed based on a generative adversarial network.

6. The human body tracking method according to any one of claims 1 to 5, characterized in that: The human body tracking of the target person includes: Performing face detection on the current image frame of the surveillance video to determine the face area of ​​the target person in the current image frame; Determining a portrait region of the target person in the current image frame based on the face region; Based on the portrait area, the visibility data of the motion data of the target person is updated, and the portrait area in the next image frame is tracked based on the visibility data.

7. A human body tracking device, characterized in that: include: A target determination unit, configured to determine a target person and perform body tracking on the target person; an area determination unit, configured to, if tracking fails and the reason for the tracking failure is crossing a monitoring area, determine a target monitoring area from a plurality of candidate monitoring areas based on a movement direction of the target person, wherein the movement direction is obtained based on historical movement data of the target person, and the plurality of candidate monitoring areas are obtained by grid division; A human body tracking unit, configured to track the target person based on the surveillance video within the target monitoring area; The human body tracking of the target person further includes: If tracking fails and the reason for the failure is that the target is in disguise, then new persons in the target monitoring area are identified based on the portraits of the residents in the target monitoring area and the portraits of the detection persons in the target monitoring area, where the detection persons are obtained by performing person detection on the surveillance video in the target monitoring area; The identities of the newly added persons are matched with the target persons, and the newly added persons who are successfully matched are used as the target persons for body tracking.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the human body tracking method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the human body tracking method according to any one of claims 1 to 6 are implemented.

Images