A data processing method, device, medium and equipment for pedestrian retrieval

Abstract

The present invention relates to the field of data processing, and in particular to a data processing method, device, medium and equipment for pedestrian retrieval, which characterizes the first clustering reliability corresponding to the second initial pedestrian segmentation map set in combination with the third image similarity between the second initial pedestrian segmentation maps in the second initial pedestrian segmentation map set, and then characterizes the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first clustering reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set, thereby improving the accuracy of the third correlation degree, and then improving the accuracy of the third target acquisition video, thereby improving the retrieval accuracy and tracking accuracy of the target pedestrian.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a data processing method, device, medium and equipment for pedestrian retrieval. Background Art

[0002] Faced with the massive amount of video data in the field of public security, if we only rely on manual review, analysis and processing of these videos one by one, the workload is very huge and the business efficiency is extremely low. Therefore, the existing technology relies on face recognition technology and the travel trajectory of the target object to reduce the search range of the surveillance video and improve the tracking efficiency. However, the daily travel trajectory of the target object is complicated, and when the target object deliberately avoids the face appearing in the surveillance camera, the probability of the target object's face being detected will be greatly reduced, and the search range of the surveillance video will be greatly increased, thereby reducing the tracking efficiency and accuracy, and it is difficult to meet the tracking requirements.

[0003] Therefore, how to improve tracking accuracy becomes an urgent problem to be solved. Summary of the invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is a data processing method for pedestrian retrieval, which includes the following steps:

[0005] The second initial collected video corresponding to the target pedestrian is obtained, as well as the predicted trajectory points and the second video collection device corresponding to each second initial collected video.

[0006] Image segmentation is performed on each second initial collected video to obtain a plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video.

[0007] All second initial pedestrian segmentation maps corresponding to all second initial collected videos are clustered to obtain K2 sets of second initial pedestrian segmentation maps, where K2 is an integer greater than 0.

[0008] For any second initial pedestrian segmentation map set, calculate the third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine the average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as the first clustering reliability corresponding to the current second initial pedestrian segmentation map set.

[0009] According to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquired video, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set are obtained.

[0010] According to the first clustering reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set, a third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian is obtained.

[0011] According to the third correlation degrees corresponding to all the second initial pedestrian segmentation map sets, a third target pedestrian segmentation map set is screened out from the K2 second initial pedestrian segmentation map sets.

[0012] The second initial collected video corresponding to the third target pedestrian segmentation map set is determined as the third target collected video for retrieving the target pedestrian.

[0013] The present invention also provides a data processing device for pedestrian retrieval, the data processing device comprising:

[0014] A data acquisition module, used to acquire a second initial acquisition video corresponding to the target pedestrian, and a predicted trajectory point and a second video acquisition device corresponding to each second initial acquisition video;

[0015] A data processing module, used to perform image segmentation on each second initial collected video, and obtain a plurality of second initial pedestrian segmentation images corresponding to each second initial collected video;

[0016] An image clustering module, used for clustering all second initial pedestrian segmentation maps corresponding to all second initial collected videos, and obtaining K2 sets of second initial pedestrian segmentation maps, where K2 refers to an integer greater than 0;

[0017] A first clustering reliability calculation module is used to calculate, for any second initial pedestrian segmentation map set, a third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine an average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as a first clustering reliability corresponding to the current second initial pedestrian segmentation map set;

[0018] A quantity acquisition module, used to acquire the quantity of trajectory points and the quantity of devices corresponding to the current second initial pedestrian segmentation map set according to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquisition video;

[0019] The third correlation degree acquisition module is used to obtain the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set.

[0020] The third set screening module is used to screen out a third target pedestrian segmentation map set from K2 second initial pedestrian segmentation map sets according to the third correlation degrees corresponding to all second initial pedestrian segmentation map sets.

[0021] The third video screening module is used to determine the second initial collected video corresponding to the third target pedestrian segmentation map set as the third target collected video for retrieving the target pedestrian.

[0022] The present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned data processing method for pedestrian retrieval.

[0023] The present invention also provides an electronic device, comprising a processor and the above-mentioned non-transitory computer-readable storage medium.

[0024] The present invention has at least the following beneficial effects: combining the third image similarity between the second initial pedestrian segmentation maps in the second initial pedestrian segmentation map set to characterize the first clustering reliability corresponding to the second initial pedestrian segmentation map set, and then combining the first clustering reliability corresponding to the second initial pedestrian segmentation map set, according to the first clustering reliability corresponding to the current second initial pedestrian segmentation map set, the number of trajectory points and the number of devices, to characterize the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian, thereby improving the accuracy of the third correlation degree, and then improving the accuracy of the third target acquisition video, thereby improving the retrieval accuracy and tracking accuracy of the target pedestrian. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 A flowchart of a data processing method for pedestrian retrieval provided in Embodiment 1 of the present invention;

[0027] Figure 2 A flowchart of a pedestrian retrieval method based on video data provided in Embodiment 2 of the present invention;

[0028] Figure 3 A flowchart of a pedestrian retrieval method based on trajectory data provided in Embodiment 3 of the present invention;

[0029] Figure 4 A schematic diagram of the structure of a data processing device for pedestrian retrieval provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It is understood that, where appropriate, the above-mentioned terms used to distinguish similar objects can be interchanged so that the present invention can also implement other embodiments other than the above-mentioned illustrated embodiments or described embodiments. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] This embodiment 1 provides a data processing method for pedestrian retrieval, and the data processing method for pedestrian retrieval includes: Figure 1 As shown:

[0034] S301, obtaining a second initial captured video corresponding to a target pedestrian, and a predicted trajectory point and a second video capturing device corresponding to each second initial captured video.

[0035] Among them, the target pedestrian refers to the pedestrian whose movement trajectory, behavioral activities, peer contacts, etc. within a certain period of time need to be retrieved.

[0036] The second initial collected video may be a surveillance video collected according to the predicted trajectory of the target pedestrian.

[0037] In order to retrieve the target pedestrian's movement trajectory, behavioral activities, peer contacts and other behaviors within a certain period of time, this embodiment first obtains the second initial captured video corresponding to the target pedestrian, as well as the predicted trajectory points and second video capture devices corresponding to each second initial captured video as the basis for retrieving the target pedestrian.

[0038] In a specific implementation, obtaining the second initial collected video corresponding to the target pedestrian further includes the following steps:

[0039] A second predicted trajectory set corresponding to the target pedestrian is obtained, wherein the second predicted trajectory set includes M2 second predicted trajectory points and a second predicted passing time period corresponding to each second predicted trajectory point, and M2 refers to an integer greater than 0.

[0040] According to the second predicted trajectory set, T second video acquisition devices corresponding to each second predicted trajectory point and the second initial acquisition video of each second video acquisition device within the corresponding second predicted elapsed time period are obtained, where T refers to the total number of second video acquisition devices.

[0041] Among them, the second video acquisition device can refer to a surveillance camera, which is used to collect behavior data of pedestrians, vehicles and other objects within a certain range.

[0042] In the above, the complete predicted trajectory of the target pedestrian is characterized based on the second predicted trajectory set, and the corresponding second initial collected video is obtained according to the second predicted trajectory set as the video basis for retrieving the target pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0043] In a specific implementation, obtaining the second predicted trajectory set corresponding to the target pedestrian further includes the following steps:

[0044] The preset trajectory point collection device and the trained trajectory prediction model are obtained, wherein the trajectory point collection device includes a wifi device, a public mobile communication base station and a GPS device.

[0045] According to the wifi device connected to the target pedestrian and the scanned wifi devices, a first connection range and a first connection time period corresponding to each connected wifi device, and a scanning range and a scanning time period corresponding to each scanned wifi device are obtained.

[0046] According to the public mobile communication base station connected to the target pedestrian, a second connection range and a second connection time period corresponding to each connected public mobile communication base station are acquired.

[0047] According to the GPS device, a GPS data set corresponding to the target pedestrian is acquired, wherein the GPS data set includes the GPS positioning data of the target pedestrian and the positioning time corresponding to each GPS positioning data.

[0048] According to all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data, a second actual trajectory point corresponding to the target pedestrian is obtained.

[0049] According to all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times, the second actual passing time period corresponding to each second actual trajectory point is acquired.

[0050] All the second actual trajectory points and the second actual elapsed time period corresponding to each second actual trajectory point are input into the trained trajectory prediction model to obtain a second predicted trajectory set.

[0051] Among them, the connection range and scanning range of the fixed wifi device are fixed, and the connection range of the public mobile communication base station is also determined, and the GPS device carried or used by the target pedestrian can report the location data of the target pedestrian. Therefore, the location area of ​​the target pedestrian can be obtained based on the wifi device connected or scanned by the target pedestrian, the public mobile communication base station connected to the target pedestrian, and the GPS device carried or used by the target pedestrian, as the basis for obtaining the second target acquisition video for retrieving the target pedestrian.

[0052] The first connection range, scanning range, second connection range and GPS positioning data can all represent the location information of the target pedestrian. Therefore, all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data can be combined to obtain several second actual trajectory points corresponding to the target pedestrian, and further based on all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times, the second actual passing time period corresponding to each second actual trajectory point can be obtained as the basis for obtaining the second target acquisition video of the retrieved target pedestrian.

[0053] The trained trajectory prediction model is used to extract and analyze the features of the input second actual trajectory points and the second actual elapsed time period, and output the second predicted trajectory points corresponding to the target pedestrian and the second predicted elapsed time period corresponding to each second predicted trajectory point, thereby obtaining a second predicted trajectory set as the basis for obtaining the second target acquisition video for retrieving the target pedestrian.

[0054] In the above, the location information of the target pedestrian is characterized according to the first connection range, the scanning range, the second connection range and the GPS positioning data, and a number of second actual trajectory points corresponding to the target pedestrian are obtained in combination with all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data. The second actual passing time period corresponding to each second actual trajectory point is obtained in combination with the first connection time period, the scanning time period, the second connection time period and the positioning time, and the actual trajectory information of the target pedestrian is clarified. The complete predicted trajectory of the target pedestrian is obtained in combination with the trained trajectory prediction model, thereby improving the accuracy of the second predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0055] S302: Perform image segmentation on each second initial collected video to obtain a plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video.

[0056] Among them, the second initial collected video includes not only the behavior video of pedestrians, but also the video information of other interference factors such as vehicles, buildings, animals, etc. Therefore, each frame of the second initial collected video is firstly segmented, and the second initial pedestrian segmentation map is obtained as the processing target to eliminate the influence of other interference factors on the retrieval results.

[0057] In a specific implementation, performing image segmentation on each second initial collected video, and obtaining a plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video and a collection time corresponding to each second initial pedestrian segmentation map further includes the following steps:

[0058] According to the preset second frame rate, a plurality of second acquired image frames corresponding to each second initial acquired video and the acquisition time corresponding to each second acquired image frame are acquired.

[0059] Image segmentation is performed on each second acquired image frame according to a preset image segmentation algorithm, and a plurality of second initial pedestrian segmentation maps corresponding to each second acquired image frame are obtained.

[0060] For any second initial pedestrian segmentation map, the acquisition time corresponding to the second acquisition image frame corresponding to any second initial pedestrian segmentation map is determined as the acquisition time corresponding to the any second initial pedestrian segmentation map.

[0061] The preset second frame rate is determined by the corresponding second video acquisition device.

[0062] Among them, those skilled in the art know that any image segmentation algorithm and any clustering algorithm in the prior art fall within the protection scope of the present invention, and will not be described in detail here.

[0063] In the above, image segmentation is performed on each frame of the second initial captured video, and the second initial pedestrian segmentation map is obtained as the processing target, thereby eliminating the influence of other interference factors on the retrieval results, thereby improving the retrieval accuracy of the target pedestrian.

[0064] S303: Cluster all second initial pedestrian segmentation maps corresponding to all second initial collected videos to obtain K2 sets of second initial pedestrian segmentation maps, where K2 is an integer greater than 0.

[0065] Among them, all the second initial pedestrian segmentation maps are clustered to obtain K2 second initial pedestrian segmentation map sets, so as to characterize the correlation between the same pedestrian in different second initial collected videos and the difference between different pedestrians according to the second initial pedestrian segmentation map sets.

[0066] S304, for any second initial pedestrian segmentation map set, calculate the third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine the average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as the first clustering reliability corresponding to the current second initial pedestrian segmentation map set.

[0067] Among them, in the same second initial pedestrian segmentation map set, the higher the similarity between the second initial pedestrian segmentation maps, the higher the clustering effect of the corresponding second initial pedestrian segmentation map set, that is, the higher the reliability of the first clustering.

[0068] S305: According to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquired video, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set are acquired.

[0069] Among them, according to the predicted trajectory points and second video acquisition devices corresponding to each second initial pedestrian segmentation map in the current second initial pedestrian segmentation map set, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set are obtained as the basis for characterizing the richness of the data source of the current second initial pedestrian segmentation map.

[0070] S306: Obtain a third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set.

[0071] Among them, the larger the number of trajectory points and the number of devices corresponding to each second initial pedestrian segmentation map set are, the higher the data richness and reliability of the second initial pedestrian segmentation map set is.

[0072] Therefore, the first clustering reliability, the number of trajectory points and the number of devices corresponding to each second initial pedestrian segmentation map set are positively correlated with the corresponding third association degree.

[0073] In a specific implementation, according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set, obtaining the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian further includes the following steps:

[0074] For the j-th second initial pedestrian segmentation map set, according to the first clustering reliability Z corresponding to the j-th second initial pedestrian segmentation map set j , the number of trajectory points GJ j and the number of devices P j , get the third correlation degree between the jth second initial pedestrian segmentation map set and the target pedestrian:

[0075] Y 3j =(2 / (1+e^(-(α1×P j +α2×GJ j )))-1)×Z j .

[0076] Where Y 3j It refers to the third correlation degree corresponding to the jth second initial pedestrian segmentation map set, P j GJ refers to the number of devices corresponding to the jth second initial pedestrian segmentation map set. j refers to the number of trajectory points corresponding to the jth second initial pedestrian segmentation map set, Z j It refers to the first clustering reliability corresponding to the j-th second initial pedestrian segmentation map set, α1 refers to the first preset weight, α2 refers to the second preset weight, and e is an exponential function.

[0077] As described above, combined with the first clustering reliability, the number of trajectory points and the number of devices corresponding to each second initial pedestrian segmentation map set, the third degree of association between each second initial pedestrian segmentation map and the target pedestrian is calculated, which improves the calculation accuracy of the third degree of association and thereby improves the retrieval accuracy of the target pedestrian.

[0078] S307 : Filter out a third target pedestrian segmentation map set from the K2 second initial pedestrian segmentation map sets according to the third association degrees corresponding to all the second initial pedestrian segmentation map sets.

[0079] In a specific implementation, according to the third correlation degrees corresponding to all the second initial pedestrian segmentation map sets, selecting the third target pedestrian segmentation map set from the K2 second initial pedestrian segmentation map sets further includes the following steps:

[0080] The third association degrees corresponding to the K2 second initial pedestrian segmentation map sets are compared, and the second initial pedestrian segmentation map set corresponding to the largest third association degree is determined as the third target pedestrian segmentation map set.

[0081] S308: Determine the second initial collected video corresponding to the third target pedestrian segmentation map set as the third target collected video for retrieving the target pedestrian.

[0082] Among them, the second initial collected video corresponding to the third target pedestrian segmentation map set is determined as the third target collected video for retrieving the target pedestrian. The movement trajectory, behavioral activities, peer contact and other behaviors of the target pedestrian can be analyzed through the third target collected video to provide data support for finding the target object, thereby improving the accuracy of tracking the person.

[0083] In a specific implementation, the data processing method further includes the following steps:

[0084] S01, obtaining the initial description text corresponding to the target pedestrian, the trained feature extraction model and the acquisition time corresponding to each second initial pedestrian segmentation map.

[0085] Among them, the same pedestrian may appear successively in the acquisition range of multiple second video acquisition devices, and thus appear successively in multiple second initial acquisition videos. Therefore, the acquisition time corresponding to each second initial pedestrian segmentation map is obtained to simultaneously provide the trajectory path information and time information of the target pedestrian, thereby improving the richness and accuracy of the trajectory information.

[0086] In a specific implementation, the initial description text includes a preset first face description text, a preset first body shape description text, and a preset first walking posture description text of the target pedestrian.

[0087] The first face description text is used to represent the facial feature information of the target pedestrian, such as round face, phoenix eyes, sword-shaped eyebrows, etc. The first body shape description text is used to represent the body shape information of the target pedestrian, such as broad shoulders, long legs, bow legs, hunchback, etc. The first walking posture description text is used to represent the walking posture information of the target pedestrian, such as walking with feet turned inward or outward. The specific initial description text can be set by the implementer according to the actual situation.

[0088] S02, based on the acquisition time, clustering all the second initial pedestrian segmentation maps corresponding to all the second initial acquisition videos to obtain K2 third initial pedestrian segmentation map sets, wherein the second initial pedestrian segmentation maps in each third initial pedestrian segmentation map set are sorted according to the corresponding acquisition time.

[0089] Among them, walking is a continuous action, and there are differences in the walking postures of different pedestrians when walking continuously. Therefore, this embodiment sorts the second initial pedestrian segmentation map in each third initial pedestrian segmentation map set according to the acquisition time, which not only integrates the trajectory path information and time information of the target pedestrian, but also compared with the pedestrian segmentation map set corresponding to the pedestrian segmentation map set that is not sorted according to time, the third initial pedestrian segmentation map set can more effectively characterize the walking posture of the pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0090] S03. For any third initial pedestrian segmentation map set, calculate the fifth image similarity between any two second initial pedestrian segmentation maps in the current third initial pedestrian segmentation map set, and determine the average value of all fifth image similarities as the second clustering reliability corresponding to the current third initial pedestrian segmentation map set.

[0091] S04: Input the current third initial pedestrian segmentation map set into the trained feature extraction model to obtain pedestrian feature description text corresponding to the current third initial pedestrian segmentation map set.

[0092] In a specific implementation, the pedestrian feature description text includes a second face description text, a second body shape description text, and a second walking posture description text of the pedestrian in the corresponding third initial pedestrian segmentation image set.

[0093] The trained feature extraction model is used to extract and analyze the features of the input third initial pedestrian segmentation map set, and output the corresponding pedestrian feature description text.

[0094] Corresponding to the initial description text of the target pedestrian, the pedestrian feature description text includes the second face description text, the second body shape description text and the second walking posture description text of the pedestrian in the corresponding third initial pedestrian segmentation map set, which is used to characterize the facial feature information, body shape information and walking posture information of the corresponding pedestrian.

[0095] As described above, the pedestrian feature description text corresponding to each third initial pedestrian segmentation map set is extracted according to the trained feature extraction model to characterize the facial feature information, body shape information and walking posture information of the corresponding pedestrian, which serves as the feature basis for retrieving the target pedestrian.

[0096] S05: Obtain a sixth similarity between the current third initial pedestrian segmentation map set and the target pedestrian according to the pedestrian feature description text and the initial description text.

[0097] Among them, those skilled in the art know that any text similarity algorithm in the prior art falls within the protection scope of the present invention, and will not be described in detail here.

[0098] S06, obtaining a fourth correlation degree between the current third initial pedestrian segmentation map and the target pedestrian according to the second clustering reliability, the sixth similarity, the number of predicted trajectory points and the number of second initial collected videos corresponding to the current third initial pedestrian segmentation map set.

[0099] Among them, the greater the sixth similarity corresponding to each third initial pedestrian segmentation map set, the higher the degree of association between the third initial pedestrian segmentation map set and the target pedestrian. Therefore, the second clustering reliability, the sixth similarity, the number of predicted trajectory points, and the number of second initial collected videos corresponding to each third initial pedestrian segmentation map set are all positively correlated with the corresponding fourth degree of association.

[0100] S07 , selecting a fourth target pedestrian segmentation map set from K2 third initial pedestrian segmentation map sets according to fourth association degrees corresponding to all third initial pedestrian segmentation map sets.

[0101] S08: Determine the second initial collected video corresponding to the fourth target pedestrian segmentation map set as the fourth target collected video for retrieving the target pedestrian.

[0102] Among them, the second initial collected video corresponding to the fourth target pedestrian segmentation map set is determined as the fourth target collected video for retrieving the target pedestrian. The movement trajectory, behavioral activities, peer contacts and other behaviors of the target pedestrian can be analyzed through the fourth target collected video to provide data support for finding the target object, thereby improving the accuracy of tracking the person.

[0103] As described above, the second initial pedestrian segmentation map is sorted in combination with the acquisition time, the trajectory path information and time information of the target pedestrian are integrated, and the walking posture of the pedestrian is effectively characterized, so as to calculate the sixth similarity between the third initial pedestrian segmentation map set and the target pedestrian in combination with the facial feature information, body shape information and walking posture information of the pedestrian, and further combine the second clustering reliability, the sixth similarity, the number of predicted trajectory points and the number of the second initial acquired videos to obtain the fourth correlation degree corresponding to the third initial pedestrian segmentation map set, thereby screening out the fourth target acquired video, thereby improving the retrieval accuracy and tracking accuracy of the target pedestrian.

[0104] In the above, the third image similarity between the second initial pedestrian segmentation maps in the second initial pedestrian segmentation map set is combined to characterize the first clustering reliability corresponding to the second initial pedestrian segmentation map set, and then combined with the first clustering reliability corresponding to the second initial pedestrian segmentation map set, according to the first clustering reliability corresponding to the current second initial pedestrian segmentation map set, the number of trajectory points and the number of devices, the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian is characterized, thereby improving the accuracy of the third correlation degree, and then improving the accuracy of the third target acquisition video, thereby improving the retrieval accuracy and tracking accuracy of the target pedestrian.

[0105] Embodiment 2

[0106] The second embodiment provides a pedestrian retrieval method based on video data. The pedestrian retrieval method based on video data includes the following steps: Figure 2 As shown:

[0107] S101, obtaining a first predicted trajectory set corresponding to a target pedestrian, wherein the first predicted trajectory set includes M1 first predicted trajectory points and a first predicted passing time period corresponding to each first predicted trajectory point, where M1 refers to an integer greater than 0.

[0108] Among them, the target pedestrian refers to the pedestrian whose movement trajectory, behavioral activities, peer contacts, etc. within a certain period of time need to be retrieved.

[0109] In order to retrieve the target pedestrian's movement trajectory, behavioral activities, peer contacts and other behaviors within a certain period of time, this embodiment first obtains a first predicted trajectory set corresponding to the target pedestrian within a preset period of time, which is used to predict the trajectory route of the target pedestrian, and then collects the video corresponding to the first predicted trajectory set as the basis for retrieving the target pedestrian.

[0110] In a specific implementation, obtaining a first predicted trajectory set corresponding to the target pedestrian further includes the following steps:

[0111] Obtain the preset trajectory point collection device and the trained trajectory prediction model.

[0112] According to the trajectory point collection device, a first trajectory set corresponding to the target pedestrian is obtained, wherein the first trajectory set includes N first actual trajectory points of the target pedestrian collected by the trajectory point collection device, and a first actual passing time period corresponding to each first actual trajectory point, where N refers to an integer greater than 0.

[0113] The first trajectory set is input into the trained trajectory prediction model to obtain M1 first predicted trajectory points corresponding to the target pedestrian and a first predicted elapsed time period corresponding to each first predicted trajectory point.

[0114] A first predicted trajectory set is acquired according to the M1 first predicted trajectory points and the first predicted elapsed time period corresponding to each first predicted trajectory point.

[0115] Among them, the track point acquisition device can refer to a device such as a wifi device, a public mobile communication base station and a GPS device that can obtain the location area of ​​the target pedestrian. Specifically, the connection range and scanning range of the fixed wifi device are fixed, and the connection range of the public mobile communication base station is also determined, and the GPS device carried or used by the target pedestrian can report the location data of the target pedestrian. Therefore, the location area of ​​the target pedestrian can be obtained based on the wifi device connected or scanned by the target pedestrian, the public mobile communication base station connected to the target pedestrian, and the GPS device carried or used by the target pedestrian, as the basis for obtaining the first target acquisition video for retrieving the target pedestrian.

[0116] The specific trajectory point collection device can be selected by the implementer according to the actual situation.

[0117] The trained trajectory prediction model is used to extract and analyze the features of the first actual trajectory point and the first actual elapsed time period in the input first trajectory set, and output M1 first predicted trajectory points corresponding to the target pedestrian and the first predicted elapsed time period corresponding to each first predicted trajectory point, thereby obtaining the first predicted trajectory set as the basis for obtaining the first target acquisition video for retrieving the target pedestrian.

[0118] In the above, the trajectory of the target pedestrian is predicted based on the preset trajectory point acquisition device and the trained trajectory prediction model, and the first predicted trajectory set is obtained, which clarifies the video source when retrieving the target pedestrian, provides a data basis for obtaining the first target acquisition video for retrieving the target pedestrian, and improves the retrieval accuracy of the target pedestrian.

[0119] In a specific implementation, the track point collection device includes a wifi device, a public mobile communication base station and a GPS device. According to the track point collection device, obtaining the first track set corresponding to the target pedestrian also includes the following steps:

[0120] The wifi devices connected to the target pedestrian, the first connection range and the first connection time period corresponding to each connected wifi device, as well as the wifi devices scanned by the target pedestrian, the scanning range and the scanning time period corresponding to each scanned wifi device are obtained.

[0121] The public mobile communication base station to which the target pedestrian is connected, and the second connection range and second connection time period corresponding to each public mobile communication base station are acquired.

[0122] The GPS device corresponding to the target pedestrian and the GPS data set collected by each GPS device are obtained, wherein the GPS data set includes the GPS positioning data of the target pedestrian and the positioning time corresponding to each GPS positioning data.

[0123] According to all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data, a first actual trajectory point corresponding to the target pedestrian is obtained.

[0124] According to all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times, the first actual passing time period corresponding to each first actual track point is acquired.

[0125] Among them, the connection range and scanning range of the fixed wifi device are fixed, so the first actual trajectory point and the corresponding first actual passing time period corresponding to the target pedestrian can be determined according to the wifi device connected to the target pedestrian and the scanned wifi device; the connection range of the public mobile communication base station is determined, so the second connection range and the second connection time period corresponding to each connected public mobile communication base station can be obtained according to the public mobile communication base station connected to the target pedestrian; the GPS device carried or used by the target pedestrian can report the location data of the target pedestrian, so the GPS positioning data of the target pedestrian and the positioning time corresponding to each GPS positioning data can be obtained according to the GPS device, which serves as the basis for predicting the trajectory of the target pedestrian.

[0126] In a specific implementation, according to all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data, obtaining the first actual track point corresponding to the target pedestrian further includes the following steps:

[0127] According to all the first connection ranges and all the scanning ranges, n1 first candidate trajectory points corresponding to the target pedestrian are obtained, where n1 refers to an integer greater than 0.

[0128] According to all the second connection ranges, n2 second candidate trajectory points corresponding to the target pedestrian are obtained, where n2 is an integer greater than 0.

[0129] According to n1 first candidate trajectory points, n2 second candidate trajectory points and all GPS positioning data, a first actual trajectory point corresponding to the target pedestrian is obtained.

[0130] Among them, the first candidate trajectory point is a location range determined based on the actual location information of the target pedestrian represented by the wifi device, the second candidate trajectory point is a location range determined based on the actual location information of the target pedestrian represented by the public mobile communication base station, and the GPS positioning data may refer to the longitude and latitude of the actual location of the target pedestrian. By combining all the first candidate trajectory points, all the second candidate trajectory points and all the GPS positioning data, the first actual trajectory point representing the actual location information of the target pedestrian is obtained.

[0131] In the above, the first candidate trajectory point is determined based on the actual location information of the target pedestrian represented by the wifi device, the second candidate trajectory point is determined based on the actual location information of the target pedestrian represented by the public mobile communication base station, and the first actual trajectory point is obtained in combination with the longitude and latitude of the actual location of the target pedestrian, thereby improving the accuracy of obtaining the first actual trajectory point.

[0132] In a specific implementation, obtaining n1 first candidate trajectory points corresponding to the target pedestrian according to all the first connection ranges and all the scanning ranges further includes the following steps:

[0133] If any first connection range has no intersection with other first connection ranges and the scanning range, any first connection range is taken as a first candidate trajectory point.

[0134] If any scanning range has no intersection with other scanning ranges and the first connection range, any scanning range is taken as a first candidate trajectory point.

[0135] If any first connection range intersects with other first connection ranges or scanning ranges, any first connection range and other first connection ranges or scanning ranges intersecting with any first connection range are taken together as a first candidate trajectory point.

[0136] If any scanning range has an intersection with other scanning ranges or the first connection range, any scanning range and other scanning ranges or the first connection range intersecting with any first scanning range are taken together as a first candidate trajectory point.

[0137] Traverse all first connection ranges and all scanning ranges to obtain n1 first candidate trajectory points corresponding to the target pedestrian.

[0138] As described above, based on the positional relationship between all the first connection ranges and all the scanning ranges, several non-intersecting first candidate trajectory points are obtained, thereby clarifying part of the actual trajectory information of the target pedestrian, providing a data basis for predicting the complete trajectory of the target pedestrian, and improving the accuracy of the first predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0139] In a specific implementation, obtaining n2 second candidate trajectory points corresponding to the target pedestrian according to all the second connection ranges further includes the following steps:

[0140] If any second connection range has no intersection with any other second connection range, any second connection range is taken as a second candidate trajectory point.

[0141] If any second connection range intersects with other second connection ranges, any second connection range and other second connection ranges intersecting with any second connection range are taken together as a second candidate trajectory point.

[0142] Traverse all the second connection ranges to obtain n2 second candidate trajectory points corresponding to the target pedestrian.

[0143] As described above, based on the positional relationship between all the second connection ranges, several non-intersecting second candidate trajectory points are obtained, thereby clarifying part of the actual trajectory information of the target pedestrian, providing a data basis for predicting the complete trajectory of the target pedestrian, and improving the accuracy of the first predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0144] In a specific implementation, obtaining the first actual trajectory point corresponding to the target pedestrian according to n1 first candidate trajectory points, n2 second candidate trajectory points and all GPS positioning data further includes the following steps:

[0145] If any first candidate trajectory point has no intersection with other first candidate trajectory points and second candidate trajectory points, any first candidate trajectory point is taken as a first actual trajectory point.

[0146] If any second candidate trajectory point has no intersection with other second candidate trajectory points and the first candidate trajectory point, any second candidate trajectory point is taken as a first actual trajectory point.

[0147] If any first candidate trajectory point intersects with other first candidate trajectory points or second candidate trajectory points, any first candidate trajectory point and other first candidate trajectory points or second candidate trajectory points intersecting with any first candidate trajectory point are taken together as a first actual trajectory point.

[0148] If any second candidate trajectory point intersects with other second candidate trajectory points or first candidate trajectory points, any second candidate trajectory point and other second candidate trajectory points or first candidate trajectory points intersecting with any second candidate trajectory point are taken together as a first actual trajectory point.

[0149] If any GPS positioning data is not within the range of the first candidate trajectory point and the second candidate trajectory point, a third candidate trajectory point corresponding to any GPS positioning data is obtained according to the preset radius and the GPS positioning data.

[0150] Each third candidate trajectory point is taken as a first actual trajectory point.

[0151] Traverse n1 first candidate trajectory points, n2 second candidate trajectory points and all GPS positioning data to obtain the first actual trajectory point corresponding to the target pedestrian.

[0152] As described above, based on the positional relationship between all the first candidate trajectory points and all the second candidate trajectory points, combined with the GPS positioning data and the positional relationship between the first candidate trajectory points and the second candidate trajectory points, the first actual trajectory point is obtained, thereby clarifying all the actual trajectory information of the target pedestrian, providing a data basis for predicting the complete trajectory of the target pedestrian, and improving the accuracy of the first predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0153] In a specific implementation, according to all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times, obtaining the actual passing time period corresponding to each first actual trajectory point further includes the following steps:

[0154] According to the first connection range, scanning range, second connection range and GPS positioning data corresponding to each first actual track point, the first connection time period, scanning time period, second connection time period and positioning time corresponding to each first actual track point are obtained.

[0155] The union of the first connection time period, all scanning time periods, all second connection time periods and all positioning times corresponding to each first actual trajectory point is determined as the actual passing time period corresponding to each first actual trajectory point.

[0156] As described above, based on the positional relationship between all the first connection ranges, scanning ranges, second connection ranges and GPS positioning data, the actual time period corresponding to each first actual trajectory point is obtained, thereby clarifying the actual trajectory information of the target pedestrian, providing a data basis for predicting the complete trajectory of the target pedestrian, and improving the accuracy of the first predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0157] In a specific implementation, the pedestrian retrieval method based on video data further includes the following steps:

[0158] An initial trajectory prediction model and several real trajectory set samples are obtained, wherein each real trajectory set sample includes A real trajectory point samples of a preset pedestrian and a real passing time period sample corresponding to each real trajectory point sample.

[0159] For any real trajectory set sample, B real trajectory point samples and corresponding real time period samples in any real trajectory set sample are randomly selected to form the model input sample.

[0160] The model input sample is input into the initial trajectory prediction model to obtain the corresponding A predicted trajectory point samples and the predicted time period sample corresponding to each predicted trajectory point sample.

[0161] According to the A predicted trajectory point samples and the predicted time period sample corresponding to each predicted trajectory point sample, a predicted trajectory set sample is obtained.

[0162] According to any real trajectory set sample and the corresponding predicted trajectory set sample, the model sub-loss corresponding to any real trajectory set sample is obtained.

[0163] All true trajectory set samples are traversed, and the sum of the model sub-losses corresponding to all true trajectory set samples is determined as the total model loss of the initial trajectory prediction model.

[0164] The parameters of the initial trajectory prediction model are adjusted according to the total model loss until the model sub-loss converges to obtain the trained trajectory prediction model.

[0165] As described above, the initial trajectory prediction model is trained based on the real trajectory set samples, the accuracy of the model is characterized according to the similarity between the real trajectory set samples and the corresponding predicted trajectory set samples, and the parameters of the initial trajectory prediction model are adjusted according to the gradient descent method based on the total loss of the model, thereby improving the accuracy of the trained trajectory prediction model, thereby improving the accuracy of the first predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0166] S102: Acquire, according to the first predicted trajectory set, a first video acquisition device corresponding to each first predicted trajectory point and a first initial acquisition video of each first video acquisition device within a corresponding first predicted elapsed time period.

[0167] Among them, the first video acquisition device may refer to a surveillance camera, which is used to collect behavior data of pedestrians, vehicles and other objects within a certain range.

[0168] In the above, the first initial collected video is obtained based on the first predicted trajectory set and used as the video basis for retrieving the target pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0169] S103, obtaining a plurality of first initial pedestrian segmentation maps corresponding to each first initial collected video, and clustering all first initial pedestrian segmentation maps corresponding to all first initial collected videos to obtain K1 first initial pedestrian segmentation map sets, where K1 refers to an integer greater than 0.

[0170] Among them, the first initial collected video includes not only the behavior video of pedestrians, but also the video information of other interference factors such as vehicles, buildings, animals, etc. Therefore, each frame of the first initial collected video is firstly segmented, and the first initial pedestrian segmentation map is obtained as the processing target to eliminate the influence of other interference factors on the retrieval results.

[0171] The same pedestrian may appear successively within the acquisition range of multiple first video acquisition devices, and thus appear successively in multiple first initial acquisition videos. Therefore, all first initial pedestrian segmentation maps corresponding to all first initial acquisition videos are clustered to obtain a set of K1 first initial pedestrian segmentation maps, which are used to characterize the correlation between the same pedestrian in different first initial acquisition videos and the differences between different pedestrians.

[0172] In a specific implementation, obtaining a plurality of first initial pedestrian segmentation maps corresponding to each first initial acquisition video, and clustering all first initial pedestrian segmentation maps corresponding to all first initial acquisition videos, and obtaining a set of K1 first initial pedestrian segmentation maps further includes the following steps:

[0173] According to the preset first frame rate, a plurality of first captured image frames corresponding to each first initial captured video are acquired.

[0174] Image segmentation is performed on each first acquired image frame according to a preset image segmentation algorithm, and a plurality of first initial pedestrian segmentation maps corresponding to each first acquired image frame are obtained.

[0175] All first initial pedestrian segmentation maps corresponding to all first initial collected videos are clustered according to a preset clustering algorithm to obtain K1 first initial pedestrian segmentation map sets.

[0176] The preset first frame rate is determined by the corresponding first video acquisition device.

[0177] In the above, image segmentation is performed on the first initial collected video to obtain a first initial pedestrian segmentation map, and all the first initial pedestrian segmentation maps are clustered to obtain K1 first initial pedestrian segmentation map sets, so as to characterize the correlation between the same pedestrian in different first initial collected videos and the differences between different pedestrians according to the first initial pedestrian segmentation map sets, which serves as the basis for retrieving the target pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0178] S104: Obtain a first similarity between each set of first initial pedestrian segmentation maps and the preset image according to each first initial pedestrian segmentation map and the preset image corresponding to the target pedestrian.

[0179] Among them, the preset image may refer to an image taken while the target pedestrian is walking, which may reflect information such as the body characteristics and walking posture characteristics of the target pedestrian, and serve as a reference basis for retrieving the target pedestrian from the video.

[0180] Among them, those skilled in the art know that any image similarity algorithm in the prior art falls within the protection scope of the present invention, and will not be described in detail here.

[0181] In a specific implementation, according to each first initial pedestrian segmentation map and the preset image corresponding to the target pedestrian, obtaining the first similarity between each first initial pedestrian segmentation map set and the preset image further includes the following steps:

[0182] The first image similarity between each first initial pedestrian segmentation image and the preset image is calculated.

[0183] For any first initial pedestrian segmentation map set, an average value of first image similarities between all first initial pedestrian segmentation maps corresponding to any first initial pedestrian segmentation map set and the preset image is used as the first similarity between any first initial pedestrian segmentation map set and the preset image.

[0184] All first initial pedestrian segmentation map sets are traversed to obtain a first similarity between each first initial pedestrian segmentation map set and a preset image.

[0185] In the above, the average value of the first image similarities between all the first initial pedestrian segmentation maps corresponding to the first initial pedestrian segmentation map set and the preset image is used as the first similarity between the corresponding first initial pedestrian segmentation map set and the preset image, which provides a basis for retrieving the first initial collected video with the highest degree of association with the target pedestrian.

[0186] S105: Obtain a first correlation degree between each first initial pedestrian segmentation map and the target pedestrian according to the first similarity and the number of first video acquisition devices corresponding to each first initial pedestrian segmentation map set.

[0187] The greater the first similarity corresponding to each first initial pedestrian segmentation map set, the higher the correlation between the first initial pedestrian segmentation map set and the target pedestrian. The greater the number of first video acquisition devices corresponding to each first initial pedestrian segmentation map set, the higher the data richness and reliability of the first initial pedestrian segmentation map set.

[0188] Therefore, the first similarity corresponding to each first initial pedestrian segmentation map set and the number of corresponding first video acquisition devices are both positively correlated with the corresponding first association degree.

[0189] In a specific implementation, according to the first similarity and the number of first video acquisition devices corresponding to each first initial pedestrian segmentation map set, obtaining the first correlation degree between each first initial pedestrian segmentation map and the target pedestrian further includes the following steps:

[0190] For the i-th first initial pedestrian segmentation map set, according to the first video acquisition device corresponding to the first initial pedestrian segmentation map corresponding to the i-th first initial pedestrian segmentation map set, the first device number of the first video acquisition device corresponding to the i-th first initial pedestrian segmentation map set is obtained, where i=1, 2..., K1.

[0191] According to the first similarity and the first number of devices corresponding to the i-th first initial pedestrian segmentation map set, the first correlation degree corresponding to the i-th first initial pedestrian segmentation map set is obtained:

[0192] Y 1i =(2 / (1+e^(-S i ))-1)×X 1i .

[0193] Where Y 1i It refers to the first correlation degree corresponding to the i-th first initial pedestrian segmentation map set, S irefers to the number of first devices corresponding to the i-th first initial pedestrian segmentation map set, X 1i It refers to the first similarity corresponding to the i-th first initial pedestrian segmentation map set, and e is an exponential function.

[0194] In the above, combined with the first similarity corresponding to each first initial pedestrian segmentation map set and the corresponding number of first video acquisition devices, the first degree of association between each first initial pedestrian segmentation map and the target pedestrian is calculated, which improves the calculation accuracy of the first degree of association and thereby improves the retrieval accuracy of the target pedestrian.

[0195] S106 , selecting a first target pedestrian segmentation map set from the K1 first initial pedestrian segmentation map sets according to the first correlation degree.

[0196] In a specific implementation, according to the first correlation degree, selecting a first target pedestrian segmentation map set from K1 first initial pedestrian segmentation map sets further includes the following steps:

[0197] The first association degrees corresponding to the K1 first initial pedestrian segmentation map sets are compared, and the first initial pedestrian segmentation map set corresponding to the largest first association degree is determined as the first target pedestrian segmentation map set.

[0198] S107: Determine the first initial collected video corresponding to the first target pedestrian segmentation map set as the first target collected video for retrieving the target pedestrian.

[0199] Among them, the first initial collected video corresponding to the first target pedestrian segmentation map set is determined as the first target collected video for retrieving the target pedestrian. The movement trajectory, behavioral activities, peer contact and other behaviors of the target pedestrian can be analyzed through the first target collected video to provide data support for finding the target object, thereby improving the accuracy of tracking the person.

[0200] In the above, the first initial collected video is obtained based on the first predicted trajectory set, and the influence of interference factors on the retrieval results is eliminated through image segmentation. The correlation between the same pedestrian in different first initial collected videos and the difference between different pedestrians are characterized according to the first initial pedestrian segmentation map set. Therefore, the first similarity corresponding to each first initial pedestrian segmentation map set and the corresponding number of first video acquisition devices are combined to calculate the first correlation degree between each first initial pedestrian segmentation map and the target pedestrian, thereby improving the calculation accuracy of the first correlation degree, thereby improving the retrieval accuracy and tracking accuracy of the target pedestrian.

[0201] Embodiment 3

[0202] The third embodiment provides a pedestrian retrieval method based on trajectory data. The pedestrian retrieval method based on trajectory data includes: Figure 3 As shown:

[0203] S201, obtaining a preset track point collection device, wherein the track point collection device includes a wifi device, a public mobile communication base station and a GPS device.

[0204] Among them, the connection range and scanning range of the fixed wifi device are fixed, and the connection range of the public mobile communication base station is also determined, and the GPS device carried or used by the target pedestrian can report the location data of the target pedestrian. Therefore, the location area of ​​the target pedestrian can be obtained based on the wifi device connected or scanned by the target pedestrian, the public mobile communication base station connected to the target pedestrian, and the GPS device carried or used by the target pedestrian, as the basis for obtaining the second target acquisition video for retrieving the target pedestrian.

[0205] S202, according to the Wi-Fi device connected to the target pedestrian and the scanned Wi-Fi devices, obtain a first connection range and a first connection time period corresponding to each connected Wi-Fi device, and a scanning range and a scanning time period corresponding to each scanned Wi-Fi device.

[0206] S203: According to the public mobile communication base station to which the target pedestrian is connected, a second connection range and a second connection time period corresponding to each connected public mobile communication base station are acquired.

[0207] S204: Obtain a GPS data set corresponding to the target pedestrian according to the GPS device, wherein the GPS data set includes GPS positioning data of the target pedestrian and the positioning time corresponding to each GPS positioning data.

[0208] S205: Acquire a second actual trajectory point corresponding to the target pedestrian according to all first connection ranges, all scanning ranges, all second connection ranges and all GPS positioning data.

[0209] Among them, the first connection range, scanning range, second connection range and GPS positioning data can all represent the location information of the target pedestrian. Then, by combining all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data, several second actual trajectory points corresponding to the target pedestrian can be obtained as the basis for obtaining the second target acquisition video for retrieving the target pedestrian.

[0210] In the above, by combining all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data, the second actual trajectory point corresponding to the target pedestrian is obtained, thereby improving the accuracy of obtaining the second actual trajectory point.

[0211] S206: Acquire a second actual passing time period corresponding to each second actual trajectory point according to all first connection time periods, all scanning time periods, all second connection time periods, and all positioning times.

[0212] As described above, on the basis of the second actual trajectory point, combined with the first connection time period, scanning time period, second connection time period and positioning time, the second actual passing time period corresponding to each second actual trajectory point is obtained, which further supplements the actual trajectory information of the target pedestrian, provides a data basis for predicting the complete trajectory of the target pedestrian, and improves the accuracy of the second predicted trajectory set and the retrieval accuracy of the target pedestrian.

[0213] S207, according to all the second actual trajectory points and the second actual elapsed time period corresponding to each second actual trajectory point, obtain the second video acquisition device corresponding to each second actual trajectory point and the second initial acquisition video of each second video acquisition device in the corresponding second actual elapsed time period.

[0214] Among them, the second video acquisition device can refer to a surveillance camera, which is used to collect behavior data of pedestrians, vehicles and other objects within a certain range.

[0215] In the above, the second initial collected video is obtained based on all the second actual trajectory points and the second actual passing time period, and is used as the video basis for retrieving the target pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0216] S208: According to the preset image corresponding to the target pedestrian, a second similarity between each second initially collected video and the preset image is obtained.

[0217] In a specific implementation, according to the preset image corresponding to the target pedestrian, obtaining the second similarity between each second initial collected video and the preset image further includes the following steps:

[0218] A plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video are obtained.

[0219] According to each second initial pedestrian segmentation map and the preset image, a second image similarity between each second initial pedestrian segmentation map and the preset image is calculated.

[0220] For any second initial collected video, an average value of the second image similarities between all second initial pedestrian segmentation images corresponding to the second initial collected video and the preset image is taken as the second similarity between any second initial collected video and the preset image.

[0221] All second initially collected videos are traversed to obtain a second similarity between each second initially collected video and the preset image.

[0222] In the above, image segmentation is performed on the second initial collected video to obtain a second initial pedestrian segmentation map, and the second image similarity between each second initial pedestrian segmentation map and the preset image is calculated respectively to characterize the similarity between the pedestrian in each second initial pedestrian segmentation map and the target pedestrian, and the second similarity between each second initial collected video and the preset image is further obtained to characterize the correlation between each second initial collected video and the target pedestrian, which serves as the basis for retrieving the target pedestrian, thereby improving the retrieval accuracy of the target pedestrian.

[0223] S209: Filter out a second target collected video from all the second initial collected videos according to a comparison result between the second similarity corresponding to the second initial collected video and a preset similarity threshold.

[0224] Among them, when the second similarity corresponding to the second initial collected video is greater than the preset similarity threshold, the corresponding second initial collected video will be determined as the second target collected video, so as to analyze the movement trajectory, behavioral activities, peer contacts and other behaviors of the target pedestrian in a certain time period from the second target collected video to provide data support for finding the target object, thereby improving the accuracy of tracking personnel.

[0225] The specific value of the preset similarity threshold can be set by the implementer according to actual conditions.

[0226] In the above, combined with the location information of the target pedestrian provided by the wifi device, the public mobile communication base station and the GPS device, the second actual trajectory point and the second actual time period corresponding to the target pedestrian are obtained, the actual trajectory information of the target pedestrian is clarified, and then combined with the preset image of the target pedestrian, the second target captured video is screened out from the second initial captured video corresponding to the actual trajectory, thereby improving the accuracy of the second target captured video and thus improving the retrieval accuracy of the target pedestrian.

[0227] Embodiment 4

[0228] The fourth embodiment provides a data processing device for pedestrian retrieval, the data processing device for pedestrian retrieval includes: Figure 4 As shown:

[0229] The data acquisition module 41 is used to acquire the second initial collected video corresponding to the target pedestrian, and the predicted trajectory points and the second video acquisition device corresponding to each second initial collected video.

[0230] The data processing module 42 is used to perform image segmentation on each second initial collected video to obtain a plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video.

[0231] The image clustering module 43 is used to cluster all the second initial pedestrian segmentation maps corresponding to all the second initial collected videos to obtain K2 sets of second initial pedestrian segmentation maps, where K2 refers to an integer greater than 0.

[0232] The first clustering reliability calculation module 44 is used to calculate the third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set for any second initial pedestrian segmentation map set, and determine the average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as the first clustering reliability corresponding to the current second initial pedestrian segmentation map set.

[0233] The quantity acquisition module 45 is used to acquire the quantity of trajectory points and the quantity of devices corresponding to the current second initial pedestrian segmentation map set according to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquisition video.

[0234] The third correlation degree acquisition module 46 is used to obtain the third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set.

[0235] The third set screening module 47 is used to screen out a third target pedestrian segmentation map set from K2 second initial pedestrian segmentation map sets according to the third correlation degrees corresponding to all second initial pedestrian segmentation map sets.

[0236] The third video screening module 48 is used to determine the second initial collected video corresponding to the third target pedestrian segmentation map set as the third target collected video for retrieving the target pedestrian.

[0237] In a specific implementation, the data acquisition module 41 includes:

[0238] The second trajectory prediction submodule is used to obtain a second predicted trajectory set corresponding to the target pedestrian, wherein the second predicted trajectory set includes M2 second predicted trajectory points and a second predicted passing time period corresponding to each second predicted trajectory point, and M2 refers to an integer greater than 0.

[0239] The second video acquisition submodule is used to obtain T second video acquisition devices corresponding to each second predicted trajectory point and the second initial acquisition video of each second video acquisition device within the corresponding second predicted time period according to the second predicted trajectory set, wherein T refers to the total number of second video acquisition devices.

[0240] In a specific implementation, the second trajectory prediction submodule includes:

[0241] The data acquisition unit is used to acquire a preset trajectory point acquisition device and a trained trajectory prediction model, wherein the trajectory point acquisition device includes a wifi device, a public mobile communication base station and a GPS device.

[0242] The wifi device data acquisition submodule is used to obtain the first connection range and the first connection time period corresponding to each connected wifi device, and the scanning range and the scanning time period corresponding to each scanned wifi device according to the wifi devices connected to the target pedestrian and the scanned wifi devices.

[0243] The base station data acquisition submodule is used to acquire the second connection range and the second connection time period corresponding to each public mobile communication base station connected to the target pedestrian according to the public mobile communication base station connected to the target pedestrian.

[0244] The GPS data acquisition submodule is used to acquire a GPS data set corresponding to a target pedestrian according to a GPS device, wherein the GPS data set includes the GPS positioning data of the target pedestrian and the positioning time corresponding to each GPS positioning data.

[0245] The second actual trajectory point acquisition submodule is used to acquire the second actual trajectory point corresponding to the target pedestrian according to all the first connection ranges, all the scanning ranges, all the second connection ranges and all the GPS positioning data.

[0246] The second time period acquisition submodule is used to acquire the second actual passing time period corresponding to each second actual trajectory point according to all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times.

[0247] The second trajectory prediction submodule is used to input all the second actual trajectory points and the second actual elapsed time period corresponding to each second actual trajectory point into the trained trajectory prediction model to obtain a second predicted trajectory set.

[0248] In a specific implementation, the data processing device further includes the following modules:

[0249] The second data acquisition module is used to obtain the initial description text corresponding to the target pedestrian, the trained feature extraction model and the collection time corresponding to each second initial pedestrian segmentation map.

[0250] The segmentation map clustering module is used to cluster all second initial pedestrian segmentation maps corresponding to all second initial acquisition videos based on the acquisition time, and obtain K2 third initial pedestrian segmentation map sets, wherein the second initial pedestrian segmentation maps in each third initial pedestrian segmentation map set are sorted according to the corresponding acquisition time.

[0251] The first similarity calculation module is used to calculate the fifth image similarity between any two second initial pedestrian segmentation maps in the current third initial pedestrian segmentation map set for any third initial pedestrian segmentation map set, and determine the average value of all fifth image similarities as the second clustering reliability corresponding to the current third initial pedestrian segmentation map set.

[0252] The feature extraction module is used to input the current third initial pedestrian segmentation map set into the trained feature extraction model to obtain the pedestrian feature description text corresponding to the current third initial pedestrian segmentation map set.

[0253] The second similarity calculation module is used to obtain the sixth similarity between the current third initial pedestrian segmentation map set and the target pedestrian according to the pedestrian feature description text and the initial description text.

[0254] The fourth correlation degree acquisition module is used to obtain the fourth correlation degree between the current third initial pedestrian segmentation map and the target pedestrian based on the second clustering reliability, the sixth similarity, the number of predicted trajectory points and the number of second initial collected videos corresponding to the current third initial pedestrian segmentation map set.

[0255] The fourth set screening module is used to screen out a fourth target pedestrian segmentation map set from K2 third initial pedestrian segmentation map sets according to fourth correlation degrees corresponding to all third initial pedestrian segmentation map sets.

[0256] The fourth video screening module is used to determine the second initial collected video corresponding to the fourth target pedestrian segmentation map set as the fourth target collected video for retrieving the target pedestrian.

[0257] In a specific implementation, the third association degree acquisition module 46 includes:

[0258] The third association degree calculation submodule is used to calculate the first clustering reliability Z corresponding to the j-th second initial pedestrian segmentation map set according to the j-th second initial pedestrian segmentation map set. j , the number of trajectory points GJ j and the number of devices P j , get the third correlation degree between the jth second initial pedestrian segmentation map set and the target pedestrian:

[0259] Y 3j =(2 / (1+e^(-(α1×P j +α2×GJ j )))-1)×Z j .

[0260] Where Y 3j It refers to the third correlation degree corresponding to the jth second initial pedestrian segmentation map set, P jGJ refers to the number of devices corresponding to the jth second initial pedestrian segmentation map set. j refers to the number of trajectory points corresponding to the jth second initial pedestrian segmentation map set, Z j It refers to the first clustering reliability corresponding to the j-th second initial pedestrian segmentation map set, α1 refers to the first preset weight, α2 refers to the second preset weight, and e is an exponential function.

[0261] In a specific implementation, the third set screening module 47 includes:

[0262] The third set screening submodule is used to compare the third correlation degrees corresponding to the K2 second initial pedestrian segmentation map sets, and determine the second initial pedestrian segmentation map set corresponding to the largest third correlation degree as the third target pedestrian segmentation map set.

[0263] It should be noted that the information interaction, execution process and other contents between the above modules are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0264] Embodiment 5

[0265] Embodiment 5 of the present invention provides a non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0266] The second initial collected video corresponding to the target pedestrian is obtained, as well as the predicted trajectory points and the second video collection device corresponding to each second initial collected video.

[0267] Image segmentation is performed on each second initial collected video to obtain a plurality of second initial pedestrian segmentation maps corresponding to each second initial collected video.

[0268] All second initial pedestrian segmentation maps corresponding to all second initial collected videos are clustered to obtain K2 sets of second initial pedestrian segmentation maps, where K2 is an integer greater than 0.

[0269] For any second initial pedestrian segmentation map set, calculate the third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine the average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as the first clustering reliability corresponding to the current second initial pedestrian segmentation map set.

[0270] According to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquired video, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set are obtained.

[0271] According to the first clustering reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set, a third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian is obtained.

[0272] According to the third correlation degrees corresponding to all the second initial pedestrian segmentation map sets, a third target pedestrian segmentation map set is screened out from the K2 second initial pedestrian segmentation map sets.

[0273] The second initial collected video corresponding to the third target pedestrian segmentation map set is determined as the third target collected video for retrieving the target pedestrian.

[0274] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchronization Link) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0275] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0276] Embodiment 6

[0277] A sixth embodiment of the present invention provides an electronic device, which includes a processor and the non-transitory computer-readable storage medium in the fifth embodiment of the present invention.

[0278] The above are only preferred embodiments of the present invention, and are not intended to limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A data processing method for pedestrian retrieval, characterized in that: The data processing method comprises the following steps: Acquire a second initial collected video corresponding to the target pedestrian, and a predicted trajectory point and a second video collection device corresponding to each second initial collected video; Performing image segmentation on each second initial collected video to obtain a plurality of second initial pedestrian segmentation images corresponding to each second initial collected video; Clustering all second initial pedestrian segmentation maps corresponding to all second initial collected videos to obtain K2 sets of second initial pedestrian segmentation maps, where K2 is an integer greater than 0; For any second initial pedestrian segmentation map set, calculate the third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine the average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as the first clustering reliability corresponding to the current second initial pedestrian segmentation map set; According to the predicted trajectory points corresponding to each second initial collected video and the second video collection device, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set are obtained; Acquire a third degree of association between the current second initial pedestrian segmentation map and the target pedestrian according to the first clustering reliability, the number of trajectory points, and the number of devices corresponding to the current second initial pedestrian segmentation map set; According to the third correlation degrees corresponding to all the second initial pedestrian segmentation map sets, a third target pedestrian segmentation map set is screened out from the K2 second initial pedestrian segmentation map sets; The second initial collected video corresponding to the third target pedestrian segmentation map set is determined as the third target collected video for retrieving the target pedestrian.

2. The data processing method for pedestrian retrieval according to claim 1, characterized in that: The step of obtaining the second initial collected video corresponding to the target pedestrian also includes the following steps: A second predicted trajectory set corresponding to the target pedestrian is obtained, wherein the second predicted trajectory set includes M2 second predicted trajectory points and a second predicted passing time period corresponding to each second predicted trajectory point, where M2 refers to an integer greater than 0; According to the second predicted trajectory set, T second video acquisition devices corresponding to each second predicted trajectory point and the second initial acquisition video of each second video acquisition device within the corresponding second predicted elapsed time period are obtained, where T refers to the total number of second video acquisition devices.

3. The data processing method for pedestrian retrieval according to claim 2, characterized in that: The step of obtaining the second predicted trajectory set corresponding to the target pedestrian further includes the following steps: Obtaining a preset trajectory point collection device and a trained trajectory prediction model, wherein the trajectory point collection device includes a wifi device, a public mobile communication base station and a GPS device; According to the wifi device connected to the target pedestrian and the scanned wifi devices, a first connection range and a first connection time period corresponding to each connected wifi device, and a scanning range and a scanning time period corresponding to each scanned wifi device are obtained; According to the public mobile communication base station connected to the target pedestrian, acquiring a second connection range and a second connection time period corresponding to each connected public mobile communication base station; According to the GPS device, a GPS data set corresponding to the target pedestrian is obtained, wherein the GPS data set includes GPS positioning data of the target pedestrian and a positioning time corresponding to each GPS positioning data; Acquire a second actual trajectory point corresponding to the target pedestrian according to all first connection ranges, all scanning ranges, all second connection ranges and all GPS positioning data; According to all the first connection time periods, all the scanning time periods, all the second connection time periods and all the positioning times, a second actual passing time period corresponding to each second actual trajectory point is obtained; All the second actual trajectory points and the second actual elapsed time period corresponding to each second actual trajectory point are input into the trained trajectory prediction model to obtain a second predicted trajectory set.

4. The data processing method for pedestrian retrieval according to claim 1, characterized in that: The data processing method further comprises the following steps: Obtaining the initial description text corresponding to the target pedestrian, the trained feature extraction model, and the acquisition time corresponding to each second initial pedestrian segmentation map; Based on the acquisition time, clustering all second initial pedestrian segmentation maps corresponding to all second initial acquisition videos to obtain K2 third initial pedestrian segmentation map sets, wherein the second initial pedestrian segmentation maps in each third initial pedestrian segmentation map set are sorted according to the corresponding acquisition time; For any third initial pedestrian segmentation map set, calculate the fifth image similarity between any two second initial pedestrian segmentation maps in the current third initial pedestrian segmentation map set, and determine the average value of all fifth image similarities as the second clustering reliability corresponding to the current third initial pedestrian segmentation map set; Inputting the current third initial pedestrian segmentation map set into the trained feature extraction model to obtain pedestrian feature description text corresponding to the current third initial pedestrian segmentation map set; According to the pedestrian feature description text and the initial description text, a sixth similarity between the current third initial pedestrian segmentation map set and the target pedestrian is obtained; Obtaining a fourth degree of association between the current third initial pedestrian segmentation map and the target pedestrian according to the second clustering reliability, the sixth similarity, the number of predicted trajectory points, and the number of second initial collected videos corresponding to the current third initial pedestrian segmentation map set; According to the fourth correlation degrees corresponding to all the third initial pedestrian segmentation map sets, a fourth target pedestrian segmentation map set is screened out from the K2 third initial pedestrian segmentation map sets; The second initial collected video corresponding to the fourth target pedestrian segmentation map set is determined as the fourth target collected video for retrieving the target pedestrian.

5. The data processing method for pedestrian retrieval according to claim 4, characterized in that: The initial description text includes a preset first face description text, a preset first body shape description text and a preset first walking posture description text of the target pedestrian, and the pedestrian feature description text includes a second face description text, a second body shape description text and a second walking posture description text of the pedestrian in the corresponding second initial pedestrian segmentation map set.

6. The data processing method for pedestrian retrieval according to claim 1, characterized in that: The step of obtaining a third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set further includes the following steps: For the j-th second initial pedestrian segmentation map set, according to the first clustering reliability Z corresponding to the j-th second initial pedestrian segmentation map set j , the number of trajectory points GJ j and the number of devices P j , obtain the third correlation degree between the j-th second initial pedestrian segmentation map set and the target pedestrian: AND 3j =(2 / (1+e^(-(α1×P j +α2×GJ j )))-1)×Z j ; Where Y 3j It refers to the third correlation degree corresponding to the j-th second initial pedestrian segmentation map set, P j It refers to the number of devices corresponding to the j-th second initial pedestrian segmentation map set, GJ j refers to the number of trajectory points corresponding to the j-th second initial pedestrian segmentation map set, Z j It refers to the first clustering reliability corresponding to the j-th second initial pedestrian segmentation map set, α1 refers to the first preset weight, α2 refers to the second preset weight, and e is an exponential function.

7. The data processing method for pedestrian retrieval according to claim 6, characterized in that: The step of selecting a third target pedestrian segmentation map set from K2 second initial pedestrian segmentation map sets according to the third correlation degrees corresponding to all second initial pedestrian segmentation map sets further comprises the following steps: The third association degrees corresponding to the K2 second initial pedestrian segmentation map sets are compared, and the second initial pedestrian segmentation map set corresponding to the largest third association degree is determined as the third target pedestrian segmentation map set.

8. A data processing device for pedestrian retrieval, characterized in that: The data processing device comprises: A data acquisition module, used to acquire a second initial acquisition video corresponding to the target pedestrian, and a predicted trajectory point and a second video acquisition device corresponding to each second initial acquisition video; A data processing module, used to perform image segmentation on each second initial collected video, and obtain a plurality of second initial pedestrian segmentation images corresponding to each second initial collected video; An image clustering module, used for clustering all second initial pedestrian segmentation maps corresponding to all second initial collected videos, and obtaining K2 sets of second initial pedestrian segmentation maps, where K2 refers to an integer greater than 0; A first clustering reliability calculation module is used to calculate, for any second initial pedestrian segmentation map set, a third image similarity between any two second initial pedestrian segmentation maps in the current second initial pedestrian segmentation map set, and determine an average value of all third image similarities corresponding to the current second initial pedestrian segmentation map set as a first clustering reliability corresponding to the current second initial pedestrian segmentation map set; A quantity acquisition module, used to acquire the quantity of trajectory points and the quantity of devices corresponding to the current second initial pedestrian segmentation map set according to the predicted trajectory points and the second video acquisition devices corresponding to each second initial acquisition video; A third correlation degree acquisition module is used to acquire a third correlation degree between the current second initial pedestrian segmentation map and the target pedestrian according to the first cluster reliability, the number of trajectory points and the number of devices corresponding to the current second initial pedestrian segmentation map set; A third set screening module is used to screen out a third target pedestrian segmentation map set from the K2 second initial pedestrian segmentation map sets according to the third correlation degrees corresponding to all the second initial pedestrian segmentation map sets; The third video screening module is used to determine the second initial collected video corresponding to the third target pedestrian segmentation map set as the third target collected video for retrieving the target pedestrian.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the data processing method for pedestrian retrieval as described in any one of claims 1-7.

10. An electronic device, characterized in that: The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.

Images