Abnormal trajectory recognition method and device, electronic device, and storage medium

Abstract

The present application discloses an abnormal trajectory recognition method and device, an electronic device, and a storage medium. The method includes obtaining the trajectory sequences of different objects respectively; wherein, the trajectory sequence contains captured images suspected to belong to the same object, and the captured images in the trajectory sequence are sorted according to the capture time, and the captured images are also attached with the capture location; based on the geographical regions to which the capture locations of the captured images in the trajectory sequence belong, select the trajectory sequence as a candidate trajectory suspected of being abnormal; in response to the candidate trajectory meeting a preset abnormal condition, select the geographical region with the most frequent activities in the candidate trajectory as the first region, and select the discontinuous geographical regions in the candidate trajectory as the second region; based on the first similarity between the captured images with capture locations in the first region and the captured images with capture locations in the second region in the candidate trajectory, determine whether the candidate trajectory is a target trajectory with an abnormality. The above solution can improve the accuracy of recognizing abnormal trajectories.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular, to a method and device for identifying abnormal trajectories, an electronic device, and a storage medium. Background Art

[0002] With the progress of technology and the needs of humans for a convenient life, various face-swiping applications have begun to spread, including face-swiping for unlocking, opening doors, security checks, boarding planes, checking into hotels, and paying for medical treatment. They have taken root in all walks of life and generated a large amount of trajectory data based on human portraits.

[0003] Currently, the most common application of human portrait trajectory data is human portrait clustering, which clusters the trajectory data of human portraits belonging to the same person into one file. Due to inaccurate human portrait recognition, picture quality, picture shooting angle, and other problems in traditional human portrait clustering, face pictures of different people may be clustered into the same file. In view of this, how to improve the accuracy of identifying abnormal trajectories has become an urgent problem to be solved. Summary of the Invention

[0004] The main technical problem to be solved by the present application is to provide a method and device for identifying abnormal trajectories, an electronic device, and a storage medium, which can improve the accuracy of identifying abnormal trajectories.

[0005] To solve the above technical problem, a first aspect of the present application provides a method for identifying abnormal trajectories, including: obtaining the trajectory sequences of different objects respectively; wherein, the trajectory sequence contains captured images suspected to belong to the same object, and the captured images in the trajectory sequence are sorted according to the shooting time, and the captured images are also attached with shooting locations; selecting the trajectory sequence as a candidate trajectory suspected of being abnormal based on the geographical area to which the shooting location of each captured image in the trajectory sequence belongs; in response to the candidate trajectory satisfying a preset abnormal condition, selecting the geographical area with the most frequent activities in the candidate trajectory as the first area, and selecting the discontinuous geographical areas in the candidate trajectory as the second area; determining whether the candidate trajectory is a target trajectory with an abnormality based on a first similarity between the captured images located in the first area and the captured images located in the second area in the candidate trajectory.

[0006] To solve the above technical problems, a second aspect of the present application provides a noise - following gain device, including a trajectory acquisition module for acquiring the trajectory sequences of different objects respectively; wherein, the trajectory sequence contains captured images suspected to belong to the same object, and the captured images in the trajectory sequence are sorted by the capture time, and the captured images are also attached with the capture location; a trajectory selection module for selecting the trajectory sequence as a candidate trajectory suspected of being abnormal based on the geographical area to which the capture locations of the captured images in the trajectory sequence belong; a region selection module for, in response to the candidate trajectory satisfying a preset abnormal condition, selecting the geographical area with the most frequent activities in the candidate trajectory as the first region, and selecting the discontinuous geographical areas in the candidate trajectory as the second region; an abnormality determination module for determining whether the candidate trajectory is a target trajectory with an abnormality based on the first similarity between the captured images with capture locations in the first region and the captured images with capture locations in the second region in the candidate trajectory.

[0007] To solve the above technical problems, a third aspect of the present application provides an electronic device, including a memory and a processor coupled to each other. The memory stores program instructions, and the processor is configured to execute the program instructions to implement the abnormal trajectory recognition method in the first aspect above.

[0008] To solve the above technical problems, a fourth aspect of the present application provides a computer - readable storage medium storing program instructions that can be run by a processor, and the program instructions are used to implement the abnormal trajectory recognition method in the first aspect above.

[0009] In the above solution, the captured images suspected to belong to the same target object are archived based on the portrait features included in the captured images, and are sorted by the capture time attached to the captured images through the attached capture location to obtain the trajectory sequences of different objects respectively. By pre - dividing different geographical areas, the geographical area to which the capture location of each captured image in the trajectory sequence belongs is obtained, and the trajectory sequence is selected as a candidate trajectory suspected of being abnormal. It is judged whether the candidate trajectory satisfies the preset abnormal condition. When the candidate trajectory satisfies the preset abnormal condition, it indicates that there is a region with trajectory jumps in the candidate trajectory. The geographical area with the most frequent activities in the candidate trajectory is selected as the first region, and the discontinuous geographical areas in the candidate trajectory are selected as the second region. Whether the candidate trajectory is a target trajectory with an abnormality is determined based on the first similarity between the captured images with capture locations in the first region and the captured images with capture locations in the second region in the candidate trajectory. Therefore, the trajectory sequence of the target object can be obtained through the information attached to the captured images, and the candidate trajectory suspected of being abnormal can be quickly located based on the geographical area to which the capture locations in the trajectory sequence belong, and the abnormal trajectory is determined based on the first similarity between the captured images in the first region and the second region, improving the efficiency of abnormal trajectory recognition and the accuracy of identifying abnormal trajectories at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a schematic flowchart of an embodiment of the abnormal trajectory recognition method of the present application;

[0011] Figure 2 It is a schematic framework diagram of an embodiment of the abnormal trajectory recognition device of the present application;

[0012] Figure 3 It is a schematic framework diagram of an embodiment of the electronic device of the present application;

[0013] Figure 4 It is a schematic framework diagram of an embodiment of the computer-readable storage medium of the present application. Specific embodiments

[0014] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0015] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this article means two or more than two.

[0016] Please refer to Figure 1 , Figure 1 It is a schematic flowchart of an embodiment of the abnormal trajectory recognition method of the present application. Specifically, the following steps can be included:

[0017] Step S10: Obtain the trajectory sequences of different objects respectively.

[0018] In an implementation scenario, the captured images are obtained based on a preset time range and within a preset spatial range, and the captured images include portrait information. Since the captured images are scattered and disordered, it is necessary to cluster the obtained captured images based on the portrait information they contain, clustering the originally discrete captured images into multiple archives, with each archive belonging to a target object. Thus, multiple archives containing all the captured images suspected of belonging to the same object can be obtained. For example, multiple images captured by a camera at a certain checkpoint are A1, B1, A2, B2, C1, C2, A3, C3, C4 respectively. Through clustering based on the portrait information in the images, archives corresponding to person A, person B, and person C can be obtained. The archive of person A includes images A1, A2, A3, the archive of person B includes images B1, B2, and the archive of person C includes images C1, C2, C3, C4.

[0019] In a specific implementation scenario, clustering the captured images based on portrait features can be achieved through clustering methods, such as KMA (K-Means Algorithm), DBSCAN (Density-Based Spatial Clustering of Applications with Noise), GMM (Adaptive background mixture models for real-time tracking), etc. It should be noted that in different embodiments, the method for clustering the captured images is not limited in this application.

[0020] In an implementation scenario, the captured images are attached with corresponding shooting times and shooting locations. Based on the shooting times and shooting locations attached to the captured images of the same object, a corresponding trajectory sequence can be obtained. For example, the archive of person A includes images A1, A2, A3. Based on the shooting times and shooting locations of each image, a trajectory sequence {A1, A2, A3, A4, A5} generated by person A's activities on that day can be obtained. Through the above method, based on the portrait features included in the captured images and the shooting times and shooting locations attached to the captured images, a trajectory sequence corresponding to the target object can be obtained, the trajectory pattern of the target object can be obtained, and the trajectory area can be analyzed based on the trajectory pattern, which helps to quickly screen abnormal archives.

[0021] In an implementation scenario, after obtaining the trajectory sequence of the target object, it can be compared with the average features of the previously established archives. For example, the portrait features of trajectory A1 of person A are used to calculate the similarity with the average features of all archives. If the similarity calculated with archive B meets the pre-set threshold, then A1 is classified into archive B. If the threshold is not met after traversing all archives, a new archive C is created and A1 is classified into archive C. Subsequently, the feature comparison of other trajectory points of person A is performed in turn. Generally, all trajectory points of person A will be classified into the same archive.

[0022] Step S20: Based on the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong, select the trajectory sequence as a candidate trajectory for suspected anomaly.

[0023] In the embodiments of the present disclosure, a geographical region is a spatial unit on the earth's surface, which is divided based on geographical differences according to certain indicators and methods. Due to different purposes, different indicators and methods are used, and different types of regions are divided.

[0024] In an implementation scenario, the geographical region can be divided by the longitude and latitude of the capture bayonet of the captured image, and the upper and lower limits are directly divided into blocks based on the longitude and latitude.

[0025] In another implementation scenario, block division can be performed according to the GeoHash (geographical hash) algorithm. GeoHash is essentially a way of spatial indexing. Its basic principle is to understand the earth as a two-dimensional plane, recursively decompose the plane into smaller sub-blocks, and each sub-block has the same code within a certain range of longitude and latitude. Establishing a spatial index in the GeoHash way can improve the efficiency of longitude and latitude retrieval of spatial poi data.

[0026] It should be noted that in different embodiments, the ways and methods of geographical region division are not limited in this application.

[0027] In an implementation scenario, based on the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong, it is determined whether the trajectory sequence has a trajectory jump. When the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong to the same region, it can be considered that the trajectory sequence does not have a trajectory jump. When the geographical regions to which the shooting locations of the captured images in the trajectory sequence include at least two different regions, it can be considered that the trajectory sequence may have a trajectory jump, and thus a candidate trajectory for suspected anomaly is obtained. Through the above method, the trajectory sequence is initially screened through the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong, which can reduce the subsequent workload, so the efficiency of abnormal trajectory recognition can be improved.

[0028] In a specific implementation scenario, after obtaining the trajectory sequence of the target object, it is detected whether the geographical regions to which the shooting locations corresponding to the respective captured images included in the trajectory sequence belong are the same geographical region. When the geographical regions to which all the shooting locations belong are the same geographical region, the probability of misfiling the file trajectories that have been continuously active in one region is very small, and it can be considered that there is no abnormality in the file and it can be directly filtered out.

[0029] In another specific implementation scenario, when the geographical regions to which all the shooting locations in a certain trajectory sequence do not belong to the same geographical region, it can be considered that there may be an abnormality in this trajectory sequence, and this trajectory sequence is selected as a candidate trajectory for suspected abnormality, and further detection and screening are performed on the candidate trajectory.

[0030] Step S30: In response to the candidate trajectory satisfying the preset abnormality condition, select the geographical region with the most frequent activities in the candidate trajectory as the first region, and select the discontinuous geographical regions in the candidate trajectory as the second region.

[0031] In an implementation scenario, the preset abnormality condition is used to determine whether a trajectory jump occurs in the trajectory sequence, that is, the trajectory points in the trajectory sequence are discontinuous. After obtaining the candidate trajectory for suspected abnormality, the frequencies of the geographical regions to which the shooting locations of the respective captured images in the candidate trajectory belong are counted, and sorted according to the frequencies, the geographical region with the most frequent occurrences is obtained, and based on whether the candidate trajectory satisfies the preset abnormality condition, it is determined whether a trajectory jump occurs in the candidate trajectory. Through the above method, the preset abnormality condition is set, and by determining whether the candidate trajectory satisfies the preset abnormality condition, the result of whether there is a trajectory jump region is obtained, so that the efficiency of abnormal trajectory recognition can be improved while the accuracy of recognizing abnormal trajectories is improved.

[0032] In a specific implementation scenario, the preset abnormality condition includes at least a first sub-condition, a second sub-condition, and a third sub-condition. The first sub-condition includes: the geographical region to which the start point and / or end point of the trajectory sequence belong does not belong to the geographical region with the most frequent activities, and the geographical regions to which the remaining trajectory points of the trajectory sequence belong belong to the geographical region with the most frequent activities. For example, the trajectory sequence is {region A, most frequent activity region B,..., most frequent activity region B, region C}; the second sub-condition includes: the geographical regions to which the start point and end point of the trajectory sequence belong belong to the geographical region with the most frequent activities, and there are some trajectory points whose geographical regions do not belong to the geographical region with the most frequent activities. For example, the trajectory sequence is {most frequent activity region B,..., most frequent activity region B, region C, most frequent activity region B,..., most frequent activity region B}; the third sub-condition includes: the trajectory sequence includes only two trajectory points and belong to different geographical regions respectively. For example, the trajectory sequence is {region A, region B}.

[0033] In a specific implementation scenario, when the candidate trajectory does not meet the preset abnormal condition, it can be directly determined that the candidate trajectory is not the target trajectory with an abnormality, and it can be directly filtered out. Through the above method, by setting the preset abnormal condition, the candidate trajectories with abnormalities and those without abnormalities can be quickly classified, improving the efficiency of abnormal trajectory recognition and the accuracy of identifying abnormal trajectories at the same time.

[0034] In another specific implementation scenario, by sequentially detecting whether the candidate trajectory meets the first sub-condition, the second sub-condition, and the third sub-condition, the computing power for abnormal trajectory detection can be reduced, and the efficiency of abnormal trajectory detection can be improved. When the candidate trajectory meets the first sub-condition, the step of detecting the candidate trajectory is directly skipped, and the geographical area to which the starting point and / or the ending point belongs is used as the discontinuous geographical area in the candidate trajectory. For example, if the trajectory sequence of the candidate trajectory is {area A, most frequently active area B,..., most frequently active area B, area A}, area A is the discontinuous geographical area in the candidate trajectory, so area A is used as the second area, and area B is the most frequently active geographical area in the candidate trajectory, so area B is used as the first area; when the candidate trajectory does not meet the first sub-condition but meets the second sub-condition, the step of detecting the candidate trajectory is directly skipped, and the geographical area to which the trajectory point that does not belong to the most frequently active activity area belongs is used as the discontinuous geographical area in the candidate trajectory. For example, if the trajectory sequence of the candidate trajectory is {most frequently active area A,..., most frequently active area A, area B, most frequently active area A,..., most frequently active area A}, at this time, area A is used as the first area, and area B is used as the second area; when the candidate trajectory does not meet the first sub-condition and the second sub-condition but meets the third sub-condition, the step of detecting the candidate trajectory is skipped, and the geographical areas to which two trajectory points belong are respectively used as the most frequently active geographical area and the discontinuous geographical area in the candidate trajectory.

[0035] It should be noted that in this application, the detection order of the preset abnormal condition is not limited in different embodiments.

[0036] Step S40: Based on the first similarity between the captured images of the candidate trajectory at the shooting locations in the first area and the captured images at the shooting locations in the second area, determine whether the candidate trajectory is the target trajectory with an abnormality.

[0037] In an implementation scenario, the detection based on the candidate trajectory can obtain the geographical areas with the most frequent activities and the discontinuous geographical areas. The geographical area with the most frequent activities is taken as the first area, and the discontinuous geographical area is taken as the second area. Calculate the first similarity between all the captured images in the first area and all the captured images in the second area. The first similarity maliciously represents the similarity relationship between the captured images in the first area and the second area. The lower the similarity, the higher the possibility that the second area is an abnormal jump area. On the contrary, the higher the similarity, the higher the possibility that the second area is a non-abnormal jump area. In this way, it is determined whether the candidate trajectory is a target trajectory with anomalies. Through the above method, based on the first similarity between the captured images in the first area and the captured images in the second area, it can be determined whether the candidate trajectory is a target trajectory with anomalies. Therefore, while improving the efficiency of abnormal trajectory recognition, the accuracy of abnormal trajectory recognition can be improved.

[0038] In a specific implementation scenario, the first similarity can be calculated between all the captured images at the shooting locations in the first area and all the captured images at the shooting locations in the second area among the candidate trajectories, a similarity matrix can be constructed, and the first similarity can be obtained based on the similarity matrix. Based on the maximum first similarity in the first similarity, it is detected whether the maximum first similarity is not less than the first threshold. When the maximum similarity is not less than the first threshold, it indicates that there is a similar association between the captured images in the first area and the captured images in the second area, and thus it is determined that the candidate trajectory is not a target trajectory with anomalies. When the maximum similarity is less than the first threshold, the average value of the first similarity is obtained, as well as the first similarity between the captured images in the second area and the captured images in the first target area and the second target area respectively. The first target area and the second target area are obtained according to the trajectory sequence. The first target area is the first area before the second area, and the second target area is the first area after the second area. In this way, it is determined whether the candidate trajectory is a target trajectory with anomalies. For example, the trajectory sequence of a candidate trajectory is {A1, A2, B1, A3, A4}, where the shooting locations of A1, A2, A3, and A4 belong to area A, and the shooting location of B1 belongs to area B. When the maximum value of the first similarity is less than the first threshold, the first similarities between the captured images in the second area and the captured images in the first target area and the second target area are the similarities between A2 and B1, and between A3 and B1 respectively. Then, based on the average value of the first similarity, it is determined whether the above candidate trajectory is a target trajectory.

[0039] In a specific implementation scenario, based on the average value of the first similarity and the first similarities between the captured images in the second area and the captured images in the first target area and the second target area respectively, weighted similarity can be obtained. For example, the weighted similarity sim mean The calculation formula is as follows:

[0040] sim mean = a1 × sim avg + a2 × sim gap1 + a2sim gap2 ……(1)

[0041] In formula (1), sim avg is the average value of the first similarity, sim gap1 is the first similarity between the captured image in the second region and the captured image in the first target region, sim gap2 is the first similarity between the captured image in the second region and the captured image in the second target region, a1 is the first weight, a2 is the second weight, and 2×a2 + a1 = 1.

[0042] Specifically, the default value of the first weight can be set to 0.4, and the default value of the second weight can be set to 0.3. In some embodiments, the difference between the maximum value of the first similarity and the average value of the first similarity is used to dynamically adjust the first weight and the second weight. When the difference increases, the first weight is decreased and the second weight value is increased. When the difference decreases, the first weight value is increased and the second weight value is decreased. Through the above method, a weighted similarity that can more accurately represent the similarity relationship between the captured image in the second region and the captured image in the first region is obtained based on the first similarity, and whether the candidate trajectory is the target trajectory is judged based on the weighted similarity. Therefore, the accuracy of identifying abnormal trajectories can be improved.

[0043] In another specific implementation scenario, when the geographical region to which the starting point and / or the ending point of the trajectory sequence belongs does not belong to the geographical region with the most frequent activities, and the geographical regions to which the remaining trajectory points of the trajectory sequence belong belong to the geographical region with the most frequent activities, the weighted similarity sim mean is calculated as follows:

[0044] sim mean = a1 × sim avg + a2 × sim gap ……(2)

[0045] In formula (2), sim avg is the average value of the first similarity. When the starting point of the trajectory sequence does not belong to the geographical region with the most frequent activities, sim[[ID=4?]] gap is the first similarity between the captured image in the second region and the captured image in the second target region. When the ending point of the trajectory sequence does not belong to the geographical region with the most frequent activities, sim gap is the first similarity between the captured image in the second region and the captured image in the first target region, a1 is the first weight, a2 is the second weight, and a1 + a2 = 1.

[0046] Specifically, the default value of the first weight can be set to 0.5, and the default value of the second weight can be set to 0.5. In some embodiments, the difference between the maximum value of the first similarity and the average value of the first similarity is used to dynamically adjust the first weight and the second weight. When the difference increases, the first weight is decreased and the second weight value is increased. When the difference decreases, the first weight value is increased and the second weight value is decreased. Through the above method, a weighted similarity that can more accurately represent the similarity relationship between the captured image in the second region and the captured image in the first region is obtained based on the first similarity, and whether the candidate trajectory is the target trajectory is determined based on the weighted similarity, so the accuracy of identifying abnormal trajectories can be improved.

[0047] In a specific implementation scenario, it is determined whether the candidate trajectory is the target trajectory based on the weighted similarity and the second threshold. When the weighted similarity is not less than the second threshold, it indicates that there is no captured image of other objects inconsistent with the target object in the second region, thereby determining that the candidate trajectory is not an abnormal target trajectory. When the weighted similarity is less than the second threshold, it indicates that there is a captured image of other objects inconsistent with the target object in the second region, thereby determining that the candidate trajectory is an abnormal target trajectory. Through the above method, the candidate trajectory suspected of being abnormal is quickly located based on the geographical region to which the shooting location in the trajectory sequence belongs, and the identification of the abnormal trajectory is determined based on the first similarity between the captured images in the first region and the second region, improving the efficiency of identifying abnormal trajectories while improving the accuracy of identifying abnormal trajectories.

[0048] In an implementation scenario, after determining that the candidate trajectory is an abnormal target trajectory, all the captured images in the second region may belong to the same other object, or may belong to multiple other objects. When they belong to multiple other objects, they need to be classified and summarized. The second similarity between the captured images with shooting locations in the second region in the target trajectory is obtained. Based on the second similarity, the captured images belonging to the same object are extracted from the set of images with shooting locations in the second region in the candidate trajectory to obtain the trajectory sequence of the corresponding object.

[0049] In a specific implementation scenario, based on the second similarity, the average value and the minimum value of the second similarity are obtained, and it is detected whether the average value of the second similarity and the minimum second similarity meet the preset screening conditions. When the average value of the second similarity is not less than the third threshold and the minimum second similarity is not less than the fourth threshold, it is considered that all the captured images in the second area belong to the same other object, and the entire image set is directly extracted as the trajectory sequence belonging to the same object. When the preset screening conditions are not met, for example, when the average value of the second similarity is less than the third threshold and the minimum second similarity is less than the fourth threshold, or when the average value of the second similarity is not less than the third threshold but the minimum second similarity is less than the fourth threshold, or when the average value of the second similarity is less than the third threshold but the minimum second similarity is not less than the fourth threshold, it is considered that all the captured images in the second area belong to different other objects, and based on the second similarity, the trajectory sequences belonging to multiple objects are extracted.

[0050] In a specific implementation scenario, when the preset screening conditions are not met, the captured images that have not been extracted in the image set are selected as the images to be extracted, and the captured images that have not been extracted except the images to be extracted in the image set are respectively selected as candidate images. When the second similarity between the image to be extracted and the candidate image is not less than the fifth threshold, the candidate image is selected and filed into the trajectory sequence belonging to the same object as the image to be extracted.

[0051] In another implementation scenario, after determining that the candidate trajectory is the target trajectory with anomalies, if there is only one captured image whose shooting location is in the second area, the trajectory sequence corresponding to this captured image is directly obtained.

[0052] Based on the portrait features included in the captured images, the captured images suspected to belong to the same target object are archived, and sorted based on the shooting time attached to the shooting location in the captured images to obtain the trajectory sequences of different objects. By pre-dividing different geographical regions, the geographical regions to which the shooting locations of the captured images in the trajectory sequences belong are obtained, and the trajectory sequences are selected as candidate trajectories suspected of being abnormal. Based on the preset abnormal conditions, it is determined whether the candidate trajectories are satisfied. When the candidate trajectories meet the preset abnormal conditions, it indicates that there are areas with trajectory jumps in the candidate trajectories. The geographical region with the most frequent activities in the candidate trajectories is selected as the first region, and the discontinuous geographical regions in the candidate trajectories are selected as the second region. Based on the first similarity between the captured images with shooting locations in the first region and the captured images in the second region in the candidate trajectories, it is determined whether there is an abnormal target trajectory in the candidate trajectories. Therefore, the trajectory sequence of the target object can be obtained through the information attached to the captured images, and quickly locate to the candidate trajectories suspected of being abnormal based on the geographical regions to which the shooting locations in the trajectory sequences belong, and determine the abnormal trajectories based on the first similarity between the captured images in the first region and the second region, improving the efficiency of abnormal trajectory recognition while improving the accuracy of identifying abnormal trajectories.

[0053] Please refer to Figure 2 , Figure 2 which is a schematic framework diagram of an embodiment of the abnormal trajectory recognition device 20 of the present application. As Figure 2 shown, the abnormal trajectory recognition device 20 includes: a trajectory acquisition module 21, a trajectory selection module 22, a region selection module 23, and an abnormality determination module 24. The trajectory acquisition module 21 is used to acquire the trajectory sequences of different objects; wherein, the trajectory sequences include captured images suspected to belong to the same object, and the captured images in the trajectory sequences are sorted by shooting time, and the captured images are also attached with shooting locations; the trajectory selection module 22 is used to select the trajectory sequences as candidate trajectories suspected of being abnormal based on the geographical regions to which the shooting locations of the captured images in the trajectory sequences belong; the region selection module 23 is used to, in response to the candidate trajectories meeting the preset abnormal conditions, select the geographical region with the most frequent activities in the candidate trajectories as the first region, and select the discontinuous geographical regions in the candidate trajectories as the second region; the abnormality determination module 24 is used to determine whether the candidate trajectories are target trajectories with abnormalities based on the first similarity between the captured images with shooting locations in the first region and the captured images in the second region in the candidate trajectories.

[0054] In the above solution, the captured images suspected to belong to the same target object are archived based on the portrait features included in the captured images, and sorted according to the shooting time attached to the shooting location in the captured images to obtain the trajectory sequences of different objects respectively. By pre-dividing different geographical regions, the geographical regions to which the shooting locations of the captured images in the trajectory sequences belong are obtained, and the trajectory sequences are selected as candidate trajectories suspected of being abnormal. Based on the preset abnormal conditions, it is determined whether the candidate trajectories are satisfied. When the candidate trajectories meet the preset abnormal conditions, it indicates that there are areas with trajectory jumps in the candidate trajectories. The geographical region with the most frequent activities in the candidate trajectories is selected as the first region, and the discontinuous geographical regions in the candidate trajectories are selected as the second region. Based on the first similarity between the captured images with shooting locations in the first region and the captured images with shooting locations in the second region in the candidate trajectories, it is determined whether there is an abnormal target trajectory in the candidate trajectories. Therefore, the trajectory sequence of the target object can be obtained through the information attached to the captured images, and the candidate trajectories suspected of being abnormal can be quickly located based on the geographical regions to which the shooting locations in the trajectory sequences belong. And based on the first similarity between the captured images in the first region and the second region, the abnormal trajectories are determined, which improves the efficiency of abnormal trajectory recognition and the accuracy of identifying abnormal trajectories at the same time.

[0055] In some disclosed embodiments, the trajectory selection module 22 includes an abnormal preliminary screening sub-module for detecting whether the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong are completely the same; in response to the geographical regions to which the shooting locations of the captured images do not belong being completely the same, the trajectory sequence is selected as a candidate trajectory suspected of being abnormal.

[0056] Therefore, by preliminarily screening the trajectory sequence through the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong, the subsequent workload can be reduced, so the efficiency of abnormal trajectory recognition can be improved.

[0057] In some disclosed embodiments, the region selection module 23 includes an abnormal condition matching sub-module for sequentially detecting whether the candidate trajectories meet the first sub-condition, the second sub-condition, and the third sub-condition; in response to the candidate trajectories meeting the first sub-condition, the step of detecting the candidate trajectories is skipped, and the geographical region to which the starting point and / or the ending point belong is used as the discontinuous geographical region in the candidate trajectories; in response to the candidate trajectories not meeting the first sub-condition and meeting the second sub-condition, the step of detecting the candidate trajectories is skipped, and the geographical region to which the trajectory points that do not belong to the geographical region with the most frequent activities belong is used as the discontinuous geographical region in the candidate trajectories; in response to the candidate trajectories not meeting the first sub-condition and the second sub-condition and meeting the third sub-condition, the step of detecting the candidate trajectories is skipped, and the geographical regions to which two trajectory points belong are respectively used as the geographical region with the most frequent activities and the discontinuous geographical region in the candidate trajectories.

[0058] Therefore, by setting a preset abnormal condition and determining whether the candidate trajectory meets the preset abnormal condition, the result of whether there is a trajectory jump area can be obtained, so that the efficiency of abnormal trajectory recognition can be improved while the accuracy of abnormal trajectory recognition can be improved.

[0059] In some disclosed embodiments, the area selection module 23 further includes a trajectory detection jump-out sub-module, which is configured to directly determine that the candidate trajectory is not the target trajectory with an abnormality in response to the candidate trajectory not meeting the preset abnormal condition.

[0060] Therefore, by setting a preset abnormal condition, the candidate trajectories with abnormalities and the candidate trajectories without abnormalities can be quickly classified, so that the efficiency of abnormal trajectory recognition can be improved while the accuracy of abnormal trajectory recognition can be improved.

[0061] In some disclosed embodiments, the abnormality determination module 24 includes a target trajectory determination sub-module, which is configured to detect whether the maximum first similarity is not less than a first threshold; in response to the maximum first similarity being not less than the first threshold, determine that the candidate trajectory is not the target trajectory with an abnormality; in response to the maximum first similarity being less than the first threshold, determine whether the candidate trajectory is the target trajectory based on the average value of the first similarities and the first similarities between the captured images in the second area and the captured images in the first target area and the second target area respectively; wherein, the first target area is the first area before the second area, and the second target area is the first area after the second area.

[0062] Therefore, based on the first similarity between the captured image in the first area and the captured image in the second area, it can be determined whether the candidate trajectory is the target trajectory with an abnormality, so that the efficiency of abnormal trajectory recognition can be improved while the accuracy of abnormal trajectory recognition can be improved.

[0063] In some disclosed embodiments, the target trajectory determination sub-module includes a target trajectory judgment sub-module, which is configured to weight the average value and the first similarities between the captured images in the second area and the captured images in the first target area and the second target area respectively to obtain a weighted similarity; and determine whether the candidate trajectory is the target trajectory based on whether the weighted similarity is not less than a second threshold.

[0064] Therefore, quickly locate the candidate trajectories suspected of being abnormal based on the geographical area to which the shooting locations in the trajectory sequence belong, and determine the recognition of abnormal trajectories based on the first similarity between the captured images in the first area and the second area, so as to improve the efficiency of abnormal trajectory recognition while improving the accuracy of abnormal trajectory recognition.

[0065] In some disclosed embodiments, the abnormal trajectory recognition device 20 further includes an image extraction module, which is configured to, after determining whether a candidate trajectory is a target trajectory with an abnormality, obtain a second similarity between pairs of captured images whose shooting locations in the candidate trajectory are located in the second region; based on the second similarity, extract the captured images belonging to the same object from the set of images whose shooting locations in the candidate trajectory are located in the second region, so as to obtain a trajectory sequence of the corresponding object.

[0066] Therefore, determining whether the above-mentioned captured images belong to the same object based on the similarity between the captured images in the second region, and obtaining the corresponding trajectory sequence based on the captured images of the same object, so the accuracy of identifying abnormal trajectories can be improved.

[0067] In some disclosed embodiments, the image extraction module includes a similarity comparison sub-module, which is configured to detect whether the average value of the second similarities and the minimum second similarity meet a preset screening condition; wherein, the preset screening condition includes: the average value of the second similarities is not less than a third threshold, and the minimum second similarity is not less than a fourth threshold; in response to meeting the preset screening condition, directly extract the entire set of images as the trajectory sequence belonging to the same object; in response to not meeting the preset screening condition, based on the second similarity, extract the trajectory sequences belonging to multiple objects respectively.

[0068] Therefore, determining whether the above-mentioned captured images belong to the same object based on the similarity between the captured images in the second region, and obtaining the corresponding trajectory sequence based on the captured images of the same object, so the accuracy of identifying abnormal trajectories can be improved.

[0069] In some disclosed embodiments, the similarity comparison sub-module further includes a candidate image classification sub-module, which is configured to select the captured images not yet extracted in the set of images as the images to be extracted; respectively select the captured images not yet extracted outside the images to be extracted in the set of images as candidate images; in response to the second similarity between the image to be extracted and the candidate image being not less than a fifth threshold, select the candidate image and file it into the trajectory sequence belonging to the same object as the image to be extracted.

[0070] Therefore, determining whether the above-mentioned captured images belong to the same object based on the similarity between the captured images in the second region, and obtaining the corresponding trajectory sequence based on the captured images of the same object, so the accuracy of identifying abnormal trajectories can be improved.

[0071] Please refer to Figure 3 , Figure 3It is a schematic diagram of the framework of an embodiment of the electronic device 30 of the present application. The electronic device 30 includes a mutually coupled memory 31 and a processor 32. Program instructions are stored in the memory 31, and the processor 32 is configured to execute the program instructions to implement the steps in any of the above-described embodiments of the abnormal trajectory recognition method. Specifically, the electronic device 30 may include, but is not limited to: a server, a desktop computer, a laptop computer, a tablet computer, a smart phone, etc., which are not limited herein.

[0072] Specifically, the processor 32 is configured to control itself and the memory 31 to implement the steps in any of the above-described embodiments of the abnormal trajectory recognition method. The processor 32 may also be referred to as a CPU (Central Processing Unit). The processor 32 may be an integrated circuit chip with signal processing capabilities. The processor 32 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 32 may be implemented jointly by integrated circuit chips.

[0073] In the above solution, since the electronic device 30 archives the captured images suspected to belong to the same target object based on the portrait features included in the captured images, and sorts them based on the shooting time attached to the shooting location in the captured images to obtain the trajectory sequences of different objects respectively, divides different geographical regions in advance to obtain the geographical regions to which the shooting locations of the captured images in the trajectory sequences belong, and selects the trajectory sequences as candidate trajectories suspected of being abnormal, determines whether the candidate trajectories meet the preset abnormal conditions based on the preset abnormal conditions. When the candidate trajectories meet the preset abnormal conditions, it indicates that there are areas with trajectory jumps in the candidate trajectories, selects the geographical region with the most frequent activities in the candidate trajectories as the first region, and selects the discontinuous geographical regions in the candidate trajectories as the second region. Based on the first similarity between the captured images with shooting locations in the first region and the captured images in the second region in the candidate trajectories, it determines whether there is an abnormal target trajectory in the candidate trajectories. Therefore, it can obtain the trajectory sequence of the target object through the information attached to the captured images, quickly locate the candidate trajectories suspected of being abnormal based on the geographical regions to which the shooting locations in the trajectory sequences belong, and determine the recognition of the abnormal trajectories based on the first similarity between the captured images in the first region and the second region, improving the efficiency of abnormal trajectory recognition while improving the accuracy of recognizing abnormal trajectories.

[0074] Please refer to Figure 4 , Figure 4 which is a schematic framework diagram of an embodiment of the computer-readable storage medium 40 of the present application. The computer-readable storage medium 40 stores program instructions 41 that can be run by a processor, and the program instructions 41 are used to implement the steps in any of the above-described embodiments of the abnormal trajectory recognition method.

[0075] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be elaborated here.

[0076] [[ID=!1]]The descriptions of the above embodiments tend to emphasize the differences between the embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be elaborated in this article.

[0077] In several embodiments provided by the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0078] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units. They can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0079] In addition, each functional unit in the various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0080] It should be noted that there is a misspelling in the original text. In line , it should be "Their similarities or similarities can be referred to each other." instead of "Their similarities or similarities can be referred to each other." I have corrected this error in the translation.If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

Claims

1. An abnormal trajectory recognition method, characterized in that, Including: Obtaining the trajectory sequences of different objects respectively; Wherein, the trajectory sequence includes captured images suspected to belong to the same object, and the captured images in the trajectory sequence are sorted by the capture time, and the captured images are also attached with capture locations; Based on the geographical regions to which the capture locations of the captured images in the trajectory sequence belong, selecting the trajectory sequence as a candidate trajectory suspected of being abnormal; In response to the candidate trajectory satisfying a preset abnormal condition, selecting the geographical region with the most frequent activities in the candidate trajectory as the first region, and selecting the discontinuous geographical regions in the candidate trajectory as the second region; wherein, the preset abnormal condition includes at least a first sub-condition, a second sub-condition and a third sub-condition, the first sub-condition includes: the geographical regions to which the start point and / or end point of the trajectory sequence belong do not belong to the geographical region with the most frequent activities, and the geographical regions to which the remaining trajectory points of the trajectory sequence belong belong to the geographical region with the most frequent activities, the second sub-condition includes: the geographical regions to which the start point and end point of the trajectory sequence belong belong to the geographical region with the most frequent activities, and there are some trajectory points whose geographical regions do not belong to the geographical region with the most frequent activities, the third sub-condition includes: the trajectory sequence only includes two trajectory points, and they belong to different geographical regions respectively; Based on the first similarity between the captured images whose capture locations in the candidate trajectory are located in the first region and the captured images located in the second region, determining whether the candidate trajectory is a target trajectory with an abnormality.

2. The method according to claim 1, wherein The determining whether the candidate trajectory is a target trajectory with an abnormality based on the first similarity between the captured images whose capture locations in the candidate trajectory are located in the first region and the captured images located in the second region includes: detecting whether the maximum first similarity is not less than a first threshold; In response to the maximum first similarity being not less than the first threshold, determining that the candidate trajectory is not a target trajectory with an abnormality; In response to the maximum first similarity being less than the first threshold, based on the average value of the first similarity, and the first similarity between the captured images located in the second region and the captured images located in the first target region and the second target region respectively, determining whether the candidate trajectory is the target trajectory; Wherein, the first target region is the first region before the second region, and the second target region is the first region after the second region.

3. The method according to claim 2, wherein The determining whether the candidate trajectory is the target trajectory based on the average value of the first similarity, and the first similarity between the captured images located in the second region and the captured images located in the first target region and the second target region respectively includes: weighting based on the average value, and the first similarity between the captured images located in the second region and the captured images located in the first target region and the second target region respectively, to obtain a weighted similarity; Based on whether the weighted similarity is not less than a second threshold, determining whether the candidate trajectory is the target trajectory.

4. The method according to claim 3, characterized in that Determining whether the candidate trajectory is the target trajectory based on whether the weighted similarity is not less than a second threshold includes at least one of the following: In response to the weighted similarity being not less than the second threshold, determining that the candidate trajectory is not the target trajectory with an anomaly; In response to the weighted similarity being less than the second threshold, determining that the candidate trajectory is the target trajectory with an anomaly.

5. The method according to claim 1, characterized in that, The step of detecting whether the candidate trajectory meets the preset anomaly condition includes: sequentially detecting whether the candidate trajectory meets the first sub-condition, the second sub-condition, and the third sub-condition; In response to the candidate trajectory meeting the first sub-condition, jumping out of the step of detecting the candidate trajectory, and taking the geographical region to which the starting point and / or the ending point belongs as the discontinuous geographical region in the candidate trajectory; In response to the candidate trajectory not meeting the first sub-condition and meeting the second sub-condition, jumping out of the step of detecting the candidate trajectory, and taking the geographical region to which the trajectory point that does not belong to the geographical region of the most frequent activity belongs as the discontinuous geographical region in the candidate trajectory; In response to the candidate trajectory not meeting the first sub-condition and the second sub-condition and meeting the third sub-condition, jumping out of the step of detecting the candidate trajectory, and respectively taking the geographical regions to which two trajectory points belong as the geographical region of the most frequent activity and the discontinuous geographical region in the candidate trajectory.

6. The method according to claim 1, wherein The method further includes: in response to the candidate trajectory not meeting the preset anomaly condition, directly determining that the candidate trajectory is not the target trajectory with an anomaly.

7. The method according to claim 1, characterized in that, Selecting the trajectory sequence as a candidate trajectory with suspected anomaly based on the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong includes: detecting whether the geographical regions to which the shooting locations of the captured images in the trajectory sequence belong are completely the same; In response to the geographical regions to which the shooting locations of the captured images do not belong being completely the same, selecting the trajectory sequence as a candidate trajectory with suspected anomaly.

8. The method according to any one of claims 1 to 7, characterized in that In the case of determining that the candidate trajectory is the target trajectory with an anomaly, after determining whether the candidate trajectory is the target trajectory with an anomaly based on the first similarity between the captured images at the shooting locations in the first region and the captured images at the shooting locations in the second region in the candidate trajectory, the method further includes: obtaining the second similarity between every two of the captured images at the shooting locations in the second region in the candidate trajectory; Based on the second similarity, extracting the captured images belonging to the same object from the set of images at the shooting locations in the second region in the candidate trajectory, and obtaining the trajectory sequence corresponding to the object.

9. The method according to claim 8, wherein The extracting the captured images belonging to the same object from the set of images at the shooting locations in the second region in the candidate trajectory based on the second similarity and obtaining the trajectory sequence corresponding to the object includes: detecting whether the average value of the second similarity and the minimum second similarity meet a preset screening condition; Among them, the preset screening conditions include: the average value of the second similarity is not less than a third threshold, and the minimum second similarity is not less than a fourth threshold; In response to meeting the preset screening conditions, directly extract all of the image set as a trajectory sequence belonging to the same object; In response to not meeting the preset screening conditions, based on the second similarity, extract trajectory sequences respectively belonging to multiple objects.

10. The method according to claim 9, wherein The extracting, based on the second similarity, trajectory sequences respectively belonging to multiple objects includes: selecting unextracted captured images in the image set as images to be extracted; Respectively select unextracted captured images in the image set other than the images to be extracted as candidate images; In response to the second similarity between the image to be extracted and the candidate image being not less than a fifth threshold, select the candidate image and file it into the trajectory sequence belonging to the same object as the image to be extracted.

11. An abnormal trajectory recognition device, characterized in that, including: a trajectory acquisition module, configured to acquire trajectory sequences of different objects respectively; Among them, the trajectory sequence contains captured images suspected of belonging to the same object, and the captured images in the trajectory sequence are sorted by shooting time, and the captured images are also attached with shooting locations; a trajectory selection module, configured to select the trajectory sequence as a candidate trajectory suspected of being abnormal based on the geographical region to which the shooting locations of the captured images in the trajectory sequence belong; a region selection module, configured to, in response to the candidate trajectory meeting a preset abnormal condition, select the geographical region with the most frequent activities in the candidate trajectory as a first region, and select discontinuous geographical regions in the candidate trajectory as second regions; wherein, the preset abnormal condition at least includes a first sub-condition, a second sub-condition and a third sub-condition, the first sub-condition includes: the geographical region to which the start point and / or end point of the trajectory sequence belong does not belong to the geographical region with the most frequent activities, and the geographical regions to which the remaining trajectory points of the trajectory sequence belong belong to the geographical region with the most frequent activities, the second sub-condition includes: the geographical regions to which the start point and end point of the trajectory sequence belong belong to the geographical region with the most frequent activities, and there are some trajectory points whose geographical regions do not belong to the geographical region with the most frequent activities, the third sub-condition includes: the trajectory sequence only includes two trajectory points and they belong to different geographical regions respectively; an abnormality determination module, configured to determine whether the candidate trajectory is a target trajectory with an abnormality based on the first similarity between the captured images located in the first region and the captured images located in the second region in the candidate trajectory.

12. An electronic device, characterized in that, including a mutually coupled memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program to implement the abnormal trajectory recognition method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program can be executed by a processor to implement the abnormal trajectory recognition method according to any one of claims 1 to 10.

Images