Abnormal crowd detection method and device, equipment and storage medium

Abstract

The invention discloses an abnormal crowd detection method and device, equipment and a storage medium, belongs to the field of data processing, and is used for accurately detecting abnormal crowds in a picture. The method comprises the following steps: acquiring a first fused image in a first picture; determining pedestrian features of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image; determining a third pedestrian in the plurality of second pedestrians according to the pedestrian features of the first pedestrian and the pedestrian features of the second pedestrians based on a preset peer pedestrian determination rule; on the basis of a preset abnormal crowd determination algorithm, according to the pedestrian features of the first pedestrian, the pedestrian features of the third pedestrian and the pedestrian features of fourth pedestrians, the crowd abnormal probability of the first picture is determined so as to perform abnormal crowd detection, and the fourth pedestrians are pedestrians not the first pedestrian and not the third pedestrian in the first picture. The crowd abnormal probability is used for representing the probability of crowd abnormal gathering in the first picture.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of data processing, and particularly relates to an abnormal crowd detection method and device, equipment and a storage medium. BACKGROUND

[0002] The detection of behavior abnormality in a complex crowd environment plays an important role in the safety protection of customers in a shopping mall, leisure personnel in a square, and passengers in a waiting room. In the prior art, a video monitoring method is usually used to detect abnormal crowds, that is, a dedicated person is arranged to be on duty in a video monitoring room to achieve the detection purpose. However, since there are many monitoring scenes, and the energy and enthusiasm of the on-duty staff are limited, the abnormal crowds cannot be well detected.

[0003] Therefore, a method for accurately detecting abnormal crowds in a picture is needed. SUMMARY

[0004] The embodiments of the present application provide an abnormal crowd detection method, which can accurately detect abnormal crowds in a picture.

[0005] In a first aspect, the embodiments of the present application provide an abnormal crowd detection method, which comprises: acquiring a first fusion image in a first picture, the first picture being a picture to be detected for abnormal crowds, and the first fusion image being determined based on a visible light image and a code image of the first picture; determining a pedestrian feature of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image, the first pedestrian being a preset pedestrian to be detected, the second pedestrian being a pedestrian other than the first pedestrian in the first fusion image, and the pedestrian feature comprising a characteristic vector and an identity code of the pedestrian; determining a third pedestrian in the plurality of second pedestrians according to the pedestrian feature of the first pedestrian and the pedestrian features of the second pedestrians based on a preset same-row pedestrian determination rule, the same-row pedestrian determination rule being used to determine a pedestrian in the same row as the first pedestrian; and determining a crowd abnormality probability of the first picture according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian, and a pedestrian feature of a fourth pedestrian based on a preset abnormal crowd determination algorithm to perform abnormal crowd detection, the fourth pedestrian being a pedestrian other than the first pedestrian and the third pedestrian in the first picture, and the crowd abnormality probability being used to represent a probability of abnormal crowd gathering in the first picture.

[0006] In a second aspect, an embodiment of the present application provides an abnormal crowd detection device, which includes: a first acquisition module for acquiring a first fused image in a first picture, wherein the first picture is a picture to be subjected to abnormal crowd detection, and the first fused image is determined based on a visible light image and a detection code image of the first picture; a first determination module for determining pedestrian features of a first pedestrian and pedestrian features of multiple second pedestrians in the first fused image, wherein the first pedestrian is a preset pedestrian to be detected, and the second pedestrian is a pedestrian in the first fused image who is not the first pedestrian, and the pedestrian features include a feature vector and an identity code of the pedestrian; a second determination module for determining pedestrian features of a first pedestrian and a plurality of second pedestrians based on the preset pedestrian to be detected. Suppose a rule for determining pedestrians traveling together, and determine the third pedestrian among the multiple second pedestrians based on the pedestrian characteristics of the first pedestrian and the pedestrian characteristics of the second pedestrian. The rule for determining pedestrians traveling together is used to determine the pedestrians traveling together with the first pedestrian; a third determination module is used to determine the probability of abnormal crowds in the first picture based on the preset abnormal crowd determination algorithm and based on the pedestrian characteristics of the first pedestrian, the pedestrian characteristics of the third pedestrian and the pedestrian characteristics of the fourth pedestrian, so as to perform abnormal crowd detection. The fourth pedestrian is a pedestrian in the first picture who is not the first pedestrian and the third pedestrian. The probability of abnormal crowds is used to characterize the probability of abnormal crowd gathering in the first picture.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method described in the first aspect.

[0011] In the embodiment of the present application, the first fusion image in the first picture is obtained, the first picture is a picture to be detected for abnormal crowd, and the first fusion image is determined based on the visible light image and the code image of the first picture; the pedestrian feature of the first pedestrian and the pedestrian features of a plurality of second pedestrians in the first fusion image are determined, the first pedestrian is a preset to-be-detected pedestrian, the second pedestrian is a pedestrian other than the first pedestrian in the first fusion image, and the pedestrian feature includes a characteristic vector and an identity code of the pedestrian; based on a preset same-row pedestrian determination rule, a third pedestrian in the plurality of second pedestrians is determined according to the pedestrian feature of the first pedestrian and the pedestrian features of the second pedestrians, and the same-row pedestrian determination rule is used to determine the pedestrian in the same row with the first pedestrian; based on a preset abnormal crowd determination algorithm, a crowd anomaly probability of the first picture is determined according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of a fourth pedestrian, so as to perform abnormal crowd detection, the fourth pedestrian is a pedestrian other than the first pedestrian and the third pedestrian in the first picture, and the crowd anomaly probability is used to represent the probability of abnormal crowd gathering in the first picture, so that the abnormal crowd in the picture can be accurately detected. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a flowchart of an abnormal crowd detection method provided by an embodiment of the present application; Figure 2 is a flowchart of a second abnormal crowd detection method provided by an embodiment of the present application; Figure 3 is a flowchart of a third abnormal crowd detection method provided by an embodiment of the present application; Figure 4 is a structural diagram of an abnormal crowd detection device provided by an embodiment of the present application; Figure 5 is a structural diagram of an abnormal crowd detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0013] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0014] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0015] The abnormal crowd detection method provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios.

[0016] Figure 1 An abnormal crowd detection method provided by one embodiment of the present application is shown, which can be executed by an electronic device, which can include a server and / or a terminal device, such as a vehicle-mounted terminal or a mobile phone terminal. In other words, the method can be executed by software or hardware installed in an abnormal crowd detection device, and the method includes the following steps: Step 102: obtaining a first fusion image in a first picture.

[0017] The first picture is a picture to be detected for abnormal crowds, and the first fusion image is determined based on a visible light image and a code detection image of the first picture.

[0018] The execution subject of the abnormal crowd detection method described in the present application can be an abnormal crowd detection system or an abnormal crowd detection software, and the embodiments of the present application will be described taking the abnormal crowd detection system as an example.

[0019] The abnormal crowd detection system obtains a first fusion image in a first picture, wherein the first picture is a picture to be detected for abnormal crowds, and the first fusion image is determined according to a visible light image and a code detection image of the first picture. The first fusion image can be determined by fusing the visible light image and the code detection image, or the corresponding relationship between the visible light image and the code detection image at the current time of the first picture can be determined, and then both images are taken as part of the first fusion image. For example, the first picture can be the entrance of a company, and the first fusion image can include a visible light image captured by a camera and a code detection image obtained by a code detection image acquisition device.

[0020] Step 104: determining a pedestrian feature of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image.

[0021] The first pedestrian is a preset pedestrian to be detected, the second pedestrian is a pedestrian in the first fusion image other than the first pedestrian, and the pedestrian feature includes a feature vector of the pedestrian and an identity code of the pedestrian.

[0022] After the first fusion image is acquired, the abnormal crowd detection system determines feature information of a specific pedestrian (i.e., the first pedestrian) in the first fusion image and feature information of a non-specific pedestrian (i.e., the second pedestrian). The feature information of the pedestrian includes a feature vector of the pedestrian and an identity code of the pedestrian. The feature vector of the pedestrian can be determined according to the visible light image, and the identity code of the pedestrian can be determined according to the detection code image. It can be understood that the pedestrian feature of the first pedestrian is a pedestrian feature pre-stored in the abnormal crowd detection system. The first pedestrian can be a regionally focused pedestrian, and the first pedestrian is likely to cause abnormal crowd gathering, such as a community property management personnel, a company leader, etc. When the first pedestrian is a community property management personnel, if it is determined that the first pedestrian appears in the community and is surrounded by community residents, an abnormal gathering situation occurs, such as a water and electricity outage in the community.

[0023] The abnormal crowd detection system can first perform pedestrian recognition on the visible light image to determine the pedestrian in the visible light image, and then determine whether the pedestrian is the first pedestrian. If yes, the feature vector of the first pedestrian is determined. If no, the pedestrian is determined to be the second pedestrian, and the feature vector of the second pedestrian is determined. When determining the pedestrian in the visible light image, the abnormal detection system can acquire the feature vector in the visible light image, and compare the feature vector with a human body feature vector to determine whether it is a human body feature vector. After the first pedestrian and the second pedestrian are determined, the abnormal crowd detection system acquires the identity code of the first pedestrian and the identity code of the second pedestrian through the detection code image. The identity code can be an international mobile subscriber identity (IMSI). The abnormal crowd detection system determines the pedestrian feature of the first pedestrian based on the feature vector of the first pedestrian determined through the visible light image and the identity code of the first pedestrian determined through the detection code image, and determines the pedestrian feature of the second pedestrian based on the feature vector and the identity code of the second pedestrian. When the IMSI is used as the identity code of the pedestrian, the IMSI does not change with the change of the scene where the pedestrian is located, such as the IMSI of the pedestrian at the company gate or the community gate does not change. This can determine that the identity code of each pedestrian does not change in various scenes, thereby increasing the accuracy of pedestrian identity recognition.

[0024] In determining the feature vector in the pedestrian feature of the first pedestrian and the pedestrian feature of the second pedestrian, the abnormal crowd detection system can perform human feature vector recognition on the visible light image in the first fusion image, determine whether there is a human feature vector in the visible light image, and further determine whether the recognized human feature vector is a specific feature vector (the pedestrian feature vector of the first pedestrian). If yes, the recognized human feature vector is determined as the feature vector of the first pedestrian, and it is determined as the first pedestrian. If not, the recognized human feature vector is determined as the pedestrian feature vector of the second pedestrian, and it is determined as the second pedestrian.

[0025] For example, when performing human feature vector recognition, the abnormal crowd detection system uses a human re-identification (ReID) algorithm to identify the identity of the passing pedestrian through the surveillance camera. The ReID algorithm extracts a human feature vector f and combines a deep learning model for recognition and comparison. The feature vector f i represents the feature of the i-th pedestrian, and the extraction formula is as follows: f i = ReID(Image i ) where Image i represents the i-th frame image (i.e., the first fusion image obtained), and ReId represents a re-identification model for human feature extraction. All pedestrian feature vectors captured by the camera are saved in the abnormal crowd recognition system for subsequent specific person matching.

[0026] In determining whether the recognized human feature vector is a feature feature vector, the abnormal crowd detection system can match the feature vector extracted by the ReID algorithm when the pedestrian passes through the camera with the pre-defined specific person feature vector f target . The matching function can be defined as: S(f i , f target ) = (f i *f target ) / (|f i | * |f target | ) where S is a similarity score, usually quantified using cosine similarity. If S(fi, ftarget) is greater than a pre-set threshold, the pedestrian is identified as the first pedestrian, the time and geographic location information of the specific person is recorded, and an alarm is given.

[0027] In determining the identity codes of the first pedestrians and the second pedestrians, the abnormal crowd detection system can determine the IMSI codes corresponding to each first pedestrian and each second pedestrian, and take the IMSI codes as the identity codes of the pedestrians corresponding thereto. For example, the abnormal crowd detection system collects IMSI code information in real time near the camera through a mobile detection code base station device. Each IMSI code is accompanied by collection time T and location information L of the detection code device i . Each record is in the form of: IMSI j = (L i , T i ) wherein IMSI j represents the IMSI code collected by the jth detection code base station, L i represents the geographic position information at the time when the detection code base station collects the IMSI code, represented by longitude and latitude coordinates, and T i represents the time stamp at which the detection code base station collects the IMSI code.

[0028] Step 106: determining a third pedestrian from among the second pedestrians based on a preset same-ride pedestrian determination rule according to the pedestrian features of the first pedestrian and the pedestrian features of the second pedestrians.

[0029] wherein the same-ride pedestrian determination rule is used to determine a pedestrian who rides with the first pedestrian.

[0030] After determining the pedestrian features of the first pedestrian and the pedestrian features of the second pedestrians, the abnormal crowd detection system determines a pedestrian who rides with the first pedestrian from among the second pedestrians based on the pedestrian features of the first pedestrian and the pedestrian features of the second pedestrians through a preset same-ride pedestrian determination rule, and takes the pedestrian as the third pedestrian. The same-ride pedestrian determination rule is used to determine the same-ride pedestrian of the first pedestrian from among the pedestrians. Specifically, when the first pedestrian is a community property management personnel, the third pedestrian can be a person who temporarily works in the community property, so when a water stop occurs in the community, the community residents will not only gather around the community property management personnel to inquire about the situation, but also gather around the person who temporarily works in the community property to inquire about the situation.

[0031] The preset same-ride pedestrian determination rule can be a rule for determining the same-ride pedestrian according to distance, for example, determining the second pedestrian in the first picture who is less than a preset threshold away from the first pedestrian as the third pedestrian, or a rule for determining the same-ride pedestrian according to interaction information, for example, determining whether there is interaction information between the first pedestrian and each second pedestrian according to the pedestrian features of the second pedestrians, and taking the second pedestrian with the interaction information as the third pedestrian. The interaction feature is a feature generated by the interaction between the two (which can be a greeting, or a hand-holding, or a passing of an object, etc.).

[0032] Step 108: determining a crowd abnormality probability of the first picture based on the preset crowd abnormality determination algorithm according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian, so as to perform crowd abnormality detection.

[0033] The fourth pedestrian is a pedestrian in the first picture other than the first pedestrian and the third pedestrian, and the crowd abnormality probability is used to represent a probability of crowd abnormality gathering in the first picture.

[0034] After determining the same-pedestrian of the first pedestrian, the crowd abnormality detection system determines a crowd abnormality probability in the first picture based on the preset crowd abnormality determination algorithm according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian, so as to perform crowd abnormality detection, wherein the crowd abnormality determination algorithm is used to determine the probability of crowd abnormality gathering in the first picture, and the fourth pedestrian is a pedestrian in the first picture other than the first pedestrian and the third pedestrian.

[0035] That is, the crowd abnormality detection system first determines the first pedestrian and the second pedestrian in the first picture, then determines the third pedestrian in the second pedestrian who is the same-pedestrian with the first pedestrian, and determines the fourth pedestrian as a pedestrian in the second pedestrian other than the third pedestrian, and finally determines the crowd abnormality probability according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian.

[0036] When determining the crowd abnormality probability according to the crowd abnormality determination algorithm, not only can the crowd abnormality probability be determined according to the number of the first pedestrian, the third pedestrian and the fourth pedestrian and the preset crowd abnormality probability table, wherein the different crowd abnormality probabilities corresponding to different numbers of the first pedestrian, the third pedestrian and the fourth pedestrian are stored in the crowd abnormality probability table, but also the crowd abnormality probability can be determined by performing preset algorithm calculation according to the number of the first pedestrian, the third pedestrian and the fourth pedestrian, and the crowd abnormality probability can be determined according to the vector feature of the first pedestrian, the third pedestrian and the fourth pedestrian, such as determining the average distance between the first pedestrian, the third pedestrian and the fourth pedestrian according to the pedestrian feature of the first pedestrian, the third pedestrian and the fourth pedestrian, and determining the crowd abnormality probability corresponding to the different average distances.

[0037] After determining the crowd abnormality probability, the crowd abnormality detection system can determine whether there is crowd abnormality gathering according to the crowd abnormality probability, such as when the crowd abnormality probability is higher, it means that the crowd abnormality gathering is more likely to occur in the current picture, so the crowd abnormality detection system can determine that there is crowd abnormality gathering in the first picture when the crowd abnormality probability is greater than a preset threshold.

[0038] The embodiment of the present application provides the abnormal crowd detection method, the first fusion image in the first picture is acquired, the first picture is a picture to be detected for abnormal crowd, and the first fusion image is determined based on the visible light image and the detection code image of the first picture; the pedestrian feature of a first pedestrian and the pedestrian feature of a plurality of second pedestrians in the first fusion image are determined, the first pedestrian is a preset to-be-detected pedestrian, the second pedestrian is a pedestrian other than the first pedestrian in the first fusion image, and the pedestrian feature includes a characteristic vector and an identity code of the pedestrian; based on a preset same-row pedestrian determination rule, a third pedestrian in the plurality of second pedestrians is determined according to the pedestrian feature of the first pedestrian and the pedestrian feature of the second pedestrian, and the same-row pedestrian determination rule is used to determine the pedestrian in the same row with the first pedestrian; based on a preset abnormal crowd determination algorithm, a crowd anomaly probability of the first picture is determined according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of a fourth pedestrian, so as to perform abnormal crowd detection, the fourth pedestrian is a pedestrian other than the first pedestrian and the third pedestrian in the first picture, and the crowd anomaly probability is used to represent the probability of abnormal crowd gathering in the first picture, so that the abnormal crowd in the picture can be accurately detected.

[0039] In an implementation manner, the third pedestrian in the plurality of second pedestrians is determined according to the pedestrian feature of the first pedestrian and the pedestrian feature of the second pedestrian based on the preset same-row pedestrian determination rule (step 106), and steps A1-A3 can be performed: Step A1: acquiring a second fusion image in a second picture.

[0040] The second picture is a picture to be determined for the same-row pedestrian.

[0041] The abnormal crowd detection system can acquire a second fusion image in a second picture, and the second picture is a picture to be determined for the same-row pedestrian. That is, in order to better determine the same-row pedestrian of the first pedestrian, the abnormal crowd detection system can also acquire a fusion image from other pictures (the second picture) including the first pedestrian, and determine the same-row pedestrian of the first pedestrian according to the second fusion image.

[0042] Specifically, the time of the second fusion image acquired by the abnormal crowd detection system can be the same as the time of the first fusion image acquired, or can be different. That is, the abnormal crowd detection system can acquire fusion images at different times, and determine the same-row pedestrian of the first pedestrian according to the fusion images at different times.

[0043] Step A2: determining the pedestrian feature of the first pedestrian and the pedestrian feature of a fifth pedestrian in the second fusion image.

[0044] The fifth pedestrian is a pedestrian other than the first pedestrian in the second fusion image.

[0045] After the second fusion image is acquired, the abnormal crowd detection system determines the pedestrian feature of the first pedestrian and the pedestrian feature of the fifth pedestrian in the second fusion image, wherein the first pedestrian is the preset pedestrian in the first image that needs to be detected, and the fifth pedestrian is a pedestrian in the second image other than the first pedestrian.

[0046] That is, the abnormal crowd detection system can identify the acquired second fusion image, determine whether it includes the first pedestrian, and if it does, acquire the pedestrian features of all pedestrians included in the second fusion image and determine the pedestrian features of the pedestrians other than the first pedestrian, i.e., the fifth pedestrian.

[0047] Step A3: determining the third pedestrian based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian.

[0048] After determining the pedestrian feature of the fifth pedestrian in the second fusion image, the abnormal crowd detection system can determine the third pedestrian based on the determined pedestrian features of the plurality of fifth pedestrians and the plurality of second pedestrians. That is, the abnormal crowd detection system can determine the pedestrians other than the first pedestrian in each image through a plurality of images including the first pedestrian, and further determine the same-ride pedestrians of the first pedestrian in the first image based on the pedestrian features of the pedestrians other than the first pedestrian in the plurality of images.

[0049] Further, the abnormal crowd detection system can not only determine the third pedestrian based on the first fusion image of the first image and the second fusion image of the second image, but also determine the third pedestrian based on the first fusion image of the first image, the second fusion image of the second image, and the third fusion image of the third image. That is, the abnormal crowd detection system can acquire a plurality of fusion images including the first pedestrian, and further determine the same-ride pedestrians of the first pedestrian based on the plurality of fusion images.

[0050] Specifically, when determining the third pedestrian based on the pedestrian feature of the second pedestrian and the pedestrian feature of the fifth pedestrian, the abnormal crowd detection system can directly determine the pedestrian that exists in both the second pedestrian and the fifth pedestrian as the third pedestrian. When the number of pedestrians that exist in both the second pedestrian and the fifth pedestrian is greater than a preset threshold, the abnormal crowd detection system can further determine the same-ride pedestrian based on the interaction feature of each common pedestrian with the first pedestrian, for example, determine the pedestrian that interacts with the first pedestrian among the plurality of common pedestrians as the same-ride pedestrian of the first pedestrian, or determine the pedestrian that interacts with the first pedestrian more than a preset threshold number of times among the plurality of common pedestrians as the same-ride pedestrian of the first pedestrian.

[0051] More specifically, the third pedestrian can be determined not only according to the feature vector of the second pedestrian and the feature vector of the fifth pedestrian, but also according to the identity code of the second pedestrian and the identity code of the fifth pedestrian, for example, comparing the identity codes of the plurality of second pedestrians with the identity codes of the plurality of fifth pedestrians to determine the common identity codes as the identity code of the third pedestrian, and then determine the third pedestrian.

[0052] In an implementation manner, the third pedestrian is determined based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian (step A3), and steps B1-B2 can be performed. Step B1: determining a pedestrian appearance frequency based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian.

[0053] The pedestrian appearance frequency is used to represent the appearance frequency of each pedestrian in the fifth pedestrian and the second pedestrian.

[0054] The abnormal crowd detection system can determine a plurality of fifth pedestrians based on the pedestrian feature of the fifth pedestrian, determine a plurality of second pedestrians based on the pedestrian feature of the second pedestrian, and then determine the pedestrian appearance frequency of each pedestrian based on the plurality of fifth pedestrians and the plurality of second pedestrians. For example, when the second pedestrian in the first picture includes an A pedestrian, and the fifth pedestrian in the second picture also includes the A pedestrian, it can be determined that the appearance frequency of the A pedestrian is 2.

[0055] Step B2: determining the pedestrian with the appearance frequency greater than a preset threshold as the third pedestrian.

[0056] After determining the appearance frequency of each pedestrian, the abnormal crowd detection system can determine the pedestrian with the appearance frequency greater than a preset threshold as the third pedestrian, for example, when the pedestrian appearance frequency is greater than 2 times or 5 times, the pedestrian is determined as the third pedestrian. Further, when there is one pedestrian with an appearance frequency greater than a preset threshold, the pedestrian can be directly determined as the third pedestrian, and when there are a plurality of pedestrians with an appearance frequency greater than a preset threshold, the one or three pedestrians with the highest appearance frequency can be determined as the third pedestrian.

[0057] Specifically, the abnormal crowd detection system can determine the third crowd through a voting manner, for example, the abnormal crowd detection system votes for each identified IMSI code, and the voting mechanism can be defined as follows: when a pedestrian f target is identified, the IMSI code (identity code) associated with it IMSI j is incremented by 1, and the formula for updating its vote is: V (IMSI j ) = V (IMSI j ) + 1 After the voting ends, the abnormal pedestrian detection system can regard the IMSI code with the highest number of votes as the IMSI code of the third pedestrian, and then determine the third pedestrian.

[0058] Figure 2 is a flowchart of a second abnormal crowd detection method provided by an embodiment of the present specification, as shown in the flowchart includes: Figure 2 Step 202: Obtain a first fusion image in a first picture.

[0059] The first picture is a picture to be detected for abnormal crowds, and the first fusion image is determined based on a visible light image and a code image of the first picture.

[0060] Step 204: Determine the pedestrian features of a first pedestrian and the pedestrian features of a plurality of second pedestrians in the first fusion image.

[0061] The first pedestrian is a preset pedestrian to be detected, the second pedestrian is a pedestrian other than the first pedestrian in the first fusion image, and the pedestrian features include a feature vector and an identity code of the pedestrian.

[0062] Step 206: Obtain a second fusion image in a second picture.

[0063] The second picture is a picture to be determined for the same pedestrian.

[0064] Step 208: Determine the pedestrian features of the first pedestrian and the pedestrian features of a fifth pedestrian in the second fusion image.

[0065] The fifth pedestrian is a pedestrian other than the first pedestrian in the second fusion image.

[0066] Step 210: Determine a pedestrian appearance frequency based on the pedestrian features of the fifth pedestrian and the pedestrian features of the second pedestrian.

[0067] The pedestrian appearance frequency represents the appearance frequency of each pedestrian among the fifth pedestrian and the second pedestrian.

[0068] Step 212: Determine a third pedestrian whose appearance frequency is greater than a preset threshold.

[0069] Step 214: Determine a crowd anomaly probability of the first picture based on a preset abnormal crowd determination algorithm, according to the pedestrian features of the first pedestrian, the pedestrian features of the third pedestrian, and the pedestrian features of a fourth pedestrian, to perform abnormal crowd detection.

[0070] ​The fourth pedestrian is a pedestrian in the first picture who is not the first pedestrian and not the third pedestrian, and the crowd anomaly probability is used to represent a probability of crowd anomaly gathering in the first picture.

[0071] In the embodiment of the description, by acquiring a plurality of fusion images including the first pedestrian, and then determining the same-row pedestrian of the first pedestrian according to the fusion images, the same-row pedestrian of the first pedestrian in the first picture can be accurately acquired, and by taking the IMSI as the identity code, it is ensured that each first pedestrian has a fixed identity code in different scenes, and the accuracy of determining the same-row pedestrian and the first pedestrian is improved.

[0072] In an implementation manner, the first picture is determined according to the first pedestrian feature, the second pedestrian feature and the third pedestrian feature of the first pedestrian (step 108), which can execute steps C1-C2: Step C1: acquiring a plurality of first fusion images at a plurality of times, and determining the first pedestrian feature, the third pedestrian feature and the fourth pedestrian feature of the first pedestrian in each first fusion image.

[0073] The abnormal crowd detection system can acquire the first fusion image corresponding to each time at a plurality of times, and determine the first pedestrian feature, the third pedestrian feature and the fourth pedestrian feature of the first pedestrian in each first fusion image. That is, the abnormal crowd detection system can acquire the first fusion image of the first picture in real time, and determine the pedestrian feature of each pedestrian in the first fusion image of the first picture at each time in real time.

[0074] Step C2: determining the crowd anomaly probability of the first picture in real time according to the first pedestrian feature, the third pedestrian feature and the fourth pedestrian feature of the first pedestrian in each first fusion image based on the abnormal crowd determination algorithm.

[0075] After acquiring the pedestrian feature of each pedestrian in the first fusion image in real time, the abnormal crowd detection system can determine the crowd anomaly probability of the first picture in real time, thereby realizing the effect of detecting the abnormal crowd in real time.

[0076] Specifically, when performing real-time abnormal crowd detection on the first picture, the abnormal crowd detection system can not only acquire the first fusion image of each time and perform abnormal crowd detection on the current time based on the first fusion image corresponding to each time, but also acquire the first fusion image of each time and perform abnormal crowd detection on the current time according to the acquired first fusion images of multiple times. For example, the abnormal crowd detection system can determine the crowd abnormal probability of the current time according to the average value of the crowd abnormal probability within one hour, or determine the crowd abnormal probability of the current time according to the average number of the first pedestrian, the average number of the third pedestrian, and the average number of the fourth pedestrian within two hours.

[0077] In an implementation manner, the abnormal crowd detection system can perform steps D1-D2 when determining the crowd abnormal probability of the first picture according to the pedestrian features of the first pedestrian, the third pedestrian, and the fourth pedestrian in each first fusion image (step C2). Step D1: determining a first value, a second value, and a third value according to the pedestrian features of the first pedestrian, the third pedestrian, and the fourth pedestrian in each first fusion image.

[0078] The first value is used to represent the density of the pedestrians in the first picture, the second value is used to represent the proportion information of the first pedestrian and the third pedestrian to the pedestrians in the first picture, and the third value is used to represent the density of the first pedestrian and the third pedestrian.

[0079] The abnormal crowd detection system can determine the density of the pedestrians in the first picture (the first value), the proportion information of the first pedestrian and the third pedestrian to the pedestrians in the first picture (the second value), and the density of the first pedestrian and the third pedestrian (the third value) according to the pedestrian features of the first pedestrian, the third pedestrian, and the fourth pedestrian.

[0080] That is, the abnormal crowd detection system can determine the total number of the pedestrians in the first picture according to the pedestrian features of the first pedestrian, the third pedestrian, and the fourth pedestrian, and then determine the density of the pedestrians according to the area of the region in the first picture; determine the total number of the first pedestrian and the third pedestrian according to the number of the first pedestrian and the third pedestrian, and then determine the proportion information of the total number of the first pedestrian and the third pedestrian to the total number of the pedestrians in the first picture; and determine the density of the first pedestrian and the third pedestrian in the first picture according to the total number of the first pedestrian and the third pedestrian and the area of the region in the first picture.

[0081] Specifically, when performing real-time abnormal crowd detection on the first picture, the abnormal crowd detection system can determine the average crowd density of the first picture based on the total number of historical pedestrians; determine the average proportion information of the total number of historical pedestrians and the total number of historical pedestrians based on the total number of historical pedestrians and the total number of historical pedestrians; and determine the historical average density of the first picture based on the total number of historical pedestrians and the total number of historical pedestrians.

[0082] For example, the abnormal crowd detection system can determine the set of pedestrians in the first picture as P T = {p1, p2, … p n}, determine the area of the sub-region of the first picture as A rj , the time window as T, determine the set of identity codes as I target , the total set of identity codes as I T,R , and I T,R represents the set of all IMSI identity codes that appear in the region R within the time window T. This is the set of all IMSI information recorded by the abnormal crowd detection system, which is used to identify the first pedestrian within a certain time, where p1 is the number of pedestrians at the first time, and p2 is the number of pedestrians at the second time. The abnormal crowd detection system can determine the density aggregation index D T,R as the first value, and the formula is:

[0083] D T,R is used to measure the density of the crowd in a unit area, reflecting the abnormal aggregation of the crowd in a short period of time. Where D T,R is the density aggregation index, which is used to measure the density of the crowd in the region R within a given time window T, which reflects the density of the crowd in each region. m is the total number of sub-regions in region R. |P T，rj | represents the number of personnel or IMSI in the sub-region r j within the time window T. This value represents the number of personnel in the sub-region. A rj is the area of the sub-region r j , which is used to calculate the crowd density of the sub-region. |P T，rj | / A rj represents the crowd density of each sub-region, i.e. the number of personnel per unit area.

[0084] For example, set the entrance of the cell as the first picture, when we detect that the area of the entrance of the cell is 100 square meters, and 50 people appear in the region within 24 hours, then the density D T,R can be calculated as 50 / 100=0.5 person / square meter.

[0085] The abnormal crowd detection system can calculate the frequency aggregation index F T,R As the second value, the formula is:

[0086] Among them, F T,R Measures the frequency of recurrence of the same group in a specific area over time and detects abnormal repetitive clustering. T,R Frequency Clustering Index: This measures the frequency of repeated appearances of the same group in a specific region R within a time window T. This can be used to detect abnormal repetitive clustering behavior within a specific time period. n is the total number of people: This refers to the total number of people or IMSIs (International Mobile Equipment Identity) that appeared in region R within the time window T. It is the benchmark number used to standardize the clustering index. i Location information of the i-th person: The location of each person, usually the coordinates obtained based on some positioning technology (such as GPS). Location data is used to determine whether the individual is present in a specific area R. j is the location of region R: refers to a specific area (e.g., a sub-region), usually defined by coordinates, indicating the spatial extent of the target area. t∈T is the time point in the time window: T is a time window that contains multiple observations or measured data points. t represents a time point and indicates the observed data at that time point. δ(p i , r j , t) is an indicator function: This is an indicator function used to check whether the j Is there a position p for person i? i If the person appears in area r j and time is t, then δ(p i , r j , t) otherwise it is 0.

[0087] For example, if the gate of the community is set as the first screen, when we detect that the number of times the three first pedestrians appear at the gate of the community are 5, 4, and 3 respectively, we can calculate F T,R It is (5+4+3) / 3=4 times per person.

[0088] The abnormal crowd detection system can identify the specific IMSI clustering index S T，R Determined as the third value, the formula is:

[0089] Among them, T，R ∩I target 丨 represents the number of IMSIs in the target IMSI set that actually appear in the area, and measures the proportion of high-risk IMSIs in the abnormal population.T，R is a set of all IMSIs that appear in the area R within the time window T, which is a set of all IMSI information recorded by the surveillance system. target is a set of target IMSIs, that is, a list of IMSIs of specific targets. It is usually the IMSIs of pedestrians or specific targets that need to be noticed. T，R is a set of all people that appear in the area R within the time window T.

[0090] For example, the cell gate is set as the first picture, and 100 first pedestrians appear in the area within 24 hours, of which 30 are first pedestrians. Then P T，R is 30 / 100=0.3. If the proportion increases at a certain time, for example, the number of first pedestrians increases to 50, then P T，R is 50 / 100=0.5, indicating that there may be an abnormal gathering of people in the area.

[0091] Step D2: determining the crowd anomaly probability based on the first value, the second value, and the third value.

[0092] After determining the first value, the second value, and the third value, the abnormal crowd detection system can directly add the first value, the second value, and the third value, and determine the sum value as the crowd anomaly probability, or can weight the first value, the second value, and the third value, and determine the weighted sum value as the crowd anomaly probability.

[0093] Specifically, the crowd anomaly detection system can determine the crowd anomaly probability by P=aX+bY+cZ, where P is the crowd anomaly probability, a is the weight coefficient corresponding to the first value, X is the first value, b is the weight coefficient corresponding to the second value, Y is the second value, c is the weight coefficient corresponding to the third value, and Z is the third value.

[0094] Figure 3 is a flowchart of a third abnormal crowd detection method provided by an embodiment of the present specification, as shown in the flowchart, the flowchart includes: Figure 3 Step 302: obtaining a first fusion image in a first picture.

[0095] The first picture is a picture to be detected for abnormal crowds, and the first fusion image is determined based on a visible light image and a detection code image of the first picture.

[0096] Step 304: determining a pedestrian feature of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image.

[0097] ​The first pedestrian is a preset to-be-detected pedestrian, the second pedestrian is a pedestrian in the first fusion image other than the first pedestrian, and the pedestrian feature includes a characteristic vector and an identity code of the pedestrian.

[0098] Step 306: determining a third pedestrian in the plurality of second pedestrians based on a preset same-ride pedestrian determination rule according to the pedestrian feature of the first pedestrian and the pedestrian feature of the second pedestrian.

[0099] The same-ride pedestrian determination rule is used to determine a pedestrian who rides with the first pedestrian.

[0100] Step 308: obtaining a plurality of first fusion images at a plurality of times, and determining the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of a fourth pedestrian in each first fusion image.

[0101] The fourth pedestrian is a pedestrian in the first image other than the first pedestrian and the third pedestrian.

[0102] Step 310: determining a first value, a second value and a third value according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian in each first fusion image.

[0103] The first value is used to represent the density of pedestrians in the first image, the second value is used to represent the proportion information of the first pedestrian and the third pedestrian in the first image, and the third value is used to represent the density of the first pedestrian and the third pedestrian.

[0104] Step 312: determining the crowd anomaly probability based on the first value, the second value and the third value to perform abnormal crowd detection.

[0105] The crowd anomaly probability is used to represent the probability of abnormal crowd gathering in the first image.

[0106] In the embodiments of the description, by determining the pedestrian features of the first pedestrian, the third pedestrian and the fourth pedestrian in the first image, and then determining the crowd anomaly gathering probability according to the pedestrian features of the first pedestrian, the third pedestrian and the fourth pedestrian, the specific pedestrians in the image can be focused on, so that the abnormal gathering in the image can be detected in real time and accurately.

[0107] In an implementation manner, after the step of determining the crowd anomaly probability of the first image according to the preset abnormal crowd determination algorithm, the pedestrian feature of the first pedestrian, the pedestrian feature of the second pedestrian and the pedestrian feature of the third pedestrian (step 108), steps E1-E2 can be further performed. Step E1: determining an abnormality prevention strategy corresponding to the crowd abnormality probability.

[0108] Different abnormality prevention strategies correspond to different crowd abnormality probabilities.

[0109] After determining the crowd abnormality probability of the first picture, the abnormal crowd detection system can determine an abnormality prevention strategy corresponding to the determined crowd abnormality probability, wherein different crowd abnormality probabilities correspond to different abnormality prevention strategies.

[0110] Specifically, the abnormal crowd detection system sets different prevention strategies according to the level of the crowd abnormality probability, and the abnormality prevention strategy becomes more and more strict as the crowd abnormality probability increases. For example, for a low-probability crowd abnormality probability, the abnormality prevention strategy can correspond to dispersing through broadcasting, and for a high-probability crowd abnormality probability, the abnormality prevention strategy can correspond to dispersing through security guards.

[0111] Specifically, the crowd abnormality probability can be divided into three levels, namely low-level abnormality, medium-level abnormality and high-level abnormality, wherein the low-level abnormality is that the crowd abnormality probability is less than a first threshold, the medium-level abnormality is that the crowd abnormality probability is greater than the first threshold and less than a second threshold, and the high-level abnormality is that the crowd abnormality probability is greater than the second threshold.

[0112] Step E2: preventing abnormal crowd gathering based on the abnormality prevention strategy.

[0113] After determining the crowd abnormality probability of the first picture and the abnormality prevention strategy corresponding to the crowd abnormality probability, the abnormal crowd detection system can execute the corresponding abnormality prevention strategy, thereby preventing abnormal crowd gathering.

[0114] It should be noted that the abnormal crowd detection method provided in the embodiments of the present application can be executed by an abnormal crowd detection device or a control module in the abnormal crowd detection device for executing the abnormal crowd detection method. In the embodiments of the present application, the abnormal crowd detection device executes the abnormal crowd detection method as an example to illustrate the abnormal crowd detection device provided in the embodiments of the present application.

[0115] Figure 4 is a structural schematic diagram of the abnormal crowd detection device according to the embodiments of the present application. As shown in Figure 4 The abnormal crowd detection includes a first acquisition module 402, a first determination module 404, a second determination module 406 and a third determination module 408.

[0116] The first acquisition module 402 is configured to acquire a first fusion image in a first picture, wherein the first picture is a picture to be detected for abnormal crowd, and the first fusion image is determined based on a visible light image and a detection code image of the first picture. The first determining module 404 is configured to determine a pedestrian feature of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image, the first pedestrian being a preset pedestrian to be detected, the second pedestrians being pedestrians in the first fusion image other than the first pedestrian, and the pedestrian feature including a feature vector and an identity code of the pedestrian; The second determining module 406 is configured to determine a third pedestrian from the second pedestrians based on a preset same-ride pedestrian determining rule according to the pedestrian feature of the first pedestrian and the pedestrian features of the second pedestrians, the same-ride pedestrian determining rule being used to determine a pedestrian who rides with the first pedestrian. The third determining module 408 is configured to determine a crowd anomaly probability of the first image based on a preset abnormal crowd determining algorithm according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian, and a pedestrian feature of a fourth pedestrian, the fourth pedestrian being a pedestrian in the first image other than the first pedestrian and the third pedestrian, and the crowd anomaly probability being used to represent a probability of abnormal crowd gathering in the first image, so as to perform abnormal crowd detection.

[0117] The abnormal crowd detection apparatus in the embodiments of the present application can be an apparatus, or a component, an integrated circuit, or a chip in a terminal. The apparatus can be a mobile electronic device, or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application are not limited in this regard.

[0118] The abnormal crowd detection apparatus in the embodiments of the present application can be an apparatus having an operating system. The operating system can be an Android operating system, an ios operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0119] The abnormal crowd detection apparatus provided in the embodiments of the present application can implement the method provided in the method embodiments. Figures 1 to 3 The processes implemented in the method embodiments are not repeated here to avoid repetition.

[0120] Based on the same technical concept, the application further provides an electronic device for executing the above abnormal crowd detection method, Figure 5 A structural schematic diagram of an electronic device for implementing various embodiments of the application. The electronic device can have a large difference due to different configurations or performances, and can include a processor 502, a communications interface 504, a memory 506, and a communications bus 508, wherein the processor 502, the communications interface 504, and the memory 506 complete mutual communication through the communications bus 508. The processor 502 can call a computer program stored on the memory 506 and executable on the processor 502 to execute the following steps: Obtaining a first fusion image in a first picture, the first picture being a picture to be detected for abnormal crowd, and the first fusion image being determined based on a visible light image and a code image of the first picture; Determining a pedestrian feature of a first pedestrian and pedestrian features of a plurality of second pedestrians in the first fusion image, the first pedestrian being a preset pedestrian to be detected, and the second pedestrians being pedestrians other than the first pedestrian in the first fusion image, and the pedestrian feature including a feature vector and an identity code of the pedestrian; Determining a third pedestrian among the second pedestrians based on a preset same-pedestrian determination rule according to the pedestrian feature of the first pedestrian and the pedestrian features of the second pedestrians, the same-pedestrian determination rule being used to determine a pedestrian who is with the first pedestrian; Determining a crowd anomaly probability of the first picture based on a preset abnormal crowd determination algorithm according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian, and a pedestrian feature of a fourth pedestrian, to perform abnormal crowd detection, the fourth pedestrian being a pedestrian other than the first pedestrian and the third pedestrian in the first picture, and the crowd anomaly probability being used to represent a probability of abnormal crowd gathering in the first picture.

[0121] In an implementation manner, the determining the third pedestrian among the second pedestrians based on the preset same-pedestrian determination rule according to the pedestrian feature of the first pedestrian and the pedestrian features of the second pedestrians includes: Obtaining a second fusion image in a second picture, the second picture being a picture to be determined for same-pedestrian; Determining the pedestrian feature of the first pedestrian and a pedestrian feature of a fifth pedestrian in the second fusion image, the fifth pedestrian being a pedestrian other than the first pedestrian in the second fusion image; Determining the third pedestrian based on the pedestrian feature of the fifth pedestrian and the pedestrian features of the second pedestrians.

[0122] In an implementation manner, the determining the third pedestrian based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian comprises: determining a pedestrian appearance frequency based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian, the pedestrian appearance frequency being used to represent an appearance frequency of each of the fifth pedestrian and the second pedestrian; determining the pedestrian with the appearance frequency greater than a preset threshold as the third pedestrian.

[0123] In an implementation manner, the determining the crowd anomaly probability of the first picture according to the preset crowd anomaly determination algorithm comprises: obtaining a plurality of first fusion images at a plurality of times, and determining the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian in each of the first fusion images; determining the crowd anomaly probability of the first picture in real time according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian in each of the first fusion images based on the crowd anomaly determination algorithm.

[0124] In an implementation manner, the determining the crowd anomaly probability of the first picture in real time according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian in each of the first fusion images comprises: determining a first value, a second value and a third value according to the pedestrian feature of the first pedestrian, the pedestrian feature of the third pedestrian and the pedestrian feature of the fourth pedestrian in each of the first fusion images, the first value being used to represent a density of pedestrians in the first picture, the second value being used to represent an occupancy ratio information of the first pedestrian and the third pedestrian in the first picture, and the third value being used to represent a density of the first pedestrian and the third pedestrian; determining the crowd anomaly probability based on the first value, the second value and the third value.

[0125] In an implementation manner, after the determining the crowd anomaly probability of the first picture according to the preset crowd anomaly determination algorithm and according to the pedestrian feature of the first pedestrian, the pedestrian feature of the second pedestrian and the pedestrian feature of the third pedestrian, the method further comprises: determining an anomaly prevention strategy corresponding to the crowd anomaly probability, different anomaly prevention strategies corresponding to different crowd anomaly probabilities; The specific implementation steps of the crowd abnormal gathering prevention based on the abnormal prevention strategy can refer to the steps of the above-mentioned abnormal crowd detection method embodiments, and the same technical effects can be achieved. To avoid repetition, they will not be described here.

[0126] It should be noted that the electronic device in the embodiments of the present application includes a server, a terminal or other devices in addition to the terminal.

[0127] The above electronic device structure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the illustration, or combine certain components, or different component arrangements. For example, the input unit can include a graphics processing unit (GPU) and a microphone, and the display unit can be configured with a display panel in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit includes at least one of a touch panel and other input devices. The touch panel is also called a touch screen. Other input devices can include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, and the like, which will not be described here.

[0128] The memory can be used to store software programs and various data. The memory can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory can include a volatile memory or a non-volatile memory, or the memory can include both a volatile and a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synchlink DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM).

[0129] The processor can include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor.

[0130] The embodiment of the present application further provides a readable storage medium, and the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize each process of the above-mentioned abnormal crowd detection method embodiment, and the same technical effects can be achieved, to avoid repetition, which will not be described here.

[0131] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0132] The embodiment of the present application further provides a computer program product, which is executed by a processor to realize each process of the above abnormal crowd detection method embodiment and achieve the same technical effects. To avoid repetition, details are not described herein.

[0133] The embodiment of the present application further provides a computer program product, which is executed by a processor to realize each process of the above abnormal crowd detection method embodiment and achieve the same technical effects. To avoid repetition, details are not described herein.

[0134] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0135] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0136] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, also can be through hardware, but many cases the former is the better embodiment. Based on such understanding, the technical solutions of the present application essentially or say the part of the contribution to the prior art can be embodied in the form of software products, the computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), including a number of instructions to make a terminal (may be a mobile phone, computer, server, air conditioner, or network equipment, etc.) executes the method described in various embodiments of the present application.

[0137] The embodiments of the present application are described above in conjunction with the drawings, but the present application is not limited to the above-mentioned specific embodiments, the above-mentioned specific embodiments are only illustrative, but not limited, those skilled in the art can make many forms without departing from the purpose of the present application and the scope of the claims under the inspiration of the present application, all belong to the protection of the present application.

Claims

1. A method for detecting abnormal crowds, characterized in that: include: Acquire a first fused image in a first picture, where the first picture is a picture to be used for abnormal crowd detection, and the first fused image is determined based on a visible light image and a detection code image of the first picture; determining pedestrian features of a first pedestrian in the first fused image and pedestrian features of multiple second pedestrians, where the first pedestrian is a preset pedestrian to be detected and the second pedestrians are pedestrians in the first fused image other than the first pedestrian, the pedestrian features including feature vectors and identity codes of the pedestrians; determining a third pedestrian among the plurality of second pedestrians based on a preset pedestrian determination rule and pedestrian characteristics of the first pedestrian and the second pedestrian characteristics, wherein the pedestrian determination rule is used to determine a pedestrian traveling with the first pedestrian; Based on a preset abnormal crowd determination algorithm, the probability of the crowd being abnormal in the first picture is determined according to the pedestrian characteristics of the first pedestrian, the pedestrian characteristics of the third pedestrian, and the pedestrian characteristics of the fourth pedestrian to perform abnormal crowd detection. The fourth pedestrian is a pedestrian in the first picture who is not the first pedestrian and not the third pedestrian. The probability of the crowd being abnormal is used to characterize the probability of abnormal crowd gathering in the first picture.

2. The method according to claim 1, characterized in that The determining of a third pedestrian among the plurality of second pedestrians based on a preset pedestrian determination rule and according to the pedestrian characteristics of the first pedestrian and the pedestrian characteristics of the second pedestrian includes: Acquire a second fused image in a second picture, where the second picture is a picture for determining a fellow pedestrian; determining pedestrian features of the first pedestrian and pedestrian features of a fifth pedestrian in the second fused image, where the fifth pedestrian is a pedestrian other than the first pedestrian in the second fused image; The third pedestrian is determined based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian.

3. The method according to claim 2, characterized in that The determining the third pedestrian based on the pedestrian feature of the fifth pedestrian and the pedestrian feature of the second pedestrian includes: determining a pedestrian appearance frequency based on the pedestrian characteristics of the fifth pedestrian and the pedestrian characteristics of the second pedestrian, wherein the pedestrian appearance frequency is used to represent the appearance frequency of each of the fifth pedestrian and the second pedestrian; The pedestrian whose appearance frequency is greater than a preset threshold is determined as the third pedestrian.

4. The method according to claim 1, wherein The determining, according to a preset abnormal crowd determination algorithm, a probability of abnormal crowds in the first picture based on the pedestrian features of the first pedestrian, the pedestrian features of the second pedestrian, and the pedestrian features of the third pedestrian, includes: Acquire a plurality of the first fused images at a plurality of times, and determine pedestrian features of the first pedestrian, pedestrian features of the third pedestrian, and pedestrian features of the fourth pedestrian in each of the first fused images; Based on the abnormal crowd determination algorithm, the probability of the crowd in the first picture being abnormal is determined in real time according to the pedestrian features of the first pedestrian, the pedestrian features of the third pedestrian, and the pedestrian features of the fourth pedestrian in each of the first fused images.

5. The method according to claim 1, wherein The determining, in real time, a probability of crowd abnormality in the first picture based on the pedestrian features of the first pedestrian, the pedestrian features of the third pedestrian, and the pedestrian features of the fourth pedestrian in each of the first fused images includes: determining a first value, a second value, and a third value based on the pedestrian features of the first pedestrian, the third pedestrian, and the fourth pedestrian in each of the first fused images, wherein the first value is used to represent the density of pedestrians in the first image, the second value is used to represent the proportion of the first pedestrian and the third pedestrian to the total number of pedestrians in the first image, and the third value is used to represent the density of the first pedestrian and the third pedestrian; The probability of the population being abnormal is determined based on the first value, the second value, and the third value.

6. The method according to claim 1, wherein After determining the probability of the crowd in the first picture being abnormal based on the pedestrian features of the first pedestrian, the pedestrian features of the second pedestrian, and the pedestrian features of the third pedestrian according to the preset abnormal crowd determination algorithm, the method further includes: Determine an abnormality prevention strategy corresponding to the abnormal probability of the group, where different abnormal probabilities of the group correspond to different abnormality prevention strategies; Based on the abnormal prevention strategy, abnormal crowd gathering prevention is carried out.

7. An abnormal crowd detection device, characterized in that: include: A first acquisition module is configured to acquire a first fused image from a first picture, where the first picture is a picture to be used for abnormal crowd detection, and the first fused image is determined based on a visible light image and a detection code image of the first picture; a first determining module, configured to determine pedestrian features of a first pedestrian and pedestrian features of multiple second pedestrians in the first fused image, wherein the first pedestrian is a preset pedestrian to be detected and the second pedestrians are pedestrians other than the first pedestrian in the first fused image, the pedestrian features including a feature vector and an identity code of the pedestrian; a second determining module, configured to determine a third pedestrian among the plurality of second pedestrians based on a preset pedestrian determination rule and according to pedestrian characteristics of the first pedestrian and pedestrian characteristics of the second pedestrian, wherein the pedestrian determination rule is used to determine a pedestrian traveling with the first pedestrian; The third determination module is used to determine the probability of abnormal crowd in the first picture based on a preset abnormal crowd determination algorithm and according to the pedestrian characteristics of the first pedestrian, the pedestrian characteristics of the third pedestrian and the pedestrian characteristics of the fourth pedestrian, so as to perform abnormal crowd detection. The fourth pedestrian is a pedestrian in the first picture who is not the first pedestrian and the third pedestrian. The abnormal crowd probability is used to characterize the probability of abnormal crowd gathering in the first picture.

8. A computer device, characterized in that: The device comprises: processor; and A memory arranged to store computer-executable instructions, wherein the executable instructions are configured to be executed by the processor, and the executable instructions include instructions for executing the steps in the abnormal crowd detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is used to store computer-executable instructions, and the executable instructions enable a computer to execute the abnormal crowd detection method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the abnormal crowd detection method according to any one of claims 1 to 6.

Images