Real-time pedestrian counting method, device, camera and server

Abstract

The present application discloses a real-time pedestrian counting method, device, camera and server. The counting method comprises: acquiring M photographed images; cropping the M images to obtain N target images; using a pedestrian re-identification algorithm to calculate the highest similarity value between the current target image and the target images in the re-identification image library one by one; when the highest similarity value is lower than a first threshold, adding the current target image to the re-identification image library, and increasing the number of people by 1; when the highest similarity value is higher than the first threshold, adding the current target image together with the similar image to the corresponding personal image library, and the number of people remains unchanged; using a face recognition algorithm to perform face recognition on the re-identification image library and the personal image library; when there is a similarity value between two target images in the re-identification image library that is higher than a second threshold, reducing the number of people by 1; when there is a similarity value between two target images in the personal image library that is lower than the second threshold, increasing the number of people by 1.

Description

Technical Field

[0001] The present application belongs to the field of computer vision, and specifically relates to a real-time pedestrian counting method, device, camera with built-in AI chip, and cloud server. Background Art

[0002] Online pedestrian counting is one of the more common applications of computer vision. With the continuous advancement and development of computer vision algorithms and AI chips, as well as the urgent needs of public places such as security and supermarkets, pedestrian counting technology can provide passenger flow numbers for public places such as communities, campuses, hospitals, and airports, providing decision makers with more accurate references.

[0003] In the process of implementing this application, the inventor found that the current mainstream pedestrian online counting algorithm has many restrictions, it is difficult to distinguish pedestrians wearing similar clothes, and pedestrians who have changed clothes are often counted repeatedly. At present, there are at least the following problems: the current pedestrian detection algorithm is difficult to distinguish the same pedestrian wearing different clothes, pedestrians who have changed clothes are often counted repeatedly, and there is a technical problem of missing counts for pedestrians wearing similar clothes. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a real-time pedestrian counting method, device, camera with built-in AI chip and cloud server, which can solve the technical problems that the current pedestrian re-identification algorithm has difficulty in distinguishing the same pedestrian wearing different clothes, often counting pedestrians who have changed clothes repeatedly, and missing counting pedestrians with similar clothes.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a real-time pedestrian counting method, which is applied to a camera with a built-in AI chip or a cloud server, including:

[0007] Get the M images taken;

[0008] Using a target detection algorithm to perform target detection on each of the M images;

[0009] The M images are cropped according to the target detection result to obtain N target images, wherein each of the target images includes only one target pedestrian;

[0010] A person re-identification algorithm is used to obtain feature maps corresponding to the N target images, and the highest similarity values ​​between the current target image and the target images in the re-identification image library are calculated one by one according to the feature maps, wherein the target images in the re-identification image library are the target images that have been subjected to similarity calculation and added to the re-identification image library;

[0011] When the highest similarity value is lower than a first threshold, the target pedestrian corresponding to the current target image is defined as a new person, the current target image is added to the re-identification image library, and the number of pedestrians is increased by 1; when the highest similarity value is higher than the first threshold, the target pedestrian corresponding to the current target image is defined as a non-new person, the current target image and similar images are added to the corresponding personal image library, and the number of pedestrians remains unchanged, wherein the similar image is an image that is searched from the re-identification image library and is similar to the feature map of the current target image;

[0012] Using a face detection algorithm and a face recognition algorithm to perform face recognition on the target image in the re-recognition image library and the personal image library;

[0013] When the similarity value between two target images in the re-identification image library is higher than the second threshold, an association relationship is established between the two target images, and the number of pedestrians is reduced by 1; when the similarity value between two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

[0014] Furthermore, before acquiring the M photographed images, the method further includes:

[0015] The M images are captured by the camera, the camera is installed at a height of 3 meters, an angle of 15 degrees looking down, and a focal length greater than 6 mm.

[0016] Furthermore, the taking of M images by the camera is specifically:

[0017] The M images are captured by the camera at preset time intervals.

[0018] Further, after acquiring the M photographed images, the method further includes:

[0019] Determine the effective area through mask;

[0020] The target detection algorithm is used to perform target detection on each of the M images, specifically:

[0021] A target detection algorithm is used to perform target detection on the target within the effective area of ​​each of the M images.

[0022] Furthermore, the adopting of a person re-identification algorithm to obtain a feature map corresponding to the N target images, and calculating the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature map, specifically includes:

[0023] Using a person re-identification algorithm to obtain feature maps corresponding to the N target images;

[0024] Using a search algorithm, searching for one or more similar images similar to the feature map of the current target image from a re-identification image library;

[0025] respectively calculating similarity values ​​between the feature map of the current target image and the one or more similar images;

[0026] Determine a highest similarity value between the feature map of the current target image and the one or more similar images.

[0027] In a second aspect, an embodiment of the present application provides a real-time pedestrian counting device, which is applied to a camera with a built-in AI chip or a cloud server, including:

[0028] An acquisition module is used to acquire M captured images;

[0029] A detection module, configured to perform target detection on each of the M images using a target detection algorithm;

[0030] A cropping module, used for cropping the M images according to the target detection result to obtain N target images, wherein each of the target images includes only one target pedestrian;

[0031] a calculation module, configured to use a person re-identification algorithm to obtain a feature map corresponding to the N target images, and calculate the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature map, wherein the target images in the re-identification image library are the target images that have been subjected to similarity calculation and added to the re-identification image library;

[0032] a storage module, configured to define the target pedestrian corresponding to the current target image as a new person when the highest similarity value is lower than a first threshold value, add the current target image to the image library, and increase the number of pedestrians by 1; and define the target pedestrian corresponding to the current target image as a non-new person when the highest similarity value is higher than the first threshold value, add the current target image together with similar images to the corresponding personal image library, and keep the number of pedestrians unchanged, wherein the similar image is an image similar to the feature map of the current target image searched from the re-identification image library;

[0033] A recognition module, configured to perform face recognition on the target image in the re-recognition image library and the personal image library by using a face detection algorithm and a face recognition algorithm;

[0034] The association module is used to establish an association relationship between the two target images when the similarity value between the two target images in the re-identification image library is higher than a second threshold, and reduce the number of pedestrians by 1; when the similarity value between the two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

[0035] Furthermore, the device also includes:

[0036] The shooting module is used to shoot the M images through the camera, the camera is installed at a height of 3 meters, the installation angle is 15 degrees looking down, and the focal length is greater than 6 mm.

[0037] Furthermore, the shooting module is specifically configured to shoot the M images at preset time intervals through the camera.

[0038] Furthermore, the device also includes:

[0039] The mask module is used to determine the effective area through the mask;

[0040] The detection module is specifically configured to use a target detection algorithm to perform target detection on the target within the effective area of ​​each of the M images.

[0041] Furthermore, the calculation module specifically includes:

[0042] A feature submodule, used for adopting a person re-identification algorithm to obtain a feature map corresponding to the N target images;

[0043] A search submodule, configured to use a search algorithm to search for one or more similar images similar to the feature map of the current target image from the re-identification image library;

[0044] A calculation submodule, used to respectively calculate similarity values ​​between the feature map of the current target image and the one or more similar images;

[0045] A determination submodule is used to determine the highest similarity value between the feature map of the current target image and the one or more similar images.

[0046] In a third aspect, an embodiment of the present application provides a camera with a built-in AI chip, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the pedestrian counting method as described in the first aspect.

[0047] In a fourth aspect, an embodiment of the present application provides a cloud server, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the pedestrian counting method as described in the first aspect.

[0048] In an embodiment of the present application, on the one hand, for the same pedestrian after changing clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as two new people and entered into the database, and the number of pedestrians is one more than the actual number. After the face recognition is performed on the target image in the re-identification image library in the second stage, they will be redefined as the same person, and the number of pedestrians will be reduced by 1. On the other hand, for two pedestrians with similar clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as only one person and entered into the database, and the number of pedestrians is one less than the actual number. After the face recognition is performed on the target image in the personal image library in the second stage, they will be redefined as two people, and the number of pedestrians will be increased by 1, thereby correcting the erroneous result of repeated counting after changing clothes, and finally obtaining an accurate number of pedestrians. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for implementing human number counting provided in an embodiment of the present application;

[0050] Figure 2 It is a flowchart of another method for counting the number of people provided in an embodiment of the present application;

[0051] Figure 3 It is a structural diagram of a real-time pedestrian counting device provided in an embodiment of the present application.

[0052] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0055] The photographic device provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings. Embodiment 1

[0056] Reference Figure 1 , shows a flow chart of a method for counting the number of people provided in an embodiment of the present application, which is applied to a camera with a built-in AI chip or a cloud server. Specifically, the camera can be a Ruijian smart camera produced by Zhejiang Hanshuo Electronic Technology Co., Ltd.

[0057] Specifically, the camera with built-in AI chip contains a DSP chip and supports deep learning operators of frameworks such as Caffe, Tensorflow, and Pytorch.

[0058] Real-time pedestrian counting methods include:

[0059] S101: Acquire M captured images.

[0060] Optionally, M images are captured by a camera.

[0061] Optionally, the M images may be obtained by shooting with multiple cameras.

[0062] S102: Using a target detection algorithm, perform target detection on each of the M images.

[0063] Optionally, the target detection algorithm may be yolov3, tiny-yolo, Faster-RCNN, SSD or other algorithms.

[0064] S103: Crop the M images according to the target detection result to obtain N target images, wherein each target image includes only one target pedestrian.

[0065] It can be understood that the target detection result may include the coordinate position of the target pedestrian in the target image, the position of the target pedestrian may be identified, and the target pedestrian may be specifically selected using a virtual detection frame, and then the original image may be cropped according to the detection frame to obtain a target image that includes only one target pedestrian in each image.

[0066] Optionally, target images of a suitable size are cropped according to a confidence algorithm, and it can be further ensured that each target image includes only one target pedestrian.

[0067] S104: Using a person re-identification algorithm to obtain feature maps corresponding to N target images, and calculating the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature maps, wherein the target images in the re-identification image library are target images that have undergone similarity calculation and have been added to the re-identification image library.

[0068] Optionally, pedestrian features in N target images are identified through a Backbone network. Optionally, the pedestrian re-identification algorithm can adopt the re-identification algorithm in the JD fast-reid toolbox, reid-strong-baseline, etc.

[0069] It should be noted that the pedestrian re-identification algorithm can be used to determine whether the corresponding pedestrians in two target images are the same person.

[0070] S105: When the highest similarity value is lower than the first threshold, the target pedestrian corresponding to the current target image is defined as a new person, the current target image is added to the re-identification image library, and the number of pedestrians is increased by 1. When the highest similarity value is higher than the first threshold, the target pedestrian corresponding to the current target image is defined as a non-new person, the current target image and the similar images are added to the corresponding personal image library, and the number of pedestrians remains unchanged, wherein the similar image is an image that is searched from the re-identification image library and is similar to the feature map of the current target image.

[0071] The first threshold value may be selected according to actual needs.

[0072] Optionally, the re-identification image library and the personal image library may be two independent databases set in different storage spaces.

[0073] It can be understood that when the highest similarity value is lower than the first threshold, it means that the current target image is different from all target images in the re-identification image library. Based on this, it can be judged that the target pedestrian corresponding to the current target image is defined as appearing in the detection area for the first time, and the number of pedestrians should be increased by 1. Since the situation where the pedestrian changes clothes and enters the detection area again cannot be ruled out, the newcomer here is assigned an independent ID and added to the re-identification image library for subsequent processing.

[0074] On the contrary, when the highest similarity value is higher than the first threshold, it means that the current target image is similar to some target images in the re-identification image library. In order to prevent duplicate counting, the number of pedestrians should be temporarily kept unchanged. Since it is impossible to rule out the situation where two pedestrians with similar clothing enter the detection area one after another, the non-new people here are associated with each other and added to the personal image library for subsequent processing.

[0075] Specifically, the similarity value or similarity distance value of the two target images can be used to determine whether the corresponding pedestrians in the two target images are the same person. It can be understood that the higher the similarity value and the lower the similarity distance value, the more similar the two target images are, and the lower the similarity value and the higher the similarity distance value, the greater the difference between the two target images.

[0076] Specifically, similarity measurement methods include: cosine similarity distance, Euclidean distance, Mahalanobis distance, Hamming distance, etc. Preferably, cosine similarity distance is used.

[0077] S106: Using a face detection algorithm and a face recognition algorithm, perform face recognition on the target image in the re-recognition image library and the personal image library.

[0078] It can be understood that in this step, the target images in the image library have been processed by the first-stage pedestrian re-identification algorithm. The pedestrian re-identification algorithm preliminarily determines that each target image in the image library corresponds to a target pedestrian. However, for pedestrians who change clothes, there is an extra count, and for pedestrians with similar clothes, there is a missed count.

[0079] The facial recognition algorithm can eliminate the uncertainty factors of clothing changes and clothing similarities.

[0080] S107: When the similarity value between two target images in the re-identification image library is higher than the second threshold, the two target images are associated and the number of pedestrians is reduced by 1; when the similarity value between two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

[0081] The second threshold can be selected according to actual needs.

[0082] It is understandable that once there is a pedestrian who has changed clothes in the re-identification image library, the face recognition will find the two target images before and after the change of clothes, establish an association between the two, and count them as one person. At this time, the previous number of pedestrians needs to be reduced by 1 for correction.

[0083] Once there are pedestrians with similar clothing in the personal image library, face recognition will find two target images with similar clothing, and after face recognition, they will be judged as different pedestrians. At this time, the previous number of pedestrians needs to be added by 1 for correction.

[0084] In an embodiment of the present application, on the one hand, for the same pedestrian after changing clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as two new people and entered into the database, and the number of pedestrians is one more than the actual number. After the face recognition is performed on the target image in the re-identification image library in the second stage, they will be redefined as the same person, and the number of pedestrians will be reduced by 1. On the other hand, for two pedestrians with similar clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as only one person and entered into the database, and the number of pedestrians is one less than the actual number. After the face recognition is performed on the target image in the personal image library in the second stage, they will be redefined as two people, and the number of pedestrians will be increased by 1, thereby correcting the erroneous result of repeated counting after changing clothes, and finally obtaining an accurate number of pedestrians. Embodiment 1

[0085] Reference Figure 2 , shows a flow chart of another method for implementing human population counting provided in an embodiment of the present application, which is applied to a camera with a built-in AI chip or a cloud server. Specifically, the camera can be a Ruijian smart camera produced by Zhejiang Hanshuo Electronic Technology Co., Ltd.

[0086] Specifically, the camera with built-in AI chip contains a DSP chip and supports deep learning operators of frameworks such as Caffe, Tensorflow, and Pytorch.

[0087] Real-time pedestrian counting methods include:

[0088] S201: M images are captured by a camera at a preset time interval. The camera is mounted at a height of 3 meters, a mounting angle of 15 degrees looking down, and a focal length greater than 6 mm.

[0089] It should be noted that the pedestrian re-identification algorithm needs to obtain the complete pedestrian torso, and the camera height needs to be as high as possible, while the face recognition algorithm needs to see the face as clearly as possible and the focus needs to be as large as possible. Setting the camera height to 3 meters, the setting angle to 15 degrees looking down, and the focal length greater than 6 mm can better balance these two points.

[0090] Optionally, M images are captured by a camera at preset time intervals.

[0091] It can be understood that by capturing images at preset time intervals, there is no need for continuous video streaming, the required data information has little correlation, the data transmission bandwidth is reduced, and the energy consumption and risk of the equipment are reduced. At the same time, since there is no need for video streaming, there are no intermittent frames or discontinuous videos, which improves the accuracy of the calculation.

[0092] S202: Acquire M captured images.

[0093] Optionally, the M images include A images taken by a first camera and B images taken by a second camera, wherein the first camera is a camera with a built-in AI chip. When the method is applied to the first camera, before acquiring the M images taken at preset time intervals, the second camera transmits the B images to the first camera.

[0094] It is understandable that, when the real-time pedestrian counting method is applied to the first camera, the second camera transmits the B images it captures to the first camera, so as to complete the subsequent steps of implementing the pedestrian counting method on the first camera.

[0095] To implement the real-time pedestrian counting method on the first camera, only a camera with a built-in AI chip is needed, with low implementation cost and simple structure.

[0096] When the real-time pedestrian counting method is applied to a cloud server, the first camera transmits image A taken by it to the cloud server, and the second camera transmits image B taken by it to the cloud server, so as to complete the subsequent steps of implementing the pedestrian counting method on the cloud server.

[0097] Optionally, M is a preset value, for example, M is equal to 100. When 100 images are captured, or when a plurality of cameras cumulatively capture 100 images, the process proceeds to step S203.

[0098] S203: Determine the effective area through the mask.

[0099] When processing M images, by setting a mask according to the effective activity area of ​​the pedestrian, the torso of the pedestrian can be made complete, which is convenient for the subsequent pedestrian re-identification algorithm.

[0100] S204: Using a target detection algorithm, perform target detection on targets within a valid area of ​​each of the M images.

[0101] S205: Crop the M images according to the target detection result to obtain N target images, wherein each target image includes only one target pedestrian.

[0102] S206: Using a person re-identification algorithm to obtain feature maps corresponding to the N target images.

[0103] S207: Using a search algorithm, search for one or more similar images that are similar to the feature map of the current target image from the re-identification image library.

[0104] Optionally, the target images in the image library are processed one by one. The one-by-one processing may be processed one by one in sequence. In order to improve the processing efficiency, a preset number of images, such as three images, may be grouped together and processed one group by one in sequence.

[0105] Optionally, the search algorithm may adopt a 1:M search algorithm or an N:M search algorithm, and may find the closest 1 or N pedestrians from among the M target pedestrians.

[0106] S208: Calculate similarity values ​​between the feature map of the current target image and one or more similar images respectively.

[0107] S209: Determine the highest similarity value between the feature map of the current target image and one or more similar images.

[0108] The search algorithm can reduce the amount of calculation and improve the calculation efficiency.

[0109] S210: When the highest similarity value is lower than the first threshold, the target pedestrian corresponding to the current target image is defined as a new person, the current target image is added to the re-identification image library, and the number of pedestrians is increased by 1. When the highest similarity value is higher than the first threshold, the target pedestrian corresponding to the current target image is defined as a non-new person, the current target image and the similar images are added to the corresponding personal image library, and the number of pedestrians remains unchanged, wherein the similar image is an image searched from the re-identification image library that is similar to the feature map of the current target image.

[0110] S211: Using a face detection algorithm and a face recognition algorithm, perform face recognition on target images in the re-identification image library and the personal image library.

[0111] S212: When the similarity value between two target images in the re-identification image library is higher than the second threshold, the two target images are associated and the number of pedestrians is reduced by 1; when the similarity value between two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

[0112] In the embodiment of the present application, images are collected at preset time intervals, and continuous video streams are not required. The required data information has little correlation, which reduces the data transmission bandwidth, reduces the energy consumption and risk of the device, and because no video stream is required, there is no video intermittent frame or video discontinuity, which improves the accuracy of the calculation. The search algorithm can reduce the amount of calculation, pre-search for relatively similar images, and then make comparisons, which improves the calculation efficiency. Embodiment 3

[0113] Reference Figure 3 , shows a schematic diagram of the structure of a real-time pedestrian counting device provided in an embodiment of the present application, the real-time pedestrian counting device 30 is applied to a camera with a built-in AI chip or a cloud server, and the real-time pedestrian counting device 30 includes:

[0114] The acquisition module 301 is used to acquire M photographed images.

[0115] The detection module 302 is used to perform target detection on each of the M images using a target detection algorithm.

[0116] The cropping module 303 is used to crop the M images according to the target detection result to obtain N target images, wherein each target image includes only one target pedestrian.

[0117] The calculation module 304 is used to use a pedestrian re-identification algorithm to obtain feature maps corresponding to N target images, and calculate the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature maps, wherein the target images in the re-identification image library are target images that have undergone similarity calculation and have been added to the re-identification image library.

[0118] The storage module 305 is used to define the target pedestrian corresponding to the current target image as a new person when the highest similarity value is lower than the first threshold, add the current target image to the re-identification image library, and increase the number of pedestrians by 1; when the highest similarity value is higher than the first threshold, define the target pedestrian corresponding to the current target image as a non-new person, add the current target image together with the similar image to the corresponding personal image library, and the number of pedestrians remains unchanged, wherein the similar image is an image that is searched from the re-identification image library and has a feature map similar to the current target image.

[0119] The recognition module 306 is used to use a face detection algorithm and a face recognition algorithm to perform face recognition on the target images in the re-recognition image library and the personal image library.

[0120] The association module 307 is used to establish an association relationship between the two target images when the similarity value between the two target images in the re-identification image library is higher than the second threshold, and reduce the number of pedestrians by 1; when the similarity value between the two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

[0121] Furthermore, the real-time pedestrian counting device 30 further includes:

[0122] The shooting module 308 is used to shoot M images through a camera. The camera is installed at a height of 3 meters, an angle of 15 degrees looking down, and a focal length greater than 6 mm.

[0123] Furthermore, the shooting module 308 is specifically configured to shoot M images at preset time intervals through a camera.

[0124] Furthermore, the real-time pedestrian counting device also includes:

[0125] The mask module 309 is used to determine the effective area through the mask.

[0126] The detection module 302 is specifically configured to use a target detection algorithm to perform target detection on a target within a valid area of ​​each of the M images.

[0127] Furthermore, the calculation module 304 specifically includes:

[0128] The special submodule 3041 is used to adopt a pedestrian re-identification algorithm to obtain feature maps corresponding to N target images.

[0129] The search submodule 3042 is used to use a search algorithm to search for one or more similar images that are similar to the feature map of the current target image from the re-identification image library.

[0130] The calculation submodule 3043 is used to respectively calculate the similarity values ​​between the feature map of the current target image and one or more similar images.

[0131] The determination submodule 3044 is used to determine the highest similarity value between the feature map of the current target image and one or more similar images.

[0132] The real-time pedestrian counting device 30 provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0133] In the embodiment of the present application, pedestrians who have changed clothes are counted as two new people and included in the database when passing through the first stage of pedestrian re-identification algorithm, and the number of pedestrians is one more than the actual number. After the second stage of face recognition of the target image in the image library, they will be redefined as the same person, and the number of pedestrians will be reduced by 1, so as to correct the erroneous result of repeated counting after the change of clothes, and finally obtain the accurate number of pedestrians.

[0134] The virtual device in the embodiment of the present application may be a device, or a component, an integrated circuit, or a chip in a terminal. Embodiment 4

[0135] The embodiment of the present application also provides a camera with a built-in AI chip, including a camera, a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the pedestrian counting method in the above embodiment are implemented. And the same technical effect can be achieved, so it will not be repeated here to avoid repetition.

[0136] In an embodiment of the present application, on the one hand, for the same pedestrian after changing clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as two new people and entered into the database, and the number of pedestrians is one more than the actual number. After the face recognition is performed on the target image in the re-identification image library in the second stage, they will be redefined as the same person, and the number of pedestrians will be reduced by 1. On the other hand, for two pedestrians with similar clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as only one person and entered into the database, and the number of pedestrians is one less than the actual number. After the face recognition is performed on the target image in the personal image library in the second stage, they will be redefined as two people, and the number of pedestrians will be increased by 1, thereby correcting the erroneous result of repeated counting after changing clothes, and finally obtaining an accurate number of pedestrians. Embodiment 5

[0137] The embodiment of the present application also provides a cloud server, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and when the program or instruction is executed by the processor, the steps of the pedestrian counting method in the above embodiment are implemented. And the same technical effect can be achieved, and it will not be repeated here to avoid repetition.

[0138] In an embodiment of the present application, on the one hand, for the same pedestrian after changing clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as two new people and entered into the database, and the number of pedestrians is one more than the actual number. After the face recognition is performed on the target image in the re-identification image library in the second stage, they will be redefined as the same person, and the number of pedestrians will be reduced by 1. On the other hand, for two pedestrians with similar clothes, when passing through the first stage of the pedestrian re-identification algorithm, they are counted as only one person and entered into the database, and the number of pedestrians is one less than the actual number. After the face recognition is performed on the target image in the personal image library in the second stage, they will be redefined as two people, and the number of pedestrians will be increased by 1, thereby correcting the erroneous result of repeated counting after changing clothes, and finally obtaining an accurate number of pedestrians.

[0139] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A real-time pedestrian counting method, applied to a camera with a built-in AI chip or a cloud server, characterized in that: include: Get the M images taken; Using a target detection algorithm to perform target detection on each of the M images; The M images are cropped according to the target detection result to obtain N target images, wherein each of the target images includes only one target pedestrian; A person re-identification algorithm is used to obtain feature maps corresponding to the N target images, and the highest similarity values ​​between the current target image and the target images in the re-identification image library are calculated one by one according to the feature maps, wherein the target images in the re-identification image library are the target images that have been subjected to similarity calculation and added to the re-identification image library; When the highest similarity value is lower than a first threshold, the target pedestrian corresponding to the current target image is defined as a new person, the current target image is added to the re-identification image library, and the number of pedestrians is increased by 1; when the highest similarity value is higher than the first threshold, the target pedestrian corresponding to the current target image is defined as a non-new person, the current target image and similar images are added to the corresponding personal image library, and the number of pedestrians remains unchanged, wherein the similar image is an image that is searched from the re-identification image library and is similar to the feature map of the current target image; Using a face detection algorithm and a face recognition algorithm to perform face recognition on the target image in the re-recognition image library and the personal image library; When the similarity value between two target images in the re-identification image library is higher than the second threshold, an association relationship is established between the two target images, and the number of pedestrians is reduced by 1; when the similarity value between two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

2. The method according to claim 1, characterized in that Before acquiring the M images, the method further includes: The M images are captured by the camera, the camera is installed at a height of 3 meters, an angle of 15 degrees looking down, and a focal length greater than 6 mm.

3. The method according to claim 2, characterized in that The taking of M images by the camera is specifically: The M images are captured by the camera at preset time intervals.

4. The method according to claim 1, characterized in that: After acquiring the M images, the method further includes: Determine the effective area through mask; The target detection algorithm is used to perform target detection on each of the M images, specifically: A target detection algorithm is used to perform target detection on the target within the effective area of ​​each of the M images.

5. The method according to claim 1, characterized in that The adopting of a person re-identification algorithm to obtain a feature map corresponding to the N target images, and calculating the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature map, specifically includes: Using a person re-identification algorithm to obtain feature maps corresponding to the N target images; Using a search algorithm, searching for one or more similar images similar to the feature map of the current target image from a re-identification image library; respectively calculating similarity values ​​between the feature map of the current target image and the one or more similar images; Determine a highest similarity value between the feature map of the current target image and the one or more similar images.

6. A real-time pedestrian counting device, applied to a camera with a built-in AI chip or a cloud server, characterized in that: include: An acquisition module is used to acquire M captured images; A detection module, configured to perform target detection on each of the M images using a target detection algorithm; A cropping module, used for cropping the M images according to the target detection result to obtain N target images, wherein each of the target images includes only one target pedestrian; a calculation module, configured to use a person re-identification algorithm to obtain a feature map corresponding to the N target images, and calculate the highest similarity value between the current target image and the target images in the re-identification image library one by one according to the feature map, wherein the target images in the re-identification image library are the target images that have been subjected to similarity calculation and added to the re-identification image library; a storage module, configured to define the target pedestrian corresponding to the current target image as a new person when the highest similarity value is lower than a first threshold, add the current target image to the re-identification image library, and increase the number of pedestrians by 1; and define the target pedestrian corresponding to the current target image as a non-new person when the highest similarity value is higher than the first threshold, add the current target image together with similar images to the corresponding personal image library, and keep the number of pedestrians unchanged, wherein the similar image is an image similar to the feature map of the current target image searched from the re-identification image library; A recognition module, configured to perform face recognition on the target image in the re-recognition image library and the personal image library by using a face detection algorithm and a face recognition algorithm; The association module is used to establish an association relationship between the two target images when the similarity value between the two target images in the re-identification image library is higher than a second threshold, and reduce the number of pedestrians by 1; when the similarity value between the two target images in the personal image library is lower than the second threshold, the number of pedestrians is increased by 1.

7. The device according to claim 6, characterized in that Also includes: The shooting module is used to shoot the M images through the camera, the camera is installed at a height of 3 meters, the installation angle is 15 degrees looking down, and the focal length is greater than 6 mm.

8. The device according to claim 7, characterized in that The shooting module is specifically configured to shoot the M images at preset time intervals through the camera.

9. The device according to claim 6, characterized in that Also includes: The mask module is used to determine the effective area through the mask; The detection module is specifically used to use a target detection algorithm to perform target detection on the target within the effective area of ​​each of the M images.

10. The device according to claim 6, characterized in that The computing module specifically includes: A feature submodule, used for adopting a person re-identification algorithm to obtain a feature map corresponding to the N target images; A search submodule, configured to use a search algorithm to search for one or more similar images similar to the feature map of the current target image from the re-identification image library; A calculation submodule, used to respectively calculate similarity values ​​between the feature map of the current target image and the one or more similar images; A determination submodule is used to determine the highest similarity value between the feature map of the current target image and the one or more similar images.

11. A camera with a built-in AI chip, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the pedestrian counting method as described in any one of claims 1 to 5.

12. A cloud server, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the pedestrian counting method as described in any one of claims 1 to 5.

Images