Passenger flow counting method and device, computer equipment and storage medium

Abstract

The invention discloses a passenger flow counting method and device, computer equipment and a storage medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a monitoring video image, converting the monitoring video image into a to-be-detected picture stream, setting a first region in any to-be-detected picture in the to-be-detected picture stream, and enabling the first region to at least comprise a second region, detecting a target in a first area in any to-be-detected picture based on a target detection model, extracting at least one to-be-detected target to obtain a target preset area image corresponding to the to-be-detected target, and determining a tracker corresponding to the to-be-detected target based on the target preset area image, according to the method, passenger flow counting is determined according to the target image center point coordinates of the target tracker and the overlapping degree of the second area, and the accuracy and real-time performance of passenger flow counting are effectively improved through efficient target recognition, an accurate tracking algorithm and dynamic change monitoring.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a passenger counting method, apparatus, computer equipment, and storage medium. Background Art

[0002] As businesses deepen their analysis of customer flow data, customer flow has become a crucial indicator of store sales performance. Accurate customer flow statistics provide insights into customer visit frequency and volume, enabling businesses to optimize marketing strategies and store layouts, enhance the customer experience, and increase sales. Currently, video-based customer flow counting methods are becoming mainstream, employing a variety of advanced technologies such as face detection and person detection and tracking. While these methods have achieved impressive statistical accuracy, practical applications still present challenges. In particular, the accuracy of customer flow counting can be compromised when customers overlap or linger at the door.

[0003] Some passenger counting technologies, particularly those based on head detection, typically use object detection algorithms such as Tiny-Yolo or the SSH module to detect head frames. Path matching and tracking are then performed by calculating distances between objects (such as OKS distance and Euclidean distance). These technologies can count passenger flow based on the movement of objects. During path matching, paths are created to record the center point of each object and assign a unique ID number. By matching these paths with the detection frames, people entering and leaving the vehicle can be counted.

[0004] However, using OKS distance and Euclidean distance for target matching ignores factors such as target size and color, resulting in large matching errors in some cases, which in turn affects customer counting accuracy. This method uses the path's trajectory to determine whether it meets the counting requirements. However, for targets that wander between active and inactive areas, this can lead to miscounts or omissions. This is especially true when customers linger at the store entrance, resulting in inaccurate customer counting results. Summary of the Invention

[0005] The present application provides a passenger flow counting method to at least solve the problem of improving the accuracy of passenger flow counting in the related art.

[0006] This application provides a passenger flow counting method, including:

[0007] Obtain surveillance video images and convert them into image streams to be detected;

[0008] Setting a first area in any picture to be detected in the picture stream to be detected, wherein the first area at least includes a second area;

[0009] Based on the target detection model, detect targets within a first area in any image to be detected, extract at least one target to be detected, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is used to detect, track or classify the target to be detected;

[0010] Determining a tracker corresponding to the target to be detected based on the target preset area image, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image;

[0011] The passenger flow count is determined based on the coordinates of the center point of the target image and the overlap degree of the second area.

[0012] The present application also provides a passenger counting device, comprising:

[0013] The acquisition module is used to acquire surveillance video images and convert them into image streams to be detected;

[0014] A region division module is configured to set a first region in any image to be detected in the image stream to be detected, wherein the first region includes at least a second region;

[0015] A detection module, configured to detect targets within a first area in any image to be detected based on a target detection model, extract at least one target to be detected, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is configured to detect, track, or classify the target to be detected;

[0016] a position determination module, configured to determine a tracker corresponding to the target to be detected based on an image of a preset target area, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image;

[0017] The counting module is used to determine the passenger flow count based on the coordinates of the center point of the target image and the overlap degree of the second area.

[0018] The present application also provides a computer device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned passenger flow counting methods when executing the computer program.

[0019] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned passenger flow counting methods are implemented.

[0020] This application establishes a first region within the image to be detected, and a second region within the first region, effectively focusing on key portions of the video image, thereby optimizing the accuracy of customer flow counting. By acquiring the image to be detected from the surveillance video and identifying the target within it, the system accurately marks the preset target region, providing a clear basis for subsequent target detection. These marked targets are processed based on the target detection model, their preset region images are extracted, and trackers are set for the target preset region images. This not only detects the target but also tracks it during its movement, thereby improving the accuracy of target identification and tracking. Determining the location of the preset region images further strengthens continuous monitoring of the target's status. As the target changes dynamically within the surveillance video, the target's movement can be tracked and calculated in real time. By summarizing the changes observed by these target trackers within the surveillance video image, accurate customer flow statistics can be generated. This method, through efficient target identification, precise tracking algorithms, and dynamic monitoring, effectively improves the accuracy and real-time nature of customer flow counting, ensuring stable acquisition of customer flow data even in complex surveillance scenarios and providing businesses with accurate market analysis data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flow chart of a passenger flow counting method provided in an embodiment of the present application.

[0023] Figure 2 This is a structural block diagram of a passenger counting device provided in an embodiment of the present application.

[0024] Figure 3 A timing diagram of a passenger flow counting method provided in an embodiment of the present application.

[0025] Figure 4 This is a diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] In one embodiment, Figure 1 As shown, a passenger flow counting method is provided, comprising the following steps:

[0030] S100: Acquire surveillance video images and convert the surveillance video images into a picture stream to be detected.

[0031] Specifically, by acquiring the image to be detected, the system can extract image frames from the real-time video stream. The process of identifying the target to be detected ensures that the system can quickly and accurately identify the target object in the image, whether it is a face, a human body, or other objects, which lays the foundation for target tracking and subsequent processing. By annotating the target to be detected, the specific location and morphological characteristics of the target can be further clarified, providing clear spatial coordinates for target detection.

[0032] S200: Set a first area in any picture to be detected in a picture stream to be detected, where the first area at least includes a second area.

[0033] The first area refers to a specific area set in the image to be detected, which is the area of interest for target activity. The second area refers to the area within the first area of the image used to count passenger flow.

[0034] Specifically, a first area is set in any picture to be detected in the picture stream to be detected, and the system can focus on a specific area in the video for detailed processing without having to perform redundant analysis on the entire image. This area division can reduce the amount of calculation, improve processing efficiency, and make subsequent target detection more focused and accurate. When the target to be detected enters the first area, the system responds immediately and uses this moment as the starting point for target tracking and analysis. The target's behavior is tracked from the moment it enters the first area. The surveillance video image is converted into the picture to be detected, so that subsequent target detection and behavior analysis can be carried out more efficiently. In the process of acquiring the picture to be detected, unnecessary background information is reduced, which not only improves the image processing speed, but also reduces the waste of computing resources.

[0035] S300, based on the target detection model, detect the target in the first area of any image to be detected, extract at least one target to be detected, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is used to detect, track or classify the target to be detected.

[0036] The preset area image of the target to be detected refers to a specific area selected in the detection image for analysis and processing according to pre-set rules or requirements in the target detection task. These areas may be areas of interest for a specific target or targets, such as a person's head, body, or other key parts. The target detection model is a technology that automatically identifies and locates targets in images using deep learning algorithms, particularly convolutional neural networks (CNNs). It can not only identify the type of target but also provide the target's location coordinates in the image.

[0037] Specifically, the target detection model extracts the labeled preset area image of the target to be detected. The system can quickly identify and extract key information areas when processing complex surveillance videos. The target detection model can not only identify the target, but also combine tracking and classification functions according to actual needs to track the target in real time during its movement, ensuring continuous monitoring of the target's dynamics and precise positioning.

[0038] S400 , determining a tracker corresponding to the target to be detected based on the target preset area image, wherein the tracker at least stores image information, and the image information at least includes coordinates of a center point of the target image.

[0039] A tracker is an algorithm or tool used to continuously monitor and update the position and state of an object in a video frame. In object detection tasks, the tracker's role is to track the object's movement in subsequent video frames based on the detected object position (for example, the object's bounding box) and accurately record the object's trajectory.

[0040] Specifically, determining the target preset area image of at least one target to be detected in any image to be detected and setting a tracker for the target preset area image of the target to be detected can significantly improve the accuracy of target positioning and the stability of subsequent tracking. By clearly defining the target's preset area location, the system can reduce false detections and missed detections and provide a reliable starting point for tracking, thereby optimizing the overall tracking process. At the same time, determining the target area can reduce the range of images that need to be processed, improve processing efficiency, and save computing resources, especially in large-scale video data.

[0041] S500: Determine the passenger flow count based on the coordinates of the center point of the target image and the overlap degree of the second area.

[0042] Specifically, by continuously tracking each target and analyzing its motion changes over time, the system accurately records its entry, exit, or stay, thereby precisely calculating the passenger flow within a specific area. By dynamically adjusting the coordinates of the target image center point, the system maintains high accuracy in target recognition and counting, even in complex environments, such as when targets are obscured or overlapped, to avoid miscounts or omissions.

[0043] Setting a first region within the image to be detected and a second region within the first region effectively focuses on key parts of the video image, thereby optimizing the accuracy of customer flow counting. By acquiring the image to be detected from the surveillance video and identifying the target within it, the system accurately marks the pre-set target region, providing a clear basis for subsequent target detection. These marked targets are processed based on the target detection model, and images of the pre-set regions are extracted. This allows not only target detection but also tracking of the target during its movement, thereby improving the accuracy of target identification and tracking. Determining the location of the pre-set region images further strengthens continuous monitoring of the target's status. As the target changes dynamically within the surveillance video, the target's movement can be tracked and calculated in real time. By summarizing the changes observed by these target trackers within the surveillance video image, accurate customer flow statistics can be generated. This method, through efficient target identification, precise tracking algorithms, and dynamic monitoring, effectively improves the accuracy and real-time nature of customer flow counting, ensuring stable acquisition of customer flow data even in complex surveillance scenarios and providing businesses with accurate market analysis data.

[0044] In one embodiment, obtaining a surveillance video image and converting the surveillance video image into a picture stream to be detected includes:

[0045] In response to any target to be detected appearing in the monitoring picture, the monitoring picture is extracted as a monitoring video image until the target to be detected disappears from the monitoring picture;

[0046] At a preset frame interval, frames in the surveillance video image are extracted to obtain a number of to-be-detected pictures as a to-be-detected picture stream.

[0047] Specifically, in response to the appearance of any target to be detected in the monitoring screen, the monitoring video image is extracted until the target disappears, and then the frame images in the monitoring screen are extracted at a preset frame interval to form a stream of images to be detected. This process can effectively improve the continuity and accuracy of target detection and tracking. In this way, the system can track the dynamic changes of the target in real time, ensuring that the entire process from the appearance to the disappearance of the target is continuously monitored and analyzed. In addition, extracting images at preset frame intervals can reduce the processing of redundant data, optimize the efficiency of the data flow, reduce the computational burden, and ensure that the target information is fully acquired. This method ensures that the dynamic changes of the target in the monitoring video can be accurately captured, improving the reliability of passenger flow counting and target behavior analysis.

[0048] In one embodiment, a tracker corresponding to a target to be detected is determined based on an image of a preset target area, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image, including:

[0049] The tracker also includes: identifier and timestamp, and the image information also includes: target color distribution histogram and target image width and height;

[0050] Determine the center coordinates of the target preset area image based on the target preset area image;

[0051] Obtaining the color of the target preset area image and converting the color of the target preset area image into a hue, saturation, and brightness space to update the target preset area image;

[0052] Traversing the pixels of the updated target preset area image, and extracting the saturation value of the updated target preset area image;

[0053] Dividing the saturation value into a plurality of intervals, determining the number of pixels in the updated target preset area image corresponding to the plurality of intervals, and generating a target color distribution histogram;

[0054] Arrange the number of pixels contained in the target color distribution histogram in descending order, and determine the segments corresponding to the maximum value and the second largest value;

[0055] The middle values of the segments corresponding to the maximum and second-largest values are obtained respectively, and the centers of gravity corresponding to the two segments are calculated respectively to obtain the width and height of the target image.

[0056] Among them, the color conversion model refers to a model that converts an image from one color space to another. In this application, it refers to converting RGB or other color spaces into hue, saturation, and value (HSV) color space in order to extract the color features of the target. Hue (Hue) is the basic color of the color and is an important component in the HSV color space, indicating the type of color (such as red, blue, green, etc.). Saturation (Saturation) describes the purity or intensity of the color. In the HSV color space, the higher the saturation, the purer the color, and the lower the saturation, the darker the color. Value (Value) indicates the brightness of the color, with higher values indicating brighter colors and lower values indicating darker colors. Saturation value (Saturation Value) refers to the saturation value of the image pixel in the HSV color space, reflecting the color intensity of the target area. The target color distribution histogram (Target Color Distribution Histogram) is a histogram generated by statistically analyzing the color values of the image pixels, reflecting the distribution of colors within the target area. The number of pixels is usually counted according to different color intervals. In this application, the statistical information of the color distribution within the target area is described for target recognition and tracking. The centroid refers to the average position of a certain data distribution. For the color distribution histogram, the centroid represents the center value of the color interval, reflecting the color distribution center of the target area. Image Information refers to all important data related to the target image, which is used to describe the characteristics of the target. Target image width and height array: records the width and height of the target in the image, which is used to characterize the size and shape of the target. Target image center point coordinate array: records the center point position of the target in the image, which is used to locate the spatial position of the target. In this application, it refers to the coordinate data of the center point of the target bounding box to help the tracker determine the position of the target.

[0057] Specifically, by assigning a separate tracker to each target to be detected, the system can continuously track the target's position and dynamic changes in subsequent image frames, ensuring that the target's position in the video stream is always accurately recorded. This process avoids identity confusion in multi-target tracking, especially when targets are close together or occluded. Each target can be effectively distinguished and tracked using a specific tracker. This effectively improves the continuity and accuracy of target tracking, optimizes the overall performance of the monitoring system, and ensures the accuracy and timeliness of monitoring data. A tracker is created based on the target's preset area image. The tracker contains an identifier, a timestamp, and image information, enabling the system to continuously and accurately track each target during processing. A unique identifier is created for each target's preset area image using the identifier to distinguish the target. The time of the target's dynamic changes is recorded using the timestamp. The target image's center point coordinate array is calculated using the target's bounding box coordinates, ensuring precise localization of the target. Converting the target's preset area image to HSV color space using a color conversion model minimizes the impact of lighting changes on the target image. This conversion process enables more stable extraction of the target's hue, saturation, and brightness characteristics, providing more reliable color information for subsequent target recognition. By traversing the pixels of the image of the preset area of the new target, a detailed histogram of the color distribution of each target can be created, intuitively showing the distribution of the target color in the image. Dividing the saturation value into several intervals and calculating the number of pixels in each interval can effectively analyze the color characteristics of the target. By sorting the target color distribution histogram, the segment with the most prominent color distribution can be found. The maximum and second-largest value segments are selected and their center of gravity is calculated. This process helps the tracker determine the width, height, and shape of the target, helping to continuously track its size as the target changes, and can also effectively avoid tracking errors caused by changes in the target's appearance.

[0058] In one embodiment, determining a tracker corresponding to a target to be detected based on an image of a preset target area further includes:

[0059] Determine whether the image to be detected is the first image in the image stream to be detected;

[0060] In response to the picture to be detected being the first picture in the stream of pictures to be detected, setting a tracker for an image position of a preset target area in the picture to be detected;

[0061] In response to the picture to be detected not being the first picture in the picture stream to be detected, calculating the difference between the image information of the preset area image in the picture to be detected and the corresponding tracker history information, obtaining a minimum value among the differences, and determining whether the image information of the preset area image in the picture to be detected matches the corresponding tracker history information based on the minimum value and the preset value;

[0062] In response to the minimum value being less than the preset value, the image information of the preset target area image in the to-be-detected picture matches the tracker historical information, and the historical image information of the tracker is updated;

[0063] In response to the minimum value being greater than or equal to the preset value, the image information of the preset target area image in the picture to be detected does not match the tracker history information, and a tracker is set for the preset area image in the picture to be detected.

[0064] Specifically, the system determines whether to create a tracker for the target in the video for the first time by determining whether the image to be detected is the first image in the surveillance video. For surveillance videos, the first image is a critical starting point. By setting a tracker on the first image, the target in subsequent frames can be accurately tracked, laying the foundation for analyzing the target's motion in the video. If the image to be detected is not the first image, the system compares the preset area image in the current frame with the image information in the historical tracker. If the image information is the same, indicating that the target's characteristics have not changed, the system updates the image information in the historical tracker, avoiding the repeated creation of new trackers, saving computing resources and improving processing efficiency. If the image information is different, indicating that the current target is new, the system generates a new tracker for the new preset area image to ensure accurate identification and continuous tracking of the target. In long-running applications such as video surveillance, the target's characteristics may change due to various factors (such as occlusion, movement, or position change). Through intelligent judgment and update mechanisms, the system can promptly adjust the tracking strategy to avoid tracking failures due to target changes. This not only improves the accuracy and robustness of target tracking, but also effectively improves the real-time response capability of the system, and enables continuous and stable target monitoring and analysis, reducing unnecessary computing overhead and tracking errors, and ensuring the efficiency and reliability of the monitoring system.

[0065] In one embodiment, in response to the picture to be detected not being the first picture in the picture stream to be detected, calculating the difference between image information of a preset area image in the picture to be detected and corresponding tracker history information, obtaining a minimum value among the differences, and determining whether the image information of the preset area image in the picture to be detected matches the corresponding tracker history information based on the minimum value and a preset value, including:

[0066] Calculate the difference between the image information of the preset area image in the image to be detected and the historical image information of the tracker, where the difference is obtained based on the following formula:

[0067]

[0068]

[0069] Among them, D iIndicates the difference between the target preset area image and the i-th tracker, X represents the horizontal coordinate of the center point of the target preset area image, Y represents the vertical coordinate of the center point of the target preset area image, X i Indicates the horizontal coordinate of the latest point of the i-th tracker, Y i Indicates the ordinate of the latest point of the i-th tracker, W represents the width of the target preset area image, H represents the height of the target preset area image, W i represents the latest width of the i-th tracker, H i Represents the latest height of the i-th tracker, P1 represents the first color value in the color array, P2 represents the second color value in the color array, P 1i Represents the first color value in the latest color array of the i-th tracker, P 2i Represents the second color value in the latest color array of the i-th tracker, M 1x Indicates the horizontal coordinate value of the first centroid in the color array, M 2x Indicates the horizontal coordinate value of the second center of gravity in the color array, M 1xi Indicates the horizontal coordinate value of the first centroid in the latest color array of the i-th tracker, M 2xi Indicates the horizontal coordinate value of the second center of gravity in the latest color array of the i-th tracker, M 1y Indicates the vertical coordinate value of the first centroid in the color array, M 2y Indicates the vertical coordinate value of the second center of gravity in the color array, M 1yi Indicates the ordinate value of the first centroid in the latest color array of the i-th tracker, M 2yi It represents the ordinate value of the second centroid in the latest color array of the i-th tracker. A1 is the center point coordinate weight, A2 is the target preset area image size weight, and A3 is the target preset area image color centroid weight. The sum of A1, A2, and A3 is 1.

[0070] Among them, A1, A2 and A3 can be adjusted according to actual conditions.

[0071] Specifically, suppose the target information in the image to be detected is:

[0072] Target center coordinates: (X, Y) = (200, 300), target width and height: W = 60, H = 40, target color array: [P1, P2] = [255, 180], target color center coordinates: M 1x =160,M 2x =155,M 1y =210,M 2y =205;

[0073] Assume the target information in the history tracker:

[0074] Target center coordinates: (Xi , Y i )=(195,295);

[0075] Target width and height: W i =58,H i =42;

[0076] Target color array: [P 1i ,P 2i ]=[255,178];

[0077] Coordinates of the target color center: M 1xi =162,M 2xi =156,M 1yi =212,M 2yi =208;

[0078] Let A1 = 0.5, A2 = 0.3, A3 = 0.2;

[0079] Calculate the difference in target center point coordinates:

[0080]

[0081] Calculate the difference between the target width and height:

[0082] Size difference = |WW i ∣+∣HH i |=|60-58|+|40-42|=2+2=4;

[0083]

[0084] Difference in horizontal axis:

[0085]

[0086] Vertical axis difference:

[0087]

[0088] Compute the differences in the color array:

[0089] Color difference = |P1-P 1i ∣+∣P2-P 2i |=|255-255|+|180-178|=0+2=2;

[0090] Calculate the color change rate:

[0091] Weighted composite total rate of change:

[0092] Multiply each rate of change by the corresponding weight and sum them up to get the total rate of change.

[0093] Position change weighting:

[0094] Weighted change position = A1 × position change rate = 0.5 × 0.02 = 0.01;

[0095] Size change weighting:

[0096] Weighted dimensional change = A2 × dimensional change rate = 0.3 × 0.04 = 0.012;

[0097] Color center of gravity change weighting:

[0098] Weighted color change = A3 × color change rate = 0.2 × 0.0046 = 0.00092;

[0099] Sum up the three weighted change rates to get the total change rate:

[0100] Total rate of change = 0.01 + 0.012 + 0.00092 = 0.02292;

[0101] A threshold of 0.05 is set. If the total change rate is less than this threshold, the image under test is considered similar to the historical tracker information, and the historical tracker is updated. If the total change rate is greater than this threshold, a new tracker is generated. The total change rate is 0.02292, which is less than the threshold of 0.05. Therefore, the image under test and the historical tracker image information are similar, and the historical tracker is updated.

[0102] In one embodiment, Figure 3 As shown, the passenger flow count is determined based on the coordinates of the center point of the target image and the overlap of the second area, including:

[0103] Initialize a first array and a second array, wherein the first array is used to store image information of a preset target area to be confirmed, and the second array is used to store image information of a preset target area to be confirmed;

[0104] Sort the coordinates of the center point of the target image according to the acquisition time sequence to obtain the coordinate array of the center point of the target image;

[0105] Get the first and last coordinate points in the target image center coordinate array;

[0106] In response to the fact that the first coordinate point in the target image center point coordinate array does not overlap with the second area border and the last coordinate point overlaps with the second area border, determining whether there is a record of the tracker in the second array;

[0107] In response to there being a record of the tracker in the second array, deleting the record of the tracker before the last record in the second array;

[0108] In response to the absence of a record of the tracker in the second array, storing the tracker and the preset area image of the preset area image into the first array;

[0109] In response to the first coordinate point in the target center point coordinate array overlapping with the second area border and the last coordinate point not overlapping with the second area border, determining whether there is a record of the tracker in the first array;

[0110] In response to there being a record of the tracker in the first array, deleting the record of the tracker before the last record in the first array;

[0111] In response to the absence of a record of the tracker in the first array, the tracker of the preset area and the preset area image are stored in the second array.

[0112] Determining the passenger flow count based on the coordinates of the center point of the target image and the overlap degree of the second area also includes:

[0113] Determine whether the update interval of the tracker exceeds the preset value;

[0114] If the update interval of the response tracker exceeds a preset value, determining whether the identifier of the tracker exists in the first array;

[0115] In response to the presence of the identifier of the tracker in the first array, the passenger flow number count is incremented by one, the image is stored, and the tracker is deleted;

[0116] In response to the tracker identifier not being present in the first array, determining whether the tracker identifier is present in the second array;

[0117] In response to the absence of the tracker identifier in the second array, reacquiring the image to be detected;

[0118] In response to the presence of a tracker identifier in the second array, the count of the number of people leaving the passenger flow is increased by one, and the tracker is deleted.

[0119] Specifically, the mall installs cameras to track customers entering and leaving the mall. The mall entrance is defined as the second detection area, and the center point coordinates of the target image are used to determine whether the target enters or leaves the mall, thereby counting the number of customers.

[0120] Suppose a customer's first coordinate point is (X1, Y1) and their last coordinate point is (X2, Y2), and they overlap with the border of the mall entrance. According to the algorithm, if the first coordinate point overlaps with the border and the last coordinate point does not, the system will store the customer's tracker and image information in the "first array." During this process, the mall's customer count increases by 1, indicating that the customer has entered the mall. For example, when customer Li Ming enters the mall, the system records his target center coordinates as (100, 200), which are within the mall entrance area. The system stores his tracker in the first array, and the mall's customer count increases by 1. After customer Li Ming enters the mall, the system continues to track his coordinates. If the customer stays within the mall for a period of time, the system will update the target's coordinates, but the customer has not yet left the mall area. Therefore, the system will only continue to track Li Ming's target coordinates but will not determine whether he has entered or left the mall. After a period of time, customer Li Ming prepares to leave the mall. The system will re-determine Li Ming's coordinates. If Li Ming's first coordinate point does not overlap with the area border of the mall entrance, but his last coordinate point does, the system determines that Li Ming has left the mall and stores his tracker and image information in the "second array." During this process, the mall's customer count decreases by 1, indicating that the customer has left the mall. For example, when customer Li Ming leaves the mall, the system detects that his target center coordinates are (1200, 2200), located in the mall exit area. The system stores Li Ming's tracker in the second array. At this point, the mall's customer count decreases by 1. Suppose customer Li Ming's target tracker has not been updated for a period of time, or the system's update interval exceeds a preset value (for example, 30 seconds). In this case, the system uses the tracker's identifier to determine whether the target has entered or left the mall. If Li Ming's tracker is recorded in the first array, the system increments the mall's customer count by 1, stores the image, and deletes Li Ming's tracker. If Li Ming's tracker is recorded in the second array, the system decrements the mall's customer count by 1, stores the image, and deletes Li Ming's tracker. In this way, the number of customers in the mall will be updated in a timely manner to reflect the real-time entry and exit of customers.

[0121] The system accurately determines whether customers enter or leave pre-defined areas within the mall and updates the number of customers based on the coordinates of the target image's center point and the overlap of the area's borders. In practice, this system allows mall managers to precisely understand customer flow within the mall and make timely decisions about security and resource allocation.

[0122] In one embodiment, in response to overlap of objects to be detected in the image to be detected, the method includes:

[0123] Use the deep learning model to segment the target in the image to be detected and obtain the predicted position of the target;

[0124] The predicted position of the target to be detected is calculated through the Kalman filter algorithm and the information of the history tracker to obtain the position of the target to be detected.

[0125] Among them, the deep learning model is a type of machine learning model that uses deep neural networks for data processing and feature extraction. Through a multi-layer network architecture (including convolutional layers, pooling layers, fully connected layers, etc.), it can automatically learn features from large amounts of data and use them for tasks such as classification, regression, and generation. It can be YOLO (You Only Look Once), Faster R-CNN, or RetinaNet. The Kalman filter is a recursive filtering algorithm based on state estimation, which is used to estimate the system state (such as position, velocity, acceleration, etc.) in a dynamic system. It is based on two steps: prediction and update. The state at the previous moment is used to predict the state at the current moment, and then the prediction result is corrected based on the actual observation data.

[0126] Specifically, by combining a deep learning model with a Kalman filter, the system can efficiently segment and recover the positions of overlapping targets, significantly reducing false positives and missed detections. In multi-target tracking, the precise segmentation provided by the deep learning model provides reliable input for the Kalman filter, enabling it to accurately predict the target's position based on historical tracking data and target motion patterns, thereby improving tracking accuracy. Even if multiple targets enter an overlapping area, the system can dynamically adjust the target's position through the Kalman filter to ensure accurate tracking of each target.

[0127] In one embodiment, based on a target detection model, a target to be detected in a target image is extracted to obtain a preset area image of the target to be detected, wherein the target detection model is used to detect, track, or classify the target to be detected, including:

[0128] Based on the target detection model, the bounding box of the target to be detected is annotated to obtain the bounding box coordinates of the target to be detected;

[0129] The target to be detected is extracted according to the bounding box coordinates of the target to be detected, and an image of the target preset area is obtained.

[0130] Among them, the target to be detected is annotated by an image annotation tool. The image annotation tool refers to a tool used to manually or automatically add labels, borders or other annotations to images or video frames, obtain a target data set to be detected, input the target data set to the target detection model, and train the target detection model.

[0131] Specifically, extracting targets from images to be detected based on the target detection model can significantly improve the accuracy and efficiency of target positioning and processing. By annotating the targets with bounding boxes, the model can accurately locate the position of the target to be detected, ensuring that subsequent processing focuses on the target area rather than redundant background information. This precise target positioning allows the target in the image to be quickly and accurately extracted, forming an image of the target preset area. The system can process multi-target scenarios in real time, reducing manual intervention and improving response speed and processing accuracy in applications such as security monitoring and autonomous driving.

[0132] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0133] The embodiment of the present application also provides a passenger flow counting device, such as Figure 2 As shown, it includes: an acquisition module 610, a region division module 620, a detection module 630, a position determination module 640 and a counting module 650.

[0134] An acquisition module 610 is used to acquire surveillance video images and convert the surveillance video images into a picture stream to be detected;

[0135] A region division module 620 is configured to set a first region in any picture to be detected in the picture stream to be detected, where the first region includes at least a second region;

[0136] A detection module 630 is configured to extract at least one target to be detected from any image to be detected based on a target detection model, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is configured to detect, track, or classify the target to be detected;

[0137] A position determination module 640 is configured to determine a tracker corresponding to the target to be detected based on the target preset area image, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image;

[0138] The counting module 650 is used to determine the passenger flow count based on the coordinates of the center point of the target image and the overlap degree of the second area.

[0139] For the description of the features in the embodiment corresponding to the passenger flow counting device, reference can be made to the relevant description of the embodiment corresponding to the passenger flow counting method, which will not be repeated here.

[0140] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned passenger counting method embodiments when running.

[0141] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0142] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above passenger counting method embodiments are implemented.

[0143] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two, such as Figure 4 As shown, in order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] The above is a detailed introduction to a passenger counting method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A passenger flow counting method, characterized in that: include: Acquire surveillance video images and convert the surveillance video images into a picture stream to be detected; Setting a first area in any picture to be detected in the picture stream to be detected, where the first area at least includes a second area; Based on the target detection model, detect targets within the first area in any of the images to be detected, extract at least one target to be detected, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is used to detect, track or classify the target to be detected; Determining a tracker corresponding to the target to be detected based on the target preset area image, wherein the tracker stores at least image information, and the image information includes at least coordinates of a center point of the target image; The passenger flow count is determined according to the overlap between the coordinates of the center point of the target image and the second area.

2. A passenger flow counting method according to claim 1, characterized in that: Acquiring surveillance video images and converting the surveillance video images into a picture stream to be detected includes: In response to any target to be detected appearing in the monitoring picture, starting to extract the monitoring picture as the monitoring video image until the target to be detected disappears from the monitoring picture; Based on a preset frame interval, frame images in the monitoring video image are extracted to obtain a plurality of to-be-detected pictures as the to-be-detected picture stream.

3. A passenger flow counting method according to claim 1, characterized in that: Based on the target preset area image, a tracker corresponding to the target to be detected is determined, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image, including: The tracker further includes: an identifier and a timestamp, and the image information further includes: a target color distribution histogram and a target image width and height; Based on the target preset area image, determining the coordinates of the center point of the target preset area image; Acquire the color of the target preset area image, and convert the color of the target preset area image into a hue, saturation, and brightness space to update the target preset area image; Traversing the pixels of the updated target preset area image, and extracting the saturation value of the updated target preset area image; Dividing the saturation value into a plurality of intervals, determining the number of pixels of the updated target preset area image corresponding to the plurality of intervals, and generating the target color distribution histogram; Arrange the number of pixels included in the target color distribution histogram in descending order, and determine the segments corresponding to the maximum number and the second largest number; The middle values of the segments corresponding to the maximum value and the second largest value are obtained respectively, and the centers of gravity corresponding to the two segments are calculated respectively to obtain the width and height of the target image.

4. A passenger flow counting method according to claim 3, characterized in that: Determining a tracker corresponding to the target to be detected based on the target preset area image further includes: Determine whether the picture to be detected is the first picture in the picture stream to be detected; In response to the picture to be detected being the first picture in the picture stream to be detected, setting a tracker for the preset target area image in the picture to be detected; In response to the picture to be detected not being the first picture in the picture stream to be detected, calculating a difference between image information of the preset area image in the picture to be detected and corresponding tracker history information, obtaining a minimum value among the differences, and determining whether the image information of the preset area image in the picture to be detected matches the corresponding tracker history information based on the minimum value and a preset value; In response to the minimum value being less than a preset value, determining that the image information of the preset target area image in the to-be-detected picture matches the tracker historical information, and updating the tracker historical image information; In response to the minimum value being greater than or equal to a preset value, it is determined that the image information of the preset target area image in the picture to be detected does not match the tracker history information, and the tracker is set for the preset area image in the picture to be detected.

5. A passenger flow counting method according to claim 4, characterized in that: In response to the picture to be detected not being the first picture in the picture stream to be detected, calculating the difference between the image information of the preset area image in the picture to be detected and the corresponding tracker history information, obtaining a minimum value among the differences, and determining whether the image information of the preset area image in the picture to be detected matches the corresponding tracker history information based on the minimum value and a preset value, including: Calculate the difference between the image information of the preset area image in the image to be detected and the historical image information of the tracker, wherein the difference is obtained based on the following formula: Among them, D i Indicates the difference between the target preset area image and the i-th tracker, X represents the horizontal coordinate of the center point of the target preset area image, Y represents the vertical coordinate of the center point of the target preset area image, X i Indicates the horizontal coordinate of the latest point of the i-th tracker, Y i represents the ordinate of the latest point of the i-th tracker, W represents the image width of the target preset area, H represents the image height of the target preset area, W i represents the latest width of the i-th tracker, H i Represents the latest height of the i-th tracker, P1 represents the first color value in the color array, P2 represents the second color value in the color array, P 1i Represents the first color value in the latest color array of the i-th tracker, P 2i Represents the second color value in the latest color array of the i-th tracker, M 1x Indicates the horizontal coordinate value of the first centroid in the color array, M 2x Indicates the horizontal coordinate value of the second centroid in the color array, M 1xi Indicates the horizontal coordinate value of the first centroid in the latest color array of the i-th tracker, M 2xi Indicates the horizontal coordinate value of the second center of gravity in the latest color array of the i-th tracker, M 1y Indicates the vertical coordinate value of the first centroid in the color array, M 2y Indicates the vertical coordinate value of the second centroid in the color array, M 1yi Indicates the ordinate value of the first centroid in the latest color array of the i-th tracker, M 2yi It represents the ordinate value of the second centroid in the latest color array of the i-th tracker. A1 is the center point coordinate weight, A2 is the target preset area image size weight, and A3 is the target preset area image color centroid weight. The sum of A1, A2, and A3 is 1.

6. A passenger flow counting method according to claim 4, characterized in that: Determining a passenger flow count based on the coordinates of the center point of the target image and the overlap between the second area includes: Initialize a first array and a second array, wherein the first array is used to store image information of a preset target area to be confirmed, and the second array is used to store image information of a preset target area to be confirmed; Sorting the target image center point coordinates according to the acquisition time sequence to obtain a target image center point coordinate array; Obtain the first and last coordinate points in the target image center coordinate array; In response to the fact that, in the target image center point coordinate array, the first coordinate point does not overlap with the border of the second area and the last coordinate point overlaps with the border of the second area, determining whether there is a record of the tracker in the second array; In response to there being a record of the tracker in the second array, deleting the record of the tracker before the last record in the second array; In response to the absence of a record of the tracker in the second array, storing the tracker and the preset area image of the preset area image into the first array; In response to the first coordinate point in the target center point coordinate array overlapping with the border of the second area and the last coordinate point not overlapping with the border of the second area, determining whether there is a record of the tracker in the first array; In response to there being a record of the tracker in the first array, deleting the record of the tracker before the last record in the first array; In response to the absence of the record of the tracker in the first array, the tracker of the preset area and the preset area image are stored in the second array.

7. A passenger flow counting method according to claim 6, characterized in that: Determining the passenger flow count based on the coordinates of the center point of the target image and the overlap between the second area further includes: Determining whether the update interval of the tracker exceeds a preset value; In response to the update interval of the tracker exceeding a preset value, determining whether an identifier of the tracker exists in the first array; In response to the presence of the tracker identifier in the first array, the passenger flow number count is incremented by one, the image is stored, and the tracker is deleted; In response to the tracker identifier not existing in the first array, determining whether the tracker identifier exists in the second array; In response to the absence of the identifier of the tracker in the second array, reacquiring the picture to be detected; In response to the existence of the identifier of the tracker in the second array, the count of the number of passengers leaving the passenger flow is increased by one, and the tracker is deleted.

8. A passenger flow counting device, characterized in that: include: An acquisition module is used to acquire surveillance video images and convert the surveillance video images into a picture stream to be detected; A region division module is configured to set a first region in any picture to be detected in the picture stream to be detected, wherein the first region at least includes a second region; A detection module, configured to detect targets within a first area in any image to be detected based on a target detection model, extract at least one target to be detected, and obtain a target preset area image corresponding to the target to be detected, wherein the target detection model is configured to detect, track, or classify the target to be detected; A position determination module is used to determine a tracker corresponding to the target to be detected based on the image of the preset target area, wherein the tracker stores at least image information, and the image information includes at least the coordinates of the center point of the target image; a counting module is used to determine the passenger flow count based on the overlap between the coordinates of the center point of the target image and the second area.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images