Pedestrian flow detection method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to a people flow detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: making a mobile robot move according to a patrol path, and enabling the patrol path to comprise at least one monitoring point; acquiring relative position information of the monitoring point and the mobile robot, and judging whether the mobile robot is within the detection range of the monitoring point according to the relative position information; if the mobile robot is within the detection range, acquiring an image shot by the mobile robot, and determining the current pedestrian volume according to the image; and comparing the current human traffic with a preset human traffic threshold, and if the current human traffic is greater than or equal to the preset human traffic threshold, sending a human traffic congestion early warning to the target device. According to the invention, the method achieves the high-efficiency detection of the pedestrian flow.

Description

Technical Field

[0001] This application relates to the field of image recognition, and in particular to a method, apparatus, electronic device, and computer-readable storage medium for detecting pedestrian flow. Background Technology

[0002] Shopping malls, parks, commercial streets, stadiums, and other venues all have a limited capacity for people. However, due to holidays or events, the number of people inside these venues can easily exceed their capacity, leading to various safety issues and increasing the risk of injuries.

[0003] Currently, pedestrian traffic is monitored by deploying cameras at key locations and using surveillance algorithms. However, because the cameras are only installed in key locations, the monitoring range is limited and blind spots exist. To solve the problem of limited field of view, a large number of cameras would need to be deployed to cover a large area, but this would result in a waste of both the cameras and the recognition algorithms. Therefore, how to efficiently detect pedestrian traffic has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and computer-readable storage medium for detecting pedestrian flow, in order to at least solve the problem of how to efficiently detect pedestrian flow in the related art.

[0005] In a first aspect, embodiments of this application provide a method for detecting pedestrian traffic, including: The mobile robot is controlled to move along an inspection path, which includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information, which is associated and matched with the floor map of the inspection site. Obtain the relative position information of the monitoring point and the mobile robot, and determine whether the mobile robot is within the detection range of the monitoring point based on the relative position information; If the mobile robot is within the detection range, the adaptive image enhancement module is activated to adjust the camera device parameters and acquire continuous frame images captured by the mobile robot. The continuous frame images include visible light images and infrared images. The visible light images and infrared images are fused using a multimodal fusion algorithm. The current pedestrian flow is compared with a preset pedestrian flow threshold, which is dynamically generated based on the relative position information of the monitoring point, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is less than the preset threshold, the inspection path priority is dynamically adjusted based on the real-time pedestrian flow distribution, prioritizing the inspection of monitoring points with higher pedestrian density. If the current pedestrian flow is greater than or equal to the preset threshold, a pedestrian congestion warning is sent to the target device.

[0006] In one embodiment, acquiring the relative position information of the monitoring point and the mobile robot, and determining whether the mobile robot is within the detection range of the monitoring point based on the relative position information, includes: Obtain the three-dimensional coordinates (x2, y2, z2) of the monitoring point and the real-time three-dimensional coordinates (x1, y1, z1) of the mobile robot, where x is latitude, y is longitude, and z is altitude; The detection range of the monitoring point is defined as a spherical region with a preset radius centered on the three-dimensional coordinates; Based on the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, the relative position information of the monitoring point and the mobile robot is obtained, and the mobile robot is determined to be within the detection range based on the relative position information.

[0007] In one embodiment, acquiring consecutive frame images captured by the mobile robot and determining the current pedestrian flow based on the consecutive frame images includes: Acquire multiple consecutive frames of visible light images and synchronously acquired multiple consecutive frames of infrared images; The visible light image is processed by an adaptive image enhancement module, the processing including: dynamically adjusting the exposure according to the ambient light intensity, correcting the white balance according to the color temperature deviation, and adjusting the sensitivity according to the image noise level; Human contour recognition is performed on the processed visible light image, and human thermal radiation contour is extracted from the infrared image. The two contour features are matched and fused through a multimodal fusion algorithm to eliminate non-human interference targets and missed identification of occluded people, and to determine the number of people in each frame of the image. Based on the pedestrian flow in each frame, the average pedestrian flow in the consecutive frames is obtained, and the average value is used as the current pedestrian flow.

[0008] In one embodiment, the relative position information of the monitoring point and the mobile robot, obtained by using the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, satisfies the following:

[0009] Where x1 is the latitudinal coordinate of the mobile robot. x2 is the latitude coordinate of the monitoring point. y1 is the longitude coordinate of the mobile robot. y2 is the longitude coordinate of the monitoring point. z1 represents the altitude coordinates of the mobile robot. z2 is the elevation coordinate of the monitoring point. n is the distance between the mobile robot and the monitoring point.

[0010] In one embodiment, the current pedestrian flow is compared with the preset pedestrian flow threshold. If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to continue moving along the inspection path.

[0011] In one embodiment, the method for determining the preset pedestrian flow threshold includes: Based on the relative location information of the monitoring points, historical pedestrian flow data, and real-time environmental parameters, the preset pedestrian flow threshold for the corresponding monitoring points is determined. If the monitoring point does not have sufficient historical pedestrian flow data, an initial preset pedestrian flow threshold is generated based on the spatial characteristics of the monitoring point and combined with pedestrian flow data from similar monitoring points in the same type of venue. This threshold is then dynamically iterated and optimized as actual detection data accumulates.

[0012] In one embodiment, congestion interception emergency steps are also included: If the mobile robot fails to enter the detection range of the monitoring point within a preset time period after reaching the preset path segment corresponding to the monitoring point, it is determined that there is a risk of congestion and interception. Control the mobile robot to stop moving along the inspection path and adjust the shooting parameters of the camera equipment to cover the monitoring point area; Acquire continuous frame images of the monitoring point area captured by the camera equipment, and determine the current pedestrian flow through a multimodal fusion algorithm; If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning and the current three-dimensional coordinate information of the mobile robot will be sent to the target device. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to adjust its movement path and try to avoid congested areas to enter the detection range of the monitoring point. If multiple attempts fail, it continues to move along the inspection path.

[0013] Secondly, embodiments of this application provide a pedestrian flow detection device, comprising: The control module is used to control the mobile robot to move according to the inspection path, which includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information, which is associated and matched with the floor map of the inspection site. It is also used to set a preset time, determine whether the mobile robot enters the detection range of the monitoring point within the preset time, trigger an emergency detection mechanism, and adjust the movement parameters and shooting parameters. The judgment module is used to obtain the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, calculate the relative position information between the two, and determine whether the mobile robot is within the detection range of the monitoring point based on the relative position information; it is also used to determine whether there is a risk of congestion and interception of the mobile robot. The determination module is used to, if the mobile robot is within the detection range, activate the adaptive image enhancement module to adjust the camera device parameters, acquire continuous frame images captured by the mobile robot, the continuous frame images including visible light images and infrared images, perform feature fusion on the visible light images and infrared images using a multimodal fusion algorithm, determine the current pedestrian flow based on the fused image; it is also used to determine the current pedestrian flow based on continuous frame images captured from a distance under an emergency detection mechanism; The sending module is used to compare the current pedestrian flow with a preset pedestrian flow threshold, which is dynamically generated based on the relative location information of the monitoring points, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is less than the preset threshold, the priority of the inspection path is dynamically adjusted according to the real-time pedestrian flow distribution, prioritizing the inspection of monitoring points with higher pedestrian density. If the current pedestrian flow is greater than or equal to the preset threshold, a pedestrian congestion warning is sent to the target device. If the current pedestrian flow is less than the preset threshold, the control module is triggered to drive the robot to continue inspection. The preset threshold is dynamically determined based on the relative location information of each monitoring point. The module is also used to send a congestion warning containing the current location information of the mobile robot under an emergency detection mechanism.

[0014] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pedestrian flow detection method as described in the first aspect above.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pedestrian flow detection method as described in the first aspect above.

[0016] The pedestrian flow detection method, device, electronic device, and computer-readable storage medium provided in this application have at least the following technical effects.

[0017] By controlling a mobile robot to move along an inspection path, which includes at least one monitoring point, the relative position information of the monitoring point and the mobile robot is obtained. It is determined whether the mobile robot is within the detection range of the monitoring point. If it is, images captured by the mobile robot are acquired, and the current pedestrian flow is determined based on the images. If the current pedestrian flow is greater than or equal to a preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device to inform staff that the current pedestrian flow has reached a congestion level. By using a mobile robot to monitor pedestrian flow along the inspection path, the monitoring coverage is increased, and the mobile robot can promptly detect pedestrian congestion during the inspection, thus enabling efficient pedestrian flow monitoring.

[0018] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating a method for detecting pedestrian flow according to an exemplary embodiment; Figure 2 This is a flowchart illustrating the acquisition of relative position information between a monitoring point and a mobile robot, according to an exemplary embodiment. Figure 3 This is a flowchart illustrating the acquisition of continuous frame pedestrian flow according to an exemplary embodiment; Figure 4 This is a block diagram of a people flow detection device according to an exemplary embodiment; Figure 5 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0021] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0022] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0023] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0024] Firstly, embodiments of this application provide a method for detecting pedestrian traffic. Figure 1 This is a flowchart illustrating a method for detecting pedestrian flow according to an exemplary embodiment, such as... Figure 1 As shown, the methods for detecting pedestrian flow include: Step S101: Control the mobile robot to move along the inspection path. The inspection path includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information (x2, y2, z2), where x is latitude, y is longitude, and z is altitude. The three-dimensional coordinate information is associated and matched with the floor map of the inspection site.

[0025] The inspection path is set according to the layout of the specific location, and the entire inspection path is a closed loop, so that the mobile robot can continuously monitor the flow of people according to the inspection path.

[0026] Furthermore, the inspection path includes at least one monitoring point, each dividing the path into different segments, and each monitoring point is responsible for monitoring its corresponding segment. Since the size of each segment varies, the pedestrian flow capacity of each segment also differs. Therefore, a different preset pedestrian flow threshold is set for each monitoring point. Thus, based on the location information of the monitoring point, the segment it is responsible for is determined, thereby establishing the corresponding preset pedestrian flow threshold for that monitoring point. This allows the mobile robot to dynamically adapt to congestion conditions in different segments during subsequent inspections.

[0027] The mobile robot moves continuously along the inspection path. When it approaches a monitoring point on the inspection path, it triggers the corresponding pedestrian flow monitoring process based on the comparison between its real-time three-dimensional coordinates and the pre-stored three-dimensional coordinates of the monitoring point.

[0028] Step S102: Obtain the relative position information of the monitoring point and the mobile robot, and determine whether the mobile robot is within the detection range of the monitoring point based on the relative position information.

[0029] Figure 2 This is a flowchart illustrating the acquisition of relative position information between a monitoring point and a mobile robot, according to an exemplary embodiment, such as... Figure 2 As shown, in one embodiment, obtaining the relative position information of the monitoring point and the mobile robot includes: Step S1021: Obtain the three-dimensional coordinates (x2, y2, z2) of the monitoring point and the real-time three-dimensional coordinates (x1, y1, z1) of the mobile robot, where x is latitude, y is longitude and z is altitude.

[0030] By using the pre-stored three-dimensional coordinate data of the monitoring points and the positioning chip on the mobile robot, such as a combination of GPS and barometric altimeter, the robot can obtain its own real-time latitude, longitude, and altitude coordinates, thereby accurately determining the spatial position of the monitoring points and the mobile robot, and providing a data basis for distance calculation.

[0031] Step S1022: Based on the three-dimensional coordinates of the monitoring point, determine the detection range of the monitoring point as a spherical region with a preset radius centered on the three-dimensional coordinates.

[0032] The preset radius can be set according to the spatial characteristics of the road segment corresponding to the monitoring point. For example, the preset radius is larger in the main passage area and smaller in the narrow corridor area, so as to ensure that the detection range matches the actual monitoring needs and avoid false triggering across floors or irrelevant areas.

[0033] Step S1023: Based on the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, obtain the relative position information of the monitoring point and the mobile robot, and determine whether the mobile robot is within the detection range based on the relative position information.

[0034] Based on the three-dimensional coordinates of the monitoring point and the mobile robot, the relative position information of the monitoring point and the mobile robot is obtained while satisfying the following conditions:

[0035] Where x1 is the latitudinal coordinate of the mobile robot. x2 represents the latitude coordinates of the monitoring point. y1 is the longitude coordinate of the mobile robot. y2 is the longitude coordinate of the monitoring point. z1 represents the elevation coordinates of the mobile robot. Z2 represents the elevation coordinates of the monitoring point. n is the distance between the mobile robot and the monitoring point.

[0036] When n is less than or equal to the preset radius, the mobile robot is within the detection range of the monitoring point; when n is greater than the preset radius, the mobile robot is not within the detection range of the monitoring point.

[0037] Step S103: If the mobile robot is within the detection range, the adaptive image enhancement module is activated to adjust the camera device parameters and acquire continuous frame images captured by the mobile robot. The continuous frame images include synchronously acquired visible light images and infrared images. The visible light images and infrared images are fused using a multimodal fusion algorithm, and the current pedestrian flow is determined based on the fused image.

[0038] The mobile robot is equipped with a visible light camera and an infrared camera. When the mobile robot enters the detection range, the two cameras start simultaneously to acquire images. At the same time, the adaptive image enhancement module dynamically adjusts parameters according to the environmental conditions: it dynamically adjusts the exposure according to the ambient light intensity, extending the exposure time in low light and shortening the exposure time in strong light; it corrects the white balance according to the color temperature deviation, increasing the blue channel gain in warm light and increasing the red channel gain in cool light; and it adjusts the sensitivity according to the image noise level, reducing the sensitivity when the noise is low and appropriately increasing the sensitivity when the noise is high.

[0039] Since people are constantly moving around in a given location, and there may be obstructions and complex lighting conditions, a multimodal fusion algorithm is used to improve detection accuracy. The specific steps are as follows: Step S1031: Acquire multiple consecutive frames of visible light images and multiple consecutive frames of infrared images acquired synchronously, and acquire multiple consecutive frames of images based on the read synchronous video footage from the dual cameras, ensuring that the timestamps of the two types of images are consistent, thus providing a synchronous data foundation for feature fusion.

[0040] Step S1032: After processing the visible light image through the adaptive image enhancement module, perform personnel contour recognition; extract human thermal radiation contours from the infrared image, and use a multimodal fusion algorithm, such as the SIFT feature point matching algorithm, to match and fuse the two contour features, eliminate non-person interference targets and missed identification of occluded personnel, and determine the number of people in each frame image.

[0041] Step S1033: Based on the pedestrian flow in each frame, obtain the average pedestrian flow in the consecutive frames, and use the average pedestrian flow as the current pedestrian flow.

[0042] Based on the number of people in each frame of the image, the average number of people in multiple consecutive frames is calculated to effectively filter out errors caused by instantaneous fluctuations in the number of people and ensure the accuracy of the current number of people. For example, after acquiring images in multiple consecutive frames and obtaining the number of people in each frame according to the multimodal fusion algorithm, the current number of people is obtained by averaging.

[0043] When the mobile robot is within the detection range, it collects multimodal images through dual cameras and determines the current pedestrian flow by combining adaptive enhancement and fusion algorithms, thereby achieving accurate detection in complex environments. Furthermore, by using the mobile robot for mobile detection, the amount of detection equipment required is reduced, and the efficiency of pedestrian flow detection is improved.

[0044] The aforementioned exemplary embodiment also includes congestion interception emergency steps: If the mobile robot fails to enter the detection range of the monitoring point within a preset time period after reaching the preset path segment corresponding to the monitoring point, it is determined that there is a risk of congestion and interception. Control the mobile robot to stop moving along the inspection path, and adjust the shooting parameters of the camera equipment to cover the core area of ​​the spherical detection range of the monitoring point; Acquire continuous frame images of the monitoring point area captured by the camera equipment, and determine the current pedestrian flow through a multimodal fusion algorithm; If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning and the current three-dimensional coordinate information of the mobile robot will be sent to the target device. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to adjust its movement path and try to avoid congested areas to enter the detection range of the monitoring point. If multiple attempts fail, it continues to move along the inspection path.

[0045] Specifically, following step S1023, the following steps are also included: Step S1024: When the mobile robot moves to the preset path segment corresponding to the monitoring point according to the inspection path, the control module starts timing. If the timing reaches the preset time and the mobile robot still has not entered the detection range of the monitoring point, it is determined that there may be serious congestion in the monitoring point area, which prevents the robot from getting close and triggers the emergency detection mechanism. At this time, the control module first instructs the mobile robot to stop moving along the inspection path to avoid equipment failure or increased congestion due to forced movement; at the same time, it controls the camera to adjust the shooting parameters, zooms in on the monitoring point area through the electric zoom function, adjusts the shooting angle to cover the core area of ​​the monitoring point, and optimizes the exposure parameters to ensure image clarity. Subsequently, following the logic of step S103, continuous visible light and infrared images captured after parameter adjustment are obtained. The number of people in each frame is identified through a multimodal fusion algorithm, and the average number of people in the continuous frames is calculated as the current number of people. If the average number is greater than or equal to the preset number of people threshold for the monitoring point, a congestion warning is sent to the target device. The warning information also includes the current three-dimensional coordinates of the mobile robot and the associated floor map link, so that staff can quickly locate the congested area. If the average number is less than the preset number of people threshold, it is determined to be a non-congestion factor. The control module instructs the mobile robot to adjust its movement path and attempt to enter the detection range of the monitoring point. If it fails three times in a row, it abandons the current round of detection at the monitoring point and continues to move along the inspection path to avoid affecting the overall inspection efficiency.

[0046] The preset path segment is a preset buffer zone around the monitoring point, which is pre-stored in the control module and associated with the floor map; the preset duration can be dynamically adjusted according to the location characteristics such as the density of people at the monitoring point and the width of the path, combined with historical congestion data; the detection range is such that the distance n is always greater than the preset radius.

[0047] Step S104: Compare the current pedestrian flow with a preset pedestrian flow threshold, which is dynamically generated based on the relative location information of the monitoring points, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device. If it is less than the threshold, the inspection path priority is dynamically adjusted based on the real-time pedestrian flow distribution, prioritizing the inspection of monitoring points with higher pedestrian density. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to continue moving along the inspection path. Specifically, the dynamic generation method for the preset pedestrian flow threshold may include: For monitoring points with sufficient historical data: retrieve the historical pedestrian flow data of the monitoring point for the past 3 months through the backend server, combine it with real-time environmental parameters, such as whether it is a holiday, whether there are promotional activities, weather conditions, etc., and generate the preset pedestrian flow threshold for the current time period through a preset algorithm weighted calculation. For monitoring points without sufficient historical data, such as newly added monitoring points: based on the spatial characteristics of the monitoring point, such as area, passage width, evacuation capacity, etc., the flow data of similar monitoring points in the same type of place are matched to generate an initial preset flow threshold, and dynamically iterate and optimize as actual detection data accumulates.

[0048] The current pedestrian flow is compared with a preset pedestrian flow threshold. If the current pedestrian flow is greater than or equal to the preset threshold, a pedestrian congestion warning is sent to the target device so that staff can promptly disperse the crowd based on the warning. If the current pedestrian flow is less than the preset threshold, two scenarios are handled: if there are other monitoring points with higher pedestrian density, the inspection path priority is dynamically adjusted to prioritize those monitoring points; if not, the mobile robot continues to move along the original inspection path until it reaches the detection range of the next monitoring point to monitor pedestrian flow.

[0049] To more clearly illustrate the implementation effect of this application, a specific application scenario example is provided: In a large shopping mall, three monitoring points are set up along the inspection path. Monitoring point A corresponds to the spacious main passage on the 3rd floor, monitoring point B corresponds to the medium-sized elevator entrance on the 3rd floor, and monitoring point C corresponds to the narrow corridor on the 3rd floor. The preset pedestrian flow threshold for each monitoring point is dynamically generated based on its relative position information, historical pedestrian flow data, and real-time environmental parameters. When the mobile robot moves along the closed-loop path to the vicinity of monitoring point B, it obtains its real-time three-dimensional coordinates through the positioning chip and calculates the relative distance using the three-dimensional distance formula. If the preset radius of monitoring point B is 5m, then n>5m. The robot continues to move; when it reaches the coordinates n=0≤5m, it is determined that it has entered the detection range; then the robot's dual cameras are activated, simultaneously acquiring visible light and infrared images. After adaptive enhancement processing, the number of people in each frame is identified and the average value is calculated through a multimodal fusion algorithm to obtain the current pedestrian flow. After comparison, the average value exceeds the dynamic preset pedestrian flow threshold corresponding to monitoring point B. The robot then sends a congestion warning to the mall monitoring center through the communication module. After receiving the warning, the staff, based on the three-dimensional coordinates and floor map in the warning, promptly go to the elevator entrance on the 3rd floor to guide the flow of people and avoid congestion.

[0050] By dynamically setting preset pedestrian flow thresholds based on the 3D coordinates of monitoring points, historical data, and real-time environment, the system adapts to congestion situations in different scenarios. Simultaneously, it dynamically adjusts inspection priorities based on pedestrian flow distribution, making pedestrian flow detection more flexible and timely. Furthermore, mobile detection using robots avoids blind spots and improves the efficiency of pedestrian flow detection.

[0051] In summary, the pedestrian flow detection method provided in this application involves a mobile robot moving along an inspection path. When the mobile robot enters the spherical detection range of a monitoring point, it acquires multimodal images through dual cameras. The average pedestrian flow of consecutive frames is determined by an adaptive enhancement and fusion algorithm, and this average value is used as the current pedestrian flow. If the current pedestrian flow is greater than or equal to a dynamically generated preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device. If it is less than the threshold, the inspection priority is adjusted according to the real-time pedestrian flow distribution, or the robot continues to move along the original path. This method achieves accurate and efficient pedestrian flow detection in complex environments and avoids excessive use of equipment and blind spots in pedestrian flow detection.

[0052] Secondly, embodiments of this application provide a pedestrian flow detection device. Figure 4 This is a block diagram illustrating a people flow detection device according to an exemplary embodiment. Figure 4 As shown, the pedestrian flow detection device includes: The control module is used to control the mobile robot to move according to the inspection path, which includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information, which is associated and matched with the floor map of the inspection site. It is also used to set a preset time period, determine whether the mobile robot enters the spherical detection range of the monitoring point within the preset time period, trigger an emergency detection mechanism, and adjust the movement parameters and shooting parameters. It is also used to dynamically adjust the priority of the inspection path according to the real-time pedestrian flow distribution, driving the robot to prioritize the inspection of monitoring points with higher pedestrian density. The judgment module is used to obtain the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, based on the formula. The relative position information of the two is calculated, and the mobile robot is determined based on the relative position information to determine whether the mobile robot is within the spherical detection range of the monitoring point; it is also used to determine whether there is a risk of congestion and interception of the mobile robot; The determination module is used to, if the mobile robot is within the detection range, activate the adaptive image enhancement module to adjust the camera device parameters, acquire continuous frame images captured by the mobile robot, the continuous frame images including visible light images and infrared images, perform feature fusion on the visible light images and infrared images using a multimodal fusion algorithm, and determine the current pedestrian flow based on the fused image; it is also used to, under an emergency detection mechanism, determine the current pedestrian flow based on continuous frame images captured from a distance using a multimodal fusion algorithm; The sending module is used to compare the current pedestrian flow with a preset pedestrian flow threshold, which is dynamically generated based on the relative position information of the monitoring point, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device. If it is less than the threshold, the control module is triggered to adjust the priority of the inspection path or drive the robot to continue inspection. It is also used to send a congestion warning containing the current three-dimensional coordinate information of the mobile robot under the emergency detection mechanism.

[0053] The aforementioned modules do not operate independently but achieve collaborative control through data interaction: When the control module drives the robot to move, it sends the robot's current 3D coordinates to the judgment module in real time; the judgment module calculates the relative position based on the pre-stored 3D coordinates of the monitoring points and provides feedback on whether it has entered the spherical detection range; if it has entered the detection range, the control module triggers the determination module to activate the dual cameras and adaptive image enhancement module to collect and process multimodal images to determine the pedestrian flow; the determination module transmits the pedestrian flow data to the sending module, which compares it with a dynamically generated preset threshold. If the threshold is not reached, the control module is triggered to adjust the inspection priority or continue inspection; if the threshold is reached, an early warning is sent; in emergency scenarios, all modules collaborate to complete long-distance multimodal detection and early warning transmission, ensuring the integrity of the technical solution.

[0054] In summary, the pedestrian flow detection device provided in this application controls a mobile robot to move along an inspection path based on a floor map via a control module. This inspection path includes monitoring points with pre-stored three-dimensional coordinates. A judgment module calculates the relative position based on the three-dimensional coordinates to determine if the mobile robot is within a spherical detection range. If within the detection range, the determination module acquires multimodal images using dual cameras, and determines the current pedestrian flow through adaptive enhancement and fusion algorithms. The sending module combines dynamic thresholds to perform comparisons and trigger corresponding operations. By using a mobile robot to monitor pedestrian flow along the inspection path, the monitoring coverage is increased. Furthermore, the mobile robot can dynamically adapt to different scenarios and promptly detect pedestrian congestion, thus enabling efficient pedestrian flow monitoring.

[0055] It should be noted that the people flow detection device provided in this embodiment is used to implement the above-described embodiments, and details already described will not be repeated. As used above, terms such as "module," "unit," and "subunit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the above embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0056] Thirdly, embodiments of this application provide an electronic device, Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. (e.g.) Figure 5 As shown, the electronic device may include a processor 81 and a memory 82 storing computer program instructions.

[0057] Specifically, the processor 81 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0058] The memory 82 may include a mass storage device for data or instructions. For example, and not limitingly, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include removable or non-removable (or fixed) media. Where appropriate, the memory 82 may be internal or external to a data processing device. In a particular embodiment, the memory 82 is non-volatile memory. In a particular embodiment, the memory 82 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0059] The memory 82 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 81, including program instructions related to the three-dimensional coordinates of the monitoring point, floor map association data, historical pedestrian flow data, real-time environmental parameters, and multimodal fusion algorithms.

[0060] The processor 81 reads and executes computer program instructions stored in the memory 82 to implement any of the pedestrian flow detection methods in the above embodiments.

[0061] In one embodiment, the people flow detection device may further include a communication interface 83 and a bus 80. Wherein, as... Figure 5 As shown, the processor 81, memory 82, and communication interface 83 are connected through bus 80 and complete communication with each other.

[0062] The communication interface 83 is used to enable communication between the various modules, devices, units, and / or equipment in the embodiments of this application. The communication interface 83 can also enable data communication with other components such as external devices, image / data acquisition devices, databases, external storage, and image / data processing workstations, including the transmission of three-dimensional coordinate data, floor map data, pedestrian flow data, early warning information, etc.

[0063] Bus 80 includes hardware, software, or both, that couples the components of the people flow detection device together. Bus 80 includes, but is not limited to, at least one of the following: data bus, address bus, control bus, expansion bus, and local bus. For example, and not as a limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0064] Fourthly, embodiments of this application provide a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the pedestrian flow detection method provided in the first aspect.

[0065] The readable storage medium may be more specifically adopted, including but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0066] In a possible implementation, the present invention can also be implemented as a program product, which includes program code. When the program product is run on a terminal device, the program code is used to cause the terminal device to perform steps of implementing the pedestrian flow detection method provided in the first aspect, including steps such as three-dimensional coordinate acquisition and distance calculation, multimodal image acquisition and fusion, dynamic threshold generation, and inspection path priority adjustment.

[0067] The program code for executing the present invention can be written in any combination of one or more programming languages. The program code can be executed entirely on the user device, partially on the user device, as a standalone software package, partially on the user device and partially on a remote device, or entirely on a remote device.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method of detecting the number of people, characterized by, include: The mobile robot is controlled to move along an inspection path, which includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information, which is associated and matched with the floor map of the inspection site. Obtain the relative position information of the monitoring point and the mobile robot, and determine whether the mobile robot is within the detection range of the monitoring point based on the relative position information; If the mobile robot is within the detection range, the adaptive image enhancement module is activated to adjust the camera equipment parameters, acquire continuous frame images captured by the mobile robot, and determine the current pedestrian flow based on the continuous frame images; The current pedestrian flow is compared with a preset pedestrian flow threshold, which is dynamically generated based on the relative location information of the monitoring point, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is less than the preset threshold, the inspection path priority is dynamically adjusted based on the real-time pedestrian flow distribution, prioritizing the inspection of monitoring points with higher pedestrian density. If the current pedestrian flow is greater than or equal to the preset threshold, a pedestrian congestion warning is sent to the target device.

2. The people flow detection method according to claim 1, characterized in that, Acquiring the relative position information of the monitoring point and the mobile robot, and determining whether the mobile robot is within the detection range of the monitoring point based on the relative position information, includes: Obtain the three-dimensional coordinates (x2, y2, z2) of the monitoring point and the real-time three-dimensional coordinates (x1, y1, z1) of the mobile robot, where x is latitude, y is longitude, and z is altitude; The detection range of the monitoring point is defined as a spherical region with a preset radius centered on the three-dimensional coordinates; Based on the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, the relative position information of the monitoring point and the mobile robot is obtained, and the mobile robot is determined to be within the detection range based on the relative position information.

3. The people flow detection method according to claim 1, characterized in that, Acquiring continuous frame images captured by the mobile robot, and determining the current pedestrian flow based on the continuous frame images, includes: Acquire multiple consecutive frames of visible light images and synchronously acquired multiple consecutive frames of infrared images; The visible light image is processed by an adaptive image enhancement module, the processing including: dynamically adjusting the exposure according to the ambient light intensity, correcting the white balance according to the color temperature deviation, and adjusting the sensitivity according to the image noise level; Human contour recognition is performed on the processed visible light image, and human thermal radiation contour is extracted from the infrared image. The two contour features are matched and fused through a multimodal fusion algorithm to eliminate non-human interference targets and missed identification of occluded people, and to determine the number of people in each frame of the image. Based on the pedestrian flow in each frame, the average pedestrian flow in the consecutive frames is obtained, and the average value is used as the current pedestrian flow.

4. The method for detecting pedestrian flow according to claim 1, characterized in that, The relative position information of the monitoring point and the mobile robot, obtained based on the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, satisfies the following: Where x1 is the latitudinal coordinate of the mobile robot. x2 is the latitude coordinate of the monitoring point. y1 is the longitude coordinate of the mobile robot. y2 is the longitude coordinate of the monitoring point. z1 represents the altitude coordinates of the mobile robot. z2 is the elevation coordinate of the monitoring point. n is the distance between the mobile robot and the monitoring point.

5. The method for detecting pedestrian flow according to claim 1, characterized in that, The current pedestrian flow is compared with the preset pedestrian flow threshold. If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning is sent to the target device. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to continue moving along the inspection path.

6. The method for detecting pedestrian flow according to claim 1, characterized in that, Based on the relative location information of the monitoring points, historical pedestrian flow data, and real-time environmental parameters, the preset pedestrian flow threshold for the corresponding monitoring points is determined. If the monitoring point does not have sufficient historical pedestrian flow data, an initial preset pedestrian flow threshold is generated based on the spatial characteristics of the monitoring point and combined with pedestrian flow data from similar monitoring points in the same type of venue. This threshold is then dynamically iterated and optimized as actual detection data accumulates.

7. The pedestrian flow detection method according to claim 1, characterized in that, It also includes congestion interception emergency procedures: If the mobile robot fails to enter the detection range of the monitoring point within a preset time period after reaching the preset path segment corresponding to the monitoring point, it is determined that there is a risk of congestion and interception. Control the mobile robot to stop moving along the inspection path and adjust the shooting parameters of the camera equipment to cover the monitoring point area; Acquire continuous frame images of the monitoring point area captured by the camera equipment, and determine the current pedestrian flow through a multimodal fusion algorithm; If the current pedestrian flow is greater than or equal to the preset pedestrian flow threshold, a pedestrian congestion warning and the current three-dimensional coordinate information of the mobile robot will be sent to the target device. If the current pedestrian flow is less than the preset pedestrian flow threshold, the mobile robot is controlled to adjust its movement path and try to avoid congested areas to enter the detection range of the monitoring point. If multiple attempts fail, it continues to move along the inspection path.

8. A pedestrian flow detection device, characterized in that, include: The control module is used to control the mobile robot to move according to the inspection path, which includes at least one monitoring point. The monitoring point has pre-stored three-dimensional coordinate information, which is associated and matched with the floor map of the inspection site. It is also used to set a preset time, determine whether the mobile robot enters the detection range of the monitoring point within the preset time, trigger an emergency detection mechanism, and adjust the movement parameters and shooting parameters. The judgment module is used to obtain the three-dimensional coordinates of the monitoring point and the real-time three-dimensional coordinates of the mobile robot, calculate the relative position information between the two, and determine whether the mobile robot is within the detection range of the monitoring point based on the relative position information; it is also used to determine whether there is a risk of congestion and interception of the mobile robot. The determination module is used to activate the adaptive image enhancement module to adjust the camera equipment parameters if the mobile robot is within the detection range, and acquire continuous frame images captured by the mobile robot. The continuous frame images include visible light images and infrared images. The visible light images and infrared images are fused using a multimodal fusion algorithm. The current pedestrian flow is determined based on the fused image. It is also used in emergency detection mechanisms to determine the current pedestrian flow based on continuous frame images taken from a distance; The sending module is used to compare the current pedestrian flow with a preset pedestrian flow threshold, which is dynamically generated based on the relative location information of the monitoring points, historical pedestrian flow data, and real-time environmental parameters. If the current pedestrian flow is less than the preset threshold, the priority of the inspection path is dynamically adjusted according to the real-time pedestrian flow distribution, prioritizing the inspection of monitoring points with higher pedestrian density. If the current pedestrian flow is greater than or equal to the preset threshold, a pedestrian congestion warning is sent to the target device. If the current pedestrian flow is less than the preset threshold, the control module is triggered to drive the robot to continue inspection. The preset threshold is dynamically determined based on the relative location information of each monitoring point. The module is also used to send a congestion warning containing the current location information of the mobile robot under an emergency detection mechanism.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the pedestrian flow detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the pedestrian flow detection method as described in any one of claims 1 to 7.

Images