Personnel access detection method, apparatus, device, and storage medium
By generating foreground images and utilizing pre-selection box detection and historical trajectory matching, the problem of unstable detection by thermopile sensors in complex environments was solved, achieving low-power, high-accuracy personnel entry and exit detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAMEN MILESIGHT IOT CO LTD
- Filing Date
- 2023-11-21
- Publication Date
- 2026-07-03
Smart Images
Figure CN117809332B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of digital signal processing technology, and in particular to a method, apparatus, device, and storage medium for detecting personnel entry and exit. Background Technology
[0002] A thermopile sensor is a non-contact, point-to-point, directional electronic component used in various temperature measurement environments. The thermopile sensor collects human body temperature data in the form of a pixel array, which can be used to determine personnel entry and exit.
[0003] Current technical solutions typically involve preprocessing the collected human body temperature data array before identifying the highest temperature value based on a certain threshold to extract and segment individual targets. This approach lacks generality in environments with large numbers of people and complex entry / exit patterns, and is prone to missed detections. Furthermore, the detection performance is even more unstable in devices with limited processor computing power. Summary of the Invention
[0004] To address the aforementioned issues, this application provides a method, apparatus, device, and storage medium for personnel entry and exit detection. The logic is simple and computationally intensive, achieving low power consumption. The tracking and statistical process employs pre-selected boxes to divide multiple targets, exhibiting strong generalization and reducing the likelihood of missed detections, while maintaining stable detection performance.
[0005] Firstly, this application provides a method for detecting personnel entry and exit, the method comprising:
[0006] S1. For any frame of real-time temperature image collected in the scene, generate a real-time foreground image based on the difference in pixel values between the real-time temperature image and the background image of the scene, and perform noise reduction processing on the real-time foreground image. The background image is obtained by averaging multiple temperature images collected in the scene.
[0007] S2. Traverse the real-time foreground image after noise reduction according to the preset frame size to determine multiple pre-selected frames. For any of the pre-selected frames, take the pixel with the largest pixel value in the pre-selected frame as the new center point and adjust the position of the pre-selected frame.
[0008] S3. Calculate the score of each preselected box based on the pixel value of each pixel within each adjusted preselected box, and filter out multiple target preselected boxes based on the scores; match each target preselected box with the historical trajectory of each personnel target, and update the latest trajectory of each personnel target based on the matching results;
[0009] S4. For any of the personnel targets, calculate the movement direction of the personnel target based on the current position indicated by the latest trajectory of the personnel target and the given displacement step size;
[0010] S5. Based on the movement direction of each person target in the real-time temperature image, the number of people entering and / or exiting the scene is statistically determined.
[0011] In one possible implementation, prior to step S1, the method further includes:
[0012] Based on multiple temperature images captured in the scene, the pixel values of pixels at the same location are added together and the average value is taken to obtain the background image; step S1 includes:
[0013] S11. For any frame of real-time temperature image, generate a real-time foreground image based on the difference between the pixel values of the real-time temperature image and the background image, and set the pixel values of pixels in the real-time foreground image that are less than a given judgment threshold to 0.
[0014] S12. Perform erosion and / or dilation processing on any pixel in the real-time foreground image whose pixel value is not 0; the erosion processing includes setting the pixel value of the pixel to 0 when the pixel values of all adjacent pixels of the pixel are 0; the dilation processing includes setting the pixel value of the pixel to the average value of the pixel values of all adjacent pixels when the pixel values of all adjacent pixels of the pixel are not all 0.
[0015] In one possible implementation, the historical trajectory of any of the personnel targets includes an ID indicating the personnel target, and further includes at least one of the following: the number of times the target was detected during the historical tracking process, the number of times the target was not detected during the historical tracking process, the position coordinates of the latest pre-selected box in the historical trajectory, the activation state of the personnel target, the center point coordinates of the initial pre-selected box in the historical trajectory, and the displacement step size corresponding to the personnel target.
[0016] In one possible implementation, step S3 includes:
[0017] S31. For any adjusted preselected box, sum the pixel values of each pixel in the adjusted preselected box, and normalize them according to the format parameters of the real-time temperature image to obtain the score of the adjusted preselected box.
[0018] S32. The non-maximum suppression algorithm is used to filter the pre-selected boxes whose scores are greater than the confidence threshold, so as to obtain multiple target pre-selected boxes;
[0019] S33. Match the latest pre-selected boxes in the historical trajectory of each personnel target with each target pre-selected box in pairs, and calculate the IOU value of each matching combination;
[0020] S34. If the IOU value of the matching combination is greater than the preset value, the historical trajectory and the target preselection box corresponding to the matching combination are successfully matched. The target preselection box in the matching combination is used as the latest preselection box of the corresponding historical trajectory to obtain the latest trajectory. The remaining target preselection boxes that are not successfully matched are established as the trajectory of the new object to be tracked to obtain the latest trajectory.
[0021] For the remaining historical trajectories that failed to match, update the number of times the target was not detected during the historical tracking process. If the number of times the target was not detected during the historical tracking process exceeds a preset number, delete the remaining historical trajectories.
[0022] In one possible implementation, step S4 includes:
[0023] S41. For any of the personnel targets, calculate the predicted position coordinates of the personnel target based on the center position coordinates of the latest preselected box in the latest trajectory of the personnel target and the displacement step size corresponding to the personnel target.
[0024] S42. When the vertical coordinate indicates the direction of entry and exit, the movement direction of the personnel target is determined by subtracting the vertical coordinate of the center point of the initial pre-selected box in the latest trajectory from the vertical coordinate of the predicted position coordinate. The sign of the difference indicates the movement direction.
[0025] Secondly, a personnel entry and exit detection device is provided, which includes multiple functional modules for performing corresponding steps in the personnel entry and exit detection method provided in the first aspect.
[0026] Thirdly, a computing device is provided, comprising a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the personnel access detection method as provided in the first aspect.
[0027] In one possible implementation, the computing device further includes a battery module for powering the processor, the processor's computing power and / or power consumption metrics meeting preset conditions.
[0028] Fourthly, a computer-readable storage medium is provided, wherein at least one program is stored therein, the at least one program being executed by a processor to implement the personnel access detection method as provided in the first aspect.
[0029] The technical solution provided in this application includes at least the following technical effects:
[0030] A real-time foreground image is generated by subtracting the background image from the real-time temperature image, and then denoising is applied to the foreground image. Pixels are filtered according to a temperature threshold to divide the real-time foreground image into multiple pre-selected boxes. The pixel with the largest pixel value within each box is used as the new center point to adjust the position of the pre-selected box. Multiple target pre-selected boxes are obtained by filtering the pre-selected box scores. The target pre-selected boxes are matched with each historical trajectory to generate the latest trajectory for each person target. The movement direction of the person target is calculated based on the current position and displacement step size in the latest trajectory. The number of people entering and exiting is counted based on the movement direction. The above scheme has a simple image processing flow logic and low computational load, achieving low power consumption requirements. The tracking and statistics process uses pre-selected boxes to divide multiple targets, which has strong generalization and is less prone to missed detections, with stable detection results. Attached Figure Description
[0031] Figure 1 This is a flowchart illustrating a personnel entry and exit detection method provided in an embodiment of this application;
[0032] Figure 2 This is a flowchart illustrating another personnel entry and exit detection method provided in an embodiment of this application;
[0033] Figure 3 This is a schematic diagram of a temperature image provided in an embodiment of this application;
[0034] Figure 4 This is a schematic diagram of a personnel entry and exit detection device provided in an embodiment of this application;
[0035] Figure 5 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application. Detailed Implementation
[0036] To further illustrate the various embodiments, this application provides accompanying drawings. These drawings are part of the disclosure of this application and are mainly used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of this application. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components. In this application, the term "at least one" means one or more, and the term "multiple" means two or more; for example, "multiple persons" refers to two or more persons.
[0037] This application will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0038] Example 1
[0039] This application provides a method for detecting personnel entry and exit. Figure 1This is a flowchart illustrating a personnel entry and exit detection method provided in an embodiment of this application. Figure 2 This is a flowchart illustrating another personnel entry and exit detection method provided in this application embodiment.
[0040] In this embodiment, a thermopile sensor is used to acquire temperature images in real time within a scene. For example, the thermopile sensor is mounted overhead, collecting temperature data downwards within the scene area. When a person passes by, the temperature in the scene changes. The technical solution provided in this embodiment processes the temperature images acquired by the thermopile sensor, determines the temperature image area corresponding to a person through pre-selection box detection, tracks the movement trajectory of each person, and determines the direction of movement based on the trajectory's direction, thereby completing the counting. The following is combined with... Figure 1 and Figure 2 This application describes a personnel entry and exit detection method provided in its embodiments. See also... Figure 1 The method includes the following steps S1 to S5, which can be executed by any computing device that acquires temperature images collected by the thermopile sensor in real time.
[0041] S1. For any frame of real-time temperature image collected in the scene, generate a real-time foreground image based on the difference in pixel values between the real-time temperature image and the background image of the scene, and perform noise reduction processing on the real-time foreground image.
[0042] For example, the format of a real-time temperature image is 16*8 (number of pixels in the image width direction * number of pixels in the image width direction). The pixel value of each pixel in the real-time temperature image represents the temperature value of the corresponding point in the scene. This application provides a schematic diagram of a temperature image, see below. Figure 3 The temperature image has a size of 16*8 pixels, and the pixel value of each pixel represents the temperature.
[0043] The background image is obtained by averaging multiple temperature images captured in the scene. Specifically, based on the multiple temperature images captured in the scene, the pixel values of pixels at the same location are added together and the average value is taken to obtain the background image. For example, there may be 20 images; the specific number can be adjusted according to accuracy requirements, and this application does not limit this.
[0044] In this embodiment, the denoising process includes two parts: filtering and morphological processing. The filtering uses a threshold, and the morphological processing includes erosion and dilation. For details, see [link to relevant documentation]. Figure 2 Step S1 includes:
[0045] S11. For any frame of real-time temperature image, generate a real-time foreground image based on the difference between the pixel values of the real-time temperature image and the background image, and set the pixel values of pixels in the real-time foreground image that are less than a given judgment threshold to 0.
[0046] The judgment threshold is, for example, 30. Pixel values less than 30 are considered part of the background environment rather than the human body.
[0047] S12. Perform erosion and / or dilation processing on any pixel with a non-zero pixel value in the real-time foreground image.
[0048] The erosion process includes setting the pixel value of a pixel to 0 when all adjacent pixels of the pixel have a pixel value of 0.
[0049] The dilation process includes setting the pixel value of a pixel to the average of the pixel values of its neighboring pixels when the pixel values of the neighboring pixels are not all zero.
[0050] Through the above process, the filtering part in the previous image denoising process is weakened, and morphological processing is mainly adopted, which can effectively reduce the amount of computation, meet the low power consumption requirements, and is suitable for computing devices with limited computing power, achieving low power consumption while ensuring processing effect.
[0051] For example, a computing device for executing the technical solutions provided in this application includes a processor and a battery module. The processor can acquire and perform real-time temperature monitoring based on real-time temperature images collected by a thermopile sensor, and the battery module can continuously power the processor.
[0052] In one possible implementation, the processor is, for example, an STM32F4 series MCU. The computing device, powered by a battery module, can be stably deployed in real-world scenarios for personnel access detection. In contrast, high-performance computing devices include servers or personal computers equipped with high-performance chips such as NPGs and GPUs. These related technical solutions rely on high-performance computing devices to achieve accurate personnel access detection, lacking generalization ability on low-performance devices, resulting in high computational load and unstable performance. In comparison, the personnel access detection method provided in this application embodiment is designed for low-power computing devices with limited computing power, offering stable detection results and is particularly suitable for various edge service deployment scenarios.
[0053] Specifically, the computing power of a computing device can be measured using computing power metrics, such as the number of floating-point operations per second (FLOPs) or the number of tera-operations per second (TOPS). A higher metric value indicates higher computing power. Optionally, the computing power metrics of the computing device described in this application embodiment meet preset conditions. These preset conditions indicate a threshold for the computing power metrics of low-performance computing devices. For example, preset conditions may include: the computing power metrics being less than a first threshold.
[0054] Specifically, low-power computing devices also consume less power. Therefore, personnel entry and exit detection algorithms suitable for low-power computing devices need to minimize computational load to meet low power consumption requirements. Based on this, the power consumption index of the computing device described in this application embodiment also meets preset conditions. These preset conditions indicate a power consumption index threshold for the computing device. For example, preset conditions include: the power consumption index is less than a second threshold. Exemplarily, the power consumption index can be quantitatively calculated based on the operating frequency of the computing device's processor.
[0055] Furthermore, the computing device also includes a wireless communication module, which can periodically / in real-time report the processing results of the low-computing-power device to the backend for further data analysis. The wireless communication module may be, for example, LoRa or Bluetooth; this application does not limit its application to this type.
[0056] S2. Traverse the real-time foreground image after denoising according to the preset box size to determine multiple pre-selected boxes. For any pre-selected box, take the pixel with the largest pixel value in the pre-selected box as the new center point and adjust the position of the pre-selected box.
[0057] In this embodiment, the size of the frame is 4*4 (the number of pixels in the image width direction * the number of pixels in the image height direction). By using the point with the highest temperature within the pre-selected frame as the latest center point, the adjusted pre-selected frame can accurately contain the image area corresponding to the human body temperature.
[0058] S3. Calculate the score of each preselected box based on the pixel value of each pixel in each adjusted preselected box, and filter out multiple target preselected boxes based on the score; match each target preselected box with the historical trajectory of each personnel target, and update the latest trajectory of each personnel target based on the matching result.
[0059] In this embodiment of the application, the historical trajectory of any personnel target includes an ID indicating the personnel target. Specifically, the historical trajectory also includes at least one of the following: the number of times the target was detected during historical tracking, the number of times the target was not detected during historical tracking, the position coordinates of the latest pre-selected box in the historical trajectory, the activation state of the personnel target, the center point coordinates of the initial pre-selected box in the historical trajectory, and the displacement step size corresponding to the personnel target.
[0060] For example, the historical track stores the above information using a preset data format. The historical track uses the variable name "tracker", and one specific definition method is as follows:
[0061] tracker = {
[0062] "id":0,#ID
[0063] "acc":0,# The number of times the current ID appears
[0064] "dec":3, # The number of times the current ID has disappeared, defaults to 3, and decreases from 3 for each disappearance.
[0065] "active":0, # Whether the current ID is active
[0066] "box":[0,0,1,1],#The coordinates of the center position of the latest preselected box corresponding to the current ID (x1,y1,x2,y2)
[0067] "start_center":[0,0],# Coordinates of the center point of the current ID.
[0068] "add_point":[0,0]# Displacement step size, used for trajectory prediction
[0069] }
[0070] Of course, other different methods can be used to define the data structure of historical trajectories, and this application does not limit this.
[0071] In this embodiment of the application, based on the above examples and... Figure 2 As shown, step S3 includes:
[0072] S31. For any adjusted preselected box, sum the pixel values of each pixel within the adjusted preselected box, and normalize them according to the format parameters of the real-time temperature image to obtain the score of the adjusted preselected box.
[0073] The format parameters of the temperature image include: the bit depth of the image and the size of the preselection box. For an 8-bit image, the pixel value is 1-255 (after step S1, there are no pixels with a value of 0 within the box). Taking an 8-bit image with a (4*4) size preselection box as an example, the score = the sum of the pixel values / 255 / 16.
[0074] S32. The Non-Maximum Suppression (NMS) algorithm is used to filter the pre-selected boxes whose scores are greater than the confidence threshold, so as to obtain multiple target pre-selected boxes.
[0075] Specifically, the NMS algorithm allows for the retention of only the most accurate target bounding box for each person target, discarding the rest. Based on this, a batch of target bounding boxes with high confidence can be initially selected for subsequent trajectory matching.
[0076] S33. Match the latest pre-selected boxes in the historical trajectory of each personnel target with each target pre-selected box in pairs, and calculate the Intersection over Union (IOU) value of each matching combination.
[0077] Specifically, for each matching combination, the IOU value is calculated as: (Area of intersection between the target preselection and the latest preselection) / (Sum of the areas of the target preselection and the latest preselection). The IOU values of each matching combination are sorted by size, and those greater than a preset value are considered successful matches. The preset value is, for example, 0.1.
[0078] S34. The latest trajectory obtained based on the IOU value includes the following situations, see below. Figure 2 .
[0079] Scenario 1: Historical trajectory matching successful:
[0080] If the IOU value of the matching combination is greater than the preset value, the historical trajectory and the target preselection box corresponding to the matching combination are successfully matched. The target preselection box in the matching combination is used as the latest preselection box of the corresponding historical trajectory to obtain the latest trajectory.
[0081] In this example, the relevant parameters of the successfully matched historical trajectory are updated: acc = acc + 1; box = target detection box; dec = 3. (See above for parameter definitions)
[0082] Scenario 2: Historical trajectory lost
[0083] The remaining unmatched target pre-selected boxes are used to create new trajectories for the objects to be tracked, resulting in the latest trajectory. In this example, the relevant parameters of the newly created trajectory are updated: acc = 0; box = target detection box; dec = 3.
[0084] Scenario 3: Create a new trajectory:
[0085] For any remaining historical trajectories that failed to match, update the number of times the target was not detected during historical tracking. If the number of times the target was not detected during historical tracking exceeds a preset number, delete the remaining historical trajectory. In this example, after updating the relevant parameters of the remaining historical trajectories that failed to match, determine whether they should be deleted: dec = dec - 1; if dec < 0, then delete.
[0086] S4. For any personnel target, calculate the direction of movement of the personnel target based on the current position indicated by the latest trajectory of the personnel target and the given displacement step size.
[0087] In this embodiment of the application, a simple prediction is made on each of the latest trajectories obtained after the above matching, and then the movement direction of the person target corresponding to the trajectory is calculated based on the prediction results. Specifically, step S4 includes:
[0088] S41. For any personnel target, calculate the predicted position coordinates of the personnel target based on the center position coordinates of the latest pre-selected box in the latest trajectory of the personnel target and the displacement step size corresponding to the personnel target.
[0089] The predicted position coordinates are obtained by predicting the center position coordinates of the latest preselected box according to the corresponding displacement step. For example, when the vertical coordinate indicates the direction of movement, the direction of movement can be determined by whether the vertical coordinate of the center position of the latest preselected box exceeds the midpoint of the vertical coordinate range of the image. If it exceeds, the movement is in the direction of increasing vertical coordinate (e.g., moving forward); if it does not exceed, the movement is in the direction of decreasing vertical coordinate (e.g., moving backward).
[0090] Taking a real-time temperature image with a vertical dimension of 8 (image height) as an example, the range of the vertical coordinate is [0, 8]. If the vertical coordinate is < 4, it can be determined that its future movement direction is in the direction of decreasing vertical coordinate. Based on this, the vertical coordinate of all pre-selected boxes in the latest trajectory (including each pre-selected box determined in the historical tracking process and the target pre-selected box determined as the latest pre-selected box) is "-displacement step size". That is, the vertical coordinate of the center position of the latest pre-selected box minus the displacement step size can be used to obtain the predicted position vertical coordinate (the horizontal coordinate remains unchanged).
[0091] Similarly, if the ordinate is ≥4, it can be determined that the future movement direction is in the direction of increasing ordinate. Based on this, the ordinate of all preselected boxes in the latest trajectory (including each preselected box determined in the historical tracking process and the target preselected box determined as the latest preselected box) is "+ displacement step size". That is, the ordinate of the center position of the latest preselected box plus the displacement step size can be used to obtain the predicted position ordinate (the abscissa remains unchanged).
[0092] For example, Figure 2 Taking a displacement step size of 1 as an example, the displacement step size indicates the range of change in the vertical coordinate. Optionally, the displacement step size can be different for different personnel targets, and self-learning can be performed based on the stride differences of different personnel during the trajectory tracking process. This application does not limit this.
[0093] S42. When the vertical coordinate indicates the direction of entry and exit, determine the movement direction of the personnel target based on the difference between the vertical coordinate of the predicted position and the vertical coordinate of the center point of the initial pre-selected box in the latest trajectory. The sign of the difference indicates the movement direction.
[0094] In one possible implementation, after that, the system first determines whether the latest trajectory is a continuous movement process based on the acc recorded in the latest trajectory. If acc > a preset value (e.g., 3), then execution continues to S42.
[0095] Specifically, the initial center point coordinates are, for example, (0,0), and the predicted position coordinates are (x, y), where both x and y are greater than or equal to 0 and y is less than or equal to the image height (e.g., 8). If the difference is positive (y-0), it can be determined that the person is moving from top to bottom; if the difference is negative (0-y), it can be determined that the person is moving from bottom to top.
[0096] S5. Based on the movement direction of each person target in the real-time temperature image, calculate the number of people entering and / or exiting the scene.
[0097] In this embodiment of the application, based on the movement direction (up or down) of each person target determined in step S4, and combined with the actual situation of the scene, the number of people entering and the number of people leaving can be counted.
[0098] In one possible implementation, the personnel entry and exit detection method provided in this application can achieve an accuracy of 92% in complex detection environments, which is 50% higher than the detection accuracy of similar products.
[0099] The technical solution of this application uses morphological processing for the initial image processing weakening filtering part, which can effectively reduce the amount of computation and meet the low power consumption requirements; the detection part uses a pre-selection box method to divide multi-person targets, which has strong generalization and can effectively reduce missed detections and achieve stable detection results.
[0100] Example 2
[0101] This application provides a personnel entry / exit detection device. (See reference...) Figure 4 The personnel entering and exiting the detection device include:
[0102] The preprocessing module 41 is used to generate a real-time foreground image based on the difference in pixel values between the real-time temperature image and the background image for any frame of real-time temperature image collected in the scene, and to perform erosion and / or dilation processing on the real-time foreground image. The background image is obtained by averaging multiple temperature images collected in the scene.
[0103] The detection module 42 is used to traverse the real-time foreground image according to a preset frame size, determine multiple pre-selected frames, and for any pre-selected frame, take the pixel with the largest pixel value in the pre-selected frame as the new center point and adjust the position of the pre-selected frame.
[0104] The tracking module 43 calculates the score of each pre-selected box based on the pixel value of each pixel within each adjusted pre-selected box, and filters out multiple target pre-selected boxes based on the score; it matches each target pre-selected box with the historical trajectory of each person target, and updates the latest trajectory of each person target based on the matching result;
[0105] The statistics module 44 is used to calculate the movement direction of any of the personnel targets based on the current position indicated by the latest trajectory of the personnel target and a given displacement step size; and to statistically obtain the number of people entering and / or exiting the scene based on the movement direction of each of the personnel targets in the real-time temperature image.
[0106] In one possible implementation, the device includes more or fewer functional modules for performing all or some of the corresponding steps in the personnel entry and exit detection method described above.
[0107] The technical solution of this application uses morphological processing for the initial image processing weakening filtering part, which can effectively reduce the amount of computation and meet the low power consumption requirements; the detection part uses a pre-selection box method to divide multi-person targets, which has strong generalization and can effectively reduce missed detections and achieve stable detection results.
[0108] It should be noted that the personnel entry and exit detection device provided in the above embodiments is only illustrated by the division of the above functional modules when implementing the corresponding steps. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the personnel entry and exit detection device provided in the above embodiments and the personnel entry and exit detection method described above belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.
[0109] This application provides a computing device that can be used to perform the above-described personnel entry and exit detection method. Figure 5 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application, such as... Figure 5As shown in section (a), the computing device includes a processor 501, a memory 502, a bus 503, and a computer program stored in the memory 502 and executable on the processor 501. The processor 501 includes one or more processing cores. The memory 502 is connected to the processor 501 via the bus 503 and is used to store program instructions. When the processor executes the computer program, it implements all or part of the steps in the above-described method embodiments provided in this application.
[0110] Furthermore, as an executable solution, the aforementioned computing device can be a computer unit, which may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described computer unit structure is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., which are not limited in this application embodiment.
[0111] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.
[0112] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0113] In one possible implementation, see Figure 5 In part (b), the computing device also includes a battery module for powering the processor, whose computing power and / or power consumption meet preset conditions (see above).
[0114] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the methods described in the embodiments of this application.
[0115] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
[0116] Although this application has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to this application without departing from the spirit and scope of this application as defined by the appended claims, and all such changes shall be within the scope of protection of this application.
Claims
1. A method for detecting personnel entry and exit, characterized in that, The method includes: S1. For any frame of real-time temperature image collected in the scene, generate a real-time foreground image based on the difference in pixel values between the real-time temperature image and the background image of the scene, and perform noise reduction processing on the real-time foreground image. The background image is obtained by averaging multiple temperature images collected in the scene. S2. Traverse the real-time foreground image after noise reduction according to the preset frame size to determine multiple pre-selected frames. For any of the pre-selected frames, take the pixel with the largest pixel value in the pre-selected frame as the new center point and adjust the position of the pre-selected frame. S3. Calculate the score of each preselected box based on the pixel value of each pixel within each adjusted preselected box, and filter out multiple target preselected boxes based on the scores; match each target preselected box with the historical trajectory of each personnel target, and update the latest trajectory of each personnel target based on the matching results; S4. For any of the personnel targets, calculate the movement direction of the personnel target based on the current position indicated by the latest trajectory of the personnel target and the given displacement step size; S5. Based on the movement direction of each person target in the real-time temperature image, the number of people entering and / or exiting the scene is statistically determined.
2. The personnel entry and exit detection method according to claim 1, characterized in that, Before step S1, the method further includes: Based on multiple temperature images captured in the scene, the pixel values of pixels at the same location are added together and the average value is taken to obtain the background image; step S1 includes: S11. For any frame of real-time temperature image, generate a real-time foreground image based on the difference between the pixel values of the real-time temperature image and the background image, and set the pixel values of pixels in the real-time foreground image that are less than a given judgment threshold to 0. S12. Perform erosion and / or dilation processing on any pixel in the real-time foreground image whose pixel value is not 0; the erosion processing includes setting the pixel value of the pixel to 0 when the pixel values of all adjacent pixels of the pixel are 0; the dilation processing includes setting the pixel value of the pixel to the average value of the pixel values of all adjacent pixels when the pixel values of all adjacent pixels of the pixel are not all 0.
3. The personnel entry and exit detection method according to claim 1, characterized in that, The historical trajectory of any of the personnel targets includes an ID indicating the personnel target, and also includes at least one of the following: the number of times the target was detected during the historical tracking process, the number of times the target was not detected during the historical tracking process, the position coordinates of the latest pre-selected box in the historical trajectory, the activation status of the personnel target, the center point coordinates of the initial pre-selected box in the historical trajectory, and the displacement step size corresponding to the personnel target.
4. The personnel entry and exit detection method according to claim 3, characterized in that, Step S3 includes: S31. For any adjusted preselected box, sum the pixel values of each pixel in the adjusted preselected box, and normalize them according to the format parameters of the real-time temperature image to obtain the score of the adjusted preselected box. S32. The non-maximum suppression algorithm is used to filter the pre-selected boxes whose scores are greater than the confidence threshold, so as to obtain multiple target pre-selected boxes; S33. Match the latest pre-selected boxes in the historical trajectory of each personnel target with each target pre-selected box in pairs, and calculate the IOU value of each matching combination; S34. If the IOU value of the matching combination is greater than the preset value, the historical trajectory and the target preselection box corresponding to the matching combination are successfully matched. The target preselection box in the matching combination is used as the latest preselection box of the corresponding historical trajectory to obtain the latest trajectory. The remaining target preselection boxes that are not successfully matched are established as the trajectory of the new object to be tracked to obtain the latest trajectory. For the remaining historical trajectories that failed to match, update the number of times the target was not detected during the historical tracking process. If the number of times the target was not detected during the historical tracking process exceeds a preset number, delete the remaining historical trajectories.
5. The personnel entry and exit detection method according to claim 3, characterized in that, Step S4 includes: S41. For any of the personnel targets, calculate the predicted position coordinates of the personnel target based on the center position coordinates of the latest preselected box in the latest trajectory of the personnel target and the displacement step size corresponding to the personnel target. S42. When the vertical coordinate indicates the direction of entry and exit, the movement direction of the personnel target is determined by subtracting the vertical coordinate of the center point of the initial pre-selected box in the latest trajectory from the vertical coordinate of the predicted position coordinate. The sign of the difference indicates the movement direction.
6. A personnel entry / exit detection device, characterized in that, The device includes: The preprocessing module is used to generate a real-time foreground image based on the difference in pixel values between the real-time temperature image and the background image of the scene for any frame of real-time temperature image collected in the scene, and to perform noise reduction processing on the real-time foreground image. The background image is obtained by averaging multiple temperature images collected in the scene. The detection module is used to traverse the real-time foreground image after denoising according to a preset box size, determine multiple pre-selected boxes, and for any one of the pre-selected boxes, take the pixel with the largest pixel value in the pre-selected box as the new center point and adjust the position of the pre-selected box. The tracking module calculates the score of each pre-selected box based on the pixel value of each pixel within each adjusted pre-selected box, and filters out multiple target pre-selected boxes based on the score; it matches each target pre-selected box with the historical trajectory of each person target, and updates the latest trajectory of each person target based on the matching results; The statistics module is used to calculate the movement direction of any of the personnel targets based on the current position indicated by the latest trajectory of the personnel target and a given displacement step size; and to statistically obtain the number of people entering and / or exiting the scene based on the movement direction of each of the personnel targets in the real-time temperature image.
7. The personnel entry and exit detection device according to claim 6, characterized in that, The device further includes: The background estimation module is used to obtain the background image by summing the pixel values of pixels at the same location and taking the average value from multiple temperature images collected in the scene; the preprocessing module is used for: For any frame of real-time temperature image, a real-time foreground image is generated based on the difference in pixel values between the real-time temperature image and the background image, and the pixel values of pixels in the real-time foreground image that are less than a given judgment threshold are set to 0. Erosion and / or dilation are performed on any pixel in the real-time foreground image whose pixel value is not 0. The erosion process includes setting the pixel value of the pixel to 0 when the pixel values of all its neighboring pixels are 0. The dilation process includes setting the pixel value of the pixel to the average value of the pixel values of all its neighboring pixels when the pixel values of all its neighboring pixels are not all 0.
8. A computing device, characterized in that, It includes a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the personnel entry and exit detection method as described in any one of claims 1 to 5.
9. The computing device according to claim 8, characterized in that, The computing device also includes a battery module for powering the processor, wherein the processor's computing power and / or power consumption meet preset conditions.
10. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is executed by a processor to implement the personnel entry and exit detection method as described in any one of claims 1 to 5.