Personnel quantity statistical method, electronic equipment and computer readable storage medium

Abstract

The invention relates to the technical field of computer vision, and provides a personnel quantity statistical method, electronic equipment and a computer readable storage medium. The method comprises the steps that a monitoring video is acquired, the monitoring video is a video which is shot by a shooting device and comprises a monitoring area, the monitoring video comprises a plurality of video frames, and the monitoring area comprises an inner area and an outer area; and identifying the plurality of video frames by adopting a preset identification model. And obtaining a plurality of action tracks. The number of entering persons, the number of leaving persons and the number of passing persons are determined according to the multiple action trajectories, the number of entering persons is the number of action trajectories with the starting point in the outer area and the end point in the inner area, the number of leaving persons is the number of action trajectories with the starting point in the inner area and the end point in the outer area, and the number of passing persons is the number of starting point in the outer area; the terminal point is the number of motion tracks in the outer area. Missing counting and repeated counting of the number of people can be avoided, and the counting accuracy of the number of people is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method for counting the number of people, an electronic device, and a computer-readable storage medium. Background Art

[0002] Customer flow is not only a key indicator for evaluating business performance, but also an important basis for businesses to optimize operating strategies, improve customer experience, predict sales trends, evaluate return on investment and assist in business decision-making.

[0003] In the traditional technical solution, based on the surveillance video of IPC (Internet Protocol Camera), a virtual counting line can be set in the surveillance imaging area of ​​IPC, and the passenger flow can be determined by judging whether there is an intersection between the movement trajectory of the passing person and the virtual counting line.

[0004] However, if the virtual counting line is set at a higher position in the imaging area, it will not intersect with the movement trajectory of shorter children, and if the virtual counting line is set at a lower position in the imaging area, it will not intersect with the movement trajectory of taller adults. In addition, if the same person walks back and forth near the virtual counting line, IPC will count repeatedly, resulting in reduced accuracy of passenger flow statistics. Summary of the invention

[0005] The embodiments of the present application provide a method, device, chip, electronic device and computer-readable storage medium for counting the number of people, which can avoid omissions and repeated counting of the number of people and improve the accuracy of counting the number of people.

[0006] In the first aspect, the present application provides a method for counting the number of people, including: obtaining a surveillance video, the surveillance video is a video captured by a shooting device and containing a surveillance area, the surveillance video contains multiple video frames, and the surveillance area includes an inner area and an outer area. Using a preset recognition model, multiple video frames are recognized to obtain multiple motion trajectories, and the multiple motion trajectories correspond one-to-one to multiple people in the multiple video frames, and the multiple people are different from each other. The number of people entering, the number of people leaving, and the number of people passing are determined based on the multiple motion trajectories, the number of people entering is the number of motion trajectories with a starting point in the outer area and an end point in the inner area among the multiple motion trajectories, the number of people leaving is the number of motion trajectories with a starting point in the inner area and an end point in the outer area among the multiple motion trajectories, and the number of people passing is the number of motion trajectories with a starting point in the outer area and an end point in the outer area among the multiple motion trajectories.

[0007] In some embodiments, a preset recognition model is used to identify multiple video frames to obtain multiple action trajectories, including: using a preset recognition model to identify multiple groups of monitoring point sets corresponding to the multiple video frames, the multiple groups of monitoring point sets correspond one-to-one to the multiple video frames, the first monitoring point set includes at least one monitoring point, the at least one monitoring point is the location point of at least one person in the first video frame, at least one monitoring point corresponds one-to-one to at least one person, the first video frame is any one of the multiple video frames, and the first monitoring point set is a point set in the multiple groups of monitoring point sets corresponding to the first video frame; multiple action trajectories are determined based on the multiple groups of monitoring point sets.

[0008] In some embodiments, a preset recognition model is used to identify multiple sets of monitoring point sets corresponding to multiple video frames, including: using a preset recognition model to identify a first video frame to obtain at least one person. Identifying at least one head-shoulder frame corresponding to the at least one person, the at least one head-shoulder frame is a rectangular frame where the head and shoulders of the at least one person are located, and the at least one person corresponds to the at least one head-shoulder frame one-to-one. Determine the position coordinates of the center point of each head-shoulder frame in the at least one head-shoulder frame as at least one monitoring point to obtain a first monitoring point set.

[0009] In some embodiments, determining multiple action trajectories based on multiple sets of monitoring points includes: obtaining multiple monitoring points of a first person in multiple video frames, where the first person is any one of the multiple persons. Determining action trajectories corresponding to the first person based on the multiple monitoring points and multiple shooting times of the multiple video frames where the multiple monitoring points are located, so as to obtain multiple action trajectories.

[0010] In some embodiments, a preset recognition model is used to recognize the first video frame to obtain at least one person, including: obtaining a preset virtual recognition frame, the preset virtual recognition frame is a polygon, the preset virtual recognition frame includes a tri-fold line that has an intersection with any two non-adjacent sides of the polygon, the area of ​​the preset virtual recognition frame on one side of the tri-fold line is the virtual inner area, the area of ​​the preset virtual recognition frame on the other side of the tri-fold line is the virtual outer area, the preset virtual recognition frame corresponds to the monitoring area, the virtual inner area corresponds to the inner area, and the virtual outer area corresponds to the outer area. Based on the preset virtual recognition frame, the preset recognition model is used to recognize the first video frame to obtain at least one person.

[0011] In some embodiments, the monitoring area is the area where the entrance or exit of a door or gate is located. If the monitoring area is the area where the door is located, the preset virtual identification frame is the frame of the door, the first line segment and the second line segment of the trifold line intersect with the frame respectively, the first line segment and the second line segment are not connected, the first line segment and the second line segment are respectively connected to the third line segment, and the third line segment is the line segment in the middle position of the trifold line.

[0012] In some embodiments, the preset virtual identification frame is a quadrilateral, and the area ratio of the virtual inner region to the virtual outer region is within a preset ratio range.

[0013] In some embodiments, the method further includes: determining the number of people entering the store, and the ratio of the sum of the number of people entering the store and the number of people passing by the store is the store entry rate.

[0014] In a second aspect, the present application provides a device for counting the number of people. The device comprises:

[0015] The acquisition module is used to acquire monitoring video, where the monitoring video is a video captured by a shooting device and includes a monitoring area. The monitoring video includes multiple video frames, and the monitoring area includes an inner area and an outer area.

[0016] The processing module is used to use a preset recognition model to recognize multiple video frames to obtain multiple action trajectories, where the multiple action trajectories correspond one-to-one to multiple persons in the multiple video frames, and the multiple persons are different from each other.

[0017] The processing module is also used to determine the number of people entering, leaving and passing according to multiple motion trajectories. The number of people entering is the number of motion trajectories whose starting point is in the outer area and whose end point is in the inner area among the multiple motion trajectories. The number of people leaving is the number of motion trajectories whose starting point is in the inner area and whose end point is in the outer area among the multiple motion trajectories. The number of people passing is the number of motion trajectories whose starting point is in the outer area and whose end point is in the outer area among the multiple motion trajectories.

[0018] In a third aspect, the present application provides a chip, which is used to execute any method in the first aspect.

[0019] In a fourth aspect, the present application provides an electronic device, comprising a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement a method as described in any one of the first aspects above. Or,

[0020] The electronic device includes the chip in the third aspect.

[0021] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method as described in any one of the above-mentioned first aspects.

[0022] In the technical solution provided by the embodiment of the present application, the IPC can obtain a monitoring video, which is a video containing a monitoring area captured by a shooting device. The monitoring video contains multiple video frames, and the monitoring area includes an inner area and an outer area. A preset recognition model is used to recognize multiple video frames to obtain multiple motion trajectories corresponding to multiple personnel. Finally, the number of people entering, leaving, and passing are determined according to the multiple motion trajectories. The number of people entering is the number of motion trajectories with a starting point in the outer area and an end point in the inner area among the multiple motion trajectories, the number of people leaving is the number of motion trajectories with a starting point in the inner area and an end point in the outer area among the multiple motion trajectories, and the number of people passing is the number of motion trajectories with a starting point in the outer area and an end point in the outer area among the multiple motion trajectories. The motion trajectories of people at different heights can be identified, and the number of people can be counted through the motion routes and behavior trends of the motion trajectories in the inner and outer areas, which can avoid the omission and repeated counting of the number of people and improve the accuracy of the number of people statistics. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a schematic diagram of an application scenario of a method for counting the number of people provided in an embodiment of the present application;

[0025] Figure 2 This is a flow chart of a method for counting the number of people provided in an embodiment of the present application;

[0026] Figure 3 This is a schematic diagram of a video frame to be detected in a method for counting the number of people provided in an embodiment of the present application;

[0027] Figure 4 This is another schematic diagram of a video frame to be detected in a method for counting the number of people provided in an embodiment of the present application;

[0028] Figure 5 This is another schematic diagram of a video frame to be detected in a method for counting the number of people provided in an embodiment of the present application;

[0029] Figure 6 This is a schematic diagram of a monitoring video frame of a method for counting the number of people provided in an embodiment of the present application;

[0030] Figure 7This is another flow chart of a method for counting the number of people provided in an embodiment of the present application;

[0031] Figure 8 This is a schematic diagram of a movement trajectory of a method for counting the number of people provided in an embodiment of the present application;

[0032] Fig. 9 This is another action trajectory schematic diagram of a method for counting the number of people provided in an embodiment of the present application;

[0033] Fig.10 It is a structural schematic diagram of a device for counting the number of people provided in an embodiment of the present application;

[0034] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0036] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0037] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0038] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0039] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0041] AI (Artificial Intelligence) visual analysis is the process of analyzing and understanding images or videos using artificial intelligence technology. Usually, through a trained model, the input image or video is matched and compared with known data to extract key information, and corresponding judgments or decisions are made based on the key information. This can achieve image detection, tracking, segmentation and other functions.

[0042] IPC is used for security supervision of home security, commercial security and public security. In the field of commercial security, IPCs installed at store entrances and exits, shopping mall entrances and exits, subway entrances and exits, scenic area entrances and exits, and community entrances and exits can collect video data of the monitored area in real time, and count the number of people entering and leaving the monitored area based on AI visual analysis technology. This facilitates the analysis of peak traffic flow, realizes traffic control, and prevents stampedes.

[0043] In the traditional technical solution, IPC can set a virtual counting line in the imaging area and determine the passenger flow by judging whether there is an intersection between the movement trajectory of the passing person and the virtual counting line. Figure 1 As shown, the virtual counting line can be set at the position shown in L1 or L2. If the virtual counting line is set at a higher position in the imaging area, such as the position shown in L1, the movement trajectory (trajectory 2) of a shorter child (personnel 2) cannot intersect with L1. If the virtual counting line is set at a lower position in the imaging area, such as the position shown in L2, the movement trajectory (trajectory 1) of a taller adult (personnel 1) cannot intersect with L2.

[0044] In addition, if the virtual counting line is set at the position shown by L1, and person 1 walks back and forth near the virtual counting line L1, the movement trajectory of person 1 will repeatedly intersect with L1. IPC will add the number of times the movement trajectory of person 1 intersects with L1 to the number of passenger flow, resulting in a decrease in the accuracy of passenger flow statistics.

[0045] In view of this, an embodiment of the present application provides a method for counting the number of people, which can improve the accuracy of passenger flow statistics.

[0046] The technical solution provided in the embodiment of the present application can be applied to devices or electronic devices with image processing functions, such as IPC, computer, server, etc. The embodiment of the present application does not limit the specific type of electronic device.

[0047] For ease of description, the embodiment of the present application takes IPC as an example to exemplarily describe the specific process of personnel number counting.

[0048] Combine the following Figures 2 to 9 The examples in the description describe the technical solutions of the embodiments of the present application.

[0049] like Figure 2 As shown, the method for IPC to count the number of people based on the video frames in the imaging area may include the following steps:

[0050] Step S201: Acquire a plurality of video frames to be detected, where the plurality of video frames to be detected are video frames captured by a shooting device at different times.

[0051] In the embodiment of the present application, the shooting device may be an IPC. The IPC may acquire video data in real time to obtain a video stream, which may include multiple video frames. The multiple video frames to be detected may be multiple continuous video frames in the multiple video frames, or may be multiple video frames separated by a preset time period. The present application does not limit this.

[0052] For example, see Figure 1 , the imaging effect of one of the multiple video frames to be detected can be as follows Figure 1 shown.

[0053] Step S202: determining a monitoring area of ​​each of the multiple video frames to be detected, which is multiple monitoring images, and the multiple monitoring images correspond to the multiple video frames to be detected one by one.

[0054] In an embodiment of the present application, the monitoring area may be an area with dense personnel flow, such as the entrance and exit of a shopping mall, a scenic spot, or a subway station. The monitoring area may be an area set by the user, and the user may set an area where people tend to wander as the monitoring area based on actual conditions. The monitoring area may also be an automatically generated area, and the IPC may obtain recognition results based on a preset recognition model after recognizing multiple monitoring video frames, and divide the area with dense distribution of head and shoulder frames of people into the monitoring area. The present application does not limit the method for setting the monitoring area.

[0055] The method for the IPC to determine multiple surveillance images based on multiple video frames to be detected may include:

[0056] Step A1: Obtain a preset virtual identification frame of the first video frame to be detected, where the preset virtual identification frame is a polygon, and the preset virtual identification frame includes a tri-fold line that has an intersection with any two non-adjacent sides of the polygon, the area of ​​the preset virtual identification frame on one side of the tri-fold line is the virtual inner area, and the area of ​​the preset virtual identification frame on the other side of the tri-fold line is the virtual outer area, the preset virtual identification frame corresponds to the monitoring area, the virtual inner area corresponds to the inner area, and the virtual outer area corresponds to the outer area.

[0057] In the embodiment of the present application, the preset virtual identification frame can be a monitoring area defined by the user according to the actual application scenario, that is, the trajectory of the preset virtual identification frame can be the trajectory of the monitoring area. The virtual inner area and the virtual outer area can be areas specified by the user. Specifically, the user can divide the preset virtual identification frame into the virtual inner area and the virtual outer area by a tri-fold line.

[0058] For example, Figure 3 As shown, if the monitoring scene is a shopping mall entrance and exit, the preset virtual identification box set by the user can be close to or overlap with the shopping mall entrance and exit trajectory, the virtual inner area is the area indicated by the arrow on the trifold line, and the virtual outer area is the area on the other side of the trifold line.

[0059] For example, after the IPC recognizes multiple surveillance images based on the preset recognition model, the preset virtual recognition frame can be determined as follows: Figure 4 As shown, the preset virtual recognition frame is an area where the head and shoulder frames of people are densely distributed, and the virtual inner area and the virtual outer area can be specified by the user according to the preset virtual recognition frame generated by IPC.

[0060] In the embodiment of the present application, the preset virtual identification frame can be a quadrilateral. If the monitoring area is the area where the door frame is located, the preset virtual identification frame can be the trajectory of the door frame. The preset virtual identification frame can also be a quadrilateral that matches the trajectory of the door frame. The first line segment and the second line segment of the trifold line intersect with the door frame respectively, the first line segment and the second line segment are not connected, the first line segment and the second line segment are connected to the third line segment respectively, and the third line segment is the line segment in the middle of the trifold line. For example, continue to refer to Figure 4 , the first line segment and Figure 4 The first line segment intersects with the left side of the frame of the middle door, and the second line segment intersects with the right side of the frame of the door.

[0061] In some embodiments, if there is no reference object such as a door frame in the monitoring area, the range of the preset virtual identification frame can be close to the range of entrances and exits such as gates.

[0062] For example, Figure 5 FIG. 1 is a schematic diagram of an application scenario in a gate scenario provided by an embodiment of the present application. The range of the preset virtual recognition frame is close to the range of the gate entrance and exit.

[0063] In the embodiment of the present application, the area ratio of the virtual inner area of ​​the preset virtual identification frame to the area ratio of the virtual outer area can be within a preset ratio range, which is convenient for dividing the movement trajectory of the personnel into the inner area and the outer area, so as to ensure that the IPC can determine the number of people entering, leaving and passing according to the movement trajectory of the personnel in multiple surveillance video frames.

[0064] The ratio of the area of ​​the virtual inner area to the area of ​​the virtual outer area can be adjusted according to the shooting angle and usage scenario of the shooting device, and the present application does not limit the specific value of the preset ratio range.

[0065] Step A2: Determine an area of ​​a preset virtual recognition frame in a first video frame to be detected as a first monitoring image to obtain multiple monitoring images, wherein the first video frame to be detected is a video frame corresponding to the first monitoring image among the multiple video frames to be detected.

[0066] In an embodiment of the present application, the IPC may use the image of the preset virtual identification frame portion of the first video frame to be detected as the first monitoring image, that is, the first monitoring image is a sub-image of the first video frame to be detected.

[0067] For example, Figure 4 The first monitoring image corresponding to the first video frame to be detected can be as shown in Figure 6 In the process of counting the number of people, the IPC only needs to monitor the movement trajectory of the people in the first monitoring video frame to determine the number of people entering, leaving, and passing in the current monitoring scene.

[0068] In the embodiment of the present application, after the IPC determines multiple surveillance images that need to track the trajectory of personnel based on multiple video frames to be detected, the number of people entering, leaving, and passing in the current surveillance scene can be determined based on the multiple surveillance images. Figure 7 As shown, the method of IPC to determine the number of people entering, the number of people leaving, and the number of people passing may include the following steps:

[0069] Step S701: Acquire a monitoring video, where the monitoring video is a video captured by a shooting device and includes a monitoring area. The monitoring video includes multiple video frames, and the monitoring area includes an inner area and an outer area.

[0070] The multiple video frames may be video frames captured by the same camera (IPC) at different times. It is understandable that the multiple video frames may be video frames captured at the same angle but at different times. The multiple video frames may be multiple continuous video frames acquired by the IPC, or multiple discontinuous video frames at intervals of a certain period of time, which is not limited in this application.

[0071] Step S702: Based on a preset recognition model, multiple video frames are recognized to obtain multiple action trajectories, where the multiple action trajectories correspond one-to-one to multiple persons in the multiple video frames, and the multiple persons are different from each other.

[0072] In the embodiment of the present application, based on the preset recognition model, multiple video frames are recognized to obtain multiple action trajectories, including:

[0073] First, based on a preset recognition model, multiple sets of monitoring point sets corresponding to multiple video frames are identified, and the multiple sets of monitoring point sets correspond to multiple video frames one-to-one. The first monitoring point set includes at least one monitoring point, and the at least one monitoring point is the location point of at least one person in the first video frame. The at least one monitoring point corresponds to at least one person one-to-one. The first video frame is any one of the multiple video frames, and the first monitoring point set is a point set in the multiple sets of monitoring point sets corresponding to the first video frame. Then, multiple action trajectories are determined based on the multiple sets of monitoring point sets.

[0074] In the embodiment of the present application, the method for IPC to determine multiple groups of monitoring point sets corresponding to multiple video frames may include:

[0075] B1: Use a preset recognition model to recognize the first video frame and obtain at least one person.

[0076] The preset recognition model can be a trained CNN (Convolutional Neural Networks) model, YOLO (You Only Look Once) model, SSD (Single Shot MultiBox Detector) model, etc., which is not limited in this application.

[0077] B2: Identify at least one head-shoulder frame corresponding to at least one person, where the at least one head-shoulder frame is a rectangular frame where the head and shoulders of the at least one person are located, and the at least one person corresponds to the at least one head-shoulder frame one-to-one.

[0078] IPC can determine the rectangular frame where the head and shoulders of a person in a surveillance video frame are located based on a preset recognition model.

[0079] For example, see Figure 6 , IPC determines multiple head and shoulder frames in the first video frame based on a preset recognition model, such as Figure 6 As shown in the rectangular box.

[0080] B3: Determine the position coordinates of the center point of each head-shoulder frame in at least one head-shoulder frame as at least one monitoring point, and obtain a first monitoring point set.

[0081] In the embodiment of the present application, the monitoring point can be the position coordinates of the center point of the rectangle, that is, the monitoring point is the intersection of the two diagonals of the head-shoulder frame. The monitoring point is the center point of the head-shoulder frame corresponding to the valid person identified by the IPC. It is understandable that if there are only areas such as clothing corners in the monitoring video frame that cannot be confirmed as valid personnel, the IPC cannot identify the head-shoulder frame, that is, it cannot be confirmed as a valid person.

[0082] For example, see Figure 6 , the multiple monitoring points in the first monitoring point set corresponding to the first video frame are as follows Figure 6 As shown by the multiple circular black dots in the figure.

[0083] In the embodiment of the present application, the method for the IPC to determine multiple action trajectories based on multiple sets of monitoring points may include:

[0084] C1: Acquire multiple monitoring points of a first person in multiple video frames, where the first person is any one of the multiple persons.

[0085] In the embodiment of the present application, the IPC can track the movement trajectory of the person based on the preset recognition model. Specifically, the IPC can track the trajectory of the person through multiple monitoring points of the same person in multiple surveillance video frames at different times.

[0086] C2: Determine the action trajectory corresponding to the first person according to the multiple monitoring points and the multiple shooting times of the multiple video frames where the multiple monitoring points are located, so as to obtain multiple action trajectories.

[0087] It can be understood that the movement trajectory of the same person can be a movement trajectory generated by IPC based on multiple shooting times corresponding to multiple monitoring points.

[0088] For example, Figure 8 As shown, the multiple monitoring points corresponding to person A can be shown as the multiple black dots corresponding to track 1. The multiple monitoring points corresponding to person B can be shown as the multiple black dots corresponding to track 2. The multiple monitoring points corresponding to person C can be shown as the multiple black dots corresponding to track 3.

[0089] Step S703: Determine the number of people entering, leaving and passing according to multiple motion trajectories, the number of people entering is the number of motion trajectories whose starting point is in the outer area and whose end point is in the inner area among the multiple motion trajectories, the number of people leaving is the number of motion trajectories whose starting point is in the inner area and whose end point is in the outer area among the multiple motion trajectories, and the number of people passing is the number of motion trajectories whose starting point is in the outer area and whose end point is in the outer area among the multiple motion trajectories.

[0090] In the embodiment of the present application, the IPC may count the persons corresponding to the movement trajectory whose starting point is in the outer area and whose end point is in the inner area as entering persons.

[0091] For example, see Figure 8 ,Trajectory 1 is an action trajectory corresponding to the entering person.

[0092] IPC can count the people whose movement trajectories are within the starting point area and whose end points are outside the starting point area as those who have left.

[0093] For example, see Figure 8 ,Trajectory 2 is an action trajectory corresponding to the leaving personnel.

[0094] IPC can count people whose movement trajectories are outside the starting point and whose end points are outside the area as passing people.

[0095] For example, see Figure 8 ,Trajectory 3 is an action trajectory corresponding to the passing person.

[0096] IPC can determine the flow of people in the monitoring area according to the number of people entering, leaving, and passing in a unit time period, so as to monitor the flow of people. IPC can also determine the store entry rate according to the number of people entering and passing. For example, the calculation formula for the store entry rate can be:

[0097] Store entry rate = number of people entering / (number of people entering + number of people passing by);

[0098] In addition, the technical solution provided by the embodiment of the present application can also avoid the problem of repeatedly counting people wandering near the monitoring area. Fig. 9 As shown, track 6 is a movement track corresponding to a person wandering near the monitoring area. IPC tracks the movement track of the wandering person and determines the starting point of track 4 within the area. If the end point is within the area, track 4 will not be counted as a person entering, leaving, or passing by. This can improve the accuracy of passenger flow statistics.

[0099] In the technical solution provided in the embodiment of the present application, IPC can calculate the store entry rate and predict data such as sales trends, so as to help merchants optimize their operating strategies, increase sales and reduce operating costs.

[0100] In the technical solution provided by the embodiment of the present application, the IPC can obtain a monitoring video, which is a video containing a monitoring area captured by a shooting device. The monitoring video contains multiple video frames, and the monitoring area includes an inner area and an outer area. A preset recognition model is used to recognize multiple video frames to obtain multiple motion trajectories corresponding to multiple personnel. Finally, the number of people entering, leaving, and passing are determined according to the multiple motion trajectories. The number of people entering is the number of motion trajectories with a starting point in the outer area and an end point in the inner area among the multiple motion trajectories, the number of people leaving is the number of motion trajectories with a starting point in the inner area and an end point in the outer area among the multiple motion trajectories, and the number of people passing is the number of motion trajectories with a starting point in the outer area and an end point in the outer area among the multiple motion trajectories. The motion trajectories of people at different heights can be identified, and the number of people can be counted through the motion routes and behavior trends of the motion trajectories in the inner and outer areas, which can avoid the omission and repeated counting of the number of people and improve the accuracy of the number of people statistics.

[0101] It should be understood that, under the premise of no logical conflict, the above-mentioned various application embodiments can be implemented in combination with each other to meet actual application requirements. The specific embodiments or implementation plans obtained after these combinations still fall within the protection scope of this application.

[0102] Corresponding to the personnel counting method in the above embodiment, the embodiment of the present application also provides a personnel counting device, which can be implemented by software, hardware or a combination of both to become part or all of a computer device, and is used to execute the steps in the personnel counting method in the above embodiment.

[0103] Fig.10 A schematic structural diagram of a device 100 for counting the number of people provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0104] Reference Fig.10 The device 100 includes an acquisition module 1010 and a processing module 1020.

[0105] The acquisition module 1010 is used to acquire a surveillance video, where the surveillance video is a video captured by a shooting device and includes a surveillance area, where the surveillance area includes an inner area and an outer area.

[0106] The processing module 1020 is used to use a preset recognition model to recognize multiple video frames to obtain multiple action trajectories. The multiple action trajectories correspond one-to-one to multiple persons in the multiple video frames, and the multiple persons are different from each other.

[0107] The processing module 1020 is also used to determine the number of people entering, the number of people leaving, and the number of people passing based on multiple motion trajectories. The number of people entering is the number of motion trajectories whose starting point is in the outer area and whose end point is in the inner area among the multiple motion trajectories. The number of people leaving is the number of motion trajectories whose starting point is in the inner area and whose end point is in the outer area among the multiple motion trajectories. The number of people passing is the number of motion trajectories whose starting point is in the outer area and whose end point is in the outer area among the multiple motion trajectories.

[0108] In some embodiments, the processing module 1020 is specifically used to: use a preset recognition model to identify multiple groups of monitoring point sets corresponding to multiple video frames, the multiple groups of monitoring point sets correspond one-to-one to the multiple video frames, the first monitoring point set includes at least one monitoring point, at least one monitoring point is the location point of at least one person in the first video frame, at least one monitoring point corresponds one-to-one to at least one person, the first video frame is any one of the multiple video frames, and the first monitoring point set is a point set in the multiple groups of monitoring point sets that corresponds to the first video frame; determine multiple action trajectories based on the multiple groups of monitoring point sets.

[0109] In some embodiments, the processing module 1020 is specifically used to: use a preset recognition model to identify the first video frame to obtain at least one person. Identify at least one head-shoulder frame corresponding to the at least one person, the at least one head-shoulder frame is a rectangular frame where the head and shoulders of the at least one person are located, and the at least one person corresponds to the at least one head-shoulder frame one-to-one. Determine the position coordinates of the center point of each head-shoulder frame in the at least one head-shoulder frame as at least one monitoring point, and obtain a first monitoring point set.

[0110] In some embodiments, the processing module 1020 is specifically used to: obtain multiple monitoring points of a first person in multiple video frames, where the first person is any one of the multiple persons, and determine the action trajectory corresponding to the first person according to the multiple monitoring points and the multiple shooting times of the multiple video frames where the multiple monitoring points are located, so as to obtain multiple action trajectories.

[0111] In some embodiments, the acquisition module 1010 is specifically used to: acquire a preset virtual identification frame, the preset virtual identification frame is a polygon, the preset virtual identification frame includes a tri-fold line that has an intersection with any two non-adjacent sides of the polygon, the area of ​​the preset virtual identification frame on one side of the tri-fold line is the virtual inner area, the area of ​​the preset virtual identification frame on the other side of the tri-fold line is the virtual outer area, the preset virtual identification frame corresponds to the monitoring area, the virtual inner area corresponds to the inner area, and the virtual outer area corresponds to the outer area. Based on the preset virtual identification frame, the preset recognition model is used to identify the first video frame to obtain at least one person.

[0112] In some embodiments, the monitoring area is the area where the entrance or exit of a door or gate is located. If the monitoring area is the area where the door is located, the preset virtual identification frame is the frame of the door, the first line segment and the second line segment of the trifold line intersect with the frame respectively, the first line segment and the second line segment are not connected, the first line segment and the second line segment are respectively connected to the third line segment, and the third line segment is the line segment in the middle position of the trifold line.

[0113] In some embodiments, the preset virtual identification frame is a quadrilateral, and the area ratio of the virtual inner region to the virtual outer region is within a preset ratio range.

[0114] In some implementations, the processing module 1020 is further used to: determine the number of people entering the store, and the ratio of the sum of the number of people entering the store and the number of people passing by the store is the store entry rate.

[0115] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0116] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0117] Based on the same inventive concept, an embodiment of the present application also provides an electronic device.

[0118] Fig.11 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.11 As shown, the electronic device 11 of this embodiment includes: at least one processor 1110 ( Fig.11 Only one is shown in the figure), memory 1120, and communication module 1140, memory 1120 stores computer program 1130 that may be run on processor 1110. When processor 1110 executes computer program 1130, steps in the above-mentioned personnel counting method embodiment are implemented, such as Figure 2 201 to step 202 as shown. Alternatively, Figure 7 From step 701 to step 703, the processor 1110 executes the computer program 1130 to implement the functions of each module / unit in the above-mentioned device embodiments, for example Fig.10 The functions of modules 1010 to 1120 are shown, and the communication module 1140 can be a separate communication unit for communicating with an external server or terminal device.

[0119] The electronic device 11 may include, but is not limited to: a processor 1110 and a memory 1120. Those skilled in the art will appreciate that Fig.11 It is only an example of the electronic device 11 and does not constitute a limitation of the electronic device 11. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 11 may also include an input sending device, a network access device, a bus, etc.

[0120] The processor 1110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0121] In some embodiments, the memory 1120 may be an internal storage unit of the electronic device 11, such as a hard disk or memory of the electronic device 11. The memory 1120 may also be an external storage device of the electronic device 11, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 11. The memory 1120 may also include both an internal storage unit of the electronic device 11 and an external storage device. The memory 1120 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program 1130, etc. The memory 1120 may also be used to temporarily store data that has been sent or is to be sent.

[0122] In addition, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0123] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on an electronic device, the electronic device executes the steps in the above-mentioned method embodiments.

[0124] An embodiment of the present application provides a chip, which includes a processor and a memory. The memory stores a computer program, and when the computer program is executed by the processor, the steps in the above-mentioned method embodiments are implemented.

[0125] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the steps in the above-mentioned method embodiments.

[0126] It should be understood that the processor mentioned in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0127] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (Direct Rambus RAM, DR RAM).

[0128] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0129] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0131] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a large-screen device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0135] Finally, it should be noted that the above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for counting the number of people, characterized in that: The method comprises: Acquire a surveillance video, where the surveillance video is a video captured by a shooting device and includes a surveillance area, the surveillance video includes a plurality of video frames, and the surveillance area includes an inner area and an outer area; Using a preset recognition model to recognize the multiple video frames, a plurality of action trajectories are obtained, wherein the multiple action trajectories correspond one-to-one to a plurality of persons in the multiple video frames, and the multiple persons are different from each other; The number of people entering, the number of people leaving, and the number of people passing are determined based on the multiple motion trajectories, the number of people entering is the number of motion trajectories whose starting point is in the outer area and whose end point is in the inner area among the multiple motion trajectories, the number of people leaving is the number of motion trajectories whose starting point is in the inner area and whose end point is in the outer area among the multiple motion trajectories, and the number of people passing is the number of motion trajectories whose starting point is in the outer area and whose end point is in the outer area among the multiple motion trajectories.

2. The method for counting the number of people according to claim 1, characterized in that: The method of using a preset recognition model to recognize the multiple video frames and obtain multiple action trajectories includes: Using a preset recognition model, identifying multiple groups of monitoring point sets corresponding to the multiple video frames, the multiple groups of monitoring point sets corresponding to the multiple video frames one-to-one, a first monitoring point set including at least one monitoring point, the at least one monitoring point being a location point where at least one person in the first video frame is located, the at least one monitoring point corresponding to the at least one person one-to-one, the first video frame being any one of the multiple video frames, and the first monitoring point set being a point set in the multiple groups of monitoring point sets corresponding to the first video frame; Based on the multiple sets of monitoring points, multiple action trajectories are determined.

3. The method for counting the number of people according to claim 2, characterized in that: The adopting of a preset recognition model to identify the multiple groups of monitoring point sets corresponding to the multiple video frames includes: Using a preset recognition model to recognize the first video frame, to obtain the at least one person; Identify at least one head-shoulder frame corresponding to the at least one person, the at least one head-shoulder frame being a rectangular frame where the head and shoulders of the at least one person are located, and the at least one person corresponds to the at least one head-shoulder frame one-to-one; The position coordinates of the center point of each of the at least one head-shoulder frame are determined as the at least one monitoring point, and the first monitoring point set is obtained.

4. The method for counting the number of people according to claim 3, characterized in that: Determining a plurality of action trajectories according to the plurality of monitoring point sets includes: Acquire multiple monitoring points of a first person in the multiple video frames, where the first person is any one of the multiple persons; According to the multiple monitoring points and the multiple shooting times of the multiple video frames where the multiple monitoring points are located, the action trajectories corresponding to the first person are determined to obtain the multiple action trajectories.

5. The method for counting the number of people according to claim 3, characterized in that: The using a preset recognition model to recognize the first video frame to obtain the at least one person includes: Obtain a preset virtual identification frame, the preset virtual identification frame is a polygon, the preset virtual identification frame includes a tri-fold line that has an intersection with any two non-adjacent sides of the polygon, the area of ​​the preset virtual identification frame on one side of the tri-fold line is a virtual inner area, the area of ​​the preset virtual identification frame on the other side of the tri-fold line is a virtual outer area, the preset virtual identification frame corresponds to the monitoring area, the virtual inner area corresponds to the inner area, and the virtual outer area corresponds to the outer area; Based on the preset virtual recognition frame, the preset recognition model is used to recognize the first video frame to obtain the at least one person.

6. The method for counting the number of people according to claim 5, characterized in that: The monitoring area is the area where the entrance and exit of the door or gate are located; If the monitored area is the area where the door is located, the preset virtual identification frame is the border of the door, the first line segment and the second line segment of the trifold line respectively intersect with the border, the first line segment and the second line segment are not connected, the first line segment and the second line segment are respectively connected to the third line segment, and the third line segment is the line segment in the middle position of the trifold line.

7. The method for counting the number of people according to claim 5, characterized in that: The preset virtual identification frame is a quadrilateral, and the area ratio of the virtual inner area to the virtual outer area is within a preset ratio range.

8. The method for counting the number of people according to claim 1, characterized in that: The method further comprises: The number of people entering the store is determined, and the ratio of the sum of the number of people entering the store and the number of people passing by the store is the store entry rate.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is configured to execute a computer program stored in the memory to implement the method as claimed in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method as described in any one of claims 1 to 8 is implemented.

Images