A queuing analysis method and apparatus

Abstract

The embodiment of the present disclosure provides a queuing analysis method and device, wherein the method comprises: detecting image frames in a video stream, and determining at least one target human body in a queuing queue in the image frames; tracking the target human bodies detected in a plurality of image frames in the video stream, and determining a tracking identifier of each target human body, wherein the tracking identifier is used for marking the same target human body in different image frames; and determining a queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one image frame. The method can obtain more abundant queuing information when analyzing the queuing, has stronger practicability, and has better adaptability to various queuing scenes.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer vision, and in particular, to a queuing analysis method and device. BACKGROUND

[0002] Queuing events are everywhere in life, such as queuing at the checkout counter of a supermarket, queuing to buy tickets at the subway, queuing for security check, queuing at the canteen, etc. Traditional queuing analysis methods, such as queuing for a number method, are not sufficient for the above various queuing scenarios, and the information about the queuing situation that can be obtained is limited. SUMMARY

[0003] Therefore, the embodiments of the present disclosure provide at least one queuing analysis method and device.

[0004] Specifically, the embodiments of the present disclosure are implemented by the following technical solutions:

[0005] In a first aspect, a queuing analysis method is provided, and the method comprises:

[0006] detecting image frames in a video stream to determine at least one target human body in a queuing queue in the image frames;

[0007] tracking the target human bodies detected in a plurality of the image frames in the video stream to determine a tracking identifier of each of the target human bodies, the tracking identifier being used to mark a same target human body in different image frames;

[0008] determining a queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames.

[0009] In some optional embodiments, the detecting the image frames in the video stream to determine at least one target human body in a queuing queue in the image frames comprises:

[0010] detecting the image frames in the video stream to obtain a detection box of each human body in the image frames;

[0011] in response to the detection box being in a preset queuing area in the image frame, determining that the human body corresponding to the detection box is the target human body in the queuing queue.

[0012] In some optional embodiments, the queuing analysis result comprises the number of people in the queuing queue; and the determining the queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames comprises:

[0013] determining the number of people in the queuing queue at a collection time corresponding to the image frame according to the number of tracking identifiers of the target human bodies in the image frame.

[0014] In some optional embodiments, the queuing analysis result comprises a queuing waiting time; and the determining of the queuing analysis result of the queuing queue according to the tracking identification of the target human body in at least one of the image frames comprises:

[0015] For an image frame in which the target human body marked by any tracking identification is at the tail of the queuing queue, determining a queuing start time of the target human body marked by the tracking identification;

[0016] For an image frame in which the target human body marked by any tracking identification is at the head of the queuing queue, determining a queuing end time of the target human body marked by the tracking identification;

[0017] Determining a queuing waiting time according to the queuing start time and the queuing end time of the target human body.

[0018] In some optional embodiments, the queuing analysis result comprises a service using time; and the determining of the queuing analysis result of the queuing queue according to the tracking identification of the target human body in at least one of the image frames comprises:

[0019] For any tracking identification, determining a plurality of image frames in which the target human body marked by the tracking identification is at the head of the queuing queue;

[0020] Determining a capture time of an image frame with the earliest capture time in the plurality of image frames as a service start time of the target human body marked by the tracking identification;

[0021] Determining a service end time of the target human body marked by the tracking identification based on detecting that the target human body marked by the tracking identification leaves the queuing queue;

[0022] Determining a service using time of the target human body according to the service start time and the service end time of the target human body.

[0023] In some optional embodiments, the method further comprises:

[0024] Extracting first feature information of a first target human body in a first image frame, and extracting second feature information of a second target human body in a second image frame adjacent to the first image frame; wherein the first target human body is a human body at the head of the queuing queue in the first image frame, and the second target human body is a human body at the head of the queuing queue in the second image frame;

[0025] In response to a similarity of the first feature information and the second feature information being less than a similarity threshold, determining that the first target human body leaves the queuing queue.

[0026] In some optional embodiments, the method further comprises:

[0027] In response to determining that the second target human body leaves the queuing queue, a service use time corresponding to the second target human body is determined according to a time when the first target human body leaves the queuing queue and a time when the second target human body leaves the queuing queue.

[0028] In some optional embodiments, the determining, according to the tracking identifiers of the target human bodies in at least one of the image frames, of the queuing analysis result of the queuing queue includes:

[0029] In response to the number of the tracking identifiers of the target human bodies in the adjacent image frames being different, and the target human body newly marked by the tracking identifier newly added in the image frame with the later acquisition time not being located at the tail of the queuing queue, it is determined that the target human body newly marked by the tracking identifier is cutting in line in the queuing queue.

[0030] In a second aspect, a queuing analysis apparatus is provided, and the apparatus includes:

[0031] a human body detection module configured to detect image frames in a video stream, and determine at least one target human body in a queuing queue in the image frames;

[0032] a human body tracking module configured to track the target human bodies detected in a plurality of the image frames in the video stream, and determine a tracking identifier of each of the target human bodies, the tracking identifier being used to mark a same target human body in different image frames;

[0033] a result analysis module configured to determine a queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames.

[0034] In a third aspect, an electronic device is provided, and the device includes a memory and a processor, the memory being configured to store computer instructions executable on the processor, and the processor being configured to implement the queuing analysis method according to any one of the embodiments of the present disclosure when the computer instructions are executed.

[0035] In a fourth aspect, a computer readable storage medium is provided, and the medium has stored thereon a computer program, the program being executable by a processor to implement the queuing analysis method according to any one of the embodiments of the present disclosure.

[0036] In a fifth aspect, a computer program product is provided, and the product includes computer programs / instructions, the computer programs / instructions being executable by a processor to implement the queuing analysis method according to any one of the embodiments of the present disclosure.

[0037] The queuing analysis method provided by the technical scheme of the embodiments of the present disclosure can track and analyze the target human body of the queuing queue through the video stream in the queuing scene, can obtain more abundant queuing information, has stronger practicability, is not limited by the queuing scene, has good adaptability to various queuing scenes, can control the queuing information, so as to optimize the allocation of resources such as manpower and material resources for the queuing crowd, thereby greatly improving the service efficiency and reducing the cost. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical scheme in the one or more embodiments of the present disclosure or the related technology, the drawings needed to be used in the embodiment or related technology description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the one or more embodiments of the present disclosure, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0039] Figure 1 is a flowchart of a queuing analysis method according to at least one embodiment of the present disclosure;

[0040] Figure 2 is a flowchart of another queuing analysis method according to at least one embodiment of the present disclosure;

[0041] Figure 2A is a schematic diagram of a subway queuing scene according to at least one embodiment of the present disclosure;

[0042] Figure 2B is a schematic diagram of a queuing scene according to at least one embodiment of the present disclosure;

[0043] Figure 2C is a statistical diagram of similarity according to at least one embodiment of the present disclosure;

[0044] Figure 3 is a block diagram of a queuing analysis device according to at least one embodiment of the present disclosure;

[0045] Figure 4 is a block diagram of another queuing analysis device according to at least one embodiment of the present disclosure;

[0046] Figure 5 is a schematic diagram of a hardware structure of an electronic device according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0048] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0049] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0050] Queuing is a common occurrence in daily life, and service providers need to track user queueing patterns to optimize service configuration. Existing queue management solutions primarily employ two methods: 1. Using queue numbers to count the number of people in line. This method is limited by the scenario and lacks flexibility. It's only suitable for small-scale queues and not for large-scale scenarios like amusement parks or zoos. 2. Using mobile devices to count the number of people in line. This method requires users to actively connect to the mobile network and is also limited by localized spaces, such as restrooms or enclosed venue corridors. Furthermore, since people not queuing can also access the mobile network, the accuracy is low, and the available queue information is limited.

[0051] In view of this, at least one embodiment of the present disclosure provides a queuing analysis method. When performing queuing analysis on a queuing queue, the method analyzes the video stream in the queuing scenario, thus avoiding the limitations of the queuing scenario and allowing control over the queuing information.

[0052] like Figure 1 As shown, Figure 1 This is a flowchart illustrating at least one embodiment of a queuing analysis method, which may include the following processing:

[0053] In step 102, an image frame in a video stream is detected to determine at least one target human body in a queuing queue.

[0054] In this embodiment, the video stream is composed of multiple image frames collected from a queuing scene. The video stream can be obtained by real-time monitoring of the queuing queue, or can be a video recorded after the queuing queue is monitored.

[0055] In this step, the image frame can be detected as a whole, or the queuing area in the image frame can be detected.

[0056] This embodiment does not limit the detection method of the image frame in the video stream. For example, the image frame can be detected by a neural network, or can be detected by other methods.

[0057] In step 104, the target human body detected in the multiple image frames in the video stream is tracked to determine the tracking identifier of each target human body.

[0058] The tracking identifier is used to mark the same target human body in different image frames. The same target human body can exist in different video frames. Tracking the target human body in multiple image frames can determine the position of the same target human body in different image frames and mark the target human body with a tracking identifier.

[0059] This embodiment does not limit the method used to track the target human body. For example, the kalman filter tracking algorithm, the siamsRPN-based tracking algorithm, etc. can be used to track the target human body.

[0060] In step 106, the queuing analysis result of the queuing queue is determined according to the tracking identifier of the target human body in at least one image frame.

[0061] The queuing analysis result can include a preliminary analysis result obtained by analyzing each image frame, and can also include a result obtained by further summarizing the preliminary analysis result.

[0062] For example, the queuing analysis result can include the number of people in the queuing queue. According to the number of tracking identifiers of the target human body in the image frame, the number of people in the queuing queue at the collection time corresponding to the image frame can be determined. Furthermore, the number of people in the queuing queue in multiple image frames of the video stream can be further analyzed to obtain the number of people in the queuing queue in different time periods, so as to obtain information such as the peak value of the number of people in the queuing queue, the peak time, the valley value, and the valley time.

[0063] The queuing analysis method provided by the technical scheme of the embodiments of the present disclosure can track and analyze the target human body in the queuing queue through the video stream in the queuing scene, can obtain richer queuing information, has stronger practicability, is not limited by the queuing scene, has good adaptability to various queuing scenes, can control the queuing information, so as to optimize the allocation of resources such as manpower and material resources for the queuing crowd, thereby greatly improving the service efficiency and reducing the cost.

[0064] Figure 2 The flowchart of the queuing analysis method provided for at least one embodiment of the present disclosure, which combines Figure 2A The process of detecting the image frame in the video stream to perform queuing analysis is described in more detail in the subway queuing scene as shown in the figure. Figure 2 As shown in the figure, the method can include the following processing, and it should be noted that the execution order of each step is not limited in the present embodiment.

[0065] In step 202, the image frame in the video stream is detected to obtain the detection box of each human body in the image frame.

[0066] In this step, the image frame in the video stream can be detected by using a human body detection network, which is a pre-trained neural network for detecting human bodies, and the detection box of each human body in the image frame is obtained. In other embodiments, a human head key point detection network can also be used to detect the image frame in the video stream, and the detection box of the head of each human body in the image frame is obtained. Alternatively, the detection box of other parts of the human body can also be detected, such as the foot, leg, etc.

[0067] The image frame is input into the human body detection network, and the detection box of each human body in the image frame is output, which contains coordinate information for indicating the position of the human body.

[0068] In step 204, in response to the detection box being in the pre-set queuing area in the image frame, it is determined that the human body corresponding to the detection box is the target human body in the queuing queue.

[0069] The position and size of the queuing area can be previously demarcated in the video screen collected by the camera near the subway ticket vending machine. In particular, for a scene where multiple queuing queues may exist, the queuing queue of the queuing area required for queuing analysis can be specified by demarcating the queuing area in the image frame of the video stream.

[0070] Generally, the perspective position of the camera collecting the video stream in the queuing scene is fixed, and the queuing area can be calibrated once for the video stream collected by the camera. One or several edges of the queuing area can also be selected and the direction of the team is indicated.

[0071] In this embodiment, the queue area in the image frame can be an area in a quadrilateral frame composed of black lines as shown in Figure 2A , and the direction of the queue is shown by the arrow. It can also be an area in a frame composed of black lines as shown in Figure 2B , and the direction of the queue is shown by the arrow.

[0072] In this step, whether the target human body corresponding to the detection frame is in the queue can be determined by detecting whether the feature point in the frame is in the queue area. In other examples, the determination can also be made according to whether the edge of the detection frame is in the queue area or the degree of overlap between the detection frame and the queue area.

[0073] The feature point is a point in the internal area of the detection frame or a point on the edge of the detection frame. The selection of the feature point is not limited in this embodiment. For example, the feature point can be the midpoint of the lower edge of the detection frame, as shown by the white circle on the detection frame in 2A. The feature point can also be the lower left corner or the lower right corner of the detection frame.

[0074] For the detection frame with the feature point in the queue area, it is determined that the target human body corresponding to the detection frame is also in the queue area, i.e., the target human body is in the queue.

[0075] In step 206, tracking is performed based on the target human bodies detected in the plurality of image frames in the video stream, and a tracking identifier of each target human body is determined.

[0076] It should be noted that for a target human body in the queue in a certain image frame, the target human body can not be in the queue in other image frames because it has not started to queue, but is in an area outside the queue or outside the queue area in the image frame. Therefore, when tracking the target human body, the detection frame of the target human body in the queue area and the detection frame of the human body around the queue area can be tracked, i.e., a tracking algorithm is used to track each human body in the video image, the same human body at each time is associated, and the tracking identifier (trackID) of each target human body is determined.

[0077] The tracking identifier can be represented by a number, such as the numbers 0-6 in Figure 2A mark the seven target human bodies in the queue. Illustratively, the tracking identifier can be determined according to the time sequence in which the target human body appears in the image of the video stream, such as the target human body 1 marked by the tracking identifier 1 can be earlier than the target human body 5 marked by the tracking identifier 5, but the target human body 1 starts to queue after the target human body 5 enters the queue.

[0078] The position of the target human body marked by the tracking identifier can be determined using the position of the feature point, the position of the center of the region occupied by the detection frame, or the position of other points on the detection frame.

[0079] In step 208, a queuing analysis result of the queuing queue is determined according to the tracking identifier of the target human body in at least one of the image frames.

[0080] By performing sequence analysis on the changes of the tracking identifiers in the queuing area at each time, the position, the serial number, the real-time waiting time, and other information of each target human body in the queuing queue at each time can be obtained. The queuing analysis result can be obtained by analyzing and counting the information.

[0081] For example, the queuing analysis result can include the queuing waiting time and the service using time of each target human body, the number of people in the queuing queue at each time, and the average queuing time and the average service using time.

[0082] For example, the queuing start time of the target human body marked by the tracking identifier can be determined for the image frame in which the target human body marked by the tracking identifier is at the tail of the queuing queue. The queuing end time of the target human body marked by the tracking identifier can be determined for the image frame in which the target human body marked by the tracking identifier is at the head of the queuing queue. The queuing waiting time can be determined according to the queuing start time and the queuing end time of the target human body.

[0083] Specifically, when calculating the queuing waiting time of the target human body, the acquisition time of the image frame with the earliest acquisition time among a plurality of image frames in which the target human body marked by the tracking identifier is at the tail of the queuing queue can be determined as the queuing start time of the target human body marked by the tracking identifier.

[0084] For a plurality of image frames in which the target human body marked by the tracking identifier is at the head of the queuing queue, the acquisition time of the image frame with the earliest acquisition time among the plurality of image frames can be determined as the queuing end time of the target human body marked by the tracking identifier.

[0085] The queuing waiting time of the target human body can be determined according to the queuing start time of the target human body and the queuing end time of the target human body.

[0086] Generally, the target human body located at the end of the out-of-queue direction, i.e., the end of the direction indicated by the arrow, is using the service provided by the subway ticket vending machine, such as ticket taking, ticket purchasing, or query service. This target human body can be considered as the target human body at the head of the queuing queue. On the contrary, the target human body located at the other end of the queue is the target human body at the tail of the queuing queue.

[0087] According to the position of the tracking mark of a certain target person, a plurality of image frames can be determined in which the target person is located at the end of the queue, and the image frame with the earliest acquisition time is the image frame in which the target person marked by the tracking mark just enters the queue, and the acquisition time of the image frame is recorded as the queue start time of the target person.

[0088] Similarly, according to the position of the tracking mark of the target person, a plurality of image frames can be determined in which the target person is located at the head of the queue, and the image frame with the earliest acquisition time is the image frame in which the target person marked by the tracking mark just enters the head of the queue to prepare to use the subway ticket machine, and the acquisition time of the image frame is recorded as the queue end time of the target person, which can also be referred to as the service start time of the target person.

[0089] Subtracting the queue start time from the queue end time of the target person can obtain the queue waiting time of the target person.

[0090] In addition, based on the queue waiting times of a plurality of target persons in the queue, the queue waiting time of a target person who just enters the queue can be estimated. For example, the average queue waiting time of the plurality of target persons can be used as the estimated queue waiting time, or the queue waiting time can be estimated according to an equation fitted according to the relationship between the queue waiting times and the number of waiting persons.

[0091] For example, for any tracking mark, a plurality of image frames can be determined in which the target person marked by the tracking mark is located at the head of the queue.

[0092] The acquisition time of the image frame with the earliest acquisition time in the plurality of image frames is determined as the service start time of the target person marked by the tracking mark.

[0093] Based on the detection that the target person marked by the tracking mark leaves the queue, the service end time of the target person marked by the tracking mark is determined. For example, when the target person appears at the head of the queue for the last time, it is considered that the target person leaves the queue, and the acquisition time of the image frame with the latest acquisition time in the plurality of image frames is determined as the service end time of the target person marked by the tracking mark.

[0094] According to the service start time and the service end time of the target person, the service use time of the target person is determined.

[0095] The method of calculating the service use time is similar to the method of calculating the queue waiting time in the above example, and the queue end time of the target person can be used as the service start time of the target person.

[0096] The position of the tracking mark of the target human body in the image frame in which the target human body is located at the head of the queuing queue is determined according to the target human body, and the image frame with the latest acquisition time is the image frame in which the target human body marked by the tracking mark has used the subway ticket vending machine and is ready to leave the queuing queue, and the acquisition time of the image frame is recorded as the service end time of the target human body.

[0097] The service start time and the service end time of the target human body are subtracted to obtain the service use time of the target human body.

[0098] The queuing waiting time and the service use time of each target human body are calculated, and the average queuing time and the average service use time are further obtained.

[0099] For example, in response to the number of tracking marks of the target human body in adjacent image frames being different, and the target human body marked by the tracking mark newly added in the image frame with the later acquisition time not being located at the tail of the queuing queue, it is determined that the target human body in the queuing queue cuts in line.

[0100] For two adjacent image frames in the video stream, when the number of tracking marks in the queuing queue is different, it is generally because a target human body newly enters the queuing queue, or a target human body newly leaves the queuing queue. Normally, the target human body newly entering the queuing queue should be located at the tail of the queuing queue, if the target human body marked by the tracking mark newly added in the image frame with the later acquisition time is not located at the tail of the queuing queue, it is determined that the target human body newly entering the queuing queue cuts in line.

[0101] The queuing analysis method provided by the technical scheme of the embodiment of the present disclosure can calibrate the position and size of the queuing area in the image frames in the video stream, and track and analyze the target human body in the queuing queue in the queuing area to be detected, which can be applied to various types of queuing scenes, flexibly configure the queuing area to be analyzed, and control the queuing information in a targeted manner, and has good adaptability to various queuing scenes, so as to optimize the allocation of resources such as manpower and material resources for the queuing crowd by the service operator, thereby greatly improving the service efficiency and reducing the cost.

[0102] In the above embodiment, when it is judged whether the target human body at the head of the queuing queue is replaced, that is, whether the target human body is just entering the head of the queuing queue or just leaving the head of the queuing queue, the position of the tracking mark is used for judgment. However, sometimes the tracking mark at the head of the queuing queue will jump, for example, when two target human bodies are very close to each other, or when the human body tracking result is inaccurate, so the method in the above embodiment may make a wrong judgment on whether the target human body at the head is changed, and further affect the accuracy of the queuing analysis result.

[0103] In an embodiment, on the basis of the above-mentioned embodiments, the present disclosure provides a queuing analysis method for making the queuing analysis result more accurate, which adopts a method of comparing the ReID (Person re-identification, pedestrian re-identification) features of the target person at the head of the queuing queue when judging whether the target person at the head of the queuing queue leaves, and after step 104 or step 206 of the above-mentioned embodiments, the method of the above-mentioned embodiments further comprises:

[0104] extracting first feature information of the first target person in the first image frame and second feature information of the second target person in the second image frame, comparing the first feature information and the second feature information, and determining that the first target person leaves the queuing queue in response to the similarity of the first feature information and the second feature information being less than a similarity threshold.

[0105] wherein the first target person is the person at the head of the queuing queue in the first image frame, the second target person is the person at the head of the queuing queue in the second image frame, the first image frame and the second image frame are adjacent image frames in the video stream, and the collection time of the first image frame is before the collection time of the second image frame.

[0106] The first feature information and the second feature information can generally be the ReID feature information of the target person, and the ReID feature can include various types of attributes of the target person, such as clothing, body posture, hairstyle, and font attributes.

[0107] Generally, the people in the queuing queue are divided into three states:

[0108] Enter the queue: for each new trackID appearing at the tail of the queuing queue, we consider it as a new person entering the queuing queue.

[0109] In the queue: the person who continuously exists in the queuing queue is considered to be queuing in the queuing queue.

[0110] Leave the queue: the person who leaves the queuing queue. This is crucial for the analysis and judgment of the queuing queue. Since the trackID may jump, we need to compare the ReID features of the target person at the head of the last frame of the video stream with the ReID features of the target person at the head of the current frame to determine whether the target person at the head of the last frame leaves the queuing queue.

[0111] In a specific implementation, the pedestrian re-identification technology can be used to extract the ReID features of the target person at the head of each image frame in the video stream, and compare the ReID features of the target person at the head of two adjacent image frames to obtain the similarity, i.e., compare the first feature information of the first target person in the first image frame with the second feature information of the second target person in the second image frame. The greater the similarity, the greater the possibility that the first target person and the second target person are the same target person, and vice versa.

[0112] For example, for a certain video stream, the results of comparing the similarity of each image frame with the adjacent previous image frame are as shown in Figure 2C In Figure 2C , the abscissa is the serial number of the image frame, "0" represents the starting point of the video stream, "500" represents the 500th image frame in the video stream, and "2000" represents the 2000th image frame in the video stream. The ordinate is the similarity, and the closer the similarity is to "1.0", the more likely it is that the head of the two adjacent image frames is the same person.

[0113] The similarity threshold can be set to 0.5. If the similarity is less than 0.5, it is considered that the head of the two adjacent image frames is not the same person. The first target person at the head of the previous frame has already left the queue at the time corresponding to the current frame. The collection time corresponding to the image frame of the previous frame can be determined as the service end time of the first target person, and the collection time corresponding to the image frame of the current frame can be determined as the service start time, or the end of queuing time, of the second target person.

[0114] Like the method of determining that the first target person leaves the queuing queue, the second target person can also be determined to leave the queuing queue at a certain image frame corresponding to the collection time.

[0115] In response to determining that the second target person leaves the queuing queue, the service use time of the second target person is determined by subtracting the time when the second target person leaves the queuing queue from the time when the first target person leaves the queuing queue.

[0116] Similarly, the queuing service time of the target person can also be calculated according to the queuing end time of the target person determined in the method in this embodiment and the queuing start time of the target person obtained by the method in the previous embodiment.

[0117] The method of using the ReID feature comparison judges whether the head target human body leaves in the embodiment, effectively avoids the situation that the target human body is judged to be out of the queue due to inaccurate tracking, improves the accuracy of the queue analysis, and thus more accurately counts the queuing waiting time and service use time when analyzing the queuing area, so as to better improve the customer queuing experience according to the queuing analysis result.

[0118] As Figure 3 shown, Figure 3 is a block diagram of a queuing analysis device according to at least one embodiment of the present disclosure, and the device comprises:

[0119] The human body detection module 31 is configured to detect image frames in a video stream and determine at least one target human body in a queuing queue in the image frames.

[0120] The human body tracking module 32 is configured to track the target human bodies detected in a plurality of image frames in the video stream and determine a tracking identifier of each target human body, wherein the tracking identifier is used to mark the same target human body in different image frames.

[0121] The result analysis module 33 is configured to determine a queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one image frame.

[0122] In one example, the human body detection module 31 is specifically configured to detect image frames in a video stream to obtain a detection frame of each human body in the image frames, and determine a human body corresponding to the detection frame as the target human body in the queuing queue in response to the detection frame being in a preset queuing area in the image frames.

[0123] In one example, the queuing analysis result includes the number of people in the queuing queue, and the result analysis module 33 is specifically configured to determine the number of people in the queuing queue at a collection time corresponding to the image frames according to the number of tracking identifiers of the target human bodies in the image frames.

[0124] In one example, the queuing analysis result includes a queuing waiting time, and the result analysis module 33 is specifically configured to determine a queuing start time of a target human body marked by any tracking identifier in an image frame in which the target human body is at the tail of the queuing queue, determine a queuing end time of the target human body marked by any tracking identifier in an image frame in which the target human body is at the head of the queuing queue, and determine the queuing waiting time according to the queuing start time and the queuing end time of the target human body.

[0125] In one example, the queuing analysis results include service usage time; the result analysis module 33 is specifically used for: for any of the tracking identifiers, determining multiple image frames in which the target human body marked by the tracking identifier is at the head of the queuing queue; determining the acquisition time of the image frame with the earliest acquisition time among the multiple image frames as the service start time of the target human body marked by the tracking identifier; determining the service end time of the target human body marked by the tracking identifier based on detecting that the target human body marked by the tracking identifier has left the queuing queue; and determining the service usage time of the target human body according to the service start time and service end time of the target human body.

[0126] In one example, the result analysis module 33 is specifically used to: in response to the fact that the number of tracking markers of the target human body in adjacent image frames is different, and the target human body marked by the newly added tracking marker in the image frame with a later acquisition time is not located at the tail of the queue, determine that the target human body marked by the newly added tracking marker in the queue has jumped the queue.

[0127] In one example, such as Figure 4 As shown, the device further includes a feature comparison module 34.

[0128] Extract first feature information of a first target human body in a first image frame, and extract second feature information of a second target human body in an adjacent second image frame; wherein, the first target human body is the human body at the head of the queue in the first image frame, and the second target human body is the human body at the head of the queue in the second image frame; in response to the similarity between the first feature information and the second feature information being less than a similarity threshold, determine that the first target human body leaves the queue.

[0129] In one example, the result analysis module 33 is specifically used to: in response to determining that the second target human body leaves the queuing queue, determine the service usage time corresponding to the second target human body based on the time when the first target human body leaves the queuing queue and the time when the second target human body leaves the queuing queue.

[0130] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0131] This disclosure also provides an electronic device, such as... Figure 5 As shown, the electronic device includes a memory 51 and a processor 52. The memory 51 is used to store computer instructions that can be run on the processor, and the processor 52 is used to implement the queuing analysis method according to any embodiment of the present disclosure when executing the computer instructions.

[0132] The embodiment of the present disclosure further provides a computer program product, which comprises computer programs / instructions, and the computer programs / instructions realize the queuing analysis method in any embodiment of the present disclosure when executed by a processor.

[0133] The embodiment of the present disclosure further provides a computer readable storage medium, which stores a computer program, and the program realizes the queuing analysis method in any embodiment of the present disclosure when executed by a processor.

[0134] For the device embodiment, since it basically corresponds to the method embodiment, the related parts are described in the part of the method embodiment. The device embodiment described above is only illustrative, wherein the modules described as separate components can or can not be physically separated, and the components shown as modules can or can not be physical modules, that is, they can be located in one place or distributed on multiple network modules. Part or all of the modules can be selected to achieve the purpose of the scheme of the present specification according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0135] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or possible.

[0136] Other embodiments of the present specification will be apparent to those skilled in the art after consideration of the specification and practice of the application disclosed herein. The present specification is intended to cover any variations, uses or adaptations of the present specification following, in general, the principles of the present specification and including such departures from the present specification as come within the known and customary practice of the art to which the present specification pertains or the like. The specification is to be regarded as illustrative, rather than restrictive, and the true scope and spirit of the present specification are indicated by the following claims.

[0137] It should be understood that the present specification is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made to the present specification without departing from the scope thereof. The scope of the present specification is limited only by the claims that follow.

[0138] The above only describes the preferred embodiments of the present specification and does not limit the present specification, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the protection scope of the present specification.

Claims

1. A method of queue analysis, characterized by, The method comprises: detecting image frames in a video stream to determine at least one target human body in a queuing queue in the image frames; tracking the target human bodies detected in a plurality of the image frames in the video stream to determine a tracking identifier of each of the target human bodies, the tracking identifier being used to mark the same target human body in different image frames; the tracking identifier being determined according to the time sequence in which the target human body appears in the frames of the video stream; determining a queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames; the determining of the queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames comprises: in response to the number of the tracking identifiers of the target human bodies in adjacent image frames being different and the target human body marked by the tracking identifier newly added in the image frame collected later not being located at the tail of the queuing queue, determining that the target human body marked by the tracking identifier newly added in the queuing queue cuts in line; extracting first feature information of a first target human body in a first image frame and extracting second feature information of a second target human body in an adjacent second image frame; wherein the first target human body is a human body at the head of the queuing queue in the first image frame, and the second target human body is a human body at the head of the queuing queue in the second image frame; in response to the similarity of the first feature information and the second feature information being less than a similarity threshold, determining that the first target human body leaves the queuing queue; the first feature information and the second feature information are information of pedestrian re-identification features of the target human body.

2. The method of claim 1, wherein, The detecting of the image frames in the video stream to determine at least one target human body in a queuing queue in the image frames comprises: detecting image frames in a video stream to obtain a detection frame of each human body in the image frames; in response to the detection frame being in a preset queuing area in the image frame, determining that the human body corresponding to the detection frame is the target human body in the queuing queue.

3. The method of claim 1, wherein, The queuing analysis result comprises the number of people in the queuing queue; The determining of the queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames comprises: determining the number of people in the queuing queue at the collection time corresponding to the image frame according to the number of the tracking identifiers of the target human bodies in the image frame.

4. The method of claim 1, wherein, The queuing analysis result comprises queuing waiting time; the determining of the queuing analysis result of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames comprises: for any image frame in which a target human body marked by a tracking identifier is at the tail of the queuing queue, determining the queuing start time of the target human body marked by the tracking identifier; for any image frame in which a target human body marked by a tracking identifier is at the head of the queuing queue, determining the queuing end time of the target human body marked by the tracking identifier; determining the queuing waiting time according to the queuing start time and the queuing end time of the target human body.

5. The method of claim 1, wherein, The queuing analysis result comprises service use time; The method further comprises: In response to determining that the second target human body leaves the queuing queue, determining the service use time corresponding to the second target human body according to the time when the first target human body leaves the queuing queue and the time when the second target human body leaves the queuing queue. The device comprises: A human body detection module configured to detect image frames in a video stream and determine at least one target human body in a queuing queue in the image frames; A human body tracking module configured to track the target human bodies detected in a plurality of the image frames in the video stream and determine a tracking identifier for each of the target human bodies, the tracking identifier being used to mark a same target human body in different image frames; the tracking identifier is determined according to the time sequence in which the target human body appears in the video stream; 6. The method of claim 1, wherein, A result analysis module configured to determine queuing analysis results of the queuing queue according to the tracking identifiers of the target human bodies in at least one of the image frames; The result analysis module is specifically configured to: in response to the number of the tracking identifiers of the target human bodies in adjacent image frames being different and a target human body marked by a newly added tracking identifier not being located at the tail of the queuing queue in an image frame collected later in time, determine that the target human body marked by the newly added tracking identifier cuts in line in the queuing queue.

7. A queuing analysis apparatus characterized by comprising: The device further comprises: A feature comparison module configured to extract first feature information of a first target human body in a first image frame and extract second feature information of a second target human body in an adjacent second image frame; the first target human body is a human body at the head of a queuing queue in the first image frame, and the second target human body is a human body at the head of a queuing queue in the second image frame; in response to the similarity between the first feature information and the second feature information being less than a similarity threshold, it is determined that the first target human body leaves the queuing queue; the first feature information and the second feature information are information of pedestrian re-identification features of target human bodies. The device comprises a memory and a processor, the memory is configured to store computer instructions executable on the processor, and the processor is configured to implement the method of any one of claims 1 to 6 when executing the computer instructions. The computer program / instructions implement the method of any one of claims 1 to 6 when executed by a processor. The program implements the method of any one of claims 1 to 6 when executed by a processor. The program implements the method of any one of claims 1 to 6 when executed by a processor. ​ 8. An electronic device, comprising: ​ 9. A computer program product comprising computer programs / instructions, characterized in that, ​ 10. A computer-readable storage medium having stored thereon a computer program, characterized in that, ​

Images