Queuing duration evaluation method and device, computer equipment and storage medium

Abstract

The invention relates to a queuing duration evaluation method and device, computer equipment and a storage medium, and can be applied to the fields of artificial intelligence and financial science and technology. The method comprises the following steps: acquiring a figure coordinate of each dynamic target in a monitoring image; performing posture recognition on each dynamic target, and determining a queuing target in the dynamic targets according to a posture recognition result; performing clustering processing on the figure coordinates of each queuing target to determine a queuing team corresponding to each queuing target, and obtaining the number of queuing people of each queuing team; and determining the corresponding relationship between the queuing windows and the queuing teams, and evaluating the queuing duration of each queuing window according to the number of queuing people corresponding to each queuing window. By adopting the method, people in the monitoring image can be detected in real time by utilizing computer vision and a clustering algorithm, queuing people and non-queuing people are accurately distinguished, and the number of queuing people is calculated, so that the queuing time is accurately predicted.

Description

Technical Field

[0001] The present application relates to the field of computer technologies, and particularly to a queuing duration evaluation method, apparatus, computer device, storage medium, and computer program product, which can be applied to the fields of artificial intelligence and fintech. Background Art

[0002] With the development of society, people's life rhythm is getting faster and faster, and their sense of time is getting stronger. In many places, especially in offline service outlets such as banks, hospitals, and government service halls, people often need to queue up to wait for services. However, since it takes a certain amount of time for staff to handle services, there are many people queuing up during peak service periods, resulting in long waiting times. To reduce queuing time and improve the service experience, many people have started to pay attention to the prediction of queuing time so as to plan service time in advance. Currently, there are some methods for predicting queuing time, such as methods based on historical data and methods based on sensors. However, these methods have their own limitations. The method based on historical data can only make inferences based on historical queuing times and it is difficult to accurately obtain the current queuing time; although the method based on sensors can monitor queuing situations in real time, it requires the installation of multiple sensor devices and may be affected by environmental factors, resulting in inaccurate sensing data.

[0003] Currently, the prediction accuracy of queuing time is not high. Summary of the Invention

[0004] Based on this, it is necessary to provide a queuing duration evaluation method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the prediction accuracy of queuing time for the above technical problems.

[0005] In a first aspect, the present application provides a queuing duration evaluation method, including:

[0006] Obtain the human coordinates of each moving target in a monitored image; the monitored image includes at least one moving target and at least one queuing window;

[0007] Perform pose recognition on each moving target, and determine queuing targets among the moving targets according to the pose recognition results;

[0008] Perform clustering processing on the human coordinates of each queuing target to determine the queuing queue corresponding to each queuing target, and obtain the number of people queuing in each queuing queue;

[0009] Determine the corresponding relationship between the queuing window and the queuing queue, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0010] In one embodiment, before obtaining the person coordinates of each dynamic target in the surveillance image, the following steps are further included:

[0011] Obtain the surveillance video of the target scene and acquire video frame images from the surveillance video;

[0012] According to the calibrated pixel points on the video frame image and the geographical coordinates corresponding to the calibrated pixel points and the geographical space coordinates, obtain the spatial two-dimensional coordinate mapping model between the surveillance video and the target scene; the spatial two-dimensional coordinate mapping model is used to determine the person coordinates of the dynamic targets in the surveillance image.

[0013] In one embodiment, obtaining the person coordinates of each dynamic target in the surveillance image includes:

[0014] Process the surveillance image using a human pose estimation model to obtain an anchor box corresponding to each dynamic target;

[0015] According to the anchor box corresponding to each dynamic target, determine the two-dimensional spatial coordinates of each dynamic target as the person coordinates of each dynamic target.

[0016] In one embodiment, perform pose recognition on each dynamic target, and according to the pose recognition result, determine the queuing targets among the dynamic targets, including:

[0017] Process the surveillance image using a human pose estimation model to obtain an anchor box corresponding to each dynamic target;

[0018] In the anchor box corresponding to each dynamic target, obtain the key point coordinates of the dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates, and ankle coordinates;

[0019] According to the key point coordinates of each dynamic target, perform pose recognition on each dynamic target respectively;

[0020] In the case where the pose recognition result is a standing pose, regard the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, regard the corresponding dynamic target as a non-queuing target.

[0021] In one embodiment, perform clustering processing on the person coordinates of each queuing target to determine the queuing team corresponding to each queuing target and obtain the number of people queuing in each queuing team, including:

[0022] Configure a density-based clustering algorithm according to the preset neighborhood radius parameter;

[0023] Respectively take the person coordinates of each queuing target as a data point, and use the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point;

[0024] Take each cluster as a queue, and determine the number of people queuing in each queue according to the number of data points included in each cluster respectively.

[0025] In one embodiment, determining the correspondence between the queuing window and the queue, and evaluating the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window, includes:

[0026] Obtain the window coordinates of each queuing window in the monitoring image;

[0027] Calculate the average person coordinates of each queue according to the person coordinates of the queuing targets in each queue respectively;

[0028] Determine the correspondence between the queuing window and the queue according to the average person coordinates of each queue and the window coordinates of each queuing window;

[0029] Obtain the reference waiting duration of each queuing window, and evaluate the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of people queuing respectively.

[0030] In a second aspect, the present application also provides a queuing duration evaluation device, including:

[0031] An acquisition module, configured to acquire the person coordinates of each moving target in the monitoring image; the monitoring image includes at least one moving target and at least one queuing window;

[0032] An identification module, configured to perform pose identification on each moving target, and determine a queuing target among the moving targets according to the pose identification result;

[0033] A clustering module, configured to perform clustering processing on the person coordinates of each queuing target to determine the queue corresponding to each queuing target, and obtain the number of people queuing in each queue;

[0034] An evaluation module, configured to determine the correspondence between the queuing window and the queue, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0035] In a third aspect, the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0036] Obtain the person coordinates of each moving target in the monitoring image; the monitoring image includes at least one moving target and at least one queuing window;

[0037] Perform pose recognition on each dynamic target, and determine queuing targets among the dynamic targets according to the pose recognition results;

[0038] Perform clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtain the number of people queuing in each queuing line;

[0039] Determine the corresponding relationship between the queuing windows and the queuing lines, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0040] Fourthly, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0041] Obtain the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window;

[0042] Perform pose recognition on each dynamic target, and determine queuing targets among the dynamic targets according to the pose recognition results;

[0043] Perform clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtain the number of people queuing in each queuing line;

[0044] Determine the corresponding relationship between the queuing windows and the queuing lines, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0045] Fifthly, the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0046] Obtain the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window;

[0047] Perform pose recognition on each dynamic target, and determine queuing targets among the dynamic targets according to the pose recognition results;

[0048] Perform clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtain the number of people queuing in each queuing line;

[0049] Determine the corresponding relationship between the queuing windows and the queuing lines, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0050] The above queuing duration evaluation method, device, computer device, storage medium, and computer program product obtain the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window; perform pose recognition on each dynamic target, and determine the queuing target among the dynamic targets according to the pose recognition result; perform clustering processing on the person coordinates of each queuing target to determine the queuing team corresponding to each queuing target, and obtain the number of people queuing in each queuing team; determine the corresponding relationship between the queuing window and the queuing team, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window. Using computer vision and clustering algorithms to detect people in the monitoring image in real time, accurately distinguish queuing people from non-queuing people, calculate the number of people queuing, and thus accurately predict the queuing time. Brief Description of the Drawings

[0051] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0052] Figure 1 It is an application environment diagram of the queuing duration evaluation method in an embodiment;

[0053] Figure 2 It is a flowchart of the queuing duration evaluation method in an embodiment;

[0054] Figure 3 It is an effect diagram of the queuing duration evaluation method in an embodiment;

[0055] Figure 4 It is a structural block diagram of the queuing duration evaluation device in an embodiment;

[0056] Figure 5 It is an internal structure diagram of a computer device in an embodiment. Detailed Description of the Embodiments

[0057] In order to make the objectives, technical solutions, and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The queuing duration evaluation method provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0059] In an exemplary embodiment, as Figure 2 shown, a queuing duration evaluation method is provided. Taking the method applied to Figure 1 the terminal 102 in it as an example for illustration, it includes the following steps 202 to step 208. Among them:

[0060] Step 202, obtain the human coordinates of each moving target in the monitoring image; the monitoring image includes at least one moving target and at least one queuing window.

[0061] Among them, the moving target refers to the people in the monitoring image, and the queuing window can be, but is not limited to, bank business windows, cafeteria food collection windows, cinema ticket check-in gates, and supermarket cash desks.

[0062] Optionally, first construct a mapping model between the video monitoring image and the geospatial data to obtain the two-dimensional human coordinate points where each person in the monitoring image is located, as Figure 3 shown.

[0063] Step 204, perform pose recognition on each moving target, and determine the queuing target among the moving targets according to the pose recognition result.

[0064] Optionally, taking the cafeteria as an example, since the queuing line at the cafeteria window is close to the dining location, it is necessary to obtain the human pose according to the human pose estimation algorithm, as Figure 3 shown, by distinguishing the differences between standing and sitting postures, it is judged whether the current person is queuing for meals or sitting and dining.

[0065] Step 206, perform clustering processing on the human coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtain the number of people queuing in each queuing line.

[0066] Optionally, continuing with the above example, as Figure 3As shown, by virtue of the position information of the people queuing for meals, the DBSCAN clustering algorithm can be used to cluster each person's coordinates into clusters, determine multiple queues based on the multiple clusters obtained by clustering, and then determine the number of people queuing in each queue according to the number of person coordinates included in each cluster.

[0067] Step 208: Determine the correspondence between the queuing windows and the queues, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0068] Optionally, according to the distance between the queuing window and the queue, determine the queuing window corresponding to each queue. In this way, the queuing waiting time can be predicted according to the reference waiting duration for processing a customer transaction of each queuing window and the corresponding number of queuing people. Among them, the reference waiting duration can be, but is not limited to, the service processing duration, the preparation duration for selling food, the ticket issuing duration, the charging duration, which specifically depends on the queuing window and is not limited here.

[0069] In the above queuing duration evaluation method, obtain the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window; perform pose recognition on each dynamic target, and determine the queuing target among the dynamic targets according to the pose recognition result; perform clustering processing on the person coordinates of each queuing target to determine the queue corresponding to each queuing target and obtain the number of people queuing in each queue; determine the correspondence between the queuing window and the queue, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window. Use computer vision and clustering algorithms to detect people in the monitoring image in real time, accurately distinguish queuing people from non-queuing people, calculate the number of queuing people, and thus accurately predict the queuing time. Through this method, people can plan the queuing time in advance and avoid long waits.

[0070] In one embodiment, before obtaining the person coordinates of each dynamic target in the monitoring image, it further includes: obtaining the monitoring video of the target scene and obtaining the video frame image from the monitoring video; obtaining the spatial two-dimensional coordinate mapping model between the monitoring video and the target scene according to the calibration pixel points on the video frame image and the geographical coordinates corresponding to the calibration pixel points; the spatial two-dimensional coordinate mapping model is used to determine the person coordinates of the dynamic target in the monitoring image.

[0071] Optionally, the monitored video information projects the real geospatial entity information onto a two-dimensional image plane. Since the imaging is two-dimensional, the three-dimensional spatio-temporal information contained in the real geospatial entity is lost. By establishing a two-dimensional coordinate mapping model between the video surveillance image and the geospatial space, the conversion between the two-dimensional geospatial coordinates and the two-dimensional image pixel coordinates can be realized. Since the camera is fixed and will capture the same geoscene for a long time, that is, the background of the surveillance scene is unchanged, and only the dynamic foreground targets change. Therefore, map the unchanged background to the geospatial data, find the geospatial coordinates corresponding to each pixel point of the background on the image, and on this basis, the position information of the dynamic targets in a single-frame image can be extracted.

[0072] In this embodiment, by establishing a mapping model between the surveillance image and the two-dimensional space coordinates, the conversion between the two-dimensional geospatial coordinates and the two-dimensional image pixel coordinates can be realized.

[0073] In one embodiment, obtaining the person coordinates of each dynamic target in the surveillance image includes: processing the surveillance image using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; determining the two-dimensional space coordinates of each dynamic target according to the anchor box corresponding to each dynamic target as the person coordinates of each dynamic target.

[0074] Furthermore, perform pose recognition on each dynamic target, and determine queuing targets among the dynamic targets according to the pose recognition results, including: processing the surveillance image using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; respectively obtaining the key point coordinates of the dynamic target in the anchor box corresponding to each dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates, and ankle coordinates; performing pose recognition on each dynamic target respectively according to the key point coordinates of each dynamic target; in the case where the pose recognition result is a standing pose, taking the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, taking the corresponding dynamic target as a non-queuing target.

[0075] Optionally, YOLO-NAS Pose is a deep learning-based human pose estimation model that combines the advantages of two models, YOLOv3 and NASNet. YOLOv3 is an anchor-based object detection model that can quickly and accurately detect objects in images. NASNet is an automated machine learning model that can automatically search for the optimal network architecture. YOLO-NAS Pose combines these two models, enabling it to not only quickly detect objects in images but also automatically generate suitable anchor boxes based on the shape and size of the objects, thereby improving the accuracy of pose estimation. The specific workflow of YOLO-NAS Pose is as follows: First, YOLOv3 processes the input image to obtain a series of candidate boxes. Then, YOLO-NAS Pose automatically generates suitable anchor boxes based on the size and shape of the candidate boxes. Next, YOLO-NAS Pose uses NASNet to perform pose estimation on each candidate box to obtain the key points of each person. Finally, YOLO-NAS Pose integrates all the key points of the people to obtain the final pose estimation result. According to the position of the anchor box in the surveillance image, through a two-dimensional coordinate mapping model, the two-dimensional coordinate points where the person is located are calculated; based on the coordinate information of the shoulders, hips, knees, ankles, etc. obtained from the pose estimation, it is determined whether the person is in a standing or sitting pose.

[0076] In this embodiment, a deep learning-based human pose estimation model is adopted, which can identify the coordinates of each dynamic target person in the surveillance image and perform pose recognition on each dynamic target. According to the pose recognition result, queuing targets can be determined among the dynamic targets.

[0077] In one embodiment, clustering processing is performed on the person coordinates of each queuing target to determine the queuing team corresponding to each queuing target and obtain the number of people queuing in each queuing team, including: configuring a density-based clustering algorithm according to a preset neighborhood radius parameter; respectively taking the person coordinates of each queuing target as a data point, and using the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; respectively taking each cluster as a queuing team, and respectively determining the number of people queuing in each queuing team according to the number of data points included in each cluster.

[0078] Optionally, the DBSCAN clustering algorithm is a density-based clustering algorithm that can divide high-density regions in a dataset into different clusters. The workflow of the DBSCAN clustering algorithm is as follows: First, define a parameter ε representing the neighborhood radius. Then, for each data point, check if there are enough data points in its neighborhood. If so, consider this data point as a core point. Next, for each core point, find all the data points in its neighborhood and add these data points to the same cluster. Finally, repeat the above process until all data points are assigned to a certain cluster. The DBSCAN clustering algorithm does not require specifying the number of clusters in advance but automatically determines the number of clusters according to the characteristics of the dataset; it can handle clusters of any shape without assuming that the shape of the cluster is circular or spherical, making it suitable for clustering in a queue scenario; it can also handle discontinuous data without misjudging discontinuous data as part of a cluster. Therefore, obtain the two-dimensional coordinates of all people in a standing posture and regard them as points in a plane. Use the DBSCAN clustering algorithm to cluster all the above points. The clustering result can be regarded as queues in multiple windows, and obtain the number of people queuing in each queue.

[0079] In this embodiment, by using a clustering algorithm to cluster the coordinate points of each person in the queuing target, it is possible to distinguish queues for each person's coordinates, thereby calculating the number of people queuing in each queue.

[0080] In one embodiment, to determine the correspondence between queuing windows and queues, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window, the following steps are included: Obtain the window coordinates of each queuing window in the monitoring image; calculate the average coordinate of each person in each queuing queue according to the coordinate of each person in the queuing queue respectively; determine the correspondence between the queuing window and the queuing queue according to the average coordinate of each queuing queue and the window coordinates of each queuing window; obtain the reference waiting duration of each queuing window, and evaluate the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of people queuing respectively.

[0081] Optionally, according to the position of the average coordinate of the people in the queue, determine which window in the cafeteria it belongs to, and combine the average duration of selling food at this window. This average duration can be judged according to historical experience or set manually. Combining the number of people queuing, the queuing waiting duration can be obtained.

[0082] In this embodiment, the human pose estimation model based on deep learning can accurately identify and locate human poses; using the DBSCAN clustering algorithm, it can automatically identify and group queuing queues without manual intervention, improving the accuracy of the number of people queuing. Finally, combining the historical experience data of each window and the detected number of people queuing, the queuing waiting duration can be predicted more accurately.

[0083] In an exemplary embodiment, a method for evaluating the queuing duration of banking services includes:

[0084] Obtain the surveillance video of the banking service hall, and acquire video frame images from the surveillance video; according to the calibrated pixel points on the video frame images and the geographical coordinates corresponding to the calibrated pixel points and the geographical space coordinates, obtain the spatial two-dimensional coordinate mapping model between the surveillance video and the banking service hall; the spatial two-dimensional coordinate mapping model is used to determine the human coordinates of the dynamic targets in the surveillance image.

[0085] Use the human pose estimation model to process the surveillance image to obtain an anchor box corresponding to each dynamic target; according to the anchor box corresponding to each dynamic target, determine the two-dimensional spatial coordinates of each dynamic target as the human coordinates of each dynamic target. The surveillance image includes at least one dynamic target and at least one service handling window.

[0086] Use the human pose estimation model to process the surveillance image to obtain an anchor box corresponding to each dynamic target; respectively obtain the key point coordinates of the dynamic target in each anchor box corresponding to the dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates, and ankle coordinates; according to the key point coordinates of each dynamic target, perform pose recognition on each dynamic target respectively; in the case where the pose recognition result is a standing pose, use the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, use the corresponding dynamic target as a non-queuing target.

[0087] Configure the density-based clustering algorithm according to the preset neighborhood radius parameter; respectively use the human coordinates of each queuing target as a data point, and use the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; respectively use each cluster as a queuing line, and respectively determine the number of people queuing in each queuing line according to the number of data points included in each cluster.

[0088] Obtain the window coordinates of each service handling window in the surveillance image; respectively calculate the average human coordinates of each queuing line according to the human coordinates of the queuing targets in each queuing line; according to the average human coordinates of each queuing line and the window coordinates of each service handling window, determine the corresponding relationship between the service handling window and the queuing line; obtain the service handling duration of each service handling window, and respectively evaluate the queuing duration of each service handling window according to the service handling duration of each service handling window and the corresponding number of queuing people.

[0089] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless specifically stated herein, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0090] Based on the same inventive concept, an embodiment of the present application further provides a queuing duration evaluation device for implementing the queuing duration evaluation method described above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the queuing duration evaluation device provided below can refer to the limitations on the queuing duration evaluation method in the above text, and will not be repeated here.

[0091] In an exemplary embodiment, as Figure 4 shown, a queuing duration evaluation device 400 is provided, including: an acquisition module 401, an identification module 402, a clustering module 403, and an evaluation module 404, where:

[0092] The acquisition module 401 is configured to acquire the human coordinates of each moving target in the monitoring image; the monitoring image includes at least one moving target and at least one queuing window;

[0093] The identification module 402 is configured to perform pose recognition on each moving target, and determine a queuing target among the moving targets according to the pose recognition result;

[0094] The clustering module 403 is configured to perform clustering processing on the human coordinates of each queuing target to determine the queuing team corresponding to each queuing target, and obtain the number of people queuing in each queuing team;

[0095] The evaluation module 404 is configured to determine the corresponding relationship between the queuing window and the queuing team, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0096] In one embodiment, the acquisition module 401 is further configured to acquire a surveillance video of a target scene, and acquire video frame images from the surveillance video; according to the calibrated pixel points on the video frame images and the geographical coordinates corresponding to the calibrated pixel points and the geographical space coordinates, acquire a two-dimensional spatial coordinate mapping model between the surveillance video and the target scene; the two-dimensional spatial coordinate mapping model is used to determine the person coordinates of the dynamic targets in the surveillance image.

[0097] In one embodiment, the acquisition module 401 is further configured to process the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; according to the anchor box corresponding to each dynamic target, determine the two-dimensional spatial coordinates of each dynamic target as the person coordinates of each dynamic target.

[0098] In one embodiment, the recognition module 402 is further configured to process the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; in each anchor box corresponding to each dynamic target, acquire the key point coordinates of the dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates, and ankle coordinates; according to the key point coordinates of each dynamic target, perform pose recognition on each dynamic target respectively; in the case where the pose recognition result is a standing pose, regard the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, regard the corresponding dynamic target as a non-queuing target.

[0099] In one embodiment, the clustering module 403 is further configured to configure a density-based clustering algorithm according to a preset neighborhood radius parameter; respectively use the person coordinates of each queuing target as a data point, and use the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; respectively regard each cluster as a queuing line, and respectively determine the number of people queuing in each queuing line according to the number of data points included in each cluster.

[0100] In one embodiment, the evaluation module 404 is further configured to acquire the window coordinates of each queuing window in the surveillance image; respectively calculate the average person coordinates of each queuing line according to the person coordinates of the queuing targets in each queuing line; according to the average person coordinates of each queuing line and the window coordinates of each queuing window, determine the correspondence between the queuing window and the queuing line; acquire the reference waiting duration of each queuing window, and respectively evaluate the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of queuing people.

[0101] Each module in the above queuing duration evaluation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0102] In an exemplary embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a queuing duration evaluation method.

[0103] Those skilled in the art can understand that Figure 5 the structure shown in

[0104] is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different component layout.

[0105] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining a surveillance video of a target scene, and obtaining video frame images from the surveillance video; obtaining a spatial two-dimensional coordinate mapping model between the surveillance video and the target scene according to the calibrated pixel points on the video frame images and the geographical coordinates corresponding to the calibrated pixel points and the geographical space coordinates; the spatial two-dimensional coordinate mapping model is used to determine the person coordinates of the dynamic targets in the surveillance image.

[0106] In one embodiment, when the processor executes the computer program, the following steps are further implemented: processing the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; determining the two-dimensional spatial coordinates of each dynamic target according to the anchor box corresponding to each dynamic target as the person coordinates of each dynamic target.

[0107] In one embodiment, when the processor executes the computer program, the following steps are further implemented: processing the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; respectively obtaining the key point coordinates of the dynamic target in each anchor box corresponding to each dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates and ankle coordinates; respectively performing pose recognition on each dynamic target according to the key point coordinates of each dynamic target; in the case where the pose recognition result is a standing pose, regarding the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, regarding the corresponding dynamic target as a non-queuing target.

[0108] In one embodiment, when the processor executes the computer program, the following steps are further implemented: configuring a density-based clustering algorithm according to a preset neighborhood radius parameter; respectively using the person coordinates of each queuing target as a data point, and performing clustering processing on all data points by using the density-based clustering algorithm to obtain at least one cluster; each cluster includes at least one data point; respectively regarding each cluster as a queuing line, and respectively determining the number of people queuing in each queuing line according to the number of data points included in each cluster.

[0109] In one embodiment, when the processor executes the computer program, the following steps are further implemented: obtaining the window coordinates of each queuing window in the surveillance image; respectively calculating the average person coordinates of each queuing line according to the person coordinates of the queuing targets in each queuing line; determining the corresponding relationship between the queuing window and the queuing line according to the average person coordinates of each queuing line and the window coordinates of each queuing window; obtaining the reference waiting duration of each queuing window, and respectively evaluating the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of people queuing.

[0110] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: obtaining the person coordinates of each dynamic target in a surveillance image; the surveillance image includes at least one dynamic target and at least one queuing window; performing pose recognition on each dynamic target, and determining a queuing target among the dynamic targets according to the pose recognition result; performing clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtaining the number of people queuing in each queuing line; determining the corresponding relationship between the queuing window and the queuing line, and evaluating the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

[0111] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining a surveillance video of a target scene, and obtaining video frame images from the surveillance video; obtaining a spatial two-dimensional coordinate mapping model between the surveillance video and the target scene according to the calibration pixel points on the video frame images and the geographical coordinates corresponding to the calibration pixel points; the spatial two-dimensional coordinate mapping model is used to determine the person coordinates of the dynamic targets in the surveillance image.

[0112] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; determining the two-dimensional spatial coordinates of each dynamic target according to the anchor box corresponding to each dynamic target as the person coordinates of each dynamic target.

[0113] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the surveillance image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; obtaining the key point coordinates of the dynamic target in each anchor box corresponding to each dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates and ankle coordinates; performing pose recognition on each dynamic target respectively according to the key point coordinates of each dynamic target; in the case where the pose recognition result is a standing pose, taking the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, taking the corresponding dynamic target as a non-queuing target.

[0114] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: configuring a density-based clustering algorithm according to a preset neighborhood radius parameter; respectively taking the person coordinates of each queuing target as a data point, and using the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; respectively taking each cluster as a queuing line, and respectively determining the number of people queuing in each queuing line according to the number of data points included in each cluster.

[0115] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining the window coordinates of each queuing window in the monitoring image; respectively calculating the average person coordinates of each queuing line according to the person coordinates of the queuing targets in each queuing line; determining the corresponding relationship between the queuing window and the queuing line according to the average person coordinates of each queuing line and the window coordinates of each queuing window; obtaining the reference waiting duration of each queuing window, and respectively evaluating the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of queuing people.

[0116] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor, implements the following steps: obtaining the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window; performing pose recognition on each dynamic target, and determining queuing targets among the dynamic targets according to the pose recognition results; performing clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtaining the number of queuing people in each queuing line; determining the corresponding relationship between the queuing window and the queuing line, and evaluating the queuing duration of each queuing window according to the number of queuing people corresponding to each queuing window.

[0117] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining a monitoring video of the target scene, and obtaining video frame images from the monitoring video; obtaining a spatial two-dimensional coordinate mapping model between the monitoring video and the target scene according to the calibrated pixel points on the video frame images and the geographical coordinates corresponding to the calibrated pixel points; the spatial two-dimensional coordinate mapping model is used to determine the person coordinates of the dynamic targets in the monitoring image.

[0118] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the monitoring image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; determining the two-dimensional spatial coordinates of each dynamic target according to the anchor box corresponding to each dynamic target, as the person coordinates of each dynamic target.

[0119] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: processing the monitoring image by using a human body pose estimation model to obtain an anchor box corresponding to each dynamic target; respectively obtaining the key point coordinates of the dynamic target in the anchor box corresponding to each dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates, and ankle coordinates; respectively performing pose recognition on each dynamic target according to the key point coordinates of each dynamic target; in the case where the pose recognition result is a standing pose, regarding the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, regarding the corresponding dynamic target as a non-queuing target.

[0120] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: configuring a density-based clustering algorithm according to a preset neighborhood radius parameter; respectively taking the person coordinates of each queuing target as a data point, and using the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; respectively regarding each cluster as a queuing line, and respectively determining the number of people queuing in each queuing line according to the number of data points included in each cluster.

[0121] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented: obtaining the window coordinates of each queuing window in the monitoring image; respectively calculating the average person coordinates of each queuing line according to the person coordinates of the queuing targets in each queuing line; determining the correspondence between the queuing window and the queuing line according to the average person coordinates of each queuing line and the window coordinates of each queuing window; obtaining the reference waiting duration of each queuing window, and respectively evaluating the queuing duration of each queuing window according to the reference waiting duration of each queuing window and the corresponding number of people queuing.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0123] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0124] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0125] The above-described embodiments only represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application should be subject to the appended claims.

Claims

1. A method for evaluating queuing duration, characterized in that, The method includes: Obtaining the human coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window; Performing pose recognition on each dynamic target, and determining queuing targets among the dynamic targets according to the pose recognition results; Performing clustering processing on the human coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtaining the number of people queuing in each queuing line; Determining the corresponding relationship between the queuing window and the queuing line, and evaluating the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

2. The method according to claim 1, wherein Before obtaining the human coordinates of each dynamic target in the monitoring image, it further includes: Obtaining the monitoring video of the target scene, and obtaining video frame images from the monitoring video; According to the calibrated pixel points on the video frame image and the geographical coordinates corresponding to the calibrated pixel points and the geographical space coordinates, obtaining a spatial two-dimensional coordinate mapping model between the monitoring video and the target scene; the spatial two-dimensional coordinate mapping model is used to determine the human coordinates of the dynamic targets in the monitoring image.

3. The method according to claim 1, wherein The obtaining of the human coordinates of each dynamic target in the monitoring image includes: Processing the monitoring image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; Determining the two-dimensional spatial coordinates of each dynamic target according to the anchor box corresponding to each dynamic target as the human coordinates of each dynamic target.

4. The method according to claim 1, characterized in that The performing of pose recognition on each dynamic target and determining queuing targets among the dynamic targets according to the pose recognition results includes: Processing the monitoring image by using a human pose estimation model to obtain an anchor box corresponding to each dynamic target; Respectively obtaining the key point coordinates of the dynamic target in the anchor box corresponding to each dynamic target; the key point coordinates include shoulder coordinates, hip coordinates, knee coordinates and ankle coordinates; Performing pose recognition on each dynamic target respectively according to the key point coordinates of each dynamic target; In the case where the pose recognition result is a standing pose, taking the corresponding dynamic target as a queuing target, and in the case where the pose recognition result is a sitting pose, taking the corresponding dynamic target as a non-queuing target.

5. The method according to claim 1, characterized in that, The performing of clustering processing on the human coordinates of each queuing target to determine the queuing line corresponding to each queuing target and obtaining the number of people queuing in each queuing line includes: Configuring a density-based clustering algorithm according to a preset neighborhood radius parameter; Respectively taking the human coordinates of each queuing target as a data point, and using the density-based clustering algorithm to perform clustering processing on all data points to obtain at least one cluster; each cluster includes at least one data point; Respectively taking each cluster as a queuing line, and respectively determining the number of people queuing in each queuing line according to the number of data points included in each cluster.

6. The method according to claim 1, wherein The determining of the corresponding relationship between the queuing window and the queuing line includes: Obtaining the window coordinates of each queuing window in the monitoring image; Respectively calculating the average human coordinates of each queuing line according to the human coordinates of the queuing targets in each queuing line; Determine the corresponding relationship between the queuing window and the queuing line according to the average person coordinates of each queuing line and the window coordinates of each queuing window.

7. The method according to claim 1, characterized in that Evaluating the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window includes: Obtain the reference waiting duration of each queuing window, and evaluate the queuing duration of each queuing window respectively according to the reference waiting duration of each queuing window and the corresponding number of people queuing.

8. A queuing duration evaluation device, characterized in that, The device includes: An acquisition module, configured to acquire the person coordinates of each dynamic target in the monitoring image; the monitoring image includes at least one dynamic target and at least one queuing window; An identification module, configured to perform pose identification on each dynamic target, and determine a queuing target among the dynamic targets according to the pose identification result; A clustering module, configured to perform clustering processing on the person coordinates of each queuing target to determine the queuing line corresponding to each queuing target, and obtain the number of people queuing in each queuing line; An evaluation module, configured to determine the corresponding relationship between the queuing window and the queuing line, and evaluate the queuing duration of each queuing window according to the number of people queuing corresponding to each queuing window.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images