A method and system for detecting when customer service representatives in a business hall get up, based on a smart camera.

Abstract

This invention relates to the field of artificial intelligence detection technology, specifically providing a method and system for detecting whether a customer service representative in a sales office is standing up, based on a smart camera. The aim is to solve the problems of low efficiency and poor real-time performance in existing methods of supervising customer service representative etiquette by playing back surveillance videos. The method involves drawing rectangular frames representing the positions of customers and sales representatives on the smart camera's monitoring screen. The smart camera uses human shape recognition, positioning, and tracking technologies to obtain human shape data. The platform uses this data to calculate whether there are any changes in the human shape within the customer's position frame. Combined with the platform's standing-up detection technology, the system automatically and in real-time detects whether a customer service representative is standing up while attending to a customer, and automatically generates a report. This allows supervisors to understand clearly and intuitively whether sales representatives have performed proper etiquette in real-time, improving the efficiency of supervisors' work and ensuring real-time performance.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence detection technology, specifically relating to a method and system for detecting when a customer service representative in a business hall gets up, based on a smart camera. Background Technology

[0002] Telecommunications, mobile, power, and post office service halls represent the image of state-owned enterprises. The service etiquette of the staff in these halls not only reflects citizens' recognition of the service providers but also, to some extent, reflects the progress of our society and the development of the country. Currently, almost all service halls are equipped with smart cameras to monitor the service etiquette of the staff. However, the traditional method of supervision involves dedicated supervisors reviewing the surveillance video playback to identify errors and non-standard service etiquette by the staff; this method is inefficient and lacks real-time updates. Summary of the Invention

[0003] This invention provides a method and system for detecting when a customer service representative in a business hall gets up, based on a smart camera. It aims to solve the problems of low efficiency and poor real-time performance in the existing technology of supervising the service etiquette of customer service representatives by watching video playback.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present invention provides a method for detecting when a customer service representative in a business hall gets up, based on a smart camera, comprising the following steps:

[0006] S001. Draw a location rectangle for each agent position in the monitoring screen of the smart camera; the location rectangle includes the customer location rectangle and the agent position rectangle;

[0007] S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform.

[0008] S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point;

[0009] S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node.

[0010] S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed.

[0011] S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue.

[0012] S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue.

[0013] S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction.

[0014] S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

[0015] A further solution: In step S001, a planar coordinate system including the x-axis and y-axis is established; the position of the position rectangle in the planar coordinate system is represented as [(x1, y1), (x2, y2)]; (x1, y1) is the position coordinate of the upper left corner of the position rectangle, and (x2, y2) is the position coordinate of the lower right corner of the position rectangle.

[0016] The coordinates (x1, y1) of the top-left corner of the position rectangle and the coordinates (x2, y2) of the bottom-right corner of the position rectangle are normalized to a value between 0 and 10000. The normalization formula is as follows:

[0017]

[0018]

[0019]

[0020]

[0021] Wherein, (x′1, y′1) represents the position coordinates of the upper left corner of the normalized position rectangle; (x′2, y′2) represents the position coordinates of the lower right corner of the normalized position rectangle; W represents the width of the smart camera monitoring screen; and H represents the height of the smart camera monitoring screen.

[0022] Based on the above solution, since the size of the monitoring screen of the smart camera is inconsistent with the size of the detection video screen displayed on the platform, the position coordinates of the upper left corner and the lower right corner of the position rectangle are normalized so that the subsequent calculation of the coordinates is on the same screen size.

[0023] A further solution: In step S002, the human figure data includes the position and human figure ID of all human figures in each frame of the image; and a unique and different ID is assigned to each frame of the image for different customers or different customer service representatives.

[0024] The position of the human figure is represented by a human figure rectangle, and the position of the human figure rectangle in the planar coordinate system is represented as [(x3, y3), (x4, y4)]; (x3, y3) is the position coordinate of the upper left corner of the human figure rectangle, and (x4, y4) is the position coordinate of the lower right corner of the human figure rectangle; the human figure rectangle includes a customer rectangle and a seated employee rectangle; the position coordinates (x3, y3) of the upper left corner of the human figure rectangle and the position coordinates (x4, y4) of the lower right corner of the human figure rectangle are normalized as described in the above scheme; the position coordinates of the upper left corner of the normalized human figure rectangle are (x′3, y′3), and the position coordinates of the lower right corner of the normalized human figure rectangle are (x′4, y′4).

[0025] A further solution: In step S002, the time interval for the smart camera to report the aggregated human data to the platform is 3 seconds; the rule for aggregating the human data is:

[0026] For the same humanoid ID, the humanoid location coordinate data from later times overwrites the humanoid location coordinate data from earlier times.

[0027] For humanoid IDs with different location coordinates, the latest humanoid location data for each ID is aggregated within 3 seconds.

[0028] Based on the above solution, the smart camera aggregates human data from each frame every 3 seconds, reducing the interaction between the smart camera and the platform, alleviating the pressure on the platform, and reducing traffic usage.

[0029] A further solution: In step S003, the platform calculates whether there is a person in each customer location rectangle using the normalized customer rectangle coordinates and the normalized customer location rectangle coordinates; the calculation steps are as follows:

[0030] Step 1: Calculate the area S of the intersection of the normalized customer rectangle and the customer location rectangle. interS inter The calculation formula is:

[0031] S inter =max((min(y′4,y′2)-max(y′3,y′1)),0)*max((min(x′4,x′2)-max(x′3,x′1)),0);

[0032] Wherein, the normalized customer rectangle is represented in the plane coordinate system as [(x′3,y′3), (x′4,y′4)]; the normalized customer location rectangle is represented in the plane coordinate system as [(x′1,y′1), (x′2,y′2)]; min(,) represents the minimum value among two values; max(,) represents the maximum value among two values;

[0033] Step 2: Calculate the area S of the normalized customer location rectangle. B S B The calculation formula is:

[0034] S B =(x′2-x′1)*(y′2-y′1);

[0035] Step 3: Calculate S inter The proportion P of the customer location rectangle after normalization is calculated using the following formula:

[0036]

[0037] If P is greater than a preset threshold, then it is considered that there is a person in the customer location rectangle; otherwise, it is considered that there is no person in the customer location rectangle.

[0038] A further solution: In step S006, the detection data includes the serial number of the detected smart camera device, the start time of the detected smart camera monitoring video, the end time of the detected smart camera monitoring video, and the rectangular frame of the seated employee's position where the standing detection needs to be performed.

[0039] A further solution: In step S008, the monitoring video to be queried is determined based on the serial number of the detected smart camera device, the start time and end time of the detected smart camera monitoring video in the detection data; the frame extraction standard of the monitoring video is to extract one frame every second, and sort them according to the time sequence of the images.

[0040] Based on the above scheme, the smart camera device serial number, smart camera monitoring video start time and end time in the detection data enable the platform to quickly and accurately determine the monitoring video to be queried, improve the efficiency of standing detection, and adjust the duration of the monitoring video to be queried.

[0041] The platform extracts frames from the monitoring video at a frequency of one frame per second, which not only improves the efficiency of standing up detection but also does not reduce the accuracy of standing up detection.

[0042] A further solution: In step S009, the standard for the platform to detect whether someone has stood up in each image extracted from the surveillance video is as follows:

[0043] The platform extracts the heads of the people in the rectangular frame of the seated employee's position in each image, and sequentially judges the change in the position of the heads in the rectangular frame of the seated employee's position in each image; if the head rises from low to high, and the difference between the highest and lowest points is greater than a set threshold, then it is considered that the seated employee has stood up; otherwise, it is considered that the seated employee has not stood up.

[0044] Based on the above scheme, since the actual situation in the business hall is more complicated and there are obstructions, there are situations where human figures or faces cannot be recognized, while heads can almost always be seen. Therefore, the accuracy of detecting changes in the position of heads is higher than that of detecting human figures or faces.

[0045] Further proposed solution: The detection results include whether the agent stood up, relevant screenshots proving that they stood up, and the detection time of whether they stood up.

[0046] Based on the above scheme, the three data points in the test results—whether the call center staff stood up, the relevant screenshots proving that they stood up, and the time of the test—allow supervisors to clearly and directly understand whether the call center staff's service etiquette is standardized, thereby saving time and improving the efficiency of the supervision work.

[0047] Secondly, the present invention provides a system for detecting when a customer service representative in a business hall gets up, based on a smart camera, comprising: a smart camera and a platform, and performing the following steps:

[0048] S001. Draw a location rectangle for each agent position in the monitoring screen of the smart camera; the location rectangle includes the customer location rectangle and the agent position rectangle;

[0049] S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform.

[0050] S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point;

[0051] S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node.

[0052] S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed.

[0053] S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue.

[0054] S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue.

[0055] S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction.

[0056] S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

[0057] The beneficial effects of this invention are as follows:

[0058] In this invention, since the smart camera's monitoring video feed contains a large number of people, customer and agent position rectangles are drawn in the video feed to define customer and agent positions. This allows for the judgment of their behavior based on their identity, accurately determining whether the agent should greet or see off a customer who has come to conduct business or is leaving after completing their transaction. Because the smart camera performs real-time monitoring and acquires a large amount of human data, the smart camera sends aggregated human data to the platform at a certain frequency, reducing the number of interactions between the smart camera and the platform. This effectively alleviates the pressure on the platform without compromising accuracy, ensuring its normal operation. Furthermore, in the standard for determining whether a standing detection is needed, taking the transition from an empty to a occupied state as an example, an empty state lasting 10 seconds avoids misjudging a customer's arrival by others passing by; an occupied state also requires a 10-second duration. This avoids misjudgment by others passing by and provides sufficient time for the customer to walk into the customer position rectangle and sit down. This allows the platform to accurately determine whether a standing detection is needed.

[0059] This invention utilizes human figure recognition, positioning, and tracking technology obtained from the intelligent camera to obtain human figure data. The platform uses this data to calculate whether there are any changes in the human figure within the rectangular frame of each customer's location. Combined with the platform's standing-up detection technology, the platform automatically and in real-time detects whether the sales staff in the business hall have stood up while serving customers, and automatically generates a report. Supervisors can understand whether the sales staff have performed standardized service etiquette in a real-time, clear, and intuitive manner through the report, improving the efficiency of supervisors' work and ensuring real-time performance. Attached Figure Description

[0060] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This invention relates to a method for detecting when a customer service representative in a business hall gets up based on a smart camera, and a flowchart of the method for detecting getting up in the system.

[0062] Figure 2 This is a schematic diagram of a method for detecting when a customer service representative in a business hall gets up based on a smart camera, and a rectangular frame representing the position drawn in the monitoring screen of the smart camera in the system.

[0063] Figure 3This is a schematic diagram of the structure of a method for detecting when a customer service representative in a business hall gets up, based on a smart camera, and the system for detecting when an employee gets up. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the protection scope of the present invention.

[0065] Example 1:

[0066] like Figure 1 As shown, this embodiment provides a method for detecting when a customer service representative in a business hall gets up based on a smart camera, including the following steps:

[0067] S001. Draw a location rectangle for each agent position in the monitoring screen of the smart camera; the location rectangle includes the customer location rectangle and the agent position rectangle;

[0068] Among them, such as Figure 2 As shown, the customer position rectangle for seat 1 is represented as 1-1, and the employee position rectangle is represented as 1-2; the customer position rectangle for seat 2 is represented as 2-1, and the employee position rectangle is represented as 2-2.

[0069] S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform.

[0070] S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point;

[0071] S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node.

[0072] S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed.

[0073] S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue.

[0074] S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue.

[0075] The delay time can be set to 1 minute; because in the subsequent steps, the platform needs to query the monitoring video data after the current time point, so a delay is required.

[0076] S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction.

[0077] S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

[0078] One of the improved solutions is as follows:

[0079] In step S001, a planar coordinate system including the x-axis and y-axis is established; the position of the position rectangle in the planar coordinate system is represented as [(x1, y1), (x2, y2)]; (x1, y1) is the position coordinate of the upper left corner of the position rectangle, and (x2, y2) is the position coordinate of the lower right corner of the position rectangle.

[0080] The coordinates (x1, y1) of the top-left corner and the coordinates (x2, y2) of the bottom-right corner of the position rectangle are normalized to a value between 0 and 10000. The formula for normalization is as follows:

[0081]

[0082]

[0083]

[0084]

[0085] Wherein, (x′1, y′1) represents the position coordinates of the upper left corner of the normalized position rectangle; (x′2, y′2) represents the position coordinates of the lower right corner of the normalized position rectangle; W represents the width of the smart camera monitoring screen; and H represents the height of the smart camera monitoring screen.

[0086] Specifically, a planar coordinate system including the x-axis and y-axis is established, with the upper left corner of the video screen monitored by the smart camera as the origin (0, 0), the positive direction of the x-axis to the right, and the positive direction of the y-axis downward.

[0087] In step S002, the human data includes the location of all human figures in each frame of the image and the human figure ID (Identity Document); a unique and different ID is assigned to each frame of the image for different customers or different customer service representatives.

[0088] The position of the human figure is represented by a human figure rectangle, and the position of the human figure rectangle in the planar coordinate system is represented as [(x3, y3), (x4, y4)]; (x3, y3) is the position coordinate of the upper left corner of the human figure rectangle, and (x4, y4) is the position coordinate of the lower right corner of the human figure rectangle; the human figure rectangle includes a customer rectangle and a seated employee rectangle; the position coordinates of the upper left corner (x3, y3) and the position coordinates of the lower right corner (x4, y4) of the human figure rectangle are normalized as described above; the position coordinates of the upper left corner of the normalized human figure rectangle are (x′3, y′3), and the position coordinates of the lower right corner of the normalized human figure rectangle are (x′4, y′4).

[0089] In step S002, the smart camera reports the aggregated human data to the platform at intervals of 3 seconds; the rules for aggregating the human data are as follows:

[0090] For the same humanoid ID, the humanoid location coordinate data from later times overwrites the humanoid location coordinate data from earlier times.

[0091] For humanoid IDs with different location coordinates, the latest humanoid location data for each ID is aggregated within 3 seconds.

[0092] In step S003, the platform calculates whether there is a person in each customer location rectangle using the normalized customer rectangle coordinates and the normalized customer location rectangle coordinates; the calculation steps are as follows:

[0093] Step 1: Calculate the area S of the intersection of the normalized customer rectangle and the customer location rectangle. inter S inter The calculation formula is:

[0094] S inter =max((min(y′4,y′2)-max(y′3,y′1)),0)*max((min(x′4,x′2)-max(x′3,x′1)),0);

[0095] Wherein, the normalized customer rectangle is represented in the plane coordinate system as [(x′3,y′3), (x′4,y′4)]; the normalized customer location rectangle is represented in the plane coordinate system as [(x′1,y′1), (x′2,y′2)]; min(,) represents the minimum value among two values; max(,) represents the maximum value among two values;

[0096] Step 2: Calculate the area S of the normalized customer location rectangle. B S B The calculation formula is:

[0097] S B =(x′2-x′1)*(y′2-y′1);

[0098] Step 3: Calculate S inter The proportion P of the customer location rectangle after normalization is calculated using the following formula:

[0099]

[0100] If P is greater than a preset threshold, then it is considered that there is a person in the customer location rectangle; otherwise, it is considered that there is no person in the customer location rectangle.

[0101] In step S006, the detection data includes the serial number of the detected smart camera device, the start time of the detected smart camera monitoring video, the end time of the detected smart camera monitoring video, and the rectangular frame of the seated employee's position where the standing detection needs to be performed.

[0102] Specifically, the start time of the detected smart camera surveillance video is 20 seconds prior to the point when the video changes from being occupied to being unoccupied or from being unoccupied to being occupied; the end time of the detected smart camera surveillance video is 5 seconds prior to the point when the video changes from being occupied to being unoccupied or from being unoccupied to being occupied.

[0103] In step S008, the monitoring video to be queried is determined based on the serial number of the smart camera device detected in the detection data, the start time and end time of the detected smart camera monitoring video; the frame extraction standard of the monitoring video is to extract one frame every second, and sort them according to the time sequence of the images.

[0104] Specifically, the smart camera to be queried is determined based on the serial number of the detected smart camera device; the video segment and duration of the monitoring video to be queried are determined based on the start and end times of the monitored video from the detected smart camera; and approximately 25 images are extracted based on the frame extraction standard of the monitoring video.

[0105] In step S009, the standard for the platform to detect whether someone has stood up in each image extracted from the surveillance video is as follows:

[0106] The platform extracts the heads of the people in the rectangular frame of the seated employee's position in each image, and sequentially judges the change in the position of the heads in the rectangular frame of the seated employee's position in each image; if the head rises from low to high, and the difference between the highest and lowest points is greater than a set threshold, then it is considered that the seated employee has stood up; otherwise, it is considered that the seated employee has not stood up.

[0107] Specifically, the platform uses a trained head detection model to detect whether each image extracted from the surveillance video has been stood up.

[0108] The test results include whether the seated staff member stood up, relevant screenshots as proof of standing up, and the time of the test.

[0109] Example 2:

[0110] like Figure 3 As shown, this embodiment provides a system for detecting when a customer service representative in a business hall gets up, based on a smart camera. The system includes a smart camera and a platform, and performs the following steps:

[0111] S001. Draw a location rectangle for each agent position in the monitoring screen of the smart camera; the location rectangle includes the customer location rectangle and the agent position rectangle;

[0112] S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform.

[0113] S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point;

[0114] S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node.

[0115] S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed.

[0116] S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue.

[0117] S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue.

[0118] S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction.

[0119] S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

[0120] This invention is not limited to the above-described optional embodiments. Anyone can derive other various forms of products under the guidance of this invention. However, regardless of any changes made in their shape or structure, any technical solution that falls within the scope of the claims of this invention shall be protected by this invention.

Claims

1. A method for detecting when a customer service representative in a business hall gets up, based on a smart camera, characterized in that: Includes the following steps: S001. Draw a position rectangle for each seat position in the monitoring screen of the smart camera; The location rectangle includes the customer location rectangle and the agent location rectangle; S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform. S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point; In step S003, the platform calculates whether there is a person in each customer location rectangle using the normalized customer rectangle coordinates and the normalized customer location rectangle coordinates; The calculation steps are as follows: Step 1: Calculate the area S of the intersection of the normalized customer rectangle and the customer location rectangle. inter S inter The calculation formula is: S inter <max((min(y′4, y′2)-max(y′3, y′1))),0)*max((min(x′4, x′2)-max(x′3, x′1)),0); Wherein, the normalized customer rectangle is represented in the plane coordinate system as [(x′3,y′3), (x′4,y′4)]; the normalized customer location rectangle is represented in the plane coordinate system as [(x′1,y′1), (x′2,y′2)]; min(,) represents the minimum value among two values; max(,) represents the maximum value among two values; Step 2: Calculate the area S of the normalized customer location rectangle. B S B The calculation formula is: S B =(x′2-x′1)*(y′2-y′1); Step 3: Calculate S inter The proportion P of the customer location rectangle after normalization is calculated using the following formula: If P is greater than a preset threshold, then it is considered that there is a person in the customer location rectangle; otherwise, it is considered that there is no person in the customer location rectangle. S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node. S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed. S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue. S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue. S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction. S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

2. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 1, is characterized in that: In step S001, a planar coordinate system including the x-axis and y-axis is established; the position of the position rectangle in the planar coordinate system is represented as [(x1, y1), (x2, y2)]; (x1, y1) is the position coordinate of the upper left corner of the position rectangle, and (x2, y2) is the position coordinate of the lower right corner of the position rectangle; The coordinates (x1, y1) of the top-left corner and the coordinates (x2, y2) of the bottom-right corner of the position rectangle are normalized to a value between 0 and 10000. The formula for normalization is as follows: Wherein, (x1′, y1′) represents the position coordinates of the upper left corner of the normalized position rectangle; (x2′, y2′) represents the position coordinates of the lower right corner of the normalized position rectangle; W represents the width of the smart camera monitoring screen; and H represents the height of the smart camera monitoring screen.

3. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 2, is characterized in that: In step S002, the human figure data includes the position and human figure ID of all human figures in each frame of the image; and a unique and different ID is assigned to each frame of the image for different customers or different customer service representatives. The position of the human figure is represented by a human figure rectangle, and the position of the human figure rectangle in the planar coordinate system is represented as [(x3, y3), (x4, y4)]; (x3, y3) is the position coordinate of the upper left corner of the human figure rectangle, and (x4, y4) is the position coordinate of the lower right corner of the human figure rectangle; the human figure rectangle includes a customer rectangle and a seated employee rectangle; the position coordinates (x3, y3) of the upper left corner of the human figure rectangle and the position coordinates (x4, y4) of the lower right corner of the human figure rectangle are normalized as described in claim 2; the position coordinates of the upper left corner of the normalized human figure rectangle are (x′3, y′3), and the position coordinates of the lower right corner of the normalized human figure rectangle are (x′4, y′4).

4. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 3, is characterized in that: In step S002, the smart camera reports the aggregated human data to the platform at intervals of 3 seconds; the rules for aggregating the human data are as follows: For the same humanoid ID, the humanoid location coordinate data from later times will overwrite the humanoid location coordinate data from earlier times. For humanoid IDs with different location coordinates, the latest humanoid location data for each ID is aggregated within 3 seconds.

5. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 1, is characterized in that: In step S006, the detection data includes the serial number of the detected smart camera device, the start time of the detected smart camera monitoring video, the end time of the detected smart camera monitoring video, and the rectangular frame of the seated employee's position where the standing detection needs to be performed.

6. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 5, is characterized in that: In step S008, the monitoring video to be queried is determined based on the serial number of the smart camera device detected in the detection data, the start time and end time of the detected smart camera monitoring video; the frame extraction standard of the monitoring video is to extract one frame every second, and sort them according to the time sequence of the images.

7. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 1, is characterized in that: In step S009, the standard for the platform to detect whether someone has stood up in each image extracted from the surveillance video is as follows: The platform extracts the heads of the people in the rectangular frame of the seated employee's position in each image, and sequentially judges the change in the position of the heads in the rectangular frame of the seated employee's position in each image; if the head rises from low to high, and the difference between the highest and lowest points is greater than a set threshold, then it is considered that the seated employee has stood up; otherwise, it is considered that the seated employee has not stood up.

8. The method for detecting when a customer service representative in a business hall gets up based on a smart camera, as described in claim 7, is characterized in that: The test results include whether the seated staff member stood up, relevant screenshots as proof of standing up, and the time of the test.

9. A system for detecting when a customer service representative in a business hall gets up, based on a smart camera, characterized in that: include: The smart camera and platform will then perform the following steps: S001. Draw a position rectangle for each seat position in the monitoring screen of the smart camera; The location rectangle includes the customer location rectangle and the agent location rectangle; S002. The intelligent camera performs human figure recognition, positioning and tracking on each frame of the captured image to obtain human figure data, and periodically reports the aggregated human figure data to the platform. S003. After receiving the aggregated human data, the platform calculates whether there is a person in each of the customer location rectangles at the current time point; In step S003, the platform calculates whether there is a person in each customer location rectangle using the normalized customer rectangle coordinates and the normalized customer location rectangle coordinates; The calculation steps are as follows: Step 1: Calculate the area S of the intersection of the normalized customer rectangle and the customer location rectangle. inter S inter The calculation formula is: S inter <max((min(y′4, y′2)-max(y′3, y′1))),0)*max((min(x′4, x′2)-max(x′3, x′1)),0); Wherein, the normalized customer rectangle is represented in the plane coordinate system as [(x′3,y′3), (x′4,y′4)]; the normalized customer location rectangle is represented in the plane coordinate system as [(x′1,y′1), (x′2,y′2)]; min(,) represents the minimum value among two values; max(,) represents the maximum value among two values; Step 2: Calculate the area S of the normalized customer location rectangle. B S B The calculation formula is: S B =(x′2-x′1)*(y′2-y′1); Step 3: Calculate S inter The proportion P of the customer location rectangle after normalization is calculated using the following formula: If P is greater than a preset threshold, then it is considered that there is a person in the customer location rectangle; otherwise, it is considered that there is no person in the customer location rectangle. S004. The platform calculates whether the human figure in each customer location rectangle has changed based on whether there is a person in each customer location rectangle in the current time node and in combination with whether there is a person in each customer location rectangle in the previous time node. S005. The platform determines whether to perform a stand-up detection based on whether the human figure in each customer location rectangle changes. The criteria for determining whether a stand-up detection is needed are as follows: if the state changes from empty to occupied, and the empty state lasts for 10 seconds while the occupied state lasts for 10 seconds, then a stand-up detection is needed; if the state changes from occupied to empty, and the occupied state lasts for 10 seconds while the empty state lasts for 10 seconds, then a stand-up detection is needed; otherwise, a stand-up detection is not needed. S006. If standing up detection is not required, the platform will not be triggered to send the detection data to the delayed message queue. S007. If a standing detection is required, the platform is triggered to send the detection data to the delayed message queue. S008. The platform consumes the detection data in the delayed message queue, queries the monitoring video based on the detection data, and performs frame extraction. S009. The platform performs a detection on each image extracted from the surveillance video to determine whether the person has stood up, outputs and saves the detection results, and generates a report.

Images