Intelligent prompting system and method for business hall, storage medium and computer equipment

Abstract

The invention discloses a business hall intelligent prompting system and method, a storage medium and computer equipment. The system comprises an image acquisition module, an image comprehensive processing module and a service management and control module. The image acquisition module is used for performing image acquisition operation based on each image acquisition device deployed in the business hall to obtain an image acquisition result corresponding to each image acquisition device, and pushing each image acquisition result to the image comprehensive processing module; the image comprehensive processing module is used for respectively calling the image processing units corresponding to the image acquisition devices based on the image acquisition devices, acquiring supervision objects in corresponding image acquisition results by using the image processing units, determining the supervision results of the supervision objects based on preset supervision standards of the image acquisition results, and sending the supervision results to the image acquisition devices; the supervision result is sent to the service management and control module; and the service management and control module is used for identifying the supervision terminal corresponding to each supervision result, and generating supervision prompt information of each supervision terminal based on the identification result and the supervision result.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly to an intelligent prompting system and method for business halls, a storage medium, and a computer device. Background Art

[0002] With the rapid development of information technology, especially the wide application of artificial intelligence and big data technologies, the operation efficiency and service quality of all walks of life have been significantly improved. In the service industry, especially in business hall scenarios such as financial services, power services, and public services, the customer experience and service quality are directly related to the enterprise's brand image and market competitiveness. The traditional business hall service mode relies on manual monitoring and management, and has problems such as lagging response, incomplete monitoring, and low service efficiency. Therefore, how to use modern technical means to achieve intelligent management of business halls and improve service quality and customer satisfaction has become an urgent problem to be solved. Summary of the Invention

[0003] In view of this, this application provides an intelligent prompting system and method for business halls, a storage medium, and a computer device. Through highly intelligent image recognition and processing algorithms, it realizes comprehensive automated monitoring of business handling in business halls, can greatly improve the timeliness and accuracy of supervision, and is conducive to improving the service quality and customer satisfaction of business halls.

[0004] According to one aspect of this application, an intelligent prompting system for business halls is provided, including an image acquisition module, an image comprehensive processing module, and a service control module;

[0005] The image acquisition module is used to perform image acquisition operations on the image acquisition areas corresponding to each image acquisition device based on the image acquisition devices deployed in the business hall, obtain the image acquisition results corresponding to each image acquisition device, and push the image acquisition results to the image comprehensive processing module, where the image acquisition areas include the business handling areas corresponding to each business handler;

[0006] The image comprehensive processing module is used to respectively call the image processing unit corresponding to the image acquisition device based on the image acquisition device corresponding to each image acquisition result, use the image processing unit to obtain the supervision object in the corresponding image acquisition result, determine the supervision result corresponding to the supervision object based on the preset supervision standard corresponding to the image acquisition result, and send the supervision results corresponding to each image acquisition result to the service control module;

[0007] The service control module is used to identify the supervision terminal corresponding to each supervision result, and generate the supervision prompt information corresponding to each supervision terminal based on the identification result and the supervision result, and send the supervision prompt information to the corresponding supervision terminal.

[0008] According to another aspect of the present application, there is provided a smart prompting method for a business hall, including:

[0009] The image acquisition module, based on each image acquisition device deployed in the business hall, performs image acquisition operations on the image acquisition area corresponding to each image acquisition device to obtain the image acquisition results corresponding to each image acquisition device, and pushes each image acquisition result to the image comprehensive processing module, where the image acquisition area includes the business handling areas corresponding to each business handler;

[0010] The image comprehensive processing module, based on the image acquisition device corresponding to each image acquisition result, respectively calls the image processing unit corresponding to the image acquisition device, uses the image processing unit to obtain the supervision objects in the corresponding image acquisition result, determines the supervision results corresponding to the supervision objects based on the preset supervision standards corresponding to the image acquisition result, and sends the supervision results corresponding to each image acquisition result to the service control module;

[0011] The service control module identifies the supervision terminals corresponding to each supervision result, and based on the identification result and the supervision result, generates the supervision prompt information corresponding to each supervision terminal, and sends the supervision prompt information to the corresponding supervision terminal.

[0012] According to another aspect of the present application, there is provided a storage medium, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned smart prompting method for a business hall is implemented.

[0013] According to still another aspect of the present application, there is provided a computer device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, and when the processor executes the program, the above-mentioned smart prompting method for a business hall is implemented.

[0014] By means of the above technical solutions, a smart prompting system and method for a business hall, a storage medium, and a computer device provided by the present application achieve comprehensive automated monitoring of business handling in the business hall through highly intelligent image recognition and processing algorithms, can greatly improve the timeliness and accuracy of supervision, and are beneficial to improving the service quality and customer satisfaction of the business hall.

[0015] The above description is only an overview of the technical solutions of the present application. In order to be able to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features, and advantages of the present application more obvious and understandable, the specific embodiments of the present application are specifically given below. Brief Description of the Drawings

[0016] The accompanying drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0017] Figure 1 It shows a schematic structural diagram of an intelligent reminder system for a business hall provided by an embodiment of the present application;

[0018] Figure 2 It shows a schematic flow diagram of an intelligent reminder method for a business hall provided by an embodiment of the present application;

[0019] Figure 3 It shows a schematic diagram of the device structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0020] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments. It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other.

[0021] In this embodiment, an intelligent reminder system for a business hall is provided, as Figure 1 shown. The system includes an image acquisition module, an image comprehensive processing module, and a service control module;

[0022] The image acquisition module is used to perform image acquisition operations on the image acquisition areas corresponding to each image acquisition device based on the image acquisition devices deployed in the business hall, obtain the image acquisition results corresponding to each image acquisition device, and push the image acquisition results to the image comprehensive processing module, where the image acquisition areas include the business handling areas corresponding to each business handler;

[0023] The image comprehensive processing module is used to respectively call the image processing unit corresponding to the image acquisition device based on the image acquisition device corresponding to each image acquisition result, use the image processing unit to obtain the supervised objects in the corresponding image acquisition result, determine the supervision results corresponding to the supervised objects based on the preset supervision standards corresponding to the image acquisition results, and send the supervision results corresponding to each image acquisition result to the service control module;

[0024] The service control module is used to identify the supervision terminals corresponding to each supervision result, and generate supervision reminder information corresponding to each supervision terminal based on the identification result and the supervision result, and send the supervision reminder information to the corresponding supervision terminals.

[0025] An intelligent prompt system for business halls provided by an embodiment of the present application. By integrating three core modules: image acquisition, image comprehensive processing, and service control, the system aims to achieve intelligent monitoring and management of business handling personnel (i.e., staff handling business) in the business hall. The system uses advanced image recognition and processing technologies to automatically capture and analyze the behaviors and service qualities of business handling personnel in the business handling area, providing instant and accurate supervision prompt information for the business hall, thereby effectively improving service efficiency and customer satisfaction.

[0026] Before performing intelligent prompts through the intelligent prompt system for business halls, image acquisition devices (such as high-definition cameras) can be deployed in areas within the business hall that need to be supervised. These image acquisition devices are carefully arranged to ensure coverage of all key business handling areas, such as business handling counters, self-service machine areas, etc. Each image acquisition device is responsible for the image acquisition task of its corresponding area. Here, the image acquisition device can continuously acquire images in real time or acquire images at a preset frequency.

[0027] After the intelligent prompt system for business halls starts working, the image acquisition module in it can obtain the image acquisition results of each image acquisition device in real time. These image acquisition results are then immediately pushed to the image comprehensive processing module for further analysis.

[0028] Next, the image comprehensive processing module intelligently invokes the corresponding image processing units according to the image acquisition devices corresponding to each image acquisition result. These image processing units can include advanced image processing algorithms such as face recognition, behavior recognition, and object detection, which are used to extract the supervised objects from the image acquisition results. After invoking the image processing units, it can automatically identify the supervised objects in the image acquisition results, such as business handling personnel and their behavior characteristics, etc. This is the basis for determining the subsequent supervision results. The system also has a set of preset supervision standards built in, and these supervision standards can cover multiple aspects such as dressing normativity standards. Based on these supervision standards, the image processing units can evaluate each identified supervised object one by one and generate corresponding supervision results. For example, if the supervised object is the dressing and status of business handling personnel, and the preset supervision standard is the dressing normativity standard for business handling personnel, then the supervision result can be that the dressing is normative or non-normative. Whenever a new supervision result is generated, it can then be sent to the service control module to provide data support for generating subsequent supervision reminder information. In addition, it should be noted that each image acquisition device can correspond to multiple image processing units, and each image processing unit is used to perform one image processing task. For example, if the image acquisition device is deployed on the business hall counter to collect images of business handling personnel, and there is supervision on the dressing normativity and whether they are sleeping on duty for business handling personnel, then the image processing units can include two, namely the image processing unit for performing dressing normativity recognition and the image processing unit for performing sleeping on duty recognition. Additionally, since the supervision targets corresponding to image acquisition devices deployed in different locations may be different, the image processing units corresponding to different image acquisition devices may be different.

[0029] Whenever the service control module receives a supervision result, it can first identify the supervision terminal corresponding to the supervision result, and based on the identification result and the supervision result, intelligently generate supervision reminder information. The supervision reminder information can include reminders such as non-standard clothing, sleeping on duty, and service attitude. The generated supervision reminder information will be immediately pushed to the corresponding supervision terminal through an appropriate method (such as pop-up reminder, SMS notification, etc.). Among them, the supervision terminal can include the supervision terminal corresponding to the business handling personnel, which is used to remind the business handling personnel of their own problems, and can also include the supervision terminal of the superior supervision personnel of the business handling personnel, which is used to understand the work situation of the business handling personnel in real time. Specifically, the supervision terminal corresponding to each supervision result can be set according to actual needs. The supervision terminal can be a PC terminal, a mobile client, etc.

[0030] By applying the technical solution of this embodiment, through highly intelligent image recognition and processing algorithms, comprehensive automated monitoring of business handling in the business hall is achieved, which can greatly improve the timeliness and accuracy of supervision, and is beneficial to improving the service quality and customer satisfaction of the business hall.

[0031] In an embodiment of the present application, optionally, the image acquisition device includes a first image acquisition device deployed on the counter of the business hall, and the image processing unit corresponding to the first image acquisition device includes a personnel clothing recognition unit; the personnel clothing recognition unit is configured to: determine the business handling personnel corresponding to the first image acquisition device, and extract a human region from the image acquisition result corresponding to the first image acquisition device, and based on a preset segmentation ratio, determine a clothing region to be evaluated from the human region through a clothing extraction frame; based on the business handling personnel, recognize the clothing attributes corresponding to the clothing region to be evaluated, and determine a preset supervision standard corresponding to the clothing attributes from a preset database; extract features from the clothing region to be evaluated to obtain an extraction result, and calculate a first feature similarity between the extraction result and the supervision features corresponding to the preset supervision standard, and obtain a first supervision result according to the first feature similarity.

[0032] In this embodiment, some tellers may go to work in casual clothes, privately modify the uniform style, or wear exaggerated accessories, and there are problems with wearing work ID cards, such as not wearing a work ID card, wearing it improperly, or the work ID card being damaged. This makes it difficult for customers to quickly know the identity of the staff, affecting the service experience and communication efficiency. Therefore, the image acquisition device in the business hall may include a first image acquisition device, which may be deployed on the counter, mainly facing the business handling personnel in front of the counter, and is used to acquire images of the business handling personnel. Specifically, a first image acquisition device may be set on the counter of each business handling personnel. The image processing unit corresponding to the first image acquisition device may include a personnel clothing recognition unit, and the personnel clothing recognition unit is used to supervise and evaluate the clothing of the business handling personnel. Specifically, the specific working process of the personnel clothing recognition unit is as follows:

[0033] First, identify the business handling personnel corresponding to the first image acquisition device. Specifically, the shift schedule list of the business hall may be obtained, and the personnel identification of the business handling personnel located at the counter where the first image acquisition device is deployed may be extracted from it, so as to determine the business handling personnel corresponding to the first image acquisition device. In addition, the human region may also be extracted from the image acquisition result corresponding to the first image acquisition device. Here, the extraction of the human region may be realized based on image segmentation technology to separate the human from the background. The human region may also be extracted by other means, which is not limited herein.

[0034] Next, in order to accurately evaluate the dress of business handling personnel, the personnel dress recognition unit adopts a preset segmentation ratio. This ratio can be determined based on factors such as ergonomics, general or regulatory requirements of clothing design, etc. Based on the preset segmentation ratio, the image processing unit uses a clothing extraction frame to demarcate a specific clothing area to be evaluated from the person area. For example, through the preset segmentation ratio, the placement position of the upper border or the lower border of the clothing extraction frame is determined from the person area, and then the clothing extraction frame is placed correspondingly, thereby determining the position of the clothing extraction frame, and further taking the area in the clothing extraction frame as the clothing area to be evaluated. This area can include key dress parts such as jackets, trousers, skirts, etc. The size of the clothing extraction frame can be determined based on statistical experience.

[0035] Then, the personnel dress recognition unit can determine the clothing attributes of the clothing area to be evaluated corresponding to the business handling personnel according to the personnel identifier corresponding to the business handling personnel. In one embodiment, the gender and / or level corresponding to the business handling personnel can be determined according to the personnel identifier corresponding to the business handling personnel, and the determined gender and / or level are used as clothing attributes. Once the clothing attributes are recognized, the personnel dress recognition unit can search the preset database for the preset regulatory standards corresponding to the attributes. These preset regulatory standards can be formulated by business halls, industry organizations or regulatory agencies for regulating the dress of business handling personnel.

[0036] Subsequently, the personnel dress recognition unit extracts features from the clothing area to be evaluated to obtain a series of data describing the clothing features. For example, color and hue features: used to evaluate the overall coordination of the clothing color and whether it meets the color requirements stipulated by the business hall or the industry; texture and pattern features: used to evaluate the style, size, position, etc. of the texture and pattern on the clothing and whether they meet the stipulated dress standards; style and tailoring features: used to evaluate whether the style of the clothing (such as suits, shirts, uniforms, etc.) and the tailoring (such as fit, smoothness of lines, etc.) meet the regulations. In addition, other features can also be included.

[0037] Furthermore, the extracted features are compared with the regulatory features corresponding to the preset regulatory standards to calculate the similarity between them (i.e., the first feature similarity). This similarity measures the degree of compliance between the dress of business handling personnel and the regulatory standards. It should be noted that each preset regulatory standard stores the corresponding regulatory features in advance, and the feature dimension of the regulatory features of the preset regulatory standards is the same as the dimension of the features extracted by the personnel dress recognition unit. Based on the first feature similarity, the personnel dress recognition unit can output the first regulatory result. The first regulatory result can specifically be a simple pass / fail judgment or a more detailed score or rating.

[0038] It should be noted that the image comprehensive processing module can input the acquisition result of each frame of the received image into the personnel clothing recognition unit, and the personnel clothing recognition unit can recognize the clothing standardization for each frame.

[0039] Through the combination of the first image acquisition device and the personnel clothing recognition unit in the embodiment of the present application, the automatic supervision and evaluation of the clothing of the business handling personnel at the business hall counter are realized. This automatic supervision method helps to improve the recognition accuracy of the clothing standardization in the business hall. When it is found that the clothing of the business handling personnel is not standardized, the corresponding supervision terminal of the relevant supervisors is notified in time, so as to discover problems and make improvements in time, which is beneficial to enhancing the user experience.

[0040] In the embodiment of the present application, optionally, the image processing unit corresponding to the first image acquisition device further includes a personnel status recognition unit; the personnel status recognition unit is used for: based on the image acquisition results of consecutive frames corresponding to the first image acquisition device, determining the action sequence of the business handling personnel, as well as the facial features and body posture features of the business handling personnel in each frame of the image acquisition result, where the facial features include at least one of the eye aspect ratio feature, the gray value change feature of the eye area, and the head tilt angle feature, and the body posture features include the body tilt angle feature and / or the head-body relative position feature; performing feature fusion on the action sequence, the facial features, and the body posture features to obtain the first fusion feature corresponding to the business handling personnel; obtaining the preset supervision standard of the working state, and calculating the second feature similarity between the first fusion feature and the supervision feature corresponding to the preset supervision standard of the working state, and obtaining the second supervision result according to the second feature similarity.

[0041] In this embodiment, it is also possible that the business hall teller may fall asleep on duty due to fatigue and difficulty in concentrating during working hours; or during some periods with less business volume, the teller's vigilance relaxes and may fall asleep on duty, which not only leads to service interruption and business delay, but also may reduce customer satisfaction and trigger negative evaluations. Therefore, the image processing unit corresponding to the first image acquisition device may further include a personnel status recognition unit. The personnel status recognition unit recognizes and analyzes the actions, facial features, and body posture features of the business handling personnel based on the consecutive frame image acquisition results provided by the first image acquisition device, so as to evaluate their working states. The first image acquisition device continuously acquires images of the business handling personnel at the counter position to form consecutive frame image acquisition results. These consecutive frame images provide a dynamic data basis for personnel status recognition. The number of consecutive frames can be set according to experience. Specifically, the specific working process of the personnel status recognition unit is as follows:

[0042] Determination of Action Sequence. By analyzing information such as the position and action changes of the business handling personnel in the continuous frame image acquisition results, their action sequence is determined. The action sequence reflects the behavioral activities of the business handling personnel over a period of time. The action sequence can be generated based on the following method: Extract the position features, action features, and morphological features of the business handling personnel in each frame of image acquisition results. Among them, the position features are used to reflect the position information of the business handling personnel in the image, such as coordinates, regions, etc.; the action features are used to reflect the movement trajectories, speeds, accelerations, etc. of various body parts; the morphological features are used to reflect body postures, gestures, facial expressions, etc. Based on the extracted features, the personnel status recognition unit applies action recognition algorithms to recognize the specific actions in each frame of image. These algorithms can include machine learning models (such as convolutional neural network CNN, recurrent neural network RNN, etc.), which are trained to recognize various preset action categories. Once the actions in each frame of image are recognized, the personnel status recognition unit can connect these actions in chronological order to form an action sequence. This action sequence reflects the dynamic behavior of the business handling personnel over a period of time.

[0043] Extraction of Facial Features. Aspect Ratio Feature of Eyes: By analyzing the pixel dimensions of the eye region, calculate the aspect ratio of the eyes, which is used to evaluate the opening and closing state of the eyes. Gray Value Change Feature of Eye Region: Monitor the change of the gray value in the eye region over time, which is used to identify states such as blinking and fatigue. Head Tilt Angle Feature: Measure the tilt angle of the head relative to the vertical line, which is used to evaluate the degree of concentration or fatigue state.

[0044] Extraction of Body Posture Features. Body Tilt Angle Feature: Measure the tilt angle of the body relative to the vertical line, which is used to evaluate the stability of the standing or sitting posture. Relative Position Feature of Head and Body: Analyze the relative position relationship between the head and the body, which is used to evaluate the correctness of the posture.

[0045] It should be noted that the above three parts, namely the determination of the action sequence, the extraction of facial features, and the extraction of body posture features, can be carried out simultaneously.

[0046] Furthermore, fuse the action sequence, facial features, and body posture features to form the first fusion feature. The fusion process can involve technical means such as feature weighting and feature dimensionality reduction to improve the effectiveness and robustness of the features. Then, the personnel status recognition unit obtains the regulatory standards related to the working status from the preset database. These preset regulatory standards can include action specifications, facial expression requirements, body posture standards, etc.

[0047] Subsequently, the first fusion feature is compared with the supervision feature corresponding to the preset supervision standard (pre-stored in the preset database), and the similarity between them (i.e., the second feature similarity) is calculated. The similarity calculation can involve technical means such as distance metrics and similarity functions.

[0048] Finally, based on the second feature similarity, the personnel status recognition unit generates a second supervision result. The second supervision result can include the work status assessment of the business handling personnel (such as focused, fatigued, distracted, etc.), whether they comply with the work specifications, etc.

[0049] Through the comprehensive analysis of the action sequence, facial features, and body posture features in the embodiments of the present application, a comprehensive assessment of the work status of business handling personnel is achieved. This assessment method not only improves the accuracy and objectivity of supervision but also helps to timely detect potential work problems, such as fatigue driving and distracted operation.

[0050] In addition, the personnel status recognition unit can also include a pre-trained YOLOv8 model. By inputting each frame of the image into the pre-trained YOLOv8 model for detection, the detection box and the corresponding behavior categories, such as working, sleeping, etc., can be output.

[0051] In the embodiments of the present application, optionally, the image acquisition device includes a second image acquisition device deployed in the business handling area of the business hall, and the image processing unit corresponding to the second image acquisition device includes a personnel on-duty recognition unit; the personnel on-duty recognition unit is used to: determine the business handling personnel corresponding to the second image acquisition device, and determine whether the image acquisition result corresponding to the second image acquisition device contains a person. When a person is included, extract the first facial feature corresponding to the person, and use the second facial feature of the business handling personnel corresponding to the second image acquisition device as the preset supervision standard, calculate the third feature similarity between the first facial feature and the second facial feature, and determine the third supervision result according to the third feature similarity; correspondingly, the supervision terminal includes an on-duty status supervision terminal corresponding to the third supervision result; the on-duty status supervision terminal is used to: when receiving the supervision prompt information corresponding to the third supervision result, analyze the on-duty status of the business handling personnel indicated by the supervision prompt information corresponding to the third supervision result, and determine whether the on-duty status is consistent with the current display status of the service instrument corresponding to the second image acquisition device. When they are inconsistent, send an on-duty status update instruction to enable the service instrument to update the current display status.

[0052] In this embodiment, it is also possible to automatically identify whether the business handling personnel are on duty through the intelligent prompting system of the business hall. Specifically, the image acquisition device in the business hall may further include a second image acquisition device, which may be deployed in the business handling area of the business hall, such as on the counter, on the roof corresponding to the business handling area, etc. When it is deployed on the counter, the second image acquisition device may be the same acquisition device as the first image acquisition device at this time. When the second image acquisition device is different from the first image acquisition device, a first image acquisition device may be set for each counter corresponding to a business handling personnel, and a second image acquisition device may be set for the business handling area corresponding to the business handling personnel. The image processing unit corresponding to the second image acquisition device may include a personnel on-duty identification unit, which is used to identify and process the information in the image to determine whether the business handling personnel are on duty.

[0053] Specifically, the specific working process of the personnel on-duty identification unit is as follows:

[0054] First, identify the business handling personnel corresponding to the second image acquisition device. Similarly, the shift schedule list of the business hall can be obtained, and the personnel identifier of the business handling personnel located in the business handling area where the second image acquisition device is deployed can be extracted from it, so as to determine the business handling personnel corresponding to the second image acquisition device. In addition, it is also possible to analyze the image captured by the second image acquisition device to determine whether it contains a person. If it does, further extract the first facial feature of the person. Here, the algorithm for person extraction can be an algorithm in the prior art, which is not limited here. In addition, the second facial feature corresponding to the business handling personnel can be obtained from the preset database, and the second facial feature is used as the preset supervision standard, and the extracted first facial feature is compared with the second facial feature to calculate the third feature similarity between the two. This similarity is used to represent the matching degree between the two facial features. Here, the facial features corresponding to each business handling personnel can be pre-stored in the preset database, and the extraction method of the facial features of the business handling personnel in the preset database is the same as the extraction method of the first facial feature.

[0055] Next, based on the third feature similarity, determine whether the business handling personnel are on duty. Specifically, if the third feature similarity exceeds the preset threshold, it can be determined that they are on duty; otherwise, it is determined that they are not on duty. The determination result is used as the third supervision result.

[0056] When the image acquisition device in the business hall includes a second image acquisition device, the supervision terminal in the business hall may further include an on-duty status supervision terminal corresponding to the third supervision result. When the on-duty status supervision terminal receives the supervision prompt information corresponding to the third supervision result, it parses the information to obtain the on-duty status of the business handling personnel. Further, the on-duty status parsed by the on-duty status supervision terminal is compared with the current display status of the service instrument corresponding to the second image acquisition device. Here, the service instrument refers to a device used to display whether the business handling personnel are on duty, such as a display screen, etc. Through the service instrument, users can see whether the business handling area can handle business normally. If the comparison result shows that the on-duty status is inconsistent (for example, the third supervision result shows on duty, but the service instrument shows off duty), the on-duty status supervision terminal will send an on-duty status update instruction to the service instrument to update its display status and ensure the accuracy of the information.

[0057] The embodiment of the present application realizes the automatic supervision of the on-duty status of personnel in the business handling area of the business hall through image acquisition and facial feature recognition technologies; by comparing and updating with the display status of the service instrument, it ensures the accuracy and real-time nature of the status information of the business handling personnel, helps improve the operation efficiency and customer satisfaction of the business hall, and at the same time reduces the cost and error of manual supervision.

[0058] In the embodiment of the present application, optionally, the system further includes a plurality of voice processing modules, and each voice processing module is deployed on the counter corresponding to each business handling personnel in the business hall; the voice processing module is used for: picking up the conversation between the business handling personnel and the user, obtaining the voice characteristics corresponding to the business handling personnel, and based on the voice characteristics, cutting the conversation to obtain a first sub-conversation corresponding to the business handling personnel and a second sub-conversation corresponding to the user; based on linear predictive coding, respectively extracting a first sub-feature from the first sub-conversation and a second sub-feature from the second sub-conversation; through an automatic speech recognition algorithm, respectively decoding the first sub-feature and the second sub-feature to obtain a first sub-text and a second sub-text; based on a brute-force matching algorithm, respectively performing text matching on the first sub-text and the second sub-text to obtain a first matching result and a second matching result; according to the first matching result, determining the first emotion index of the business handling personnel, and according to the second matching result, determining the second emotion index of the business handling personnel, and determining a fourth supervision result according to the first emotion index and the second emotion index.

[0059] In this embodiment, multiple voice processing modules can also be set in the intelligent prompting system of the business hall to evaluate the interaction between business handling personnel and users, especially to monitor the emotional indexes of business handling personnel and users. Specifically, each voice processing module can be deployed on the counter corresponding to each business handling personnel in the business hall. In this way, the communication between each business handling personnel and users can be independently monitored and analyzed, which helps to obtain more accurate and targeted data. Specifically, the specific working process of the voice processing module is as follows:

[0060] First, the voice processing module can pick up the conversation between business handling personnel and users. By analyzing the audio signal of the conversation, the voice features corresponding to the audio signal are extracted. In addition, the voice features (pre-stored) of the business handling personnel corresponding to this voice processing module can also be obtained from the preset database. According to the voice features of the business handling personnel, they are matched with the voice features corresponding to the whole conversation, so as to identify which parts are corresponding to the business handling personnel and which parts are corresponding to the users. In this way, the above conversation can be cut to obtain the first sub-conversation corresponding to the business handling personnel and the second sub-conversation corresponding to the users.

[0061] Next, the first sub-features are extracted from the first sub-conversation and the second sub-features are extracted from the second sub-conversation respectively through Linear Predictive Coding (LPC). Here, the first sub-features and the second sub-features can include features such as encoded LPC Cepstrum Coefficients, Line Spectrum Pairs (LSPs), and Reflection Coefficients.

[0062] Subsequently, through the Automatic Speech Recognition (ASR) algorithm, the extracted first sub-features and second sub-features are decoded into text forms, namely the first sub-text and the second sub-text. This step is the key to converting the voice signal into analyzable text data.

[0063] Next, through the brute force matching (BF, Brute Force Algorithm) algorithm, the first sub-text and the second sub-text are respectively matched to obtain a first matching result and a second matching result. Specifically, the first sub-text, the second sub-text, and the sensitive word library (such as sensitive words like "what's the attitude", "how many times have I said", "delay", "business suspended for handling", etc.) can be compared character by character to obtain the first matching result corresponding to the first sub-text and the second matching result corresponding to the second sub-text. According to the first matching result, it can be known which sensitive words are included in the first sub-text; according to the second matching result, it can be known which sensitive words are included in the second sub-text.

[0064] After that, according to the first matching result and the second matching result, the first emotional index and the second emotional index of the business handler are respectively determined. In one embodiment, the more sensitive words included in the matching result, the larger the value of the corresponding emotional index, indicating that the emotion is more excited and unfriendly.

[0065] Finally, by combining the first emotional index and the second emotional index, a fourth supervision result is determined. This result is used to evaluate aspects such as the service quality and emotional management ability of the business handler, and can also reflect the emotions of users during the business handling process. The first emotional index and the second emotional index can be directly used as the fourth supervision result, or the emotional categories (calm, irritable, angry, enraged, etc.) of the business handler and the user can be determined according to the first emotional index and the second emotional index, and the determined emotional categories can be used as the fourth supervision result.

[0066] After that, the voice processing module can also send the fourth supervision result to the service control module for subsequent operations.

[0067] The embodiment of this application provides a real-time and objective emotion monitoring and service quality evaluation tool for the business hall through automated voice analysis means, which helps to timely discover and handle potential negative emotions or conflict situations, and improve the user experience and service quality.

[0068] In an embodiment of the present application, optionally, the image comprehensive processing module is further configured to: determine the frame extraction frequency corresponding to each image acquisition device based on a preset frame extraction frequency calculation model, and perform frame extraction from the corresponding image acquisition result based on the frame extraction frequency, and use the frame extraction result as the updated image acquisition result, so as to use the image processing unit to obtain the supervision object in the updated image acquisition result; wherein, for each image acquisition device, the frame extraction frequency calculation process of the preset frame extraction frequency calculation model is as follows: obtain the image acquisition area, preset supervision items, business handling personnel of the image acquisition device, and the historical supervision results corresponding to the preset supervision items; obtain the unique identifier and user evaluation portrait corresponding to the business handling personnel, and generate the target description information of the business handling personnel according to the unique identifier and the user evaluation portrait; input the image acquisition area, the preset supervision items, the target description information, and the historical supervision results into the input layer of the preset frame extraction frequency calculation model, and respectively extract the acquisition area feature corresponding to the image acquisition area, the supervision item feature corresponding to the preset supervision item, the supervision object feature corresponding to the target description information, and the supervision result feature corresponding to the historical supervision result through the feature extraction layer of the preset frame extraction frequency calculation model, and perform feature fusion on the acquisition area feature, the supervision item feature, the supervision object feature, and the supervision result feature through the feature fusion layer of the preset frame extraction frequency calculation model to obtain a second fusion feature, and output the frame extraction frequency corresponding to the second fusion feature through the result output layer of the preset frame extraction frequency calculation model.

[0069] In this embodiment, the image comprehensive processing module in the business hall intelligent prompt system can also determine the frame extraction frequency of each image acquisition device based on a preset frame extraction frequency calculation model, and perform frame extraction from the corresponding image acquisition result to update the image acquisition result, and then the image processing unit uses these updated results to identify or analyze the supervision object. That is to say, through the preset frame extraction frequency calculation model, the frame extraction frequency corresponding to each image acquisition device can be calculated. When the subsequent image processing unit processes the image acquisition result, it is not necessary to process each frame of the image acquired by the image acquisition device, but only need to process the frame-extracted image, which can greatly improve the image processing efficiency and reduce unnecessary image processing work. Specifically, for each image acquisition device, the calculation process of the preset frame extraction frequency calculation model can be as follows:

[0070] First, after receiving the image acquisition result collected by the image acquisition device, the image comprehensive processing module can obtain the image acquisition area corresponding to the image acquisition device, the preset supervision items, the business handling personnel, and the historical supervision results corresponding to the preset supervision items. Among them, the image acquisition area refers to the geographical or physical area covered by the image acquisition device, which can be specifically represented by a preset identifier of the image acquisition area; the preset supervision items refer to the specific content or objects to be supervised, such as personnel behavior, etc., such as the above-mentioned sleeping-on-duty supervision item, dressing code supervision item, etc.; the business handling personnel are the staff who handle the business, which can be specifically represented by the unique identifier of the business handling personnel; the historical supervision results are the previous supervision records or results for this supervision item, such as the sleeping-on-duty supervision result and dressing code compliance supervision result of this supervision item.

[0071] Next, the image comprehensive processing module generates target description information by obtaining the unique identifier of the business handling personnel (such as employee number, ID number, etc.) and the user evaluation portrait (the evaluation and feedback of the user, such as service attitude evaluation, business handling professionalism evaluation, etc.). The target description information is used to indicate the user evaluation situation of the business handling personnel. This helps to more accurately identify and analyze the behavior of the business handling personnel. For business handling personnel with good user evaluations, the supervision frequency can be appropriately reduced.

[0072] Furthermore, the image comprehensive processing module inputs the image acquisition area, the preset supervision items, the target description information, and the historical supervision results into the input layer of the preset frame extraction frequency calculation model. After receiving the above information, the input layer of the preset frame extraction frequency calculation model can perform feature extraction through the feature extraction layer. Specifically, the acquisition area features are the information reflecting the characteristics of the image acquisition area, such as the environmental layout, lighting conditions, and the area to which it belongs; the supervision item features are the characteristics related to the preset supervision items, such as the type of supervision object, behavior pattern, and supervision item name; the supervision object features are the features extracted based on the target description information of the business handling personnel, such as service attitude and business handling speed; the supervision result features are the information extracted from the historical supervision results, including the number of anomalies and the anomaly frequency, etc. (if the anomalies occur frequently, the supervision frequency can be increased). After the feature extraction is completed, the feature fusion layer of the preset frame extraction frequency calculation model fuses the above features to form the second fused feature. This process aims to integrate information from different dimensions to improve the accuracy and effectiveness of the frame extraction frequency calculation. Subsequently, the second fused feature is input into the result output layer of the preset frame extraction frequency calculation model, and finally, the frame extraction frequency corresponding to each image acquisition device is output. This frequency is dynamically determined based on multiple factors such as the current supervision requirements, historical supervision results, and the characteristics of the business handling personnel.

[0073] Subsequently, based on the calculated frame extraction frequency, the image comprehensive processing module can extract frames from the original image acquisition results of each image acquisition device. The frame extraction results are used as the updated image acquisition results and are transmitted to the image processing unit for subsequent processing, such as the identification of supervised objects, behavior analysis, etc.

[0074] In the embodiment of the present application, the frame extraction frequency is dynamically adjusted according to factors such as supervision requirements, historical results, and characteristics of business handling personnel, improving the flexibility of image data acquisition and processing; dynamically calculating a reasonable frame extraction frequency reduces unnecessary data acquisition and processing, and reduces system resource consumption and costs.

[0075] In the embodiment of the present application, optionally, the service control module is further configured to: after identifying the supervision terminal corresponding to each supervision result, retrieve the reminder information template and the detailed information template corresponding to the supervision terminal, and respectively extract from the supervision result the first text to be filled corresponding to the reminder information template, and the second text to be filled corresponding to the detailed information template; fill the first text to be filled into the reminder information template to obtain the reminder text corresponding to the supervision result, and fill the second text to be filled into the detailed information template to obtain the detailed text corresponding to the supervision result; send the reminder text and the detailed text as supervision reminder information to the corresponding supervision terminal, so that after receiving the reminder text and the detailed text, the supervision terminal directly pops up the reminder text, and displays the detailed text when receiving the trigger instruction of the reminder text.

[0076] In this embodiment, the process of the service control module generating supervision reminder information is as follows:

[0077] First, identify the supervision terminal associated with each supervision result. Once the supervision terminal is identified, retrieve two templates corresponding to the supervision terminal: the reminder information template and the detailed information template. Then, extract two types of text from each supervision result: one is the first text to be filled for filling the reminder information template, and the other is the second text to be filled for filling the detailed information template. Subsequently, fill the extracted text into the corresponding templates to generate the final reminder text and detailed text. Among them, the reminder text is used to remind the supervision personnel corresponding to the supervision terminal of the main information, such as only including supervision items (dress, personnel status, etc.), business handling personnel identification, etc.; the detailed text is used to provide all the information required for supervision.

[0078] Furthermore, the generated reminder text and detail text can be used as regulatory prompt information and sent to the corresponding regulatory terminals. After receiving the reminder text, the regulatory terminals will immediately pop up and display it. This instant feedback mechanism helps to quickly attract the attention of regulatory personnel and prompt them to take actions. When the regulatory personnel trigger the reminder text (e.g., click or select to view more information), the detail text will be displayed. This design allows users to delve into the details of the regulatory results as needed without being overwhelmed by a large amount of information at the initial stage.

[0079] Through templatization and automated processing in the embodiments of this application, the service control module can quickly generate and send customized regulatory information, significantly improving work efficiency; the instant pop-up of the reminder text and the on-demand display of the detail text provide an intuitive and user-friendly way to obtain information, enhancing the information processing ability and response speed of regulatory personnel, thus achieving the efficient, accurate, and personalized communication of regulatory information.

[0080] In the embodiments of this application, optionally, after receiving each regulatory result, the service control module can also filter the regulatory results, and only generate regulatory prompt information for those abnormal regulatory results and send them to the matching regulatory terminals. For example, only abnormal regulatory results such as improper dressing and business handling personnel sleeping on the job are used as the regulatory results to be sent to the regulatory terminals.

[0081] Further, as Figure 1 a specific implementation of the method, the embodiments of this application provide an intelligent reminder method for business halls, as Figure 2 shown, this method includes:

[0082] Step 101, the image acquisition module performs image acquisition operations on the image acquisition areas corresponding to each image acquisition device deployed in the business hall based on each image acquisition device, obtains the image acquisition results corresponding to each image acquisition device, and pushes each image acquisition result to the image comprehensive processing module, where the image acquisition areas include the business handling areas corresponding to each business handling personnel;

[0083] Step 102, based on the image acquisition device corresponding to each image acquisition result, the image comprehensive processing module respectively calls the image processing unit corresponding to the image acquisition device, uses the image processing unit to obtain the regulatory objects in the corresponding image acquisition result, determines the regulatory results corresponding to the regulatory objects based on the preset regulatory standards corresponding to the image acquisition result, and sends the regulatory results corresponding to each image acquisition result to the service control module;

[0084] Step 103, the service control module identifies the supervision terminals corresponding to each supervision result, and based on the identification result and the supervision result, generates supervision prompt information corresponding to each supervision terminal, and sends the supervision prompt information to the corresponding supervision terminal.

[0085] Optionally, the image acquisition device includes a first image acquisition device deployed on the counter of the business hall, and the image processing unit corresponding to the first image acquisition device includes a personnel clothing recognition unit; the method further includes:

[0086] The personnel clothing recognition unit determines the business handling personnel corresponding to the first image acquisition device, extracts the human region from the image acquisition result corresponding to the first image acquisition device, and based on a preset segmentation ratio, determines the clothing region to be evaluated from the human region through a clothing extraction frame;

[0087] Based on the business handling personnel, identify the clothing attributes corresponding to the clothing region to be evaluated, and determine the preset supervision standards corresponding to the clothing attributes from a preset database;

[0088] Extract features from the clothing region to be evaluated to obtain an extraction result, calculate the first feature similarity between the extraction result and the supervision features corresponding to the preset supervision standards, and obtain a first supervision result according to the first feature similarity.

[0089] Optionally, the image processing unit corresponding to the first image acquisition device further includes a personnel status recognition unit; the method further includes:

[0090] The personnel status recognition unit determines the action sequence of the business handling personnel and the facial features and body posture features of the business handling personnel in each frame of the image acquisition result based on the continuous frame image acquisition result corresponding to the first image acquisition device, wherein the facial features include at least one of the eye aspect ratio feature, the gray value change feature of the eye region, and the head tilt angle feature, and the body posture features include the body tilt angle feature and / or the head-body relative position feature;

[0091] Fuse the action sequence, the facial features, and the body posture features to obtain a first fusion feature corresponding to the business handling personnel;

[0092] Obtain the preset supervision standards for the working state, calculate the second feature similarity between the first fusion feature and the supervision features corresponding to the preset supervision standards for the working state, and obtain a second supervision result according to the second feature similarity.

[0093] Optionally, the image acquisition device includes a second image acquisition device deployed in the business handling area of the business hall, and the image processing unit corresponding to the second image acquisition device includes a personnel on-duty recognition unit; the method further includes:

[0094] The personnel on-duty recognition unit determines the business handling personnel corresponding to the second image acquisition device, and determines whether the image acquisition result corresponding to the second image acquisition device contains a person. When a person is included, the first facial feature corresponding to the person is extracted, and the second facial feature of the business handling personnel corresponding to the second image acquisition device is used as a preset supervision standard. The third feature similarity between the first facial feature and the second facial feature is calculated, and a third supervision result is determined according to the third feature similarity.

[0095] Correspondingly, the supervision terminal includes an on-duty status supervision terminal corresponding to the third supervision result; the method further includes:

[0096] When the on-duty status supervision terminal receives the supervision prompt information corresponding to the third supervision result, it analyzes the on-duty status of the business handling personnel indicated by the supervision prompt information corresponding to the third supervision result, and determines whether the on-duty status is consistent with the current display status of the service instrument corresponding to the second image acquisition device. When they are inconsistent, an on-duty status update instruction is sent to enable the service instrument to update the current display status.

[0097] Optionally, the system further includes a plurality of voice processing modules, and each voice processing module is deployed on the counter corresponding to each business handling personnel in the business hall; the method further includes:

[0098] The voice processing module picks up the conversation between the business handling personnel and the user, obtains the voice feature corresponding to the business handling personnel, and based on the voice feature, cuts the conversation to obtain a first sub-conversation corresponding to the business handling personnel and a second sub-conversation corresponding to the user.

[0099] Based on linear predictive coding, a first sub-feature is extracted from the first sub-conversation respectively, and a second sub-feature is extracted from the second sub-conversation.

[0100] Through an automatic speech recognition algorithm, the first sub-feature and the second sub-feature are decoded respectively to obtain a first sub-text and a second sub-text.

[0101] Based on a brute-force matching algorithm, the first sub-text and the second sub-text are text-matched respectively to obtain a first matching result and a second matching result.

[0102] Determine a first emotion index of the business handler according to the first matching result, and determine a second emotion index of the business handler according to the second matching result. Determine a fourth supervision result according to the first emotion index and the second emotion index.

[0103] Optionally, the method further includes:

[0104] The image comprehensive processing module determines a frame extraction frequency corresponding to each image acquisition device based on a preset frame extraction frequency calculation model, and performs frame extraction from the corresponding image acquisition result based on the frame extraction frequency, and uses the frame extraction result as the updated image acquisition result, so as to use the image processing unit to obtain a supervision object in the updated image acquisition result;

[0105] Among them, for each image acquisition device, the frame extraction frequency calculation process of the preset frame extraction frequency calculation model is as follows:

[0106] Obtain the image acquisition area, preset supervision items, business handlers of the image acquisition device, and historical supervision results corresponding to the preset supervision items;

[0107] Obtain a unique identifier corresponding to the business handler and a user evaluation portrait, and generate target description information of the business handler according to the unique identifier and the user evaluation portrait;

[0108] Input the image acquisition area, the preset supervision items, the target description information, and the historical supervision results into the input layer of the preset frame extraction frequency calculation model, and respectively extract the acquisition area features corresponding to the image acquisition area, the supervision item features corresponding to the preset supervision items, the supervision object features corresponding to the target description information, and the supervision result features corresponding to the historical supervision results through the feature extraction layer of the preset frame extraction frequency calculation model. Through the feature fusion layer of the preset frame extraction frequency calculation model, perform feature fusion on the acquisition area features, the supervision item features, the supervision object features, and the supervision result features to obtain second fusion features, and output a frame extraction frequency corresponding to the second fusion features through the result output layer of the preset frame extraction frequency calculation model.

[0109] Optionally, generating supervision prompt information corresponding to each supervision terminal based on the recognition result and the supervision result, and sending the supervision prompt information to the corresponding supervision terminal includes:

[0110] Retrieve a reminder information template and a detail information template corresponding to the supervision terminal, and respectively extract first text to be filled in corresponding to the reminder information template and second text to be filled in corresponding to the detail information template from the supervision result;

[0111] Fill the first text to be filled into the reminder information template to obtain the reminder text corresponding to the supervision result, and fill the second text to be filled into the detail information template to obtain the detail text corresponding to the supervision result;

[0112] Send the reminder text and the detail text as supervision reminder information to the corresponding supervision terminal, so that after receiving the reminder text and the detail text, the supervision terminal directly pops up the reminder text, and when receiving the trigger instruction of the reminder text, displays the detail text.

[0113] It should be noted that for other corresponding descriptions of each functional unit involved in the intelligent reminder method for business halls provided in the embodiments of the present application, reference can be made to Figure 1 the corresponding description in the system, which will not be elaborated here.

[0114] The embodiments of the present application also provide a computer device, which can specifically be a personal computer, a server, a network device, etc. As Figure 3 shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may further include an input / output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements the steps in the method embodiments.

[0115] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0116] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium can be non-volatile or volatile, and stores a computer program, which when executed by a processor, implements the steps in the above method embodiments.

[0117] In one embodiment, a computer program product is provided, including a computer program, which when executed by a processor, implements the steps in the above method embodiments.

[0118] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties.

[0119] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., and are not limited thereto.

[0120] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0121] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An intelligent reminder system for business halls, characterized in that, It includes an image acquisition module, an image comprehensive processing module, and a service control module; The image acquisition module is used to perform image acquisition operations on the image acquisition areas corresponding to each image acquisition device based on the various image acquisition devices deployed in the business hall, obtain the image acquisition results corresponding to each image acquisition device, and push the image acquisition results to the image comprehensive processing module. Among them, the image acquisition areas include the business handling areas corresponding to each business handling personnel; The image comprehensive processing module is used to respectively call the image processing unit corresponding to the image acquisition device based on the image acquisition device corresponding to each image acquisition result, use the image processing unit to obtain the supervision objects in the corresponding image acquisition result, determine the supervision results corresponding to the supervision objects based on the preset supervision standards corresponding to the image acquisition result, and send the supervision results corresponding to each image acquisition result to the service control module; The service control module is used to identify the supervision terminals corresponding to each supervision result, and generate supervision prompt information corresponding to each supervision terminal based on the identification result and the supervision result, and send the supervision prompt information to the corresponding supervision terminals.

2. The system according to claim 1, wherein The image acquisition device includes a first image acquisition device deployed on the counter of the business hall, and the image processing unit corresponding to the first image acquisition device includes a personnel clothing recognition unit; the personnel clothing recognition unit is used for: Determine the business handling personnel corresponding to the first image acquisition device, and extract the person area from the image acquisition result corresponding to the first image acquisition device. Based on a preset segmentation ratio, determine the clothing area to be evaluated from the person area through a clothing extraction frame; Based on the business handling personnel, identify the clothing attributes corresponding to the clothing area to be evaluated, and determine the preset supervision standards corresponding to the clothing attributes from a preset database; Extract features from the clothing area to be evaluated to obtain an extraction result, calculate the first feature similarity between the extraction result and the supervision features corresponding to the preset supervision standards, and obtain a first supervision result according to the first feature similarity.

3. The system according to claim 2, wherein The image processing unit corresponding to the first image acquisition device further includes a personnel state recognition unit; the personnel state recognition unit is used for: Based on the image acquisition results of consecutive frames corresponding to the first image acquisition device, determine the action sequence of the business handling personnel, and the facial features and body posture features of the business handling personnel in each frame of image acquisition result. Among them, the facial features include at least one of the eye aspect ratio feature, the gray value change feature of the eye area, and the head tilt angle feature, and the body posture features include the body tilt angle feature and / or the head-body relative position feature; Fuse the action sequence, the facial features, and the body posture features to obtain the first fusion feature corresponding to the business handling personnel; Obtain the preset supervision standard for the working status, calculate the second feature similarity between the first fusion feature and the supervision feature corresponding to the preset supervision standard of the working status, and obtain the second supervision result according to the second feature similarity.

4. The system according to claim 1, wherein The image acquisition device includes a second image acquisition device deployed in the business handling area of the business hall, and the image processing unit corresponding to the second image acquisition device includes a personnel on-duty identification unit; the personnel on-duty identification unit is used for: Determine the business handling personnel corresponding to the second image acquisition device, and determine whether the image acquisition result corresponding to the second image acquisition device contains a person. When a person is included, extract the first facial feature corresponding to the person, and use the second facial feature of the business handling personnel corresponding to the second image acquisition device as the preset supervision standard, calculate the third feature similarity between the first facial feature and the second facial feature, and determine the third supervision result according to the third feature similarity; Correspondingly, the supervision terminal includes an on-duty status supervision terminal corresponding to the third supervision result; the on-duty status supervision terminal is used for: When receiving the supervision prompt information corresponding to the third supervision result, analyze the on-duty status of the business handling personnel indicated by the supervision prompt information corresponding to the third supervision result, and judge whether the on-duty status is consistent with the current display status of the service instrument corresponding to the second image acquisition device. When they are inconsistent, send an on-duty status update instruction to enable the service instrument to update the current display status.

5. The system according to claim 1, wherein The system further includes a plurality of voice processing modules, and each voice processing module is deployed on the counter corresponding to each business handling personnel in the business hall; the voice processing module is used for: Pick up the conversation between the business handling personnel and the user, obtain the voice feature corresponding to the business handling personnel, and based on the voice feature, cut the conversation to obtain the first sub-conversation corresponding to the business handling personnel and the second sub-conversation corresponding to the user; Based on linear predictive coding, extract the first sub-feature from the first sub-conversation and the second sub-feature from the second sub-conversation respectively; Decode the first sub-feature and the second sub-feature respectively through an automatic speech recognition algorithm to obtain the first sub-text and the second sub-text; Based on a brute-force matching algorithm, perform text matching on the first sub-text and the second sub-text respectively to obtain the first matching result and the second matching result; Determine the first emotion index of the business handling personnel according to the first matching result, and determine the second emotion index of the business handling personnel according to the second matching result. Determine the fourth supervision result according to the first emotion index and the second emotion index.

6. The system according to claim 1, wherein The image comprehensive processing module is further used for: Based on a preset frame extraction frequency calculation model, determine the frame extraction frequency corresponding to each image acquisition device, and based on the frame extraction frequency, perform frame extraction from the corresponding image acquisition result, and use the frame extraction result as the updated image acquisition result to enable the image processing unit to obtain the supervision object in the updated image acquisition result; Among them, for each image acquisition device, the frame extraction frequency calculation process of the preset frame extraction frequency calculation model is as follows: Obtain the image acquisition area, preset supervision items, business handling personnel of the image acquisition device, and the historical supervision results corresponding to the preset supervision items; Obtain the unique identifier corresponding to the business handling personnel and the user evaluation portrait, and generate the target description information of the business handling personnel according to the unique identifier and the user evaluation portrait; Input the image acquisition area, the preset supervision items, the target description information, and the historical supervision results into the input layer of the preset frame extraction frequency calculation model, and respectively extract the acquisition area features corresponding to the image acquisition area, the supervision item features corresponding to the preset supervision items, the supervision object features corresponding to the target description information, and the supervision result features corresponding to the historical supervision results through the feature extraction layer of the preset frame extraction frequency calculation model. Through the feature fusion layer of the preset frame extraction frequency calculation model, fuse the acquisition area features, the supervision item features, the supervision object features, and the supervision result features to obtain the second fusion feature, and output the frame extraction frequency corresponding to the second fusion feature through the result output layer of the preset frame extraction frequency calculation model.

7. The system according to any one of claims 1 to 6, characterized in that The service control module is further configured to: After identifying the supervision terminals corresponding to each supervision result, retrieve the reminder information template and the detail information template corresponding to the supervision terminals, and respectively extract the first text to be filled corresponding to the reminder information template and the second text to be filled corresponding to the detail information template from the supervision results; Fill the first text to be filled into the reminder information template to obtain the reminder text corresponding to the supervision result, and fill the second text to be filled into the detail information template to obtain the detail text corresponding to the supervision result; Send the reminder text and the detail text as supervision reminder information to the corresponding supervision terminals, so that after receiving the reminder text and the detail text, the supervision terminals directly pop up the reminder text, and display the detail text when receiving the trigger instruction of the reminder text.

8. An intelligent prompting method for a business hall, characterized in that, It includes: The image acquisition module performs image acquisition operations on the image acquisition areas corresponding to each image acquisition device based on the image acquisition devices deployed inside the business hall, obtains the image acquisition results corresponding to each image acquisition device, and pushes the image acquisition results to the image comprehensive processing module, where the image acquisition areas include the business handling areas corresponding to each business handling personnel; The image comprehensive processing module respectively calls the image processing unit corresponding to the image acquisition device based on each image acquisition device corresponding to the image acquisition result, uses the image processing unit to obtain the supervision object in the corresponding image acquisition result, determines the supervision result corresponding to the supervision object based on the preset supervision standard corresponding to the image acquisition result, and sends the supervision results corresponding to each image acquisition result to the service control module; The service control module identifies the supervision terminals corresponding to each supervision result, and generates supervision prompt information corresponding to each supervision terminal based on the identification result and the supervision result, and sends the supervision prompt information to the corresponding supervision terminals.

9. A storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by a processor, the method described in claim 8 is implemented.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, the method described in claim 8 is implemented.

Images