Counter behavior detection method, device and equipment and medium

Abstract

The invention discloses a counter behavior detection method and device, equipment and a medium. The method comprises the following steps: after a to-be-detected video and a to-be-detected audio are obtained, determining an action key point sequence, a dialogue text, movement information and decibel information of a target counter; determining the probability that various preset actions occur in the target counter; determining the probability of occurrence of each type of preset interaction state in the target counter; detecting whether the occurrence probability of each type of safety abnormal behavior in the target counter is greater than the safety threshold value of each type of safety abnormal behavior; and when it is detected that the probability that the target safety abnormal behavior occurs at the target counter is greater than a safety threshold, determining that the target safety abnormal behavior occurs at the target counter, and providing alarm information for a monitoring user. According to the embodiment of the invention, whether the safety abnormal behavior occurs in the counter or not can be automatically, comprehensively and accurately detected based on the multi-dimensional information related to the business personnel and the customers at the counter.

Description

Technical Field

[0001] This invention relates to the field of financial technology, and in particular to a method, apparatus, equipment and medium for detecting counter behavior. Background Technology

[0002] Financial institutions typically have counters for handling their business. Staff at these counters interact with customers. To ensure the safety of staff, customers, and related items at the counters, it's necessary to monitor activity and detect any unusual behavior, addressing any issues promptly.

[0003] In related technologies, a common counter behavior detection solution involves technical personnel in financial institutions observing and analyzing video footage from the counter to determine if any abnormal behavior is occurring. However, this counter behavior detection solution relies on manual operation, which compromises accuracy. It cannot comprehensively and accurately detect abnormal behavior at the counter, thus failing to effectively protect the safety of staff, customers, or related items located there. Summary of the Invention

[0004] This invention provides a counter behavior detection method, device, equipment, and medium to solve the problems in related technologies where counter behavior detection schemes rely on manual operation, cannot guarantee accuracy, cannot comprehensively and accurately detect whether any abnormal security behavior is occurring at the counter, and are difficult to effectively protect the safety of financial institution staff, customers conducting business, and related items.

[0005] According to one aspect of the present invention, a counter behavior detection method is provided, comprising:

[0006] After obtaining the video and audio to be detected corresponding to the target counter, the key action point sequence, dialogue text, movement information and decibel information of the target counter are determined based on the video and audio to be detected.

[0007] Based on the sequence of key action points, determine the probability of each preset action occurring at the target counter;

[0008] Based on the dialogue text, determine the probability of each preset interaction state occurring at the target counter;

[0009] Based on the probability of various preset actions and preset interaction states occurring in the target counter, the movement information, and the decibel information, the probability of various abnormal security behaviors occurring in the target counter is determined, and it is detected whether the probability of various abnormal security behaviors occurring in the target counter is greater than the security threshold of various abnormal security behaviors.

[0010] When the probability of a target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, it is determined that the target security anomaly has occurred at the target counter, and the alarm information of the target security anomaly is provided to the monitoring user of the target counter.

[0011] According to another aspect of the present invention, a counter behavior detection device is provided, comprising:

[0012] The counter information determination module is used to determine the action key point sequence, dialogue text, movement information and decibel information of the target counter after obtaining the video and audio to be detected corresponding to the target counter;

[0013] The first probability determination module is used to determine the probability of various preset actions occurring in the target counter based on the sequence of key action points.

[0014] The second probability determination module is used to determine the probability of various preset interaction states occurring at the target counter based on the dialogue text.

[0015] The behavior detection module is used to determine the probability of various security abnormal behaviors occurring at the target counter based on the probability of various preset actions and various preset interaction states occurring at the target counter, the movement information, and the decibel information, and to detect whether the probability of various security abnormal behaviors occurring at the target counter is greater than the security threshold of various security abnormal behaviors.

[0016] The alarm module is used to determine that a target security anomaly has occurred at the target counter when the probability of the detected target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, and to provide alarm information of the target security anomaly to the monitoring user of the target counter.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor;

[0019] and a memory communicatively connected to the at least one processor;

[0020] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the counter behavior detection method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the counter behavior detection method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements the counter behavior detection method according to any embodiment of the present invention.

[0023] The technical solution of this invention involves, after acquiring the video and audio to be detected corresponding to the target counter, determining the action key point sequence, dialogue text, movement information, and decibel information of the target counter based on the video and audio; then, determining the probability of various preset actions occurring at the target counter based on the action key point sequence; determining the probability of various preset interaction states occurring at the target counter based on the dialogue text; determining the probability of various security anomalies occurring at the target counter based on the probability of various preset actions and interaction states occurring at the target counter, the movement information, and the decibel information, and detecting whether the probability of various security anomalies occurring at the target counter is greater than the security threshold for various security anomalies; when the probability of a target security anomaly occurring at the target counter is detected to be greater than the security threshold for the target security anomaly, it is determined that a target security anomaly has occurred at the target counter, and alarm information for the target security anomaly is provided to the target counter. This user monitoring solution addresses the shortcomings of existing counter behavior detection technologies, which rely on manual operation, lack accuracy, and fail to comprehensively and accurately detect any abnormal behavior occurring at the counter. This makes it difficult to effectively protect the safety of financial institution staff, customers, and related items. The new solution automatically determines key action sequences, dialogue text, movement information, and decibel levels at the counter based on corresponding video and audio data. It comprehensively and accurately detects any abnormal behavior occurring at the counter, leveraging multiple dimensions of information related to staff and customers at the counter. The solution promptly issues alarms upon detection of abnormal behavior, effectively protecting the safety of financial institution staff, customers, and related items.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a counter behavior detection method provided in Embodiment 1 of the present invention.

[0027] Figure 2 This is a flowchart of a counter behavior detection method provided in Embodiment 2 of the present invention.

[0028] Figure 3 This is a schematic diagram of the structure of a counter behavior detection device provided in Embodiment 3 of the present invention.

[0029] Figure 4 A schematic diagram of the structure of an electronic device for implementing the counter behavior detection method of this invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "target," "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising," "including," and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0033] The information collected in this invention is information and data authorized by the user or fully authorized by all parties. The collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data all comply with the relevant laws, regulations and standards of the relevant countries and regions, necessary confidentiality measures have been taken, and public order and good morals are not violated. Corresponding operation portals are provided for users to choose to authorize or refuse.

[0034] Example 1

[0035] Figure 1 This is a flowchart of a counter behavior detection method provided in Embodiment 1 of the present invention. This embodiment is applicable to detecting behaviors occurring at the counter, specifically detecting whether any abnormal security behavior is occurring at the counter. The method can be executed by a counter behavior detection device, which can be implemented in hardware and / or software and can be configured in an electronic device. This electronic device can be an electronic device installed in a financial institution for detecting behaviors occurring at the counter, specifically detecting whether any abnormal security behavior is occurring at the counter. Figure 1 As shown, the method includes:

[0036] Step 101: After obtaining the video and audio to be detected corresponding to the target counter, determine the action key point sequence, dialogue text, movement information and decibel information of the target counter based on the video and audio to be detected.

[0037] Optionally, the target counter can be any counter within a financial institution that needs to be checked for security anomalies. Security anomalies can refer to actions that endanger the safety of the teller, customer, or related items at the counter. Teller can be a technical staff member within the financial institution used to conduct business. Customer can be an individual needing to conduct specific business at the financial institution. Various types of security anomalies can exist, including but not limited to actions endangering the safety of teller, customer, or related items at the counter. These actions can include, but are not limited to, actions endangering property security, personal safety, and risk control. Property security anomalies can refer to actions endangering related items at the counter or the funds in a customer's account at the financial institution. Personal safety anomalies can refer to actions endangering the personal safety of teller or customer at the counter. Risk control anomalies can refer to actions by a customer at the counter using false identity information to endanger the security of their account at the financial institution or by a customer at the counter controlling the teller's business transactions.

[0038] Optionally, a video recording component is installed around the target counter. This component is used to record video of the target counter with the authorization of the sales personnel and customers located at the counter. The video of the target counter is a video recording taken by the video recording component with the authorization of the sales personnel and customers located at the counter, containing the counter, the sales personnel at the counter, and the customer. The video of the target counter consists of multiple frames of video images. Each frame of video image contains the counter, the sales personnel at the counter, and the customer. The video of the target counter involved in this invention is data authorized by the sales personnel and customers or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the video of the target counter all comply with the relevant laws, regulations, and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation interfaces for sales personnel and customers to choose to authorize or refuse. The video recording component can be a camera installed around the target counter.

[0039] Optionally, an audio recording component is installed around the target counter. This audio recording component is used to record the conversation between the salesperson and the customer at the counter, with their authorization, thus obtaining the audio of the target counter. The audio of the target counter is the audio recording of the conversation between the salesperson and the customer, recorded by the audio recording component with their authorization. The audio of the target counter involved in this invention is data authorized by the salesperson and the customer, or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the audio of the target counter all comply with the relevant laws, regulations, and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation interfaces for salespersons and customers to choose to authorize or refuse. The audio recording component can be a microphone installed around the target counter.

[0040] Optionally, the video capture component can periodically capture video of the target counter at a preset frequency, and send the video to an electronic device used for detecting behavior occurring within the counter, identifying any security anomalies. The audio recording component can periodically record conversations between sales personnel and customers at the counter, capturing audio of the target counter, and send this audio to the electronic device used for detecting behavior occurring within the counter, identifying any security anomalies. The video capture and audio recording components will capture and record simultaneously, and send the resulting video and audio of the target counter to the electronic device concurrently. The video and audio to be detected corresponding to the target counter can refer to a set of simultaneously received video and audio from the target counter.

[0041] Optionally, it can detect whether the electronic device receives the video and audio of the target counter. After each detection that the electronic device has received the video and audio of the target counter, the video and audio of the target counter received by the electronic device are obtained, thereby obtaining the video and audio to be detected corresponding to the target counter. Then, based on the video and audio to be detected corresponding to the target counter, the action key point sequence, dialogue text, movement information and decibel information of the target counter are determined.

[0042] Optionally, the motion keypoint sequence of the target counter is information that can be used to describe the body movements of the salesperson and customer located at the target counter. The motion keypoint sequence of the target counter includes the salesperson motion keypoint sequence and the customer motion keypoint sequence. The salesperson motion keypoint sequence can be a sequence obtained by arranging the coordinates of the salesperson's keypoints in each frame of the video image to be tested according to the order of the frames corresponding to the target counter. The coordinates of the salesperson's keypoints in the video image can be the coordinates of the salesperson's body keypoints in the video image. The customer motion keypoint sequence can be a sequence obtained by arranging the coordinates of the customer's keypoints in each frame of the video image to be tested according to the order of the frames corresponding to the target counter. The coordinates of the customer's keypoints in the video image can be the coordinates of the customer's body keypoints in the video image. Body keypoints can include, but are not limited to, the top of the head, tip of the nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. The coordinate position of the body keypoints can refer to the coordinates of the body keypoints in a two-dimensional image plane coordinate system established with the image center of the video image as the origin. The coordinates of key points on the human body can also refer to the coordinates of key points on the human body in a three-dimensional physical space coordinate system established with the origin of the video shooting component as the reference point.

[0043] Optionally, the dialogue text for the target counter can be text describing the conversation between the salesperson and the customer at the counter. The dialogue text for the target counter will include statements that can be used to characterize the emotional state of the salesperson and the customer at the counter.

[0044] Optionally, the movement information of the target counter includes staff movement information and customer movement information. Staff movement information can be calculated based on the coordinates of the staff's key points in each frame of the video image to be detected corresponding to the target counter, determining the displacement, velocity, acceleration, and direction of movement of the staff's body key points. Customer movement information can be calculated based on the coordinates of the customer's key points in each frame of the video image to be detected corresponding to the target counter, determining the displacement, velocity, acceleration, and direction of movement of the customer's body key points. In cases of abnormal property security behavior occurring at the counter, the speed of the customer's body key points will typically exceed a preset speed threshold. The preset speed threshold can be a pre-set speed threshold.

[0045] Optionally, the decibel information for the target counter can be information describing the volume of the voice when the salesperson and customer are conversing at the counter. In cases of unusual property security activity at the counter, the volume of the voice when the salesperson and customer are conversing at the counter is typically higher.

[0046] Optionally, based on the video to be detected and the audio to be detected, the action key point sequence, dialogue text, movement information, and decibel information of the target counter are determined, including: inputting the video to be detected to a sequence detection component to obtain the action key point sequence of the target counter output by the sequence detection component; wherein, the action key point sequence includes the action key point sequence of sales personnel and the action key point sequence of customers; inputting the audio to be detected to an audio-to-text conversion component to obtain the converted text output by the audio-to-text conversion component corresponding to the audio to be detected, and determining the converted text as the dialogue text of the target counter; inputting the video to be detected to a movement information detection component to obtain the movement information of the target counter output by the movement information detection component; wherein, the movement information includes the movement information of sales personnel and the movement information of customers; inputting the audio to be detected to a decibel detection component to obtain the decibel value of the audio to be detected output by the decibel detection component, and determining the decibel value of the audio to be detected as the decibel information of the target counter.

[0047] Therefore, based on the sequence detection component, audio-to-text conversion component, motion information detection component, and decibel detection component, the key action sequence, dialogue text, motion information, and decibel information of the counter can be quickly determined according to the video and audio to be detected corresponding to the counter. This allows for the determination of multiple dimensions of information related to the business personnel and customers at the counter, facilitating the comprehensive and accurate detection of any abnormal security behavior occurring at the counter based on the multiple dimensions of information related to the business personnel and customers at the counter.

[0048] Optionally, the sequence detection component can be a software module within an electronic device used to identify and analyze each frame of the video image in the video to be detected corresponding to the target counter, thereby obtaining the sequence of motion key points of the target counter. The input to the sequence detection component is the video to be detected corresponding to the target counter. The output of the sequence detection component is the sequence of motion key points of the target counter. The video to be detected corresponding to the target counter can be input into the sequence detection component. The sequence detection component will identify and analyze each frame of the video image in the video to be detected corresponding to the target counter, obtain the sequence of motion key points of the target counter, and then output the sequence of motion key points of the target counter. The process involves recognizing and analyzing each frame of the video image corresponding to the target counter to obtain the action key point sequence of the target counter, and then outputting the action key point sequence of the target counter. This includes: performing image recognition on each frame of the input video image corresponding to the target counter to identify the human body key points of the salesperson and customer in each frame of the video image, and determining the coordinates of the key points of the salesperson and customer in each frame of the video image; arranging the coordinates of the key points of the salesperson in each frame of the video image corresponding to the target counter according to the order of the video image corresponding to the target counter to obtain the action key point sequence of the target counter; arranging the coordinates of the key points of the customer in each frame of the video image corresponding to the target counter according to the order of the video image corresponding to the target counter to obtain the action key point sequence of the target counter; and outputting the action key point sequence of the salesperson and the action key point sequence of the target counter, i.e., the action key point sequence of the target counter. The action key point sequence of the target counter output by the sequence detection component can be obtained to determine the action key point sequence of the target counter.

[0049] Optionally, the audio-to-text conversion component can be a software module installed in an electronic device for converting audio into text. The input to the audio-to-text conversion component is audio. The output of the audio-to-text conversion component is the converted text corresponding to the audio. The converted text corresponding to the audio is the text obtained by converting the audio. The audio to be detected corresponding to the target counter can be input into the audio-to-text conversion component. The audio-to-text conversion component converts the input audio to be detected corresponding to the target counter into text, obtaining the converted text corresponding to the audio to be detected, and then outputs the converted text corresponding to the audio to be detected. Typically, the converted text corresponding to the audio to be detected is text that can be used to describe the dialogue between the salesperson and the customer at the counter, and will contain statements that can be used to characterize the emotional state of the salesperson and the customer at the counter. The converted text output by the audio-to-text conversion component corresponding to the audio to be detected can be obtained, and the converted text corresponding to the audio to be detected can be identified as the dialogue text of the target counter, thereby determining the dialogue text of the target counter.

[0050] Optionally, the motion information detection component can be a software module installed in an electronic device to identify and analyze each frame of video image in the video to be detected corresponding to the target counter, determine the key point coordinates of the salesperson and the customer in each frame of video image, and calculate the motion information of the target counter based on the key point coordinates of the salesperson and the customer in each frame of video image. The input of the motion information detection component is the video to be detected corresponding to the target counter. The output of the motion information detection component is the motion information of the target counter. The video to be detected corresponding to the target counter can be input into the motion information detection component. The motion information detection component will identify and analyze each frame of video image in the video to be detected corresponding to the target counter, determine the key point coordinates of the salesperson and the customer in each frame of video image, then calculate the salesperson motion information of the target counter based on the key point coordinates of the salesperson, and calculate the customer motion information of the target counter based on the key point coordinates of the customer in each frame of video image, and output the calculated salesperson motion information and customer motion information of the target counter, that is, output the motion information of the target counter. The motion information of the target counter output by the motion information detection component can be obtained to determine the motion information of the target counter.

[0051] Optionally, the decibel detection component can be a software module within an electronic device used to detect the decibel value of audio. The input to the decibel detection component is audio. The output of the decibel detection component is the decibel value of the audio. The audio to be detected, corresponding to the target counter, can be input into the decibel detection component. The decibel detection component analyzes and detects the input audio corresponding to the target counter, determines the decibel value of the audio to be detected, and then outputs the decibel value of the audio to be detected. Typically, the decibel value of the audio to be detected is information that can be used to describe the sound intensity when sales staff and customers at the counter are conversing. The decibel value of the audio to be detected output by the decibel detection component can be obtained, and this decibel value can be determined as the decibel information of the target counter, thereby determining the decibel information of the target counter.

[0052] Optionally, a dimensionality reduction algorithm corresponding to the action keypoint sequence of the target counter can be used to reduce the dimensionality of the action keypoint sequence of the target counter to a specified dimension. The dimensionality reduction algorithm corresponding to the action keypoint sequence of the target counter can be a pre-set algorithm for dimensionality reduction of the action keypoint sequence of the target counter.

[0053] Optionally, a dimensionality reduction algorithm corresponding to the dialogue text of the target counter can be used to reduce the dimensionality of the dialogue text of the target counter to a specified dimension. The dimensionality reduction algorithm corresponding to the dialogue text of the target counter can be a pre-set algorithm for dimensionality reduction of the dialogue text of the target counter.

[0054] Optionally, a dimensionality reduction algorithm corresponding to the movement information of the target counter can be used to reduce the dimensionality of the movement information of the target counter to a specified dimension. The dimensionality reduction algorithm corresponding to the movement information of the target counter can be a pre-set algorithm for reducing the dimensionality of the movement information of the target counter.

[0055] Optionally, a dimensionality reduction algorithm corresponding to the decibel information of the target counter can be used to reduce the dimensionality of the target counter's decibel information to a specified dimension. The dimensionality reduction algorithm corresponding to the decibel information of the target counter can be a pre-set algorithm for reducing the dimensionality of the target counter's decibel information.

[0056] Step 102: Determine the probability of various preset actions occurring at the target counter based on the sequence of key action points.

[0057] Optionally, preset actions can be physical actions that sales personnel and customers at the target counter would perform. Various preset actions can be multiple different types of physical actions that sales personnel and customers at the target counter would perform. These preset actions can include, but are not limited to, normal actions, abnormal acquisition actions, aggressive actions, fainting actions, and defensive actions. Normal actions refer to physical actions that a person at the counter would perform when no abnormal security behavior occurs at the counter. Abnormal acquisition actions refer to physical actions that a person at the counter would perform to acquire relevant items at the counter when abnormal property security behavior occurs at the counter. Aggressive actions refer to physical actions that a person at the counter would perform that endanger the personal safety of others when abnormal personal security behavior or risk control behavior occurs at the counter. Fainting actions refer to physical actions that a person at the counter would perform due to loss of consciousness when abnormal personal security behavior occurs at the counter. Defensive actions refer to physical actions that a person at the counter would perform to protect themselves when abnormal personal security behavior or property security behavior occurs at the counter. For each type of preset action, the probability of the preset action occurring at the target counter can refer to the probability that the salesperson and the customer at the target counter will perform the preset action.

[0058] Optionally, based on the sequence of key action points, the probability of various preset actions occurring at the target counter is determined, including: inputting the sequence of key action points into a pre-trained action recognition model to obtain the probability of various preset actions occurring at the target counter as output by the action recognition model.

[0059] Therefore, based on a pre-trained action recognition model, the probability of various preset actions occurring at the counter can be quickly determined by the sequence of key action points that can be used to describe the physical movements of sales personnel and customers at the counter.

[0060] Optionally, the electronic device is equipped with a pre-trained motion recognition model. This model can be used to analyze and detect the sequence of key motion points on the counter, determining the probability of various preset actions occurring on the counter. The input to the motion recognition model is the sequence of key motion points on the counter, and the output is the probability of each preset action occurring on the counter. After the sequence of key motion points on the counter is input into the motion recognition model, the model analyzes and detects the sequence, determines the probability of each preset action occurring on the counter, and then outputs the probability of each preset action occurring on the counter. The probabilities of various preset actions occurring on the counter, output by the motion recognition model, can be obtained.

[0061] Optionally, the sequence of motion key points of the target counter can be input into a pre-trained motion recognition model. The motion recognition model analyzes and detects the sequence of motion key points of the target counter, determines the probability of various preset actions occurring at the target counter, and then outputs the probability of each preset action occurring at the target counter. The probability of each preset action occurring at the target counter can be obtained from the output of the motion recognition model, thereby determining the probability of each preset action occurring at the target counter.

[0062] Optionally, multiple key point sequences of motion at multiple counters can be collected in advance. The collected key point sequences of motion at multiple counters can be used to train a machine learning model to obtain a motion recognition model. Then, the motion recognition model can be set up to detect behaviors occurring at the counters and detect whether there are any security anomalies occurring in the electronic devices at the counters.

[0063] Step 103: Determine the probability of various preset interaction states occurring at the target counter based on the dialogue text.

[0064] Optionally, the preset interaction state can be the emotional state that the sales staff and customers at the target counter would experience. Various preset interaction states can be multiple different types of emotional states that the sales staff and customers at the target counter would experience. These preset interaction states can include normal, angry, fearful, and controlling. Normal can refer to the emotional state that the staff at the counter would experience when no security abnormalities occur. Anger can refer to the aggressive emotional state that the staff at the counter would experience when property security abnormalities, personal safety abnormalities, or risk control abnormalities occur. Fear can refer to the emotional state that the staff at the counter would experience when their own safety is not guaranteed when property security abnormalities, personal safety abnormalities, or risk control abnormalities occur. Control can refer to the emotional state that the staff at the counter would experience when they are controlled to perform a specified operation when risk control abnormalities occur. For each type of preset interaction state, the probability of the preset interaction state occurring at the target counter can refer to the probability that the sales staff and customers at the target counter would experience the preset interaction state.

[0065] Optionally, based on the dialogue text, the probability of various preset interaction states occurring at the target counter is determined, including: inputting the dialogue text into a pre-trained text analysis model to obtain the probability of various preset interaction states occurring at the target counter as output by the text analysis model.

[0066] Therefore, based on a pre-trained text analysis model, the probability of various preset interaction states occurring at the counter can be quickly determined from the dialogue text containing statements that characterize the emotional states of sales staff and customers at the counter.

[0067] Optionally, the electronic device is equipped with a pre-trained text analysis model. This model can be used to analyze and detect the dialogue text at the counter, determining the probability of various preset interaction states occurring at the counter. The input to the text analysis model is the dialogue text at the counter, and the output is the probability of each preset interaction state occurring at the counter. After the dialogue text at the counter is input into the text analysis model, the model analyzes and detects the dialogue text, determines the probability of each preset interaction state occurring at the counter, and then outputs the probability of each preset interaction state occurring at the counter. By obtaining the probability of each preset interaction state occurring at the counter output by the text analysis model, the probability of each preset interaction state occurring at the counter can be determined.

[0068] Optionally, the dialogue text at the counter can be input into a pre-trained text analysis model. The text analysis model will analyze and detect the dialogue text at the counter, determine the probability of various preset interaction states occurring at the counter, and then output the probability of each preset interaction state occurring at the counter. The probabilities of various preset interaction states occurring at the counter, output by the text analysis model, can be obtained.

[0069] Optionally, conversation texts from multiple counters can be collected in advance, and the collected conversation texts from multiple counters can be used to train a machine learning model to obtain a text analysis model. The text analysis model can then be set up to detect behaviors occurring at the counters and detect whether there are any security anomalies occurring on electronic devices at the counters.

[0070] Step 104: Based on the probability of various preset actions and preset interaction states occurring in the target counter, the movement information, and the decibel information, determine the probability of various security anomalies occurring in the target counter, and detect whether the probability of various security anomalies occurring in the target counter is greater than the security threshold of various security anomalies.

[0071] Optionally, for each type of security anomaly, the probability of the security anomaly occurring in the target counter can be determined based on the probability of various preset actions and various preset interaction states occurring in the target counter, the movement information of the target counter, and the decibel information of the target counter.

[0072] Optionally, based on the probability of various preset actions and preset interaction states occurring at the target counter, the movement information, and the decibel information, the probability of various abnormal security behaviors occurring at the target counter is determined, including: constructing an action probability vector based on the probability of various preset actions occurring at the target counter; constructing an interaction state probability vector based on the probability of various preset interaction states occurring at the target counter; inputting the action probability vector, the interaction state probability vector, the movement information, and the decibel information into a pre-trained abnormal behavior classification model to obtain the probability of various abnormal security behaviors occurring at the target counter output by the abnormal behavior classification model; wherein, various abnormal security behaviors include property security abnormal behaviors, personal safety abnormal behaviors, and risk prevention and control abnormal behaviors.

[0073] Therefore, based on the abnormal behavior classification model, the probability of abnormal behaviors related to property security, personal safety, and risk prevention occurring at the counter can be quickly determined according to the action probability vector, interaction state probability vector, movement information, and decibel information.

[0074] Optionally, preset actions include normal actions, abnormal acquisition actions, attack actions, fainting actions, and defensive actions. An action probability vector can be constructed based on the probability of each action occurring at the target counter. This action probability vector can be a 1×5 row vector. The probability of a normal action occurring at the target counter is the first element of the action probability vector. The probability of an abnormal acquisition action occurring at the target counter is the second element. The probability of an attack action occurring at the target counter is the third element. The probability of a fainting action occurring at the target counter is the fourth element. The probability of a defensive action occurring at the target counter is the fifth element.

[0075] Optionally, preset interaction states include Normal, Angry, Fearful, and Controlled. An interaction state probability vector can be constructed based on the probability of each of these states occurring at the target counter. This vector can be a 1×4 row vector. The probability of Normal occurring at the target counter is the first element of the interaction state probability vector. The probability of Angry occurring at the target counter is the second element. The probability of Fear occurring at the target counter is the third element. The probability of Controlled occurring at the target counter is the fourth element.

[0076] Optionally, the electronic device is equipped with a pre-trained abnormal behavior classification model. This model can be trained using a machine learning model to analyze and detect probability vectors (constructed from the probabilities of various preset actions occurring at the counter), probability vectors (constructed from the probabilities of various preset interaction states occurring at the counter), counter movement information, and counter decibel information. The model determines the probability of abnormal property safety behavior, abnormal personal safety behavior, and abnormal risk control behavior occurring at the counter. The inputs to the abnormal behavior classification model are the probability vectors (constructed from the probabilities of various preset actions occurring at the counter), the probability vectors (constructed from the probabilities of various preset interaction states occurring at the counter), counter movement information, and counter decibel information. The outputs of the abnormal behavior classification model are the probabilities of abnormal property safety behavior, abnormal personal safety behavior, and abnormal risk control behavior occurring at the counter. After inputting the action probability vector (constructed based on the probability of various preset actions occurring at the counter), the interaction state probability vector (constructed based on the probability of various preset interaction states occurring at the counter), counter movement information, and counter decibel information into the abnormal behavior classification model, the abnormal behavior classification model analyzes and detects the action probability vector, the interaction state probability vector, the counter movement information, and the counter decibel information to determine the probability of abnormal property safety behavior, abnormal personal safety behavior, and abnormal risk prevention behavior occurring at the counter. It then outputs the probabilities of these three types of abnormal behavior occurring at the counter. The model allows users to obtain these probabilities from the abnormal behavior classification model, thereby determining the probability of these three types of abnormal behavior occurring at the counter.

[0077] Optionally, the action probability vector (constructed based on the probability of various preset actions occurring at the target counter), the interaction state probability vector (constructed based on the probability of various preset interaction states occurring at the target counter), the movement information of the target counter, and the decibel information of the target counter can be input into the abnormal behavior classification model. The abnormal behavior classification model will analyze and detect the action probability vector, the interaction state probability vector, the movement information, and the decibel information of the target counter to determine the probability of property safety abnormal behavior, personal safety abnormal behavior, and risk prevention abnormal behavior occurring at the target counter. It then outputs the probabilities of these three types of abnormal behavior. The probabilities of property safety abnormal behavior, personal safety abnormal behavior, and risk prevention abnormal behavior occurring at the target counter can be obtained from the abnormal behavior classification model.

[0078] Optionally, for each type of security anomaly, the security threshold can be a value calculated based on the probability that the calculated security anomaly would occur at the counter. Generally, when the calculated probability of a security anomaly occurring at the counter is greater than the security threshold, it can be determined that a security anomaly has occurred at the counter. When the calculated probability of a security anomaly occurring at the counter is less than or equal to the security threshold, it can be determined that no security anomaly has occurred at the counter.

[0079] Optionally, the security threshold for abnormal property security behavior can be a value calculated based on the probability that such behavior would occur at the counter. Generally, when the calculated probability of abnormal property security behavior occurring at the counter is greater than the security threshold, it can be determined that abnormal property security behavior has occurred at the counter. When the calculated probability of abnormal property security behavior occurring at the counter is less than or equal to the security threshold, it can be determined that no abnormal property security behavior has occurred at the counter. For example, the security threshold for abnormal property security behavior is 0.85.

[0080] Optionally, the safety threshold for abnormal personal safety behavior can be a value calculated based on the probability that such behavior would occur at the counter. Generally, when the calculated probability of such behavior occurring at the counter is greater than the safety threshold, it can be determined that an abnormal personal safety behavior has occurred at the counter. When the calculated probability of such behavior occurring at the counter is less than or equal to the safety threshold, it can be determined that no abnormal personal safety behavior has occurred at the counter.

[0081] Optionally, the safety threshold for preventing abnormal risk control behavior can be a value calculated based on the probability that such behavior would occur at the counter. Generally, when the calculated probability of such behavior occurring at the counter is greater than the safety threshold, it can be determined that abnormal risk control behavior has occurred at the counter. When the calculated probability of such behavior occurring at the counter is less than or equal to the safety threshold, it can be determined that no abnormal risk control behavior has occurred at the counter.

[0082] Optionally, after determining the probability of abnormal property security behavior occurring at the counter, the probability of abnormal personal safety behavior occurring at the counter, and the probability of abnormal risk control behavior occurring at the counter, it may be tested whether the probability of abnormal property security behavior occurring at the counter is greater than the safety threshold for abnormal property security behavior, the probability of abnormal personal safety behavior occurring at the counter is greater than the safety threshold for abnormal personal safety behavior, and the probability of abnormal risk control behavior occurring at the counter is greater than the safety threshold for abnormal risk control behavior.

[0083] Step 105: When the probability of detecting a target security anomaly at the target counter is greater than the security threshold of the target security anomaly, it is determined that the target security anomaly has occurred at the target counter, and the alarm information of the target security anomaly is provided to the monitoring user of the target counter.

[0084] Optionally, the target security anomaly can be any type of security anomaly. The alarm information for the target security anomaly can be information used to indicate that a target security anomaly has occurred at the target counter. The monitoring user for the target counter can be a technical staff member within the financial institution responsible for handling security anomalies occurring at the target counter. When the probability of detecting a target security anomaly at the target counter is greater than the target security anomaly threshold, it is determined that a target security anomaly has occurred at the target counter, and the alarm information for the target security anomaly is provided to the monitoring user of the target counter, thereby alerting the monitoring user to the occurrence of the target security anomaly and enabling them to handle the target security anomaly promptly. When the probability of detecting a target security anomaly at the target counter is less than or equal to the target security anomaly threshold, it is determined that no target security anomaly has occurred at the target counter.

[0085] Optionally, when the probability of detecting abnormal property security behavior at the target counter is greater than the security threshold for abnormal property security behavior, it is determined that abnormal property security behavior has occurred at the target counter. An alarm for this abnormal behavior is then provided to the monitoring user of the target counter, thereby alerting the monitoring user to the occurrence of abnormal property security behavior and enabling them to handle the abnormal behavior promptly. When the probability of detecting abnormal property security behavior at the target counter is less than or equal to the security threshold for abnormal property security behavior, it is determined that no abnormal property security behavior has occurred at the target counter.

[0086] Optionally, when the probability of detecting abnormal personal safety behavior at the target counter is greater than the safety threshold for such behavior, it is determined that an abnormal personal safety behavior has occurred at the target counter. An alarm for this behavior is then provided to the monitoring user at the target counter, prompting them to address the issue promptly. Conversely, when the probability of detecting abnormal personal safety behavior at the target counter is less than or equal to the safety threshold, it is determined that no abnormal personal safety behavior has occurred at the target counter.

[0087] Optionally, when the probability of detecting abnormal risk control behavior at the target counter is greater than the safety threshold for such behavior, it is determined that abnormal risk control behavior has occurred at the target counter. An alarm for this abnormal behavior is then provided to the monitoring user of the target counter, prompting the monitoring user to promptly address the abnormal behavior. Conversely, when the probability of detecting abnormal risk control behavior at the target counter is less than or equal to the safety threshold, it is determined that no abnormal risk control behavior has occurred at the target counter.

[0088] Optionally, when the probability of a detected abnormal security behavior occurring at the target counter is greater than the security threshold for the abnormal security behavior, it is determined that the abnormal security behavior has occurred at the target counter, and alarm information for the abnormal security behavior is provided to the monitoring user of the target counter. This includes: when the probability of a detected abnormal security behavior occurring at the target counter is greater than the security threshold for the abnormal security behavior, it is determined that the abnormal security behavior has occurred at the target counter, and the monitoring user's terminal device is controlled to display the alarm information for the abnormal security behavior. The monitoring user's terminal device can be a terminal device used by the monitoring user. The alarm information for the abnormal security behavior can be an alarm pop-up window indicating that an abnormal security behavior has occurred at the target counter.

[0089] Therefore, by displaying alarm information about abnormal security behavior on the monitoring user's terminal device, the monitoring user can be alerted that abnormal security behavior has occurred at the counter, enabling the monitoring user to handle the abnormal security behavior at the counter in a timely manner.

[0090] Optionally, when the probability of detecting abnormal property security behavior at the target counter is greater than the security threshold for abnormal property security behavior, it is determined that abnormal property security behavior has occurred at the target counter, and the terminal device of the monitoring user at the target counter is controlled to display alarm information for abnormal property security behavior. When the probability of detecting abnormal personal safety behavior at the target counter is greater than the security threshold for abnormal personal safety behavior, it is determined that abnormal personal safety behavior has occurred at the target counter, and the terminal device of the monitoring user at the target counter is controlled to display alarm information for abnormal personal safety behavior. When the probability of detecting abnormal risk control behavior at the target counter is greater than the security threshold for abnormal risk control behavior, it is determined that abnormal risk control behavior has occurred at the target counter, and the terminal device of the monitoring user at the target counter is controlled to display alarm information for abnormal risk control behavior.

[0091] Optionally, when the probability of abnormal property security behavior occurring at the counter is less than or equal to the security threshold for abnormal property security behavior, the probability of abnormal personal safety behavior occurring at the counter is less than or equal to the security threshold for abnormal personal safety behavior, and the probability of abnormal risk control behavior occurring at the counter is less than or equal to the security threshold for abnormal risk control behavior, it can be determined that no abnormal property security behavior, abnormal personal safety behavior, or abnormal risk control behavior has occurred at the target counter, and the counter behavior detection process is determined to end.

[0092] Optionally, after determining that one or more types of abnormal security behaviors, including property security abnormal behaviors, personal security abnormal behaviors, and risk prevention and control abnormal behaviors, have occurred at the target counter, the alarm information of one or more types of abnormal security behaviors has been provided to the monitoring user of the target counter, and then the counter behavior detection process is determined to be over.

[0093] The technical solution of this invention involves, after acquiring the video and audio to be detected corresponding to the target counter, determining the action key point sequence, dialogue text, movement information, and decibel information of the target counter based on the video and audio; then, determining the probability of various preset actions occurring at the target counter based on the action key point sequence; determining the probability of various preset interaction states occurring at the target counter based on the dialogue text; determining the probability of various security anomalies occurring at the target counter based on the probability of various preset actions and interaction states occurring at the target counter, the movement information, and the decibel information, and detecting whether the probability of various security anomalies occurring at the target counter is greater than the security threshold for various security anomalies; when the probability of a target security anomaly occurring at the target counter is detected to be greater than the security threshold for the target security anomaly, it is determined that a target security anomaly has occurred at the target counter, and alarm information for the target security anomaly is provided to the target counter. This user monitoring solution addresses the shortcomings of existing counter behavior detection technologies, which rely on manual operation, lack accuracy, and fail to comprehensively and accurately detect any abnormal behavior occurring at the counter. This makes it difficult to effectively protect the safety of financial institution staff, customers, and related items. The new solution automatically determines key action sequences, dialogue text, movement information, and decibel levels at the counter based on corresponding video and audio data. It comprehensively and accurately detects any abnormal behavior occurring at the counter, leveraging multiple dimensions of information related to staff and customers at the counter. The solution promptly issues alarms upon detection of abnormal behavior, effectively protecting the safety of financial institution staff, customers, and related items.

[0094] Example 2

[0095] Figure 2 This is a flowchart illustrating a counter behavior detection method provided in Embodiment 2 of the present invention. Embodiments of the present invention can be combined with various optional solutions from one or more of the above embodiments. For example... Figure 2 As shown, the method includes:

[0096] Step 201: After obtaining the video and audio to be detected corresponding to the target counter, determine the action key point sequence, dialogue text, movement information and decibel information of the target counter based on the video and audio to be detected.

[0097] Step 202: Input the sequence of key action points into a pre-trained action recognition model to obtain the probability of various preset actions occurring at the target counter, as output by the action recognition model.

[0098] Step 203: Input the dialogue text into a pre-trained text analysis model to obtain the probability of various preset interaction states occurring at the target counter, as output by the text analysis model.

[0099] Step 204: Construct an action probability vector based on the probability of various preset actions occurring at the target counter.

[0100] Step 205: Construct an interaction state probability vector based on the probability of various preset interaction states occurring in the target counter.

[0101] Step 206: Input the action probability vector, the interaction state probability vector, the movement information, and the decibel information into the pre-trained abnormal behavior classification model to obtain the probability of various types of security abnormal behaviors occurring in the target counter, as output by the abnormal behavior classification model, and detect whether the probability of various types of security abnormal behaviors occurring in the target counter is greater than the security threshold of various types of security abnormal behaviors.

[0102] Among these, various types of abnormal safety behaviors include abnormal behaviors related to property safety, abnormal behaviors related to personal safety, and abnormal behaviors related to risk prevention and control.

[0103] Step 207: When the probability of a target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, it is determined that the target security anomaly has occurred at the target counter, and the terminal device of the monitoring user at the target counter is controlled to display the alarm information of the target security anomaly.

[0104] The technical solutions of this invention can, based on a pre-trained action recognition model, quickly determine the probability of various preset actions occurring at the counter by using key action point sequences that describe the physical movements of sales personnel and customers at the counter. They can, based on a pre-trained text analysis model, quickly determine the probability of various preset interaction states occurring at the counter by using dialogue text containing statements representing the emotional states of sales personnel and customers at the counter. Furthermore, they can, based on an abnormal behavior classification model, quickly determine the probability of abnormal property safety behavior, abnormal personal safety behavior, and abnormal risk control behavior occurring at the counter by using action probability vectors, interaction state probability vectors, movement information, and decibel information. Finally, they can display alarm information about the abnormal safety behavior on the monitoring user's terminal device, alerting the monitoring user to the occurrence of abnormal safety behavior at the counter and enabling the monitoring user to promptly address such behavior.

[0105] Example 3

[0106] Figure 3 This is a schematic diagram of a counter behavior detection device provided in Embodiment 3 of the present invention. The device can be configured in an electronic device. Figure 3 As shown, the device includes: a counter information determination module 301, a first probability determination module 302, a second probability determination module 303, a behavior detection module 304, and an alarm module 305.

[0107] The system includes the following modules: a counter information determination module 301, which, after acquiring the video and audio to be detected corresponding to the target counter, determines the action key point sequence, dialogue text, movement information, and decibel information of the target counter based on the video and audio; a first probability determination module 302, which determines the probability of various preset actions occurring at the target counter based on the action key point sequence; a second probability determination module 303, which determines the probability of various preset interaction states occurring at the target counter based on the dialogue text; a behavior detection module 304, which determines the probability of various security abnormal behaviors occurring at the target counter based on the probability of various preset actions and preset interaction states occurring at the target counter, the movement information, and the decibel information, and detects whether the probability of various security abnormal behaviors occurring at the target counter is greater than the security threshold of various security abnormal behaviors; and an alarm module 305, which, when the probability of a target security abnormal behavior occurring at the target counter is detected to be greater than the security threshold of the target security abnormal behavior, determines that the target security abnormal behavior has occurred at the target counter and provides the alarm information of the target security abnormal behavior to the monitoring user of the target counter.

[0108] The technical solution of this invention involves, after acquiring the video and audio to be detected corresponding to the target counter, determining the action key point sequence, dialogue text, movement information, and decibel information of the target counter based on the video and audio; then, determining the probability of various preset actions occurring at the target counter based on the action key point sequence; determining the probability of various preset interaction states occurring at the target counter based on the dialogue text; determining the probability of various security anomalies occurring at the target counter based on the probability of various preset actions and interaction states occurring at the target counter, the movement information, and the decibel information, and detecting whether the probability of various security anomalies occurring at the target counter is greater than the security threshold for various security anomalies; when the probability of a target security anomaly occurring at the target counter is detected to be greater than the security threshold for the target security anomaly, it is determined that a target security anomaly has occurred at the target counter, and alarm information for the target security anomaly is provided to the target counter. This user monitoring solution addresses the shortcomings of existing counter behavior detection technologies, which rely on manual operation, lack accuracy, and fail to comprehensively and accurately detect any abnormal behavior occurring at the counter. This makes it difficult to effectively protect the safety of financial institution staff, customers, and related items. The new solution automatically determines key action sequences, dialogue text, movement information, and decibel levels at the counter based on corresponding video and audio data. It comprehensively and accurately detects any abnormal behavior occurring at the counter, leveraging multiple dimensions of information related to staff and customers at the counter. The solution promptly issues alarms upon detection of abnormal behavior, effectively protecting the safety of financial institution staff, customers, and related items.

[0109] In an optional embodiment of the present invention, the counter information determination module 301, when performing the operation of determining the action key point sequence, dialogue text, movement information, and decibel information of the target counter based on the video to be detected and the audio to be detected, is specifically configured to: input the video to be detected to a sequence detection component to obtain the action key point sequence of the target counter output by the sequence detection component; wherein, the action key point sequence includes the action key point sequence of sales personnel and the action key point sequence of customers; input the audio to be detected to an audio-to-text conversion component to obtain the converted text output by the audio-to-text conversion component corresponding to the audio to be detected, and determine the converted text as the dialogue text of the target counter; input the video to be detected to a movement information detection component to obtain the movement information of the target counter output by the movement information detection component; wherein, the movement information includes the movement information of sales personnel and the movement information of customers; input the audio to be detected to a decibel detection component to obtain the decibel value of the audio to be detected output by the decibel detection component, and determine the decibel value of the audio to be detected as the decibel information of the target counter.

[0110] In an optional embodiment of the present invention, the first probability determination module 302 is specifically used to: input the sequence of action key points into a pre-trained action recognition model to obtain the probability of various preset actions occurring in the target counter as output by the action recognition model.

[0111] In an optional embodiment of the present invention, the second probability determination module 303 is specifically used to: input the dialogue text into a pre-trained text analysis model to obtain the probability of various preset interaction states occurring in the target counter as output by the text analysis model.

[0112] In an optional embodiment of the present invention, the behavior detection module 304, when performing the operation of determining the probability of various types of abnormal security behaviors occurring at the target counter based on the probability of various preset actions and various preset interaction states occurring at the target counter, the movement information, and the decibel information, is specifically configured to: construct an action probability vector based on the probability of various preset actions occurring at the target counter; construct an interaction state probability vector based on the probability of various preset interaction states occurring at the target counter; input the action probability vector, the interaction state probability vector, the movement information, and the decibel information into a pre-trained abnormal behavior classification model to obtain the probability of various types of abnormal security behaviors occurring at the target counter output by the abnormal behavior classification model; wherein, the various types of abnormal security behaviors include property safety abnormal behaviors, personal safety abnormal behaviors, and risk prevention and control abnormal behaviors.

[0113] In an optional embodiment of the present invention, the alarm module 305 is specifically configured to: determine that the target security abnormality has occurred at the target counter when the probability of the detected target security abnormality is greater than the security threshold of the target security abnormality, and control the terminal device of the monitoring user of the target counter to display the alarm information of the target security abnormality.

[0114] The counter behavior detection device provided in the embodiments of the present invention can execute the counter behavior detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] Example 4

[0116] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement the counter behavior detection method of embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, electronic devices, blade electronic devices, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0117] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0118] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0119] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as counter behavior detection methods.

[0120] In some embodiments, the counter behavior detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on a heterogeneous hardware accelerator via read-only memory and / or a communication unit. When the computer program is loaded into random access memory and executed by a processor, one or more steps of the counter behavior detection method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the counter behavior detection method by any other suitable means (e.g., by means of firmware).

[0121] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices (PLCs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or electronic device.

[0123] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] To provide user interaction, the systems and techniques described herein can be implemented on a heterogeneous hardware accelerator, which includes: a display device (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the heterogeneous hardware accelerator. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or haptic feedback); and input from the user can be received in any form (including sound input, voice input, or haptic input).

[0125] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data electronic devices), or computing systems that include middleware components (e.g., application electronic devices), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0126] A computing system can include clients and electronic devices. Clients and electronic devices are generally geographically separated and typically interact via communication networks. The client-electronic device relationship is created by computer programs running on the respective computers and establishing a client-electronic device relationship between them. Electronic devices can be cloud electronic devices, also known as cloud computing electronic devices or cloud servers, which are host products within the cloud computing service system. These address the shortcomings of traditional physical hosts and virtual private server services, such as high management difficulty and weak business scalability.

[0127] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0128] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A counter behavior detection method, characterized in that, include: After obtaining the video and audio to be detected corresponding to the target counter, the key action point sequence, dialogue text, movement information and decibel information of the target counter are determined based on the video and audio to be detected. Based on the sequence of key action points, determine the probability of each preset action occurring at the target counter; Based on the dialogue text, determine the probability of each preset interaction state occurring at the target counter; Based on the probability of various preset actions and preset interaction states occurring in the target counter, the movement information, and the decibel information, the probability of various abnormal security behaviors occurring in the target counter is determined, and it is detected whether the probability of various abnormal security behaviors occurring in the target counter is greater than the security threshold of various abnormal security behaviors. When the probability of a target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, it is determined that the target security anomaly has occurred at the target counter, and the alarm information of the target security anomaly is provided to the monitoring user of the target counter.

2. The counter behavior detection method according to claim 1, characterized in that, Based on the video and audio to be detected, determine the sequence of key action points, dialogue text, movement information, and decibel information of the target counter, including: The video to be detected is input into the sequence detection component to obtain the action key point sequence of the target counter output by the sequence detection component; wherein, the action key point sequence includes the action key point sequence of the sales staff and the action key point sequence of the customer; The audio to be detected is input into the audio-to-text conversion component to obtain the converted text output by the audio-to-text conversion component corresponding to the audio to be detected, and the converted text is determined as the dialogue text of the target counter; The video to be detected is input into the mobile information detection component to obtain the mobile information of the target counter output by the mobile information detection component; wherein, the mobile information includes the mobile information of the sales staff and the mobile information of the customer; The audio to be detected is input to the decibel detection component to obtain the decibel value of the audio to be detected output by the decibel detection component, and the decibel value of the audio to be detected is determined as the decibel information of the target counter.

3. The counter behavior detection method according to claim 1, characterized in that, Based on the sequence of key action points, determine the probability of various preset actions occurring at the target counter, including: The sequence of key action points is input into a pre-trained action recognition model to obtain the probability of various preset actions occurring at the target counter, as output by the action recognition model.

4. The counter behavior detection method according to claim 1, characterized in that, Based on the dialogue text, determine the probability of various preset interaction states occurring at the target counter, including: The dialogue text is input into a pre-trained text analysis model to obtain the probability of various preset interaction states occurring at the target counter, as output by the text analysis model.

5. The counter behavior detection method according to claim 1, characterized in that, Based on the probability of various preset actions and preset interaction states occurring at the target counter, the movement information, and the decibel information, the probability of various abnormal security behaviors occurring at the target counter is determined, including: Construct an action probability vector based on the probability of various preset actions occurring at the target counter; Construct an interaction state probability vector based on the probability of various preset interaction states occurring at the target counter; The action probability vector, the interaction state probability vector, the movement information, and the decibel information are input into a pre-trained abnormal behavior classification model to obtain the probability of various types of abnormal security behaviors occurring at the target counter, as output by the abnormal behavior classification model; wherein, various types of abnormal security behaviors include property security abnormal behaviors, personal safety abnormal behaviors, and risk prevention and control abnormal behaviors.

6. The counter behavior detection method according to claim 1, characterized in that, When the probability of a detected abnormal security behavior occurring at the target counter is greater than the security threshold for the abnormal security behavior, it is determined that the abnormal security behavior has occurred at the target counter, and alarm information for the abnormal security behavior is provided to the monitoring user of the target counter, including: When the probability of a target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, it is determined that the target security anomaly has occurred at the target counter, and the terminal device of the monitoring user at the target counter is controlled to display the alarm information of the target security anomaly.

7. A counter behavior detection device, characterized in that, include: The counter information determination module is used to determine the action key point sequence, dialogue text, movement information and decibel information of the target counter after obtaining the video and audio to be detected corresponding to the target counter; The first probability determination module is used to determine the probability of various preset actions occurring in the target counter based on the sequence of key action points. The second probability determination module is used to determine the probability of various preset interaction states occurring at the target counter based on the dialogue text. The behavior detection module is used to determine the probability of various security abnormal behaviors occurring at the target counter based on the probability of various preset actions and various preset interaction states occurring at the target counter, the movement information, and the decibel information, and to detect whether the probability of various security abnormal behaviors occurring at the target counter is greater than the security threshold of various security abnormal behaviors. The alarm module is used to determine that a target security anomaly has occurred at the target counter when the probability of the detected target security anomaly occurring at the target counter is greater than the security threshold of the target security anomaly, and to provide alarm information of the target security anomaly to the monitoring user of the target counter.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the counter behavior detection method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the counter behavior detection method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the counter behavior detection method according to any one of claims 1-6.

Images