A device operation flow optimization system and device based on interaction behavior data

Abstract

This invention discloses a device operation process optimization system and device based on interactive behavior data, belonging to the field of human-computer interaction technology. The invention performs facial recognition on a designated device for the user to be operated on, recording all identified users as potential users. If several potential users exist, user filtering is performed to determine the primary user to complete the device operation; otherwise, the operation is directly performed based on the potential user. If user filtering is performed, multiple interaction judgments are conducted using camera interaction behavior data to determine whether to execute the human-computer interaction operation, thereby identifying the primary user. During the execution of the human-computer interaction operation, a simultaneous human-computer interaction operation judgment is performed to select the issued interaction behavior command. Based on the response of the potential users to the interaction behavior command, the corresponding primary user is determined to complete the device operation. This solves the problem in existing technologies where systems struggle to accurately identify target users, leading to incorrect identity matching and potential payment errors.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and in particular to a device operation process optimization system and device based on interactive behavior data. Background Technology

[0002] To promote the development of the digital economy, facial recognition payment has facilitated the construction of a cashless society, reduced the circulation and management costs of paper money, and because a face is a unique biometric feature that is difficult to forge or steal, it is widely used in scenarios such as subway entry, shopping mall payment, hospital registration, and school cafeteria, thus promoting the construction of "smart cities" and "smart campuses".

[0003] For example, the facial recognition payment management system disclosed in Chinese invention patent CN118037302A includes: a payment information input module for inputting payment information; a facial recognition module for capturing a pre-payment video when the user arrives at the payment location, for facial recognition authentication to obtain user information, and for capturing a post-payment video before the user leaves the payment location; an abnormal action determination module for judging whether the pre-payment or post-payment video is abnormal through an abnormal action machine learning model, and outputting an abnormal action signal if abnormal; a payment behavior determination module, including a risk rule storage module and a payment record determination module; and a payment record determination module including a payment information capture module and a risk decision module.

[0004] In the construction of smart campuses, payment systems are a crucial component. Facial recognition payment, as a new type of intelligent payment technology, can be integrated with other intelligent facilities on campus (such as smart access control, classroom attendance systems, and library borrowing systems) to enhance the campus's informatization and intelligentization levels. Through this technology, schools can more efficiently manage student and faculty identity verification, attendance management, and other related matters.

[0005] Facial recognition payment is part of the digital transformation of campuses, enabling them to gradually move towards a "cashless society," promoting the development of digital payments to a higher level, and further improving the school's digital infrastructure. With the application of technologies such as artificial intelligence, big data, and cloud computing, campus management will become more intelligent and flexible.

[0006] In facial recognition payment devices (especially in campus settings), human-computer interaction is not a static, singular process, but a dynamic behavioral pattern influenced by the environment and user status. Therefore, it is necessary to design and optimize the device's operation process based on interaction behavior data. The current implementation process mainly includes: environmental perception, interaction behavior data collection, decision optimization, payment processing, and user feedback. Environmental perception primarily involves activating the environmental perception module to monitor data such as lighting, noise, and user distance in real time. After the user arrives at the recognition area, the camera uses facial recognition technology to detect and identify the user's facial features for identity verification. Interaction behavior data collection... Data acquisition includes touch behavior detection, head nodding behavior detection, and voice behavior detection. Then, decision optimization is based on the collected interaction behavior data, evaluating and selecting the most suitable interaction method. Finally, payment processing and user feedback: after confirming the main user's identity, the system generates a payment request and transmits it to the payment module. At this point, the system transmits information such as the payment amount, user account, and payment method to the payment platform. After payment confirmation, the payment module processes the payment request and returns the payment result. If the payment is successful, the device provides feedback information, such as a payment success notification. Based on the payment result, the system provides corresponding feedback to the user, such as voice prompts or screen displays. By integrating multiple interaction methods (touch, head nodding, voice) and an intelligent decision engine, the system can adjust the operation process in real time according to dynamic changes in the environment and user behavior to optimize payment efficiency and accuracy.

[0007] The above-mentioned technology has at least the following technical problems:

[0008] Current face detection algorithms typically only detect faces and lack target selection logic. When multiple faces appear in the frame simultaneously, the system may randomly select or choose the face with the highest confidence level for recognition, leading to incorrect identification. When multiple users are in front of the payment machine at the same time, the system may have difficulty accurately identifying the target user, resulting in incorrect identity matching and potentially causing payment errors. Summary of the Invention

[0009] This invention provides a device operation process optimization system and device based on interactive behavior data, which can more accurately locate target payment users, thereby improving the accuracy of facial recognition payment. The technical solution provided by this application is as follows:

[0010] Firstly, a device operation process optimization system based on interactive behavior data is provided. The specific implementation of this method is as follows: Face recognition user screening module: Face recognition is performed on the user to be operated on the specified device. All recognized users are recorded as pending users. If there are several pending users, user screening is performed to determine the main user to complete the device operation; otherwise, the pending operation is directly completed based on the pending user. Device lens interaction judgment module: If user screening is performed, multiple interaction judgments are performed based on lens interaction behavior data to determine whether to execute the human-computer interaction operation, thereby determining the main user and completing the device operation. Interaction behavior selection and response module: When executing the human-computer interaction operation, human-computer interaction operation judgment is performed simultaneously to select the issued interaction behavior command, and the response of the pending user to the interaction behavior command is scored to determine the corresponding main user to complete the device operation.

[0011] In a second aspect, a device operation process optimization device based on interactive behavior data is provided. The device includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform steps of a device operation process optimization system based on interactive behavior data.

[0012] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0013] In a campus environment, facial recognition is performed on designated devices to identify users. All identified users are recorded as pending users. By setting up a "pending user" mechanism, the system can temporarily retain multiple recognition results without immediately executing payments, avoiding the risk of accidental payment triggering and improving the system's recognition accuracy and security. If there are several pending users, user screening is performed to determine the primary user to complete the device operation. This solves the technical problem in current technology where the system cannot automatically determine which user should be subject to the payment process when multiple users are detected simultaneously. Otherwise, the payment operation is completed directly based on the face of the pending user, improving the system's scenario adaptability and automation level, and avoiding human intervention.

[0014] If user screening is performed, multiple interaction judgments are made based on camera interaction behavior data to determine whether to execute human-computer interaction operations, thereby identifying the main user and completing the device operation. This solves the technical deficiency of traditional facial payment systems that lack "user intent recognition" capabilities and cannot determine whether a user is ready to pay based on their behavior. By collecting camera interaction behavior data, user intent recognition is achieved, improving the accuracy of main user screening and avoiding misselection or omission. Finally, during the execution of human-computer interaction operations, human-computer interaction operation judgment is performed simultaneously to select the issued interaction behavior command, and the response of the pending users to the interaction behavior command is scored to determine the corresponding main user to complete the device operation. This makes up for the current system's inability to quantify the effectiveness of different users' responses to commands, resulting in inaccurate screening. It effectively distinguishes between bystanders and actual payers, prevents erroneous payments, and ensures the uniqueness, security, and reliability of the final payment operation.

[0015] After an interactive behavior command is issued by a designated device, the system collects interactive behavior response data from each potential user in real time, based on the type of the command. This fills the gap in traditional systems that only perform face detection and recognition and cannot determine a user's payment intent through behavioral feedback. This provides data support for subsequent response quality assessment and primary user selection. Then, based on the interactive behavior response data of each potential user, individual scores are calculated. The degree of difference between each interactive behavior response data point and its corresponding reference data is quantified to obtain corresponding interactive behavior response indicators. This achieves an objective and numerical evaluation of interactive behavior responses, establishing comparable indicators and effectively improving the measurability of behavior response judgment. The system addresses the issues of ambiguity in primary user identification caused by the inability of existing technologies to quantify response effectiveness, ensuring computational efficiency, repeatability, and fairness. It then normalizes and superimposes various interaction response indicators to obtain the corresponding interaction response scores for potential users, improving the accuracy and stability of primary user selection. Finally, the interaction response scores of each potential user are ranked, and the user with the highest score is designated as the primary user to complete the device's pending operations. This fills the gap in traditional systems lacking data-quantified decision-making standards, compensates for the tendency to generate subjective or random judgments, and ensures the uniqueness, security, and accuracy of payment execution.

[0016] Regarding the selection of interactive behavior commands, decisions can also be made by analyzing real-time environmental monitoring data. Based on the environmental monitoring data and the reference data of each interactive behavior command, interactive behavior commands are matched, and the interactive behavior command that matches successfully is selected as the currently issued interactive behavior command. This fills the gap in the mechanism of traditional systems that use fixed interactive commands and cannot flexibly select appropriate interaction methods according to environmental changes, thereby improving the success rate and response speed of interactive commands.

[0017] If multiple successfully matched interaction commands exist, the one with the highest timeliness level is selected and issued. This compensates for the lack of a command priority judgment mechanism based on historical execution effects in existing technologies, which helps to shorten interaction latency, improve the smoothness of the payment process, and realize the self-learning and self-optimization of the interaction strategy. As usage data accumulates, the decision quality is continuously improved. If no successfully matched interaction command exists, human-computer interaction is not performed, the face recognition payment transaction is closed, and the main user is prompted to re-perform face recognition. If the main user still cannot be confirmed, the main user is prompted that face recognition has failed, and other operation modes are pushed to complete the payment operation. This prevents the system from repeatedly trying to interact in extreme environments (such as excessively low light or excessive noise) from causing infinite loops or lag, which helps to prevent system crashes or erroneous payments, enhances security, and improves the robustness of the system. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the structure of a device operation process optimization system based on interactive behavior data provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the process of performing human-computer interaction operations provided in an embodiment of the present invention;

[0021] Figure 3 This is a flowchart illustrating Embodiment 2 provided in this invention;

[0022] Figure 4 This is a flowchart of Embodiment 3 provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0024] Before providing a detailed explanation of the embodiments of this application, the application scenarios of these embodiments will be described first.

[0025] The device operation process optimization system provided in this application embodiment is mainly applied in campus payment scenarios. In campus scenarios, due to the large number of students and the concentrated time of use, when using a facial recognition payment device to pay by scanning faces, it is very likely that multiple faces will be recognized at the same time, resulting in payment face errors and payment deduction errors. Therefore, in the following description, the specified device refers to a facial recognition payment device, the payment operation to be completed is facial recognition payment, and the user generally refers to students or faculty members.

[0026] Example 1: This embodiment of the invention provides a device operation process optimization system based on interactive behavior data. This method can be implemented using a facial recognition payment device, which can be a terminal or a server. Figure 1 The diagram shown is a structural schematic of a device operation process optimization system based on interactive behavior data provided in an embodiment of the present invention. The processing flow of the system includes the following modules:

[0027] Face recognition user screening module: Based on the specified device, the system performs face recognition on the user to be operated on, and records all recognized users as pending users. If there are several pending users, the system performs user screening to determine the main user to complete the operation to be performed on the device; otherwise, the system directly completes the operation based on the pending user.

[0028] It should be noted that before designing a device operation process optimization system based on interactive behavior data, technical professionals typically pre-build a preset database to support the operation of various control strategies. This database integrates multiple key control parameters, including preset quantities, preset reliability, interaction method matching tables, preset time periods, and preset lens differences. Based on the analytical logic and actual hardware environment of a device operation process optimization system based on interactive behavior data, technical professionals have pre-set all parameter settings. The resulting preset database provides crucial data support for subsequent data uploading, storage optimization, and even automated filtering and judgment.

[0029] Device lens interaction determination module: If user screening is required, multiple interaction determinations are performed based on lens interaction behavior data to determine whether to execute human-computer interaction operations, thereby identifying the main user and completing the device operation to be executed.

[0030] The process of determining multiple interactions using camera interaction behavior data is as follows:

[0031] F1 acquires camera interaction behavior data for each user to be identified, including gaze direction angle and duration of occurrence.

[0032] Specifically, face detection algorithms such as MTCNN (Multi-task Cascaded Convolutional Networks), RetinaFace, and MediaPipe Face Mesh are used to obtain the coordinates of key facial points, including the center points of the left and right eyes, the tip of the nose, the corners of the mouth, and the chin. Head pose is estimated through 3D geometric modeling. Then, OpenCV combined with Dlib or MediaPipe Face Mesh methods are commonly used to perform PnP calculations on the detected 2D key points and predefined 3D face model points to obtain rotation and translation vectors. The calculation results include: Yaw (horizontal angle), the left and right rotation angle used to determine whether the face is looking directly at the camera, and Pitch (vertical angle), the vertical tilt angle. The square root of the sum of the squares of these two angles gives the gaze direction angle. The appearance duration describes the length of time a face appears continuously in the recognition frame. The system continuously acquires camera video streams (usually 25-30 FPS), performing face detection and ID tracking on each frame. When the same face appears... If the ID persists, record its "occurrence time period" to obtain the duration of occurrence.

[0033] F2 compares the camera interaction behavior data of each user to be determined with the corresponding standard reference data. The standard reference data includes the set gaze range and the set occurrence time difference, which are generally extracted from the preset data database and preset by the preset staff.

[0034] F3 determines the facial recognition angle based on the gaze direction and angle of each user to be identified, and the set gaze range. This mainly includes the following scenarios:

[0035] In the first case, if the gaze direction angles of multiple undetermined users fall within the set gaze range, then the duration range of the undetermined users' appearances will be compared.

[0036] Specifically, the comparison of the duration of pending user appearances includes the following:

[0037] Sort the appearance durations of each pending user (in ascending order), and select the pending user with the longest appearance duration as the main user. If the difference in appearance duration between the first and last pending user among the preset number of pending users after sorting is less than the set appearance time difference, then perform human-computer interaction to determine the main user; otherwise, use the pending user with the longest appearance duration as the main user to complete the device operation.

[0038] It should be added that the preset quantity and the set time difference are preset by staff based on historical data and experience rules within a historical time period, and are stored in the historical database in advance. When using, they can be directly read from the preset database.

[0039] In the second scenario, if there is only one user whose gaze direction falls within the set gaze range, then the user in question becomes the primary user, and the operation to be performed on the device is completed by the primary user.

[0040] In the third scenario, if no user's gaze direction falls within the set gaze range, then a human-computer interaction operation is performed to determine if the main user has completed the operation to be performed on the device.

[0041] By quantitatively analyzing interactive behavior commands and user response data, the system achieves accurate identification and intelligent selection of the primary user. After issuing action-based or voice-based interactive commands, the system collects response data from each potential user in real time and quantifies the differences with preset reference data to form objectively comparable interactive behavior response indicators. Through normalization processing and the overlay of multiple scores, the system can comprehensively evaluate the effectiveness and matching degree of each user's interaction, and automatically determine the primary user whose interactive behavior best matches the command requirements based on the ranking of the scoring results. This method effectively avoids the misjudgment problem caused by multiple users identifying simultaneously, improves the accuracy and robustness of human-computer interaction, and significantly enhances the intelligence and reliability of the device operation process.

[0042] like Figure 2 The diagram shown is a flowchart illustrating the execution of human-computer interaction operations according to an embodiment of the present invention. The specific logic is as follows: After the designated device issues an interaction behavior instruction, the interaction behavior response data of each pending user is collected in real time according to the type of the issued interaction behavior instruction. A single-item score is then performed based on the interaction behavior response data of each pending user: the difference between each interaction behavior response data and the corresponding reference data is quantified to obtain the corresponding interaction behavior response index; after data normalization processing, each interaction behavior response index is superimposed to obtain the corresponding interaction behavior response score for the pending user; the interaction behavior response scores of each pending user are sorted, and the pending user with the highest interaction behavior response score is designated as the primary user to complete the operation to be executed by the device. Through the above process, the system's recognition accuracy, operational security, and adaptive capability are improved, ensuring the efficiency and reliability of the payment process.

[0043] It should be explained that the specific methods for performing human-computer interaction operations are as follows:

[0044] After an interactive behavior command is issued to a designated device, interactive behavior response data of each user to be selected is collected in real time according to the type of interactive behavior command issued. The types of interactive behavior commands include action commands and voice commands. Action commands include, but are not limited to, touching the screen and nodding operations. Voice commands indicate voice interaction with the user to be operated through a microphone.

[0045] Individual scoring is performed based on the interaction behavior response data of each pending user: the degree of difference between each interaction behavior response data and the corresponding reference data is quantified to obtain the corresponding interaction behavior response index.

[0046] The interactive behavior response data is matched with the issued interactive behavior commands. If the interactive behavior command is a touch screen, the matched interactive behavior response data includes, but is not limited to, touch timestamps and touch durations. If the interactive behavior command is a nodding operation, the matched interactive behavior response data includes, but is not limited to, facial key point location data, action confidence, user facial region coordinates, and action duration and frequency. If the interactive behavior command is a voice interaction, the matched behavioral interaction response data includes synchronized video frame information, language confidence, voice timestamps, and language waveforms and feature vectors. The synchronized video frame information is directly read from the video stream captured by the camera. The raw voice signal is captured by the microphone, and the waveform can be directly obtained using libraries such as pyaudio, sounddevice, and librosa. The language confidence and voice timestamps can be directly output by an automatic speech recognition system (ASR). The feature vectors are obtained by acoustic feature extraction from the voice waveform, using algorithms such as MFCC (Mel Frequency Cepstral Coefficients), Log-Mel Spectrogram, and Fbank.

[0047] Specifically, the degree of difference between each interactive behavior response data and the corresponding reference data is quantified: after performing a difference calculation between each interactive behavior data and the corresponding reference data, a ratio calculation is performed with the reference data to obtain the individual score of the corresponding interactive behavior response data, namely the interactive behavior response index.

[0048] After normalizing the data of each interaction response index (Z-Score standardization), the indices are summed to obtain the interaction response score of the corresponding user to be determined.

[0049] Sort the interaction response scores of each pending user (in ascending order) and designate the pending user with the highest interaction response score as the master user to complete the operation to be performed by the device.

[0050] Interactive behavior selection and response module: When performing human-computer interaction operations, the module simultaneously performs human-computer interaction operation judgment to select the interactive behavior command to be issued, and scores the response of the pending user to the interactive behavior command to determine the corresponding main user to complete the operation to be performed on the device.

[0051] It should be noted that the human-computer interaction operation is judged simultaneously to select the interactive behavior command to be issued. The specific process is as follows:

[0052] The first step is to obtain the facial confidence scores of each potential user and sort them (from smallest to largest). If the highest-scoring facial confidence score is lower than the preset confidence score, the facial recognition payment transaction is stopped, and a voice prompt indicating that the main user has not been recognized is issued. The facial payment recognition is then restarted. If the highest-scoring facial confidence score that is re-recognized is still lower than the preset confidence score, other operation modes are pushed, including card payment mode and QR code payment mode.

[0053] Specifically, face confidence is calculated from the output probability or feature similarity of the detection or recognition model. Most modern algorithms (such as MTCNN, RetinaFace, ArcFace) can directly obtain this value without manual calibration, and the value range is generally between 0 and 1. Preset confidence is set by staff based on historical data and empirical rules to limit the minimum usable face confidence, and it is generally stored in a preset database.

[0054] The second step is to calculate the average confidence of the face with the highest ranking if it is higher than the preset confidence. Then, the average confidence is calculated based on the face confidence of each user to obtain the corresponding average confidence. The average confidence is then matched in the interaction method matching table in the preset database to obtain the corresponding interaction behavior. The interaction behavior instruction is then issued according to the interaction behavior.

[0055] It should be added that by inputting the average confidence score into the interaction matching table, the corresponding interaction behavior can be matched. This dataset is used to fit the mapping relationship between the average confidence score and the interaction behavior. It is constructed as follows: in the initial data table built based on the gradient boosting regression algorithm, the average confidence score collected in the historical time period and the interaction behavior set according to empirical rules are selected as training samples. The model is trained based on the XGBoost framework with the least squares error as the objective function, and finally the trained interaction matching table is obtained.

[0056] By introducing a face confidence level assessment and dynamic interaction mode matching mechanism, adaptive optimization and security control of the face recognition payment process are achieved. The system first calculates and sorts the face confidence scores of each potential user. When the highest confidence score falls below a preset threshold, a voice prompt is automatically triggered, and recognition is re-attempted, effectively avoiding the risk of erroneous payments due to recognition uncertainty or environmental interference. If multiple recognitions still fall below the threshold, the system proactively pushes other payment or operation modes, improving the fault tolerance of the interaction and the user experience. Once the recognition result confidence score meets the requirements, the system further calculates the average confidence score and searches for the corresponding interaction strategy in a preset interaction mode matching table to dynamically select the optimal interaction behavior (such as voice confirmation or action response). This method can intelligently adjust the interaction process based on recognition quality, achieving integrated processing of recognition, decision-making, and interaction, which helps improve the system's recognition accuracy, operational security, and adaptive capabilities, ensuring the efficiency and reliability of the payment process.

[0057] In this embodiment, by setting up a facial recognition user screening module, a device lens interaction determination module, and an interaction behavior selection and response module, accurate identification and intelligent interaction control of the main user in multi-user scenarios are achieved. First, the facial recognition user screening module can automatically identify all potential users in front of the device, avoiding misidentification problems caused by the assumption of a single user. Second, the device lens interaction determination module comprehensively judges each potential user based on multi-dimensional interaction behavior data (such as gaze direction, action response, etc.) collected by the lens, realizing dynamic screening and precise positioning of the main user. Finally, the interaction behavior selection and response module matches and provides feedback judgment based on the real-time interaction command type and user response, effectively improving the sensitivity and recognition reliability of human-computer interaction. Through the collaborative work of the three modules, the system can automatically identify the main user most likely to perform the operation in a multi-person environment, significantly reducing the false trigger rate and recognition latency, improving the security, accuracy, and naturalness of the payment and operation process, thereby achieving humanized and highly robust recognition control of intelligent devices.

[0058] like Figure 3The diagram shown is a flowchart of Embodiment 2 provided by this invention. The specific process is as follows: The number of times each user performs face recognition payment within a preset time period is obtained and recorded as the number of face recognition payment attempts. The ratio of the number of face recognition payment attempts to the length of the preset time period is used as the corresponding face payment interference frequency. If the face payment interference frequency is higher than the face payment interference level limit, the lens distance of each user is obtained for lens distance interaction determination to identify the main user. If the face payment interference frequency is not higher than the face payment interference level limit, the lens distances of each user are subtracted pairwise to obtain the corresponding lens distance difference, which is then compared with a preset lens difference. If the lens distance difference is less than the preset lens difference, the face recognition angle is determined based on the user's gaze direction angle. If the lens distance difference is not less than the preset lens difference, the user with the smallest lens distance is selected as the main user to complete the device operation. Through the above process, the security, robustness, and user experience of face payment scenarios are improved, enabling the device to complete payment operations accurately and efficiently even in complex environments.

[0059] Example 2: Since the campus scene is not crowded with students at all times and at all windows, different multi-interaction determination methods need to be adopted based on the actual number of student users. Building upon Example 1, multi-interaction determination is performed using camera interaction behavior data, and also includes:

[0060] First, obtain the number of times that a number of pending users have made facial recognition payments within a preset time period, and record them as the number of facial recognition payments.

[0061] Next, the ratio of the number of facial recognition payments to the length of a preset time period is used as the corresponding facial payment interference frequency.

[0062] Then, if the frequency of facial payment interference is higher than the facial payment interference limit value used to determine the degree of interference to the user to be operated, the camera distance of each user to be determined is obtained for camera distance interaction determination to determine the main user; wherein, the facial payment interference limit value is calculated by the preset staff based on the historical data within the historical time period to obtain the corresponding facial payment interference frequency dataset, and the value after averaging the mode in the dataset is the facial payment interference limit value.

[0063] Finally, if the frequency of interference with facial payment is not higher than the limit for the degree of interference with facial payment, then multiple interaction determinations will continue to be made based on the camera interaction behavior data.

[0064] It should be added that the specific process for determining camera distance interaction is as follows:

[0065] The lens distances of each candidate user are subtracted from each other to obtain the corresponding lens distance difference. This difference is then compared with a preset lens difference, which is used to limit the minimum difference between the lens distances of candidate users. The preset lens difference is also data set in advance by the staff, usually based on their work experience, historical data, and standard rules.

[0066] If the difference in lens distance is less than the preset lens difference, it means that the distance difference between the corresponding users to be identified is small, and the face recognition angle is determined based on the gaze direction angle of the users to be identified.

[0067] If the difference in lens distance is not less than the preset lens difference, it means that the distance difference between the corresponding pending users is large. In this case, the pending user with the smallest lens distance is selected as the main user to complete the device operation to be performed.

[0068] In this embodiment, by introducing a "face payment interference frequency" and a "lens distance interaction determination" mechanism, high-precision identification and interaction control of the main user are achieved in scenarios where multiple people appear simultaneously or there is frequent interference. The system first counts the number of times a candidate user performs face recognition payment within a preset time period and calculates the ratio of this number to the time length to obtain the face payment interference frequency, thus quantifying the degree of interference in the recognition environment. When the interference frequency exceeds a set limit, the system automatically enters the lens distance interaction determination process. By comparing the lens distances of each candidate user, the system quickly identifies the user closest to the device as the main interaction target. If the lens distance difference between users is small, further determination is made based on the gaze direction angle to ensure the accuracy of the main user's judgment. When the interference frequency is within a normal range, the system continues to rely on lens interaction behavior data for multiple interaction determinations, comprehensively identifying the main user from multi-dimensional behavioral features. This scheme can intelligently adjust the identification and determination logic according to the real-time interference level, achieving dynamic adaptation of the identification strategy and avoiding errors in main user identification caused by multiple people appearing simultaneously, false triggering, or complex environments. By introducing a quantitative index of interference frequency and a dual determination mechanism of distance and angle, the system significantly improves environmental adaptability and stability while maintaining high identification accuracy. This method effectively improves the security, robustness, and user experience of facial recognition payment scenarios, enabling devices to complete payment operations accurately and efficiently even in complex environments.

[0069] like Figure 4The diagram shown is a flowchart of Embodiment 3 provided by this invention. The specific logic is as follows: real-time acquisition of current environmental monitoring data; matching of interactive behavior commands based on the environmental monitoring data and reference data of various interactive behavior commands; selection of successfully matched interactive behavior commands as the currently issued interactive behavior command; if multiple successfully matched interactive behavior commands exist, the interactive behavior command with the highest timeliness level is selected and issued; if no successfully matched interactive behavior command exists, the main user is prompted to re-perform face recognition; if the main user still cannot be confirmed, the main user is prompted that face recognition has failed; through the above process, not only is intelligent decision-making and dynamic optimization of the human-computer interaction process realized, but also the intelligence level, response efficiency and environmental adaptability of the face recognition interaction system are improved.

[0070] Example 3: In a campus setting, the environment is dynamic, and the facial recognition mechanism is also affected by environmental factors. Therefore, when selecting the interactive behavior commands to be issued, the influence of environmental factors must be considered. Based on Example 1, the selected interactive behavior commands also include:

[0071] Real-time acquisition of current environmental monitoring data, including distance between the user and the equipment, environmental noise, and ambient light intensity.

[0072] It should be added that the distance between the user and the device is calculated using the principle of stereo vision through a binocular camera; the ambient noise is obtained through testing with the built-in microphone; and the ambient light intensity is calculated based on the brightness of the camera image.

[0073] Interactive behavior commands are matched based on environmental monitoring data and reference data for each interactive behavior command, and the interactive behavior command that successfully matches is selected as the currently issued interactive behavior command.

[0074] Interactive behavior commands are matched based on environmental monitoring data and reference data for each interactive behavior command. The specific method is as follows:

[0075] If the ambient noise level is higher than the noise threshold, action commands will be issued. The action commands will be determined synchronously based on the ambient light intensity and the distance between the user and the device. The specific corresponding situations are as follows:

[0076] If the ambient noise is higher than the noise threshold, the ambient light intensity is lower than the preset light intensity, and the distance between the user and the device is less than the reachable distance of both hands, the issued interaction command is to touch the screen; if the ambient noise is higher than the noise threshold, the distance between the user and the device is greater than the reachable distance of both hands, and the ambient light intensity is higher than the preset light intensity, the issued interaction command is to nod.

[0077] If the ambient noise is below the noise threshold, the system will simultaneously determine whether to issue a voice command based on the ambient light intensity and the distance between the user and the device. The specific corresponding situations are as follows:

[0078] If the ambient light intensity is lower than the preset light intensity, the distance between the user and the device is greater than the reachable distance of both hands, and the ambient noise is lower than the noise threshold, then the issued interactive behavior command will be voice interaction.

[0079] If multiple successfully matched interactive behavior commands exist, the interactive behavior command with the highest timeliness level will be selected and issued. The timeliness level represents the order level obtained by sorting the average execution time of each interactive behavior command within a historical time period in the historical database. The lower the average execution time of the interactive behavior command, the higher its corresponding timeliness level.

[0080] If no matching interaction command is found, no human-computer interaction will be performed, the face recognition payment transaction will be closed, and the main user will be prompted to perform face recognition again. If the main user still cannot be confirmed, the main user will be prompted that face recognition has failed, and other operation modes will be pushed to complete the payment operation.

[0081] If the interactive behavior command issued based on environmental monitoring data is inconsistent with the interactive behavior command obtained by matching the interactive behavior command in the aforementioned interactive behavior matching table, the interactive behavior command obtained by matching the interactive behavior command in the interactive behavior matching table shall be issued first.

[0082] If the interactive behavior command issued based on environmental monitoring data matches the interactive behavior command obtained by matching based on the interaction method matching table, then the corresponding interactive behavior command will continue to be issued.

[0083] In the above embodiment three, the noise threshold, preset light intensity, and reachable distance for both hands are all read from the preset database. These are preset data that are set by preset staff based on historical data and standard rules of actual scenarios, and then entered into the preset database for storage.

[0084] In this embodiment, an interactive behavior command matching and timeliness ranking mechanism driven by environmental monitoring data is introduced to achieve intelligent decision-making and dynamic optimization of the human-computer interaction process. The system acquires multi-dimensional environmental monitoring data in real time, such as the distance between the user and the device, environmental noise, and light intensity, and matches it with reference data for various interactive behavior commands to select the most suitable interactive command for the current environment. When multiple matching results exist, the system determines the timeliness level based on the average execution time of commands in the historical database, prioritizing commands with higher response speed and better execution efficiency to improve the immediacy and smoothness of the interaction. If environmental conditions cannot support the matching command, the system automatically prompts for re-identification or switching of the interaction method, enhancing the fault tolerance and security of the interaction. At the same time, by comparing the environmental matching results with the preset interaction method matching table, dual verification and priority control are achieved to ensure the stability and consistency of the interactive behavior selection. The overall solution can adaptively select the interaction strategy according to real-time environmental changes, significantly improving the intelligence level, response efficiency, and environmental adaptability of the face recognition interaction system.

[0085] This invention also provides a device operation process optimization device based on interactive behavior data, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute steps of a device operation process optimization system based on interactive behavior data.

[0086] The above-disclosed embodiments are merely some examples of the present invention and should not be construed as limiting the scope of the present invention. Therefore, any equivalent variations made in accordance with the claims of the present invention are still within the scope of the present invention.

Claims

1. A device operation process optimization system based on interactive behavior data, characterized in that, include: Face recognition user screening module: Based on the specified device, the face recognition of the user to be operated is performed, and all recognized users are recorded as pending users. If there are several pending users, user screening is performed to determine the main user to complete the operation to be performed on the device; otherwise, the operation to be performed is directly completed based on the pending user. Device lens interaction determination module: If user screening is required, multiple interaction determinations are performed based on lens interaction behavior data to determine whether to execute human-computer interaction operations, thereby identifying the main user and completing the device operation to be performed. Interactive behavior selection and response module: When performing human-computer interaction operations, it simultaneously performs human-computer interaction operation judgment to select the issued interactive behavior command, and determines the corresponding master user to complete the device operation based on the response of the pending user to the interactive behavior command. The specific process for determining multiple interactions using camera interaction behavior data is as follows: Acquire camera interaction behavior data for each user to be determined, including gaze direction angle and duration of occurrence; The camera interaction behavior data of each user to be determined is compared with the corresponding standard reference data, which includes setting the gaze range and setting the occurrence time difference; The facial recognition angle is determined based on the gaze direction and angle of each user to be identified, and the set gaze range. If the gaze direction angles of multiple pending users fall within the set gaze range, then the duration range of the pending users' appearances will be compared. If there is only one user whose gaze direction angle falls within the set gaze range, then the corresponding user becomes the primary user, and the operation to be performed by the device is completed by the primary user. If no user's gaze direction falls within the set gaze range, then a human-computer interaction operation is performed to determine if the main user has completed the operation to be performed on the device.

2. The device operation process optimization system based on interactive behavior data as described in claim 1, characterized in that, The specific content of comparing the occurrence time range of the pending users is as follows: Sort the appearance duration of each pending user and select the pending user with the highest appearance duration as the main user. If the difference in appearance duration between the first and last pending user among the preset number of pending users after sorting is less than the set appearance duration difference, then perform human-computer interaction to determine the main user. Otherwise, use the pending user with the longest appearance duration as the main user to complete the device operation.

3. The device operation process optimization system based on interactive behavior data as described in claim 2, characterized in that, The specific method for performing human-computer interaction operations is as follows: After an interactive behavior command is issued by a designated device, interactive behavior response data of each pending user is collected in real time according to the type of the issued interactive behavior command. The types of interactive behavior commands include action commands and voice commands. Individual scoring is performed based on the interaction behavior response data of each pending user: the degree of difference between each interaction behavior response data and the corresponding reference data is quantified to obtain the corresponding interaction behavior response index. After normalizing the data of each interaction response index, the scores are superimposed to obtain the interaction response scores of the corresponding user to be determined. The interaction response scores of each pending user are sorted, and the pending user with the highest interaction response score is designated as the main user to complete the operation to be performed by the device. The interactive behavior response data is matched with the issued interactive behavior instructions.

4. The device operation process optimization system based on interactive behavior data as described in claim 1, characterized in that, The process of simultaneously determining human-computer interaction operations to select the issued interactive behavior command is as follows: Obtain the facial confidence score of each pending user and sort the facial confidence scores. If the highest-ranked facial confidence score is lower than the preset confidence score, issue a voice prompt that the main user has not been recognized and re-perform facial payment recognition. If the highest-ranked facial confidence score of the re-recognized user is still lower than the preset confidence score, push other operation modes. If the confidence score of the highest-ranked face is higher than the preset confidence score, the average confidence score is calculated based on the face confidence scores of each candidate user. Then, the average confidence score is matched against the interaction method matching table in the preset database to obtain the corresponding interaction behavior method. Interaction behavior instructions are then issued according to the interaction behavior method.

5. The device operation process optimization system based on interactive behavior data as described in claim 1, characterized in that, In addition to determining multiple interactions through camera interaction behavior data, the method also includes: The number of times a certain number of users have made facial recognition payments within a preset time period is recorded as the number of facial recognition payments. The result of the ratio of the number of facial recognition payments to the length of a preset time period is used as the corresponding facial payment interference frequency. If the frequency of interference with facial payment is higher than the facial payment interference level limit value used to determine the degree of interference to the user to be operated, the camera distance of each user to be determined is obtained to perform camera distance interaction determination in order to determine the main user; If the frequency of interference with facial payment is not higher than the limit for the degree of interference with facial payment, then multiple interaction determinations will continue to be made based on the camera interaction behavior data.

6. The device operation process optimization system based on interactive behavior data as described in claim 5, characterized in that, The specific process for determining the camera distance interaction is as follows: The lens distances of each user to be determined are subtracted pairwise to obtain the corresponding lens distance differences, and then compared with a preset lens difference used to limit the minimum difference between the lens distances of the users to be determined. If the difference in lens distance is less than the preset lens difference, the face recognition angle is determined based on the gaze direction angle of the user to be determined. If the difference in lens distance is not less than the preset lens difference, then the user with the smallest lens distance will be selected as the primary user to complete the device operation.

7. The device operation process optimization system based on interactive behavior data as described in claim 4, characterized in that, In addition to the selected interactive behavior instructions, it also includes: Real-time acquisition of current environmental monitoring data, including the distance between the user and the device, environmental noise, and ambient light intensity; Based on environmental monitoring data and reference data for each interactive behavior command, the interactive behavior command is matched, and the interactive behavior command that is successfully matched is selected as the currently issued interactive behavior command. If there are multiple successfully matched interactive behavior commands, the interactive behavior command with the highest timeliness level is selected and issued. The timeliness level represents the order level obtained by sorting the average execution time of each interactive behavior command within a historical time period in the historical database. The lower the average time of the interactive behavior command, the higher the corresponding timeliness level. If no matching interaction command is found, the main user is prompted to perform face recognition again. If the main user still cannot be identified, the main user is prompted that face recognition has failed. If the interactive behavior command issued based on environmental monitoring data is inconsistent with the interactive behavior command obtained by matching based on the interaction method matching table, the interactive behavior command obtained by matching based on the interaction method matching table shall be issued first. If the interactive behavior command issued based on environmental monitoring data matches the interactive behavior command obtained by matching based on the interaction method matching table, then the corresponding interactive behavior command will continue to be issued.

8. The device operation process optimization system based on interactive behavior data as described in claim 7, characterized in that, The method for matching interactive behavior commands based on environmental monitoring data and reference data for each interactive behavior command is as follows: If the ambient noise is higher than the noise threshold, action commands will be issued, and the action commands will be determined synchronously based on the ambient light intensity and the distance between the user and the device. If the ambient noise is below the noise threshold, the system will simultaneously determine whether to issue voice commands based on the ambient light intensity and the distance between the user and the device.

9. A device for optimizing device operation processes based on interactive behavior data, characterized in that, The device operation process optimization device based on interactive behavior data includes: processor; Memory used to store processor-executable instructions; The processor is configured to perform the steps of the device operation process optimization system based on interactive behavior data as described in any one of claims 1 to 8.

Images