Cross-culture recruitment interview intelligent perception and real-time interaction optimization method and system

Abstract

The invention relates to the technical field of intelligent recognition, in particular to an intelligent perception and real-time interaction optimization method and system for cross-culture recruitment interview, and the method comprises the steps: collecting taboo data, and constructing a regional culture taboo rule library according to the three-level structure of country, category and entry; in the interview process, synchronously acquiring video streams and voice streams of an interviewer through a camera and a microphone; the video is preprocessed, a gesture recognition model is called to extract hand key points, feature vectors are generated, the similarity between the feature vectors and a target national taboo gesture template is calculated, and a limb taboo detection result is obtained; noise reduction and endpoint detection are carried out on the voice, effective voice segments are intercepted, and a language taboo detection result is generated through voice recognition in combination with a language taboo word list and emotional characteristics; and fusing the two types of detection results, outputting a comprehensive risk level according to a preset rule, generating an early warning signal and an interactive optimization suggestion when the comprehensive risk level reaches a threshold value, presenting the early warning signal and the interactive optimization suggestion on an interview interface in real time, and writing the early warning signal and the interactive optimization suggestion into a behavior database to update a rule base and generate a redisk report. According to the invention, the problems of lack of cultural taboo identification capability, language and non-language behavior analysis splitting and lack of real-time feedback and redisk mechanism in the interview process in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent recognition technology, and in particular to an intelligent perception and real-time interaction optimization method and system for cross-cultural recruitment interviews. Background Technology

[0002] With the accelerated development of economic globalization and the overseas expansion of enterprises, multinational corporations are increasingly relying on localized talent recruitment to expand into overseas markets, making cross-cultural recruitment interviews a frequent occurrence in human resource management. However, significant differences exist between different countries and regions in terms of body language (such as gestures, posture, and micro-expressions), language expression (such as word choice and tone of voice), and etiquette taboos. Traditional online interview systems typically only provide basic functions such as audio and video transmission and recording, lacking cultural databases specific to each country / region. They cannot identify culturally taboo behaviors during the interview process; for example, certain gestures, titles, or questioning methods are considered impolite or even offensive in some regions. Furthermore, existing solutions often employ single-modal recognition technology, making it difficult to collaboratively analyze visual behavior and voice content. They cannot comprehensively assess the risks of interview interactions from a multimodal perspective and lack real-time warning and guidance mechanisms when taboo behaviors occur. This makes it difficult for interviewers to adjust their expression and behavior in a timely manner, easily leading to misunderstandings by candidates, damaging the employer's brand image, and failing to meet the needs for safe, efficient, and quantifiable cross-cultural interview management. Summary of the Invention

[0003] In view of the above technical problems, the present invention provides an intelligent perception and real-time interaction optimization method and system for cross-cultural recruitment interviews. By constructing a localized cultural taboo rule base and integrating multimodal recognition and collaborative decision-making technologies, it solves the problems of lack of cultural taboo recognition ability, separation of verbal and non-verbal behavior analysis, and lack of real-time feedback and review mechanisms in the existing technology. This reduces cross-cultural communication risks and improves the cultural adaptability and interaction quality of the interview process.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0005] According to one aspect of the present invention, a method for intelligent perception and real-time interaction optimization in cross-cultural recruitment interviews is proposed, the method comprising: Collect taboo data, and based on the taboo data, construct a regionalized cultural taboo rule base in a three-level structure of national level, taboo category level and taboo item level. The taboo data comes from one of the following sources: public documents, questionnaires, and expert knowledge. The interview data is collected by simultaneously capturing the interviewer's video and audio streams through a camera and microphone during the interview process. The acquired video stream is input into the visual analysis channel, and the acquired audio stream is input into the audio analysis channel; Based on the visual analysis channel, the video stream is preprocessed, and a gesture recognition model is called to extract the coordinates of key points on the interviewer's hands. A multi-dimensional gesture feature vector is generated to represent the current gesture based on the spatial relationship between the key points on each hand. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library to calculate the similarity. Based on the comparison results of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. Based on the audio analysis channel, noise reduction and speech endpoint detection are performed on the speech stream. Valid speech segments are extracted and input into an automatic speech recognition engine to obtain transcribed text. The transcribed text is then processed for case unification, stop word removal, and synonym normalization to obtain standardized text. Based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, the standardized text is subjected to sensitive word scanning and semantic analysis. Combined with the acoustic emotion features extracted from the speech stream, language taboo detection results and emotion risk levels are generated. The physical taboo detection results and the language taboo detection results are fused together in a dual-channel manner, and the comprehensive risk level of the current interview round is output according to the preset decision rules. When the comprehensive risk level reaches the warning threshold, a warning signal and interaction optimization suggestions are generated, which include taboo type, triggering basis and recommended alternative behavior. The warning signals and the interaction optimization suggestions are presented in real time in a visual manner on the interview user interface. The occurrence time, risk level, multimodal characteristics and collection environment parameters of the corresponding taboo behaviors are written into the behavior database. The regional cultural taboo rule base is dynamically updated and a corresponding interview debriefing report is generated.

[0006] Furthermore, the construction of the aforementioned regionalized cultural taboo rule base also includes: A unique node is established for each target country or region at the country level based on the country code; In the taboo category layer, interview-related behaviors are divided into physical taboos, verbal taboos, and behavioral norms; In the taboo item layer, set a behavioral description, risk level, judgment basis link, and feasible alternative words or actions for each taboo item; A relational database is used to model the table structure of the regionalized cultural taboo rule base. By using an index with the country code and taboo category as the composite key, the target entries in the regionalized cultural taboo rule base can be retrieved.

[0007] Furthermore, after acquiring the video stream, color space conversion, grayscale conversion, and smoothing filtering are performed on each frame of the video stream, and the image size and pixel values ​​are normalized. While extracting the key points of the hands, the coordinates of the key points of the head and shoulders are extracted using a posture recognition model. The head tilt angle is calculated based on the coordinate difference between the key points of the head and the key points of the shoulders. The overlap time between the eye areas of the interviewer and the candidate is statistically analyzed using a face detection and tracking algorithm to obtain the duration of eye contact. The head tilt angle and the duration of eye contact are used as posture features. The posture features and the corresponding multi-dimensional gesture feature vectors are stored in the behavior database for the purpose of judging and reviewing behavioral norms and taboos.

[0008] Furthermore, the gesture recognition model is a hand key point detection model based on a depthwise separable convolutional network, which is used to detect the hand region from each frame of the normalized video stream and output the three-dimensional coordinates of the hand key points. During detection, the gesture recognition model calculates the curvature of each finger based on the ratio of the distance between the fingertip and the base of the finger and the center of the palm, calculates the fingertip distance feature based on the ratio of the distance between different fingertips to the length of the palm, calculates the palm tilt feature based on the angle between the palm plane and the horizontal and vertical directions, constructs the relative position feature based on the coordinate difference between selected key point pairs, and calculates the average speed of gesture movement between consecutive frames to obtain motion features, thereby forming the multidimensional gesture feature vector. After normalizing each dimension of the multidimensional gesture feature vector according to the mean and standard deviation of historical samples, the ratio of the sum of the products of the multidimensional gesture feature vector and the taboo gesture template vector in each dimension to the product of the vector magnitude is calculated. This ratio is used as the gesture similarity. When the similarity is not less than the first threshold, it is determined to be a confirmed taboo gesture. When the gesture similarity is between the second threshold and the first threshold, it is determined to be a suspected taboo gesture, and different levels of warnings are triggered respectively.

[0009] Furthermore, the microphone used when acquiring the speech stream is an omnidirectional condenser microphone with a sampling rate of not less than 48 kHz and a bit depth of not less than 16 bits. The speech stream is acquired in segments using a frame buffer through an audio acquisition library. At the beginning of the acquisition, environmental noise is recorded to construct a noise spectrum template. In the noise reduction process, the background noise is reduced by subtracting the speech spectrum of the speech stream from the noise spectrum. When performing speech endpoint detection, the audio analysis channel detects the start and end points of speech based on a joint threshold of short-time energy and zero-crossing rate. It determines and discards segments with energy and zero-crossing rate below the silence threshold for several consecutive frames, retaining only the valid speech segments.

[0010] Furthermore, the automatic speech recognition engine adopts a streaming recognition mode, which slices the effective speech segments into text in real time after recognition by a fixed time length. After removing multilingual stop words, the standardized text is matched character by character by constructing a multi-branch tree structure containing taboo words and their synonyms from the target country or region. When the path in the multi-branch tree structure reaches a node marked as a taboo word, the corresponding taboo word hit result is output. Specifically, for semantically equivalent word groups, the similarity between word vectors is calculated, and when the similarity is greater than a preset threshold, they are grouped into the same taboo category to ensure that taboo discourses with different expressions can be identified. The corresponding category and triggering context are recorded in the language taboo detection results according to the different labels of absolute taboo words and avoidable words.

[0011] Furthermore, based on the emotional polarity, subjectivity, and frequency of occurrence of anger, impatience, doubt, and interjections calculated from the standardized text, and based on the acoustic emotional features, a joint acoustic and textual emotional feature vector is constructed. The emotional feature vector is input into an emotional classification model combining a convolutional neural network and a recurrent neural network to output three emotional labels: neutral, slightly negative, and severely negative, and their corresponding probabilities. The acoustic emotional features include at least several Vermeer frequency cepstral coefficients, linear prediction coefficients, fundamental frequencies, short-time energy, and spectral centroid features. After performing sensitive word scanning and semantic analysis on the standardized text, when the semantic analysis result contains absolutely forbidden words and the sentiment label is slightly negative or severely negative, the language taboo detection result is marked as a high-risk sentiment taboo. When the semantic analysis result only contains avoidable words and the sentiment label is severely negative, the language taboo detection result is marked as a medium-risk sentiment taboo. When the sentiment label is severely negative but does not hit any forbidden words, it is only recorded as a low-risk sentiment reminder without triggering an interface warning.

[0012] Furthermore, the dual-channel collaborative fusion specifically includes: When the physical taboo detection result indicates the presence of high-risk taboo gestures in the target country or region, the sampling rate and analysis frequency of the audio analysis channel are increased, and key sensitive words are scanned and emotional intensity is assessed for the speech segments within the current time window; When the language taboo detection result indicates the presence of sensitive words or that the emotional risk level reaches the warning threshold, the image processing frame rate of the visual analysis channel is increased, and fine-grained behavior and micro-expression tracking is performed on the interviewer's facial and upper body areas to update the comprehensive risk level. The comprehensive risk level is obtained by weighting three risk components—physical taboo, language taboo, and behavioral norms—according to preset weights, and is used to determine the warning level and feedback strategy.

[0013] Furthermore, the method also includes: The warning signal is pushed to the front-end interface through a persistent connection-based communication mechanism. Within a preset time after the prohibited behavior occurs, a color-coded warning box pops up on the interview interface. The warning box displays the country or region to which the prohibited behavior belongs, the category, the risk level, and recommended alternative words or actions to guide the interviewer to make immediate adjustments. The behavioral database records the frequency, time distribution, and emotional fluctuation values ​​of various taboo behaviors. After the interview, a cultural adaptation index is calculated based on the ratio of the number of taboo behaviors to the total number of behaviors to reflect the degree of control over cultural taboos during the interview. The cultural adaptation index, taboo behavior heatmap, and priority of improvement suggestions are combined to generate the interview debriefing report.

[0014] According to another aspect of the present invention, an intelligent perception and real-time interaction optimization system for cross-cultural recruitment interviews is provided, comprising: The rule base construction module is used to collect taboo data. Based on the taboo data, a regionalized cultural taboo rule base is constructed according to a three-level structure of national level, taboo category level, and taboo item level. The taboo data comes from one of the following sources: public documents, questionnaires, and expert knowledge. A multimodal data acquisition module is used to collect interview data, which is the video stream and audio stream of the interviewer simultaneously captured by a camera and microphone during the interview process; The modal channel distribution module is used to input the acquired video stream into the visual analysis channel and the acquired audio stream into the audio analysis channel. The limb taboo detection module is used to perform image preprocessing on the video stream based on the visual analysis channel, and call the gesture recognition model to extract the coordinates of key points of the interviewer's hands. Based on the spatial positional relationship between the key points of the hands, a multi-dimensional gesture feature vector is generated to represent the current gesture. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library to calculate the similarity. Based on the comparison result of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. The language taboo detection module is used to perform noise reduction and speech endpoint detection on the speech stream based on the audio analysis channel, extract effective speech segments and input them into an automatic speech recognition engine to obtain transcribed text, perform case unification, stop word removal and synonym normalization on the transcribed text to obtain standardized text; based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, perform sensitive word scanning and semantic analysis on the standardized text, and combine the acoustic emotion features extracted from the speech stream to generate language taboo detection results and emotion risk levels; The fusion and early warning generation module is used to perform dual-channel collaborative fusion of the physical taboo detection results and the language taboo detection results, output the comprehensive risk level of the current interview round according to the preset decision rules, and generate an early warning signal and interaction optimization suggestions that include taboo type, trigger basis and recommended alternative behavior when the comprehensive risk level reaches the early warning threshold. The visualization module is used to present the warning signals and the interaction optimization suggestions in a visual manner in real time on the interview user interface, and write the corresponding taboo behavior occurrence time, risk level, multimodal characteristics and collection environment parameters into the behavior database, and dynamically update the regional cultural taboo rule base and generate the corresponding interview debriefing report.

[0015] The technical solution of the present invention has the following beneficial effects: By constructing a regionalized cultural taboo rule base covering multiple countries / regions and deeply integrating it with multimodal perception technologies such as gesture recognition, speech recognition, and sentiment analysis, the system achieves unified modeling and automatic identification of risks related to physical taboos, linguistic taboos, and behavioral norms, thereby significantly improving the accuracy of cultural risk avoidance. Through a dual-channel collaborative mechanism of visual and audio channels, when suspected or confirmed taboo behavior is detected, a warning signal and alternative wording / action suggestions are generated with a millisecond delay and pushed to the interview interface in real time, effectively compensating for the shortcomings of traditional systems that cannot provide immediate feedback. At the same time, the system structurally stores multidimensional data such as the time, type, risk level, and emotional fluctuations of taboo behavior, constructs quantitative indicators such as the cultural adaptation index, and generates a debriefing report that includes behavior frequency distribution, risk heatmap, and priority of improvement suggestions. This provides enterprises with visual and data-driven support for assessing and improving HR's cross-cultural communication capabilities and developing targeted training programs, significantly improving the safety, fairness, and communication efficiency of cross-cultural interviews. Attached Figure Description

[0016] Figure 1 This is a flowchart of an intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews, as described in this specification. Figure 2This is a structural block diagram of an intelligent perception and real-time interaction optimization system for cross-cultural recruitment interviews, as described in this specification. Figure 3 This specification describes a terminal device for implementing an intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews, as exemplified in this specification. Figure 4 This specification describes a computer-readable storage medium that stores an intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews, as exemplified in this specification. Detailed Implementation

[0017] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0018] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0019] This invention provides an intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews. (Refer to...) Figure 1 The diagram shown is a flowchart illustrating an intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews provided in an embodiment of the present invention. This method can be applied to devices such as personal computers and servers. The method can be executed by a device, which can be implemented by software and / or hardware. Specifically, the method may include the following steps S101~S107: In step S101, taboo data is collected, and based on the taboo data, a regionalized cultural taboo rule base is constructed according to a three-level structure of national level, taboo category level, and taboo item level. The taboo data comes from at least one of the following: public documents, questionnaires, and expert knowledge. The construction of the regionalized cultural taboo rule base also includes: establishing a unique node for each target country or region at the country level based on the country code; classifying interview-related behaviors into physical taboos, language taboos, and behavioral norms at the taboo category level; setting behavioral descriptions, risk levels, judgment basis links, and feasible alternative statements or actions for each taboo item at the taboo item level; and using a relational database to model the table structure of the regionalized cultural taboo rule base, and using an index with the country code and taboo category as composite keys to achieve retrieval of target items in the regionalized cultural taboo rule base.

[0020] As an example, at the level of publicly available literature, the system primarily uses data from UNESCO's website on intercultural communication, business etiquette guidelines published by commercial sections of embassies in China, and cultural guidelines published by international media. It crawls relevant page content and preprocesses and deduplicates the crawled text, for example, by merging identical entries using abstract fingerprints or hashing methods, ensuring that the retained taboo entries have high independence and authority. At the level of corporate practice and questionnaires, the system selects materials such as global recruitment culture guidelines and cross-border interview taboo manuals published by large enterprises with experience in multinational recruitment. Business personnel or language experts use a standardized input template to input fields such as "country code, taboo category, specific behavior, and risk level" in a structured manner. Simultaneously, questionnaire data is collected from local residents and long-term HR consultants through online questionnaires or offline interviews, focusing on obtaining descriptions of taboo behaviors, the degree of offense, and common scenarios in different countries or regions regarding gestures, topics, titles, tone of voice, and posture. Entries meeting the criteria in the questionnaires are then converted into structured taboo data. To address the potential discrepancies in descriptions of the same taboo behavior across different data sources, the system will label data according to its data source level during the data collection phase. Multiple records from the same country, scenario, and behavior will be cross-referenced. For example, regarding the "Brazilian thumbs-up" gesture, if a high-level public report describes it as a neutral gesture, while local experts describe it as vulgar in informal settings, the descriptions and applicable scenarios from different sources will be recorded simultaneously when the entry is generated. This information will then be integrated into the risk level setting process to improve the accuracy and applicability of the taboo data.

[0021] After obtaining the raw taboo data, the data is cleaned, categorized, and hierarchically organized according to a three-tiered structure: country level, taboo category level, and taboo item level, forming a tree-structured regional cultural taboo rule base. The country level adopts internationally recognized country and region code standards, assigning a unique country code and corresponding country name to each target country or region. It also combines publicly available information and expert knowledge to summarize the core cultural characteristics of the country or region, providing cultural background information for subsequent retrieval and display. The taboo category level is based on behavioral types in interview scenarios, stably dividing taboo behaviors in each country into three categories: physical taboos, verbal taboos, and behavioral norms. Physical taboos mainly focus on non-verbal behaviors such as gestures, physical contact, and body posture; verbal taboos focus on linguistic content such as word choice, question and answer topics, and tone of voice; and behavioral norms focus on overall behavioral style such as interruption frequency, eye contact, posture, and speaking rhythm. The taboo item layer serves as the bottom-level node, with each node corresponding to a specific taboo behavior item. Each item is assigned structured fields, including: the country code and taboo category code, a textual description of the taboo behavior, a risk level label (e.g., absolute taboo, avoidance behavior, bonus etiquette), a link to the judgment basis (pointing to specific public documents, company guidelines, or expert interview records), and feasible alternative phrases or actions in interview scenarios. These guide interviewers on how to express themselves compliantly or adjust their demeanor when identifying risks. Through this three-tiered structure, the regionalized cultural taboo rule base logically forms a top-down tree-like relationship of country, category, and item, facilitating browsing and maintenance by country and enabling quick location of the corresponding taboo item during specific judgments.

[0022] At the implementation level, the regionalized cultural taboo rule base uses a relational database for table structure modeling, mapping the country layer, taboo category layer, and taboo item layer to independent table structures linked by foreign keys. For example, a country information table can be set up to store the country code, country name, and its core cultural characteristics; a taboo category table can be set up to store the category number, category name (physical taboos, language taboos, behavioral norms), and a description of the category's applicability in interview scenarios; a taboo item table can be set up to store attribute fields such as item number, country code, category number, description of taboo behavior, risk level, judgment basis, and alternative words or actions. A foreign key relationship is established between the country code and category number fields to ensure consistency between the item layer and the upper-level nodes. In terms of index design, a composite index is established on the taboo entry table using the combination key of country code and taboo category. This allows for the retrieval of all taboo entries for a given country within milliseconds when a target country code and specified category are entered. For frequently accessed countries or regions, a caching component can be used to cache their corresponding taboo entry sets, while ensuring data consistency, to further reduce access latency. Through the standardization of field formats, constraints on primary and foreign key relationships, and the design of composite indexes, the localized cultural taboo rule base maintains structural clarity while possessing high retrieval efficiency and scalability. This provides fundamental data support for taboo identification and risk alerts during cross-cultural interviews in subsequent steps.

[0023] In step S102, interview data is collected, which is the video stream and audio stream of the interviewer being collected simultaneously through a camera and microphone during the interview process; Specifically, after acquiring the video stream, color space conversion, grayscale conversion, and smoothing filtering are performed on each frame of the video stream, and the image size and pixel values ​​are normalized.

[0024] As an explanation, the image and audio acquisition modules are activated after the interview session begins. A real-time video stream of the interviewer is continuously acquired via a camera, while an audio stream, synchronized with the video stream's timing, is acquired via a microphone. Both data streams are marked with a unified timestamp, ensuring subsequent steps are analyzed on the same timeline. For this step, the focus is on preprocessing the video stream frame by frame. Therefore, after acquiring the raw video stream, each frame is sequentially read from the acquisition module's buffer and fed into the image preprocessing pipeline for color space conversion, grayscale conversion, smoothing filtering, and size and pixel value normalization, generating standardized image data that meets the input requirements of the subsequent visual recognition model.

[0025] Specifically, the raw images captured by the camera are usually in BGR format. To ensure consistency with the input format of the subsequent visual model, a color space conversion needs to be performed to convert the BGR format image to RGB format. This conversion can be done using color space conversion functions in an image processing library. Essentially, it rearranges the channel order of each pixel, as shown in the formula: ; In other words, for any pixel, the three channel values, originally stored in the order of blue, green, and red, are reversed to the order of red, green, and blue. For example, a BGR pixel (B=100, G=150, R=200) is converted to an RGB pixel (R=200, G=150, B=100). This method ensures that subsequent models can correctly interpret the image's color information and avoids feature extraction deviations caused by incorrect channel order. After completing the color space conversion, the system performs grayscale processing on the converted image, compressing the three-channel color image into a single-channel grayscale image to reduce redundant color information and lower subsequent computational load. The grayscale value can be calculated using a weighted average method, specifically: ; Wherein, B, G, and R are the values ​​of the pixel in the blue, green, and red channels, respectively. The above weights reflect the differences in human eye sensitivity to different colors. This formula can obtain a grayscale image that is more in line with the visual characteristics of the human eye while preserving structural details.

[0026] After obtaining the grayscale image, a smoothing filter is applied to suppress noise. In interview scenarios, factors such as changes in ambient light and noise from the camera's image sensor can introduce high-frequency noise into the grayscale image. Therefore, a Gaussian filtering algorithm can be used to perform convolutional smoothing on the image. For example, a Gaussian kernel with a size of 5×5 and a standard deviation σ=1.2 can be selected, and a weighted average can be applied to each pixel and its neighborhood. This effectively removes isolated noise points while preserving important structural information such as hand contours and facial edges as much as possible, avoiding excessive blurring of edges. The output of the Gaussian filter serves as input for subsequent size and pixel value normalization processing, allowing subsequent algorithms to analyze the image on a relatively clean and smooth surface.

[0027] Regarding size and pixel value normalization, considering that subsequent gesture recognition or pose recognition models typically require images of uniform size as input, the smoothed image is scaled and cropped or padded as necessary based on the model's preset input size. For example, when the model's default input is 256×256 pixels, if the aspect ratio of the original image is inconsistent with the target aspect ratio, the image can be scaled proportionally along one side first, and then cropped or padded with black pixels with a pixel value of 0 in the other direction. This ensures that the final image meets the target size while minimizing severe stretching and deformation in key areas. After completing spatial size normalization, to improve the numerical stability of the model during training and inference, the system also normalizes the image pixel values, mapping the original pixel values ​​in the range of 0 to 255 to the interval [-1, 1]. The exemplary linear normalization formula is: ; in, The original value of a pixel in a certain channel is represented by 127.5, which is the median value. This transformation makes the mean of the normalized pixel values ​​close to 0 and the standard deviation close to 1. This is beneficial for the subsequent deep learning model to maintain a stable gradient distribution during forward inference, thereby improving the accuracy and robustness of feature extraction. After the above color space conversion, grayscale conversion, smoothing filtering, and size and pixel value normalization processes, each frame of video image is converted into a standardized image with a unified structure and numerical distribution. This provides a reliable data foundation for subsequent steps such as gesture keypoint extraction, pose analysis, and taboo behavior detection. At the same time, this step strictly adheres to the processing scope defined in this step and does not introduce the implementation details of other modules in advance.

[0028] In step S103, the acquired video stream is input into the visual analysis channel, and the acquired audio stream is input into the audio analysis channel.

[0029] After collecting and preprocessing the interview data, the data streams are split and accessed according to modality, allowing data from different modalities to be processed independently in their respective dedicated analysis channels. Specifically, to ensure time alignment and real-time performance of the video and audio streams in subsequent analysis, a unique session identifier is assigned to the current session when the interview session starts, and a timestamp marking mechanism is uniformly adopted for both data streams. During acquisition, the preprocessed video frame sequence and the original audio samples are simultaneously written to the buffer using the timestamp as an index. The video frame sequence arranged in chronological order is imported into the visual analysis channel from this buffer, while the continuous audio data corresponding to its timestamp is imported into the audio analysis channel. Demonstratively, a multi-threaded or asynchronous task queue approach can be used to configure independent processing threads for the visual analysis channel and the audio analysis channel, to avoid excessive instantaneous computational load of one modality affecting the real-time input of the other modality.

[0030] For the visual analysis channel, the input data consists of standardized video frames that have undergone color space conversion, grayscale conversion, smoothing filtering, and size and pixel value normalization in the previous step. Inputting the video stream into the visual analysis channel involves organizing and caching the standardized video frames in temporal order. This means adding image frames to the frame queue within the visual analysis channel according to their timestamps, and attaching metadata such as session identifier, frame number, timestamp, and corresponding region cropping information to each frame. This allows subsequent visual sub-modules at each level to read data from the queue frame by frame or by time window as needed. For example, the visual analysis channel can divide several consecutive frames into an analysis window based on a preset window length, providing continuous image input for subsequent gesture recognition, pose estimation, and eye behavior analysis. Simultaneously, to ensure real-time performance, flow control can be implemented by discarding some of the earliest frames or adjusting the analysis frequency when the queue length exceeds a preset threshold. This method ensures that each standardized video image frame is delivered to the subsequent processing within the visual analysis channel in an orderly and complete manner, achieving a seamless connection between visual information acquisition and analysis.

[0031] For the audio analysis channel, the input data is a continuous audio stream acquired synchronously with the video stream. The raw audio sampling data is written to the audio analysis channel's buffer queue according to a unified timestamp, maintaining the same session identifier and time reference as the visual analysis channel. This allows the audio analysis channel to accurately align audio segments with visual frames in subsequent steps based on the timestamp. Internally, the audio analysis channel can logically segment the continuous audio stream according to preset frame lengths and frame shifts, dividing the long audio stream into suitable audio segment units for analysis. Metadata such as start and end times and segment numbers are recorded for each audio segment, supporting subsequent operations such as speech recognition, sentiment analysis, and taboo detection.

[0032] In step S104, based on the visual analysis channel, the video stream is preprocessed, and the coordinates of key points on the interviewer's hand are extracted using a gesture recognition model. A multi-dimensional gesture feature vector is generated to represent the current gesture based on the spatial relationship between the key points on the hand. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library. Based on the comparison results of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. The gesture recognition model is a hand key point detection model based on a depthwise separable convolutional network, which is used to detect the hand region from each frame of the normalized video stream and output the three-dimensional coordinates of the hand key points. While extracting the key points of the hands, the coordinates of the key points of the head and shoulders are extracted using a posture recognition model. The head tilt angle is calculated based on the coordinate difference between the key points of the head and the key points of the shoulders. The overlap time between the eye areas of the interviewer and the candidate is statistically analyzed using a face detection and tracking algorithm to obtain the duration of eye contact. The head tilt angle and the duration of eye contact are used as posture features. The posture features and the corresponding multi-dimensional gesture feature vectors are stored in the behavior database for the purpose of judging and reviewing behavioral norms and taboos.

[0033] During detection, the gesture recognition model calculates the curvature of each finger based on the ratio of the distance between the fingertip and the base of the finger and the center of the palm; calculates fingertip distance features based on the ratio of the distance between different fingertips to the length of the palm; calculates palm tilt features based on the angle between the palm plane and the horizontal and vertical directions; and constructs relative position features based on the coordinate difference between selected key point pairs. It also calculates the average speed of gesture movement across multiple consecutive frames to obtain motion features, thus forming the multidimensional gesture feature vector. After normalizing each dimension of the multidimensional gesture feature vector according to the mean and standard deviation of historical samples, the ratio of the sum of the products of the multidimensional gesture feature vector and the taboo gesture template vector in each dimension to the product of the vector magnitude is calculated. This ratio is used as the gesture similarity. When the similarity is not less than the first threshold, it is determined to be a confirmed taboo gesture; when the gesture similarity is between the second threshold and the first threshold, it is determined to be a suspected taboo gesture, triggering different levels of warnings accordingly.

[0034] As explained above, this step preprocesses the acquired interviewer video stream frame by frame using a visual analysis channel. Building upon the previous steps, grayscale conversion and noise reduction are performed to make the hand contours and keypoint edges clearer, thus providing a high signal-to-noise ratio input for subsequent keypoint detection. After basic preprocessing, each processed frame is fed into a gesture recognition model based on a depthwise separable convolutional network. This model can be understood as a network structure using MobileNetV2 as the backbone network combined with a lightweight keypoint regression head. It employs depthwise convolution and pointwise convolution to significantly reduce computation while maintaining recognition accuracy, adapting to the 30fps real-time recognition requirements of interview scenarios. The model first automatically locates the hand region in the image, normalizes this region, and then regresses and outputs the three-dimensional coordinates (x, y, z) of 21 hand keypoints. Here, x and y are the normalized or mapped planar coordinates, and z is the relative depth relative to the center of the palm, used to distinguish spatial positions near or far from the camera.

[0035] After obtaining the coordinates of 21 key hand points, this step constructs a multi-dimensional gesture feature vector based on four aspects: shape, posture, relative position, and movement. First, to characterize the degree of bending of the five fingers, the distance between the fingertip and the base of each finger is used as the coordinates. and the distance between the base of the fingers and the center of the palm. Based on this, the curvature feature of the i-th finger (from thumb to little finger, corresponding to i=1–5) is defined as follows: ; in, The closer the value is to 1, the more bent the fingers are; the closer it is to 0, the more straight the fingers are. Secondly, to reflect the openness or closeness between the fingertips, the length of the palm is used as a reference. Using the normalized baseline (e.g., the distance from the wrist to the base of the middle finger), the distances between the fingertips of the index finger and thumb, middle finger and thumb, ring finger and thumb, and little finger and thumb were calculated separately. The ratio of fingertip distance to palm length yields the fingertip distance feature. ; in This is used to distinguish gestures such as a clenched fist, the OK sign, and fingers spread wide, which may have different meanings in different cultural contexts. To characterize the spatial orientation of the entire palm, this step fits the palm plane using key points such as the base of the palm, the base of the index finger, and the base of the little finger, calculating the angles of this plane relative to the horizontal and vertical directions to obtain the palm tilt characteristics. and ,in This can reflect differences in palm orientation (upward, downward, or sideways). Regarding positional relationships, several representative key point pairs are selected (e.g., index fingertip – thumb base, middle fingertip – palm center, ring finger base – palm base, thumbtip – palm center, etc.), and the coordinate differences in the x and y directions for each key point pair are calculated. , This is used to describe the local topology. Finally, to capture the dynamic changes of the gesture over time, the average velocity of the gesture movement is calculated across multiple consecutive frames. For example, the motion features are obtained over three consecutive frames using the following formula: ; Its unit can be measured in pixels per frame, used to distinguish between slow, polite gestures and fast, intense movements. Through the above feature design, the original 21 3D keypoint coordinates are compressed and mapped into a set of fixed-dimensional multi-dimensional gesture feature vectors V, such as including 5-dimensional curvature features. 4D fingertip distance ratio feature 2D palm tilt angle features Relative position features of several key points and 1D motion velocity characteristics This allows for a complete depiction of the geometric shape and motion state corresponding to the gesture in the current frame within the numerical space.

[0036] Because the physical dimensions of the aforementioned features differ (e.g., distance ratio is dimensionless, angle is in degrees, and motion speed is in pixels / frame), this embodiment standardizes the multi-dimensional gesture feature vectors to prevent any single dimension from dominating the overall result due to its large numerical range in subsequent matching. Specifically, based on pre-collected historical valid gesture samples (e.g., 1000 sets), the mean of each feature dimension in the sample set is calculated. and standard deviation Then, the Z-score normalization algorithm is used to normalize each dimension V_m of the current feature vector to obtain... ; in Let be the standardized m-th eigenvalue. After this processing, each eigenvalue dimension has a statistically significant mean of 0 and a standard deviation of 1, preserving the differences in importance of different features in the judgment of forbidden gestures while avoiding the bias caused by differences in units of measurement in similarity calculation. The standardized eigenvector is then denoted as . This serves as the input for matching template vectors in the regional cultural taboo rule base.

[0037] In collaboration with a regionalized cultural taboo rule base, this step retrieves pre-marked taboo gesture template vectors for that country / region from the rule base, based on the target country or region selected for the current interview. Both reside in the same feature space. Subsequently, the standardized feature vector V' and the taboo gesture template vector are calculated. The cosine similarity is calculated using the formula: ; in Represents the vector dot product. are their respective L2 norms, and the similarity S ranges from [0, 1]. The closer the value is to 1, the closer the current gesture is to the taboo template. The first threshold and the second threshold are preset for different countries / regions and different types of taboo gestures. In the preferred embodiment, they can be set as the first threshold 0.9 and the second threshold 0.8: when S is not less than the first threshold, it is determined that the current gesture is a confirmed taboo gesture, and the highest-level warning is triggered in combination with the risk level (such as absolute taboo) corresponding to the entry in the rule library. For example, a red warning box pops up on the interface and can be optionally accompanied by a voice prompt; when the second threshold < S < the first threshold, it is determined as a suspected taboo gesture, and a lower-level warning (such as a yellow prompt box) is triggered, prompting the HR to handle it carefully in combination with the context; when S < the second threshold, it is considered that the difference between the current gesture and the taboo template is large, regarded as a non-taboo gesture, and no warning is triggered, and only the recognition result is archived. Through this cosine similarity metric based on multi-dimensional features and the dual-threshold decision mechanism, the false alarm rate can be effectively controlled while ensuring the recall rate, so that high-risk limb taboo behaviors in cross-cultural scenarios can receive timely and graded risk warnings.

[0038] While performing the above-mentioned hand key point extraction and taboo gesture matching, the visual analysis channel will also parallelly call the pose recognition model to obtain the overall body posture information of the interviewer. Specifically, the pose recognition model detects the human skeleton in the video frame, extracts the coordinates of the head key points and the left and right shoulder key points, and calculates the head tilt angle based on this. For example, let the coordinates of the head key point be , and the coordinates of the combined shoulder key points be , then the head tilt angle can be defined as satisfying: ; Thus, the degree of left-right or front-back tilt of the head is quantified, which is used to determine the requirements for behavioral norms such as body leaning back and excessive head tilt in the candidate country's culture. At the same time, based on the face detection and tracking algorithm, the eye regions of the interviewer and the candidate are respectively located, and by statistically calculating the proportion of the overlapping duration of the two eye regions on the time axis, that is, the eye contact duration, a pose feature index reflecting the interaction quality is obtained. For example, within a given time period, if the ratio of the cumulative time of the overlapping positions of the two eye regions detected to the total visible time of the interview is too low or too high, it may constitute a potential taboo at the behavioral norm level for some cultures.

[0039] To support subsequent judgment and debriefing analysis of behavioral taboos, this step maps the obtained posture features (including head tilt angle and eye contact duration) to the corresponding multi-dimensional gesture feature vectors at specific times, storing them uniformly in the behavior database. Each record in the behavior database can be associated with a specific interview session identifier, target country / region, current frame time, taboo detection result, and risk level. During the interview, this information is used to drive the early warning module in real time, and after the interview, it can support the generation of debriefing reports such as behavior time distribution, risk event list, and cultural adaptation index. By simultaneously archiving fine-grained gesture features and body posture features, this embodiment can not only provide immediate risk warnings at the level of physical taboos, but also provide a complete data foundation for the quantitative judgment and long-term optimization of behavioral taboo issues such as whether one excessively lowers their head while speaking or whether there is a prolonged lack of eye contact.

[0040] In step S105, based on the audio analysis channel, noise reduction and speech endpoint detection are performed on the speech stream. Valid speech segments are extracted and input into the automatic speech recognition engine to obtain transcribed text. The transcribed text is then processed for case unification, stop word removal, and synonym normalization to obtain standardized text. Based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, the standardized text is subjected to sensitive word scanning and semantic analysis. Combined with the acoustic emotion features extracted from the speech stream, language taboo detection results and emotion risk levels are generated. The microphone used when acquiring the speech stream is an omnidirectional condenser microphone with a specification of no less than 48 kHz and 16-bit. The speech stream is acquired in segments using a frame buffer method through an audio acquisition library. At the beginning of the acquisition, environmental noise is recorded to construct a noise spectrum template. In the noise reduction process, the background noise is reduced by subtracting the speech spectrum of the speech stream from the noise spectrum. Specifically, after receiving the audio stream synchronously acquired in the preceding steps, the audio analysis channel first performs preprocessing optimized for the interview scenario on the entire audio signal. The system employs an omnidirectional condenser microphone and a high sampling configuration, ensuring the audio stream covers the entire frequency range of the interviewer's speech. It receives raw audio data in segments using a frame buffer through an audio acquisition library, with frame length and interval parameterized to balance real-time performance and temporal resolution. In the first few hundred milliseconds of the session, the interview room typically only experiences background noise such as air conditioning, keyboard noise, and ambient hum. Background noise segments are automatically acquired during this period, and a noise spectrum template is generated through spectral analysis. During the noise reduction stage, this noise spectrum template is used as a reference to perform frame-by-frame spectral subtraction on the subsequent audio stream. This involves subtracting the spectrum of the current audio frame from the noise spectrum in the frequency domain, significantly suppressing steady-state background noise. Typically, this reduces ambient noise by more than 20 decibels, making the audio components more prominent in subsequent endpoint detection and feature extraction. Through this "noise template first, then spectral subtraction" approach, the system can obtain relatively clean audio signals in common remote video interview environments without complex adaptive filtering.

[0041] When performing speech endpoint detection, the audio analysis channel detects the start and end points of speech based on a joint threshold of short-time energy and zero-crossing rate. It determines and discards segments with energy and zero-crossing rate below the silence threshold for several consecutive frames, retaining only the valid speech segments.

[0042] In other words, after basic noise reduction, the audio analysis channel performs speech endpoint detection on the processed speech stream to extract effective speech segments containing actual speaking content from continuous long speech. Specifically, the speech stream is divided into frames with a fixed-length time window. For each frame, two temporal features, short-time energy and zero-crossing rate, are calculated. A dual-threshold method is used to comprehensively determine whether the frame contains effective speech: short-time energy reflects the overall strength of the signal in the frame, and zero-crossing rate measures the activity of waveform changes. Both are at low levels when there is silence or background noise dominates. Subsequently, a normalized lower threshold is set for short-time energy, and a threshold of zero-crossing rate is set for the number of times per second. When the short-time energy and zero-crossing rate of several consecutive frames are both lower than the corresponding thresholds, these frames are judged to be silent or blank segments, marked as non-speech regions, and discarded in post-processing; conversely, when the short-time energy or zero-crossing rate exceeds the threshold and is maintained for a certain duration, it is considered to have entered the effective speech region, the start and end times of the speech segment are recorded, and it is extracted. In this way, useless segments such as long pauses by the interviewer and network muting can be effectively removed, and only the valid audio segments containing the questions or statements can be retained. The audio segments are then cached in a uniform audio format (such as mono, fixed sampling rate) as input for the subsequent automatic speech recognition engine.

[0043] The automatic speech recognition engine employs a streaming recognition mode. During recognition, the effective speech segments are sliced ​​into segments of fixed time lengths and transcribed into text in real time. That is, for the retained effective speech segments, this step uses an automatic speech recognition engine in streaming recognition mode for transcription processing. To adapt to cross-cultural and multilingual environments, the speech recognition engine is configured to automatically recognize multiple languages ​​and prioritizes acoustic and language models related to workplace communication and recruitment interviews to improve the transcription accuracy of terms such as "work experience" and "salary expectations." During the recognition process, each effective speech segment is further divided into continuous slices of fixed time length (e.g., 500ms), and each slice is fed into the recognition engine. After receiving the audio slices, the recognition engine quickly returns the corresponding transcribed text, thereby achieving near real-time speech-to-text conversion capabilities. For the transcribing results of the streaming output, a unified text standardization process is first performed: First, case is unified, converting all letters to a uniform form to avoid word matching failures due to case differences; second, functional particles, articles, prepositions, and other words with low semantic value are removed based on a multilingual stop word list, thereby shortening the text length and reducing the complexity of subsequent scanning; third, synonym normalization is performed, constructing a cross-cultural taboo synonym database. For semantically equivalent but superficially different word groups such as "age," "how old," and "birthyear," clustering is performed using word vector similarity. When a certain expression has a cosine similarity of cos ≥ 0.85 with a known taboo word vector, it is determined to belong to the same semantic category and is uniformly replaced with a normalized representation in the standardized text to ensure that taboo utterances with different expressions are not missed in subsequent detection.

[0044] After removing multilingual stop words, the standardized text is matched character by character by constructing a multi-branch tree structure containing taboo words and their synonyms from the target country or region. When the path in the multi-branch tree structure reaches a node marked as a taboo word, the corresponding taboo word hit result is output. Specifically, for semantically equivalent word groups, the similarity between word vectors is calculated, and when the similarity is greater than a preset threshold, they are grouped into the same taboo category to ensure that taboo discourses with different expressions can be identified. The corresponding category and triggering context are recorded in the language taboo detection results according to the different labels of absolute taboo words and avoidable words.

[0045] After obtaining the standardized text, the system performs sensitive word scanning and semantic parsing on the text based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base. To improve matching efficiency and accommodate dynamic expansion, the system uses a multi-way tree structure similar to a Trie tree to store the taboo words and their synonyms for each target country or region. Each node corresponds to a character or sub-word unit, and the leaf node marks whether it is a taboo word entry, along with category information (such as "absolute taboo word", "avoidance word", etc.). The standardized text traverses downwards step by step from the root node in the form of a character stream or sub-word sequence. When the traversal path exactly falls on a leaf node marked as a taboo word, it is considered that the corresponding taboo word entry has been hit, and the hit word, the country or region it belongs to, the taboo category, as well as the position and context sentence in the original transcription text are recorded. For the hit results marked as "absolute taboo words", it can be directly regarded as a highly sensitive behavior at the language content level; for the results marked as "avoidance words", only the word and context are temporarily recorded, and the emotional feature judgment link is used to further analyze whether its tone constitutes an actual taboo, so as to reduce false alarms for neutral or friendly expressions. <> <>

[0046] Meanwhile, the audio analysis channel also needs to extract emotion-related features from two dimensions of the speech signal itself and the text content, for constructing an emotion feature vector that combines acoustics and text. On the one hand, acoustic emotion features are extracted from the preprocessed speech stream, including at least parameters such as multi-dimensional Mel frequency cepstral coefficients (MFCC), multi-dimensional linear prediction coefficients (LPC), fundamental frequency, short-time energy, and spectral centroid. MFCC reflects the spectral envelope characteristics of speech on the Mel scale and can characterize the change in timbre; LPC reflects the resonance characteristics of the vocal tract and helps to distinguish pronunciation states such as tense and relaxed; the fundamental frequency and short-time energy can capture the changes in tone height and volume strength; the spectral centroid can represent the brightness and sharpness of the sound. These acoustic features are usually extracted at fixed time intervals (such as every 500ms) to form a feature sequence that evolves over time. On the other hand, based on the standardized text obtained by transcription, text emotion features are extracted, such as emotion polarity (a value between negative and positive), subjectivity (describing the degree of subjective evaluation contained in the statement), and the occurrence frequencies of angry words, impatient words, interrogative mood words, and exclamatory mood words. The interrogative mood words can include "ma", "ne", etc., and the exclamatory words can include "a", "ya", etc. By counting the proportion of these words in the text, the implicit doubts, accusations, or strong emotional tendencies in the interviewer's expression can be characterized. The acoustic features and text emotion features are concatenated in a predetermined order to form a sequence of emotion feature vectors with a fixed dimension, which is used as the input of the emotion classification model. <> <>

[0047] Based on the emotional polarity, subjectivity, and frequency of occurrence of anger, impatience, doubt, and interjections calculated from the standardized text, and based on the acoustic emotional features, a joint acoustic and textual emotional feature vector is constructed. The emotional feature vector is input into an emotional classification model combining a convolutional neural network and a recurrent neural network to output three emotional labels: neutral, slightly negative, and severely negative, along with their corresponding probabilities. The acoustic emotional features include at least several Vermeer frequency cepstral coefficients, linear prediction coefficients, fundamental frequencies, short-time energy, and spectral centroid features.

[0048] In the sentiment classification stage, this embodiment employs a multimodal sentiment classification model combining convolutional neural networks and recurrent neural networks. The aforementioned sentiment feature vectors are input into the model for forward inference, outputting three sentiment labels: neutral, slightly negative, and severely negative, along with their corresponding probabilities. Specifically, the acoustic feature part consists of multiple layers of one-dimensional convolutional networks and pooling layers, used to extract local sentiment patterns from the time series. After convolutional extraction, an LSTM or bidirectional LSTM layer is connected to capture the long-range temporal dependencies of sentiment. The text sentiment feature part can consist of several fully connected layers, used for compressing and nonlinearly transforming its low-dimensional sentiment indicators. The outputs of the two branches are concatenated and fused with the fully connected layers, then output a 3D probability distribution via softmax, representing the probability that the current speech segment is "neutral," "slightly negative," or "severely negative." The category with the highest probability is used as the sentiment label for that segment, while the probability value is retained as a confidence index to introduce confidence weights in subsequent risk grading.

[0049] After performing sensitive word scanning and semantic analysis on the standardized text, when the semantic analysis result contains absolutely forbidden words and the sentiment label is slightly negative or severely negative, the language taboo detection result is marked as a high-risk sentiment taboo. When the semantic analysis result only contains avoidable words and the sentiment label is severely negative, the language taboo detection result is marked as a medium-risk sentiment taboo. When the sentiment label is severely negative but does not hit any forbidden words, it is only recorded as a low-risk sentiment reminder without triggering an interface warning.

[0050] In other words, after semantic parsing and sentiment classification results are generated, the system generates language taboo detection results and sentiment risk levels based on the combination relationship between language content and sentiment state. Specifically, when the semantic parsing result shows that an absolute taboo word is hit, and the sentiment label output by the sentiment classification model is "slightly negative" or "severely negative," it is considered that the expression of negative emotions on a sensitive topic is a high-risk sentiment taboo, and the language taboo detection result is marked as high-risk. When the semantic parsing result only hits avoidable words (such as "must," "force," and other words that need to be used with caution), and the sentiment label is "severely negative," it is considered that the tone has a strong pressure or commanding tone, which has a significant risk of conflict in many cultures, and the detection result is marked as a medium-risk sentiment taboo. When the sentiment label is "severely negative" but no taboo words are hit, it means that the discourse itself does not touch on cultural taboos, but the expression is relatively intense. The system only records this situation as a low-risk sentiment reminder and does not trigger an interface warning to avoid excessively interfering with the normal interview process. Other combinations (such as neutral sentiment and no taboo words are hit) are considered risk-free. Under the constraints of the above rules, step S105 can output a comprehensive language taboo detection result that includes "whether language taboo is triggered" and "emotional risk level and its basis", providing reliable audio-side judgment information for subsequent cross-modal fusion decision-making and real-time interaction optimization.

[0051] In step S106, the physical taboo detection results and the language taboo detection results are fused together in a dual-channel manner, and the comprehensive risk level of the current interview round is output according to the preset decision rules. When the comprehensive risk level reaches the warning threshold, a warning signal and interaction optimization suggestions containing taboo type, trigger basis and recommended alternative behavior are generated. Specifically, the dual-channel collaborative fusion includes: when the physical taboo detection result indicates the presence of high-risk taboo gestures from the target country or region, increasing the sampling rate and analysis frequency of the audio analysis channel, and performing key sensitive word scanning and emotional intensity assessment on the speech segments within the current time window; when the language taboo detection result indicates the presence of sensitive words or that the emotional risk level reaches the warning threshold, increasing the image processing frame rate of the visual analysis channel, and performing fine-grained behavior and micro-expression tracking on the interviewer's facial and upper body areas to update the comprehensive risk level; the comprehensive risk level is obtained by weighting three risk components—physical taboo, language taboo, and behavioral norms—according to preset weights, and is used to determine the warning level and feedback strategy.

[0052] In step S106, the physical taboo detection results obtained in step S104 and the language taboo detection results obtained in step S105 are used as inputs. The two are aggregated on a unified timeline according to the interview rounds or time windows to form a risk description for the current time period. Specifically, within each time window, the visual analysis channel will output the physical risk component formed by whether the taboo gesture is hit, the risk level of the corresponding taboo item (such as absolute taboo, general taboo), and whether there are behavioral norm abnormalities (such as long-term head tilt, abnormal eye contact ratio, etc.); the audio analysis channel will output the language risk component formed by whether the absolute taboo word or avoidable word is hit, the language taboo category, and the emotional label (neutral, slightly negative, severely negative). In this step, numerical risk weights are assigned to physical taboos, verbal taboos, and behavioral norms. For example, "missing a taboo" is recorded as 0, "medium-risk emotional taboos" as an intermediate value, and "high-risk emotional taboos" or "confirmation of taboo gestures" as a higher value. Then, the three risk weights are weighted and summed according to preset weights to obtain the overall risk level for the current time window or interview round. The preset weights can be configured based on the company's emphasis on different risk types. For example, in countries where physical taboos are extremely sensitive, the weight of the physical taboo component can be appropriately increased, while in cultures that value politeness and tone of voice, the weight of the verbal taboo component can be increased. After normalization, the overall risk level can be mapped to discrete warning levels (e.g., "no risk", "low risk alert", "medium risk warning", "high risk warning") and compared with a global warning threshold. When the overall risk level does not reach the threshold, the system only records the risk evolution trend in the background. When the overall risk level reaches or exceeds the warning threshold, it indicates that there is significant inappropriate behavior in the cultural context of the target country or region during the current interview round, and the interviewer needs to be provided with a visual or voice warning immediately.

[0053] To enhance the sensitivity and robustness of risk detection, this step introduces a dual-channel collaborative fusion mechanism between the visual and audio analysis channels. On the limb side, when the limb taboo detection result indicates the presence of a high-risk taboo gesture from the target country or region within the current time window—for example, confirming the detection of a gesture considered insulting or extremely impolite in the local culture—it can be assumed that there is a high probability of inappropriate tone or sensitive language related to that taboo gesture on the language side. Therefore, the parameters of the audio analysis channel are dynamically adjusted. Without changing the underlying hardware sampling configuration, the analysis frame length and frame shift are shortened, increasing the analysis frequency and effectively improving the ability to analyze fine-grained speech segments. Within the current time window, the scanning of sensitive words and the assessment of emotional intensity are emphasized. For example, within a few seconds after detecting a high-risk gesture, the audio analysis channel can change the original second-level emotional assessment to accumulation over shorter time slices, allowing for rapid capture of sudden increases in tone, the appearance of abusive language, or coercive questioning techniques, thereby updating the language-side risk components and overall risk level. Conversely, on the language side, when the language taboo detection results indicate the presence of sensitive words (including absolutely taboo words or specific avoidable words) or the emotional risk level itself has reached the warning threshold (e.g., multiple consecutive segments are judged as having severe negative emotions), the system increases the image processing frame rate or analysis frame selection density of the visual analysis channel. It performs fine-grained behavioral and micro-expression tracking on the interviewer's facial and upper body areas. For example, it more frequently captures short-term changes in local muscles such as eyebrows, corners of the mouth, and eyelids, as well as changes in leaning forward, backward, and small hand movements, to determine whether there are aggressive or contemptuous body movements that accompany negative language. Through this dynamic collaborative mechanism, both the visual and audio channels automatically enter a "focused attention" state when the other shows a high-risk signal, allowing the overall risk level to be rapidly updated based on new evidence in a short time, rather than relying solely on a single channel's one-time output.

[0054] When the overall risk level reaches the warning threshold, a warning signal and interaction optimization suggestions are generated, including the taboo type, triggering basis, and recommended alternative behaviors. These are then associated with the current time window, interview round, and target country or region. Taboo types can include one or more combinations such as "physical taboos," "linguistic taboos," "high emotional risk," and "deviation from behavioral norms." The triggering basis records the specific taboo item number, the taboo gesture template or taboo word that was triggered, the emotional classification result, and the corresponding confidence or similarity information. Recommended alternative behaviors are automatically obtained from the alternative phrases or actions bound to the item in the regionalized cultural taboo rule base and filtered according to the current channel priority. For example, when linguistic taboos are predominant, alternative questions or euphemisms are prioritized, and when physical taboos are predominant, suggestions such as posture and gesture restraint are provided. Warning signals can be encapsulated as structured messages and visualized by subsequent interface modules using different colors, icons, and prompts. The feedback strategy is determined based on the overall risk level: high-risk situations receive prominent pop-ups or obvious icons, medium-risk situations receive milder badge reminders, and low-risk emotional reminders are only recorded in the background for post-interview debriefing. Through this dual-channel collaborative fusion and warning signal generation mechanism, without introducing new analytical modalities, joint assessment and tiered feedback on physical taboos, linguistic taboos, and behavioral norms are achieved. This enables timely and targeted interactive optimization suggestions for momentary high-risk behaviors during cross-cultural interviews.

[0055] In step S107, the warning signal and the interaction optimization suggestions are presented in real time in a visual manner on the interview user interface. The occurrence time, risk level, multimodal characteristics and collection environment parameters of the corresponding taboo behavior are written into the behavior database. The regional cultural taboo rule base is dynamically updated and a corresponding interview debriefing report is generated.

[0056] Specifically, the warning signal is pushed to the front-end interface through a persistent connection-based communication mechanism. Within a preset time after the prohibited behavior occurs, a color-coded warning box pops up on the interview interface. The warning box displays the country or region to which the prohibited behavior belongs, the category, the risk level, and recommended alternative words or actions to guide the interviewer to make immediate adjustments. The behavioral database records the frequency, time distribution, and emotional fluctuation values ​​of various taboo behaviors. After the interview, a cultural adaptation index is calculated based on the ratio of the number of taboo behaviors to the total number of behaviors to reflect the degree of control over cultural taboos during the interview. The cultural adaptation index, taboo behavior heatmap, and priority of improvement suggestions are combined to generate the interview debriefing report.

[0057] After obtaining the overall risk level of the current interview round through dual-channel fusion, the system first determines whether to push a warning signal and interaction optimization suggestions to the interview user interface based on whether a warning threshold has been reached. To ensure the real-time nature and continuity of the prompts, a persistent connection is established between the front-end interface and the back-end analysis service at the start of the interview. This is achieved through methods such as long connections or push-like channels to maintain bidirectional communication. When the overall risk level is determined to be medium to high risk, the back-end pushes a structured warning message to the front-end interface through this persistent connection. Within a preset time (e.g., hundreds of milliseconds to several seconds) after receiving the warning signal, the front-end displays a color-coded warning box above or to the side of the current interview screen. The color corresponds to the risk level; for example, a red background indicates high risk, a yellow background indicates medium risk, and a weaker warning style indicates low risk. The warning box displays the country or region of origin of the taboo, the taboo category (such as physical taboos, linguistic taboos, or behavioral norms), the current risk level, and recommended alternative phrases or actions retrieved from a regional cultural taboo rule base. Interviewers can quickly browse this information without interrupting the interview process and adjust their wording, tone, gestures, or posture accordingly. For high-risk events that last for a long time or are repeatedly triggered, the interface can also add features such as flashing effects or icon prompts, without exceeding the scope of this step, to alert the interviewer to systematic deviations in behavioral patterns.

[0058] While displaying warning signals to the interviewer, relevant information can be simultaneously written into the behavior database to create a traceable and statistically verifiable record of behavior. Specifically, the time of each triggered or potentially triggered taboo behavior is recorded, including the occurrence time (timestamp accurate to the frame or audio segment level), the corresponding interview session identifier, the type of taboo behavior, the overall risk level, and multimodal features and environmental parameters directly related to the event. Multimodal features may include key gesture feature vector indices used to determine the taboo behavior, the result of hitting taboo words, sentiment tags and their probability values, used to analyze the combination of "action + utterance + emotion" during debriefing. Environmental parameters may include the type of camera and microphone used at the interviewing end, current network latency and packet loss rate, estimated environmental noise, and light intensity, used to help determine whether certain errors may be caused by equipment or the environment. The behavior database is continuously updated throughout the interview process, providing a complete data foundation for subsequent statistical analysis, rule updates, and debriefing report generation. At the same time, the system can periodically count the frequency and time distribution of various taboo behaviors in the interview in the background, forming time series or time period aggregated data, which can be used to build subsequent visualization indicators such as time heatmaps.

[0059] After the interview, the records of the interview in the behavioral database are summarized and analyzed. This drives the dynamic updating of the regionalized cultural taboo rule base and generates a corresponding interview debriefing report. Regarding dynamic updates, the system tracks the trigger frequency of different taboo items, whether they recur after being warned, and whether the interviewer used recommended alternative phrases or actions in subsequent time periods. Based on these statistical results, and with approval from the management side, the risk level, applicable scenario tags, or alternative behavior descriptions of certain items can be fine-tuned. Alternatively, synonyms can be automatically mined and added to the corresponding taboo categories for frequently triggered expressions that are not currently covered by the vocabulary, gradually improving the rule base's relevance to actual cross-cultural interview scenarios. In terms of quantifying overall interview quality, the system uses the number of taboo behaviors recorded during the interview process... Total number of actions First, calculate the ratio of the two: ; Based on this, the cultural adaptation index is obtained. ; in The closer the index is to 1, the better the control over cultural taboos during the interview process. This index, along with the density of taboo behaviors at different times and of different types, can be visualized as a taboo behavior heatmap. For example, with time as the horizontal axis and taboo category as the vertical axis, the intensity of risk events can be represented by color depth, helping HR managers intuitively identify high-risk periods and high-risk behavior types in interviews. Simultaneously, based on the risk level and frequency of various taboo behaviors, combined with emotional fluctuation values, improvement suggestions can be prioritized. For example, high-risk and frequently occurring words or gestures can be prioritized for training. Finally, the cultural adaptation index, taboo behavior heatmap, and prioritized improvement suggestions are combined to generate a structured interview debriefing report. This report can be presented in a combination of charts and text descriptions, providing a basis for interviewers' personal reflection and cross-cultural communication training, and also providing data support for companies to optimize interview strategies in different countries or regions.

[0060] Based on the same line of thought, such as Figure 2 As shown, a smart perception and real-time interaction optimization system for cross-cultural recruitment interviews is provided, including: The rule base construction module 201 is used to collect taboo data and, based on the taboo data, construct a regionalized cultural taboo rule base in a three-level structure of national level, taboo category level and taboo item level. The taboo data comes from one of the following: public documents, questionnaires, and expert knowledge. The multimodal data acquisition module 202 is used to collect interview data, which is the video stream and audio stream of the interviewer simultaneously acquired through a camera and microphone during the interview process; The modal channel distribution module 203 is used to input the acquired video stream into the visual analysis channel and the acquired audio stream into the audio analysis channel; The limb taboo detection module 204 is used to perform image preprocessing on the video stream based on the visual analysis channel, and call the gesture recognition model to extract the coordinates of key points of the interviewer's hands. Based on the spatial positional relationship between the key points of the hands, a multi-dimensional gesture feature vector is generated to represent the current gesture. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library. Based on the comparison result of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. The language taboo detection module 205 is used to perform noise reduction and speech endpoint detection on the speech stream based on the audio analysis channel, extract effective speech segments and input them into an automatic speech recognition engine to obtain transcribed text, perform case unification, stop word removal and synonym normalization on the transcribed text to obtain standardized text; based on the language taboo word list and tone taboo rules pre-built in the regional cultural taboo rule base, perform sensitive word scanning and semantic analysis on the standardized text, and combine the acoustic emotion features extracted from the speech stream to generate language taboo detection results and emotion risk levels; The fusion and early warning generation module 206 is used to perform dual-channel collaborative fusion of the limb taboo detection results and the language taboo detection results, output the comprehensive risk level of the current interview round according to the preset decision rules, and generate an early warning signal and interaction optimization suggestions that include taboo type, trigger basis and recommended alternative behavior when the comprehensive risk level reaches the early warning threshold. The visualization module 207 is used to present the warning signal and the interaction optimization suggestions in a visual manner in real time on the interview user interface, and write the corresponding taboo behavior occurrence time, risk level, multimodal characteristics and collection environment parameters into the behavior database, and dynamically update the regional cultural taboo rule library and generate the corresponding interview review report.

[0061] This system constructs a regionalized cultural taboo rule base covering multiple countries / regions and deeply integrates it with multimodal perception technologies such as gesture recognition, speech recognition, and sentiment analysis. This enables unified modeling and automatic identification of risks related to physical taboos, linguistic taboos, and behavioral norms, significantly improving the accuracy of cultural risk avoidance. Through a dual-channel collaborative mechanism of visual and audio channels, when suspected or confirmed taboo behavior is detected, a warning signal and alternative wording / action suggestions are generated with millisecond-level latency and pushed to the interview interface in real time, effectively compensating for the lack of immediate feedback in traditional systems. Simultaneously, the system structurally stores multidimensional data such as the time, type, risk level, and emotional fluctuations of taboo behavior, constructing quantitative indicators such as a cultural adaptation index. It also generates a debriefing report including behavior frequency distribution, risk heatmaps, and priority of improvement suggestions. This provides visualized and data-driven support for companies to assess and improve HR's cross-cultural communication capabilities and develop targeted training programs, significantly enhancing the safety, fairness, and communication efficiency of cross-cultural interviews.

[0062] In one optional implementation, the system adopts a browser / server architecture, dividing the construction of the cultural rule base, the multimodal perception engine, the risk decision center, and the interactive optimization interface into several collaborative functional modules. The front-end uses a single-page application framework based on Vue.js to integrate visualization components and calls the MediaPipe gesture and posture recognition engine on the browser side to reduce the bandwidth pressure of video stream transmission. Simultaneously, a full-duplex persistent connection is established with the back-end via WebSocket, pushing a warning message within 200ms after prohibited behavior detection. At the interface rendering layer, a semi-transparent dynamic warning box and icon are overlaid on the video screen using HTML5Canvas. Different risk levels correspond to red, yellow, and blue color schemes and flashing styles, allowing interviewers to quickly perceive risk alerts without interrupting the interview. The backend provides REST and WebSocket services based on the Python Flask framework, responsible for task queue management, model inference invocation, and risk decision-making logic execution. The rule base and behavioral data are uniformly stored in a MySQL 8.0 database. The system can be scaled to support rule data from at least a dozen countries or regions through a distributed deployment, and ensures that the retrieval response time for a single prohibited item is no more than 50ms, in order to meet the real-time matching needs during the interview process.

[0063] Regarding rule base maintenance and entry version management, based on the existing three-level structure, a template vector field and a sensitive word field are added to the taboo entry table. These fields directly store the standard gesture key point coordinate vector `template_vector` corresponding to the entry and the list of language sensitive words associated with the taboo entry `sensitive_words`. The `create_time` and `update_time` fields record the entry creation and most recent adjustment time, facilitating retrospective and incremental release of rule versions. Rule base updates employ a "scheduled crawling + expert review + periodic incremental updates" approach: on one hand, the Python Scrapy framework automatically crawls the latest content from designated cultural research websites and authoritative media (such as the BBC Culture Guide) at fixed times each day according to a planned task. The crawled results are deduplicated using MD5 digests, ensuring a data retention rate of no less than 97% after deduplication. On the other hand, domain experts periodically review new entries, adding approved taboo behaviors and alternative solutions to the database, which are then centrally released in weekly routine updates, ensuring the rule base continuously reflects the latest changes in business etiquette and recruitment practices in various target countries or regions.

[0064] In terms of model adaptive optimization, the system utilizes historical recognition records in the behavior database to dynamically adjust the taboo gesture template and visual recognition parameters. Specifically, whenever at least 1000 gesture recognition records are collected, the misclassification rate for each taboo gesture is calculated using the formula: Misclassification rate = Number of misclassifications / Total number of recognitions. When the misclassification rate for a particular taboo gesture exceeds 8%, the corresponding multidimensional feature vector set is extracted from the misclassified samples, its mean vector is calculated, and the original template vector is replaced with this mean vector. This allows for adaptive correction of the gesture's location in the feature space. Simultaneously, by combining the recognition results of new and old templates, the tracking confidence parameter of the underlying gesture recognition engine is fine-tuned; for example, the minimum tracking confidence is increased from 0.60 to 0.65 to reduce false alarms without significantly increasing the risk of missed detections. Similarly, in the sentiment analysis branch, a fixed lightweight configuration can be used for the structure of the convolutional neural network, such as setting 32, 64, and 128 filters in three one-dimensional convolutional layers respectively, with a uniform kernel size of 3×3 and ReLU activation function. This allows the model to maintain high classification accuracy while possessing good real-time inference capabilities. For sentences containing "avoidance words," an intensity threshold can be introduced after the output layer. When the model's sentiment intensity value is greater than or equal to 0.7 and simultaneously contains such words, it is directly determined as a language-side taboo event, and this result is submitted to the risk decision center for comprehensive risk assessment.

[0065] In typical application scenarios, the system can be deployed within a company's remote recruitment platform. The front-end runs on a browser page developed using Vue.js, integrating gesture and posture recognition capabilities provided by MediaPipe v0.10.3. The back-end uses the Python Flask framework in conjunction with a MySQL database to implement multimodal data management and decision-making logic. The hardware environment includes a 1080P high-definition camera and an omnidirectional microphone with noise reduction capabilities for acquiring stable image and audio data. In an actual interview process, after the HR selects a specific country's interview mode on the interface, the system automatically loads the corresponding taboo rules and template vectors for that country. When it detects the interviewer making a thumbs-up gesture—a gesture with a risk value of 0.92 in the local culture—a red warning box pops up on the interface approximately 187ms after the gesture is captured, and the entire event is written to the behavior database for subsequent review. In a comparative test using 50 cross-cultural interviews as a sample...

[0066] The specific details of each module / unit in the above system have been described in detail in the implementation method section. For any undisclosed details, please refer to the implementation method section, and therefore will not be repeated here.

[0067] Based on the same idea, this specification also provides an intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews, such as... Figure 3 As shown.

[0068] The intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews can be the terminal device or server provided in the above embodiments.

[0069] The intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews can vary significantly due to differences in configuration or performance. It may include one or more processors 301, memory 302, and buses. The memory 302 may store one or more application programs or data. The memory 302 may include readable media in the form of volatile storage units, such as random access memory (RAM) and / or cache memory units, i.e., plug-in external hard drives, smart media cards (SMC), secure digital cards (SD), flash cards, etc., and may further include read-only storage units. The application programs stored in the memory 302 may include one or more program modules (not shown in the figure), including but not limited to: operating systems, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Furthermore, the processor 301 may be configured to communicate with the memory 302 to execute a series of computer-executable instructions stored in the memory 302 on the intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews. The intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews may also include one or more power supplies 303, one or more wired or wireless network interfaces 304, one or more I / O interfaces (input / output interfaces) 305, and one or more external devices 306 (e.g., keyboards). It may also communicate with one or more devices that enable user interaction with the device, and / or with any device that enables the device to communicate with one or more other computing devices (e.g., routers, network switches, etc.). This communication can be performed through I / O interfaces 305. Furthermore, the device can also communicate with one or more networks (e.g., local area networks (LANs)) through wired or wireless interfaces 304.

[0070] Figure 3 Only a device for intelligent sensing and real-time interaction optimization in cross-cultural recruitment interviews with components is shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on intelligent sensing and real-time interaction optimization devices for cross-cultural recruitment interviews. It may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0071] Specifically, in this embodiment, the intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews includes a memory and one or more programs. One or more programs are stored in the memory, and each program may include one or more modules. Each module may include a series of computer-executable instructions for the intelligent sensing and real-time interaction optimization device for cross-cultural recruitment interviews, and is configured to be executed by one or more processors. The program includes the following computer-executable instructions: Collect taboo data, and based on the taboo data, construct a regionalized cultural taboo rule base in a three-level structure of national level, taboo category level and taboo item level. The taboo data comes from one of the following sources: public documents, questionnaires, and expert knowledge. The interview data is collected by simultaneously capturing the interviewer's video and audio streams through a camera and microphone during the interview process. The acquired video stream is input into the visual analysis channel, and the acquired audio stream is input into the audio analysis channel; Based on the visual analysis channel, the video stream is preprocessed, and a gesture recognition model is called to extract the coordinates of key points on the interviewer's hands. A multi-dimensional gesture feature vector is generated to represent the current gesture based on the spatial relationship between the key points on each hand. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library to calculate the similarity. Based on the comparison results of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. Based on the audio analysis channel, noise reduction and speech endpoint detection are performed on the speech stream. Valid speech segments are extracted and input into an automatic speech recognition engine to obtain transcribed text. The transcribed text is then processed for case unification, stop word removal, and synonym normalization to obtain standardized text. Based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, the standardized text is subjected to sensitive word scanning and semantic analysis. Combined with the acoustic emotion features extracted from the speech stream, language taboo detection results and emotion risk levels are generated. The physical taboo detection results and the language taboo detection results are fused together in a dual-channel manner, and the comprehensive risk level of the current interview round is output according to the preset decision rules. When the comprehensive risk level reaches the warning threshold, a warning signal and interaction optimization suggestions are generated, which include taboo type, triggering basis and recommended alternative behavior. The warning signals and the interaction optimization suggestions are presented in real time in a visual manner on the interview user interface. The occurrence time, risk level, multimodal characteristics and collection environment parameters of the corresponding taboo behaviors are written into the behavior database. The regional cultural taboo rule base is dynamically updated and a corresponding interview debriefing report is generated.

[0072] Based on the same idea, exemplary embodiments of the present invention also provide a computer-readable storage medium having a program product stored thereon capable of implementing the methods described above in this specification. In some possible embodiments, various aspects of this disclosure can also be implemented as a program product including program code, which, when the program product is run on a terminal device, causes the terminal device to perform the steps according to the various exemplary embodiments of this disclosure described in the "Exemplary Methods" section above.

[0073] refer to Figure 4 As shown, a program 400 for implementing the above-described method according to an exemplary embodiment of the present disclosure is described. This program may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0074] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0075] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0076] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, CSS, and HTML, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0077] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the exemplary embodiments of this disclosure.

[0078] Furthermore, the above figures are merely illustrative representations of the processes included in the methods according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0079] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0080] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0081] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for intelligent perception and real-time interaction optimization in cross-cultural recruitment interviews, characterized in that, The method includes: Collect taboo data, and based on the taboo data, construct a regionalized cultural taboo rule base in a three-level structure of national level, taboo category level and taboo item level. The taboo data comes from one of the following sources: public documents, questionnaires, and expert knowledge. The interview data is collected by simultaneously capturing the interviewer's video and audio streams through a camera and microphone during the interview process. The acquired video stream is input into the visual analysis channel, and the acquired audio stream is input into the audio analysis channel; Based on the visual analysis channel, the video stream is preprocessed, and a gesture recognition model is called to extract the coordinates of key points on the interviewer's hands. A multi-dimensional gesture feature vector is generated to represent the current gesture based on the spatial relationship between the key points on each hand. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library to calculate the similarity. Based on the comparison results of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. Based on the audio analysis channel, noise reduction and speech endpoint detection are performed on the speech stream. Valid speech segments are extracted and input into an automatic speech recognition engine to obtain transcribed text. The transcribed text is then processed for case unification, stop word removal, and synonym normalization to obtain standardized text. Based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, the standardized text is subjected to sensitive word scanning and semantic analysis. Combined with the acoustic emotion features extracted from the speech stream, language taboo detection results and emotion risk levels are generated. The physical taboo detection results and the language taboo detection results are fused together in a dual-channel manner, and the comprehensive risk level of the current interview round is output according to the preset decision rules. When the comprehensive risk level reaches the warning threshold, a warning signal and interaction optimization suggestions are generated, which include taboo type, triggering basis and recommended alternative behavior. The warning signals and the interaction optimization suggestions are presented in real time in a visual manner on the interview user interface. The occurrence time, risk level, multimodal characteristics and collection environment parameters of the corresponding taboo behaviors are written into the behavior database. The regional cultural taboo rule base is dynamically updated and a corresponding interview debriefing report is generated.

2. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The construction of the aforementioned regionalized cultural taboo rule base also includes: A unique node is established for each target country or region at the country level based on the country code; In the taboo category layer, interview-related behaviors are divided into physical taboos, verbal taboos, and behavioral norms; In the taboo item layer, set a behavioral description, risk level, judgment basis link, and feasible alternative words or actions for each taboo item; A relational database is used to model the table structure of the regionalized cultural taboo rule base. By using an index with the country code and taboo category as the composite key, the target entries in the regionalized cultural taboo rule base can be retrieved.

3. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, After acquiring the video stream, color space conversion, grayscale conversion, and smoothing filtering are performed on each frame of the video stream, and the image size and pixel values ​​are normalized. While extracting the key points of the hands, the coordinates of the key points of the head and shoulders are extracted using a posture recognition model. The head tilt angle is calculated based on the coordinate difference between the key points of the head and the key points of the shoulders. The overlap time between the eye areas of the interviewer and the candidate is statistically analyzed using a face detection and tracking algorithm to obtain the duration of eye contact. The head tilt angle and the duration of eye contact are used as posture features. The posture features and the corresponding multi-dimensional gesture feature vectors are stored in the behavior database for the purpose of judging and reviewing behavioral norms and taboos.

4. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The gesture recognition model is a hand key point detection model based on a depthwise separable convolutional network. It is used to detect the hand region from each frame of the normalized video stream and output the three-dimensional coordinates of the hand key points. During detection, the gesture recognition model calculates the curvature of each finger based on the ratio of the distance between the fingertip and the base of the finger and the center of the palm, calculates the fingertip distance feature based on the ratio of the distance between different fingertips to the length of the palm, calculates the palm tilt feature based on the angle between the palm plane and the horizontal and vertical directions, constructs the relative position feature based on the coordinate difference between selected key point pairs, and calculates the average speed of gesture movement between consecutive frames to obtain motion features, thereby forming the multidimensional gesture feature vector. After normalizing each dimension of the multidimensional gesture feature vector according to the mean and standard deviation of historical samples, the ratio of the sum of the products of the multidimensional gesture feature vector and the taboo gesture template vector in each dimension to the product of the vector magnitude is calculated. This ratio is used as the gesture similarity. When the similarity is not less than the first threshold, it is determined to be a confirmed taboo gesture. When the gesture similarity is between the second threshold and the first threshold, it is determined to be a suspected taboo gesture, and different levels of warnings are triggered respectively.

5. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The microphone used when acquiring the speech stream is an omnidirectional condenser microphone with a sampling rate of not less than 48 kHz and a bit depth of not less than 16 bits. The speech stream is acquired in segments using a frame buffer through an audio acquisition library. At the beginning of the acquisition, environmental noise is recorded to construct a noise spectrum template. In the noise reduction process, the background noise is reduced by subtracting the speech spectrum of the speech stream from the noise spectrum. When performing speech endpoint detection, the audio analysis channel detects the start and end points of speech based on a joint threshold of short-time energy and zero-crossing rate. It determines and discards segments with energy and zero-crossing rate below the silence threshold for several consecutive frames, retaining only the valid speech segments.

6. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The automatic speech recognition engine adopts a streaming recognition mode, which slices the effective speech segments into text in real time after recognition by a fixed time length. After removing multilingual stop words, the standardized text is matched character by character by constructing a multi-branch tree structure containing taboo words and their synonyms from the target country or region. When the path in the multi-branch tree structure reaches a node marked as a taboo word, the corresponding taboo word hit result is output. Specifically, for semantically equivalent word groups, the similarity between word vectors is calculated, and when the similarity is greater than a preset threshold, they are grouped into the same taboo category to ensure that taboo discourses with different expressions can be identified. The corresponding category and triggering context are recorded in the language taboo detection results according to the different labels of absolute taboo words and avoidable words.

7. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, Based on the emotional polarity, subjectivity, and frequency of occurrence of anger, impatience, questioning, and interjections calculated from the standardized text, and based on the acoustic emotional features, a joint acoustic and textual emotional feature vector is constructed. The emotional feature vector is input into an emotional classification model combining a convolutional neural network and a recurrent neural network to output three emotional labels: neutral, slightly negative, and severely negative, and their corresponding probabilities. The acoustic emotional features include at least several Vermeer frequency cepstral coefficients, linear prediction coefficients, fundamental frequencies, short-time energy, and spectral centroid features. After performing sensitive word scanning and semantic analysis on the standardized text, when the semantic analysis result contains absolutely forbidden words and the sentiment label is slightly negative or severely negative, the language taboo detection result is marked as a high-risk sentiment taboo. When the semantic analysis result only contains avoidable words and the sentiment label is severely negative, the language taboo detection result is marked as a medium-risk sentiment taboo. When the sentiment label is severely negative but does not hit any forbidden words, it is only recorded as a low-risk sentiment reminder without triggering an interface warning.

8. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The dual-channel collaborative fusion specifically includes: When the physical taboo detection result indicates the presence of high-risk taboo gestures in the target country or region, the sampling rate and analysis frequency of the audio analysis channel are increased, and key sensitive words are scanned and emotional intensity is assessed for the speech segments within the current time window; When the language taboo detection result indicates the presence of sensitive words or that the emotional risk level reaches the warning threshold, the image processing frame rate of the visual analysis channel is increased, and fine-grained behavior and micro-expression tracking is performed on the interviewer's facial and upper body areas to update the comprehensive risk level. The comprehensive risk level is obtained by weighting three risk components—physical taboo, language taboo, and behavioral norms—according to preset weights, and is used to determine the warning level and feedback strategy.

9. The intelligent perception and real-time interaction optimization method for cross-cultural recruitment interviews according to claim 1, characterized in that, The method further includes: The warning signal is pushed to the front-end interface through a persistent connection-based communication mechanism. Within a preset time after the prohibited behavior occurs, a color-coded warning box pops up on the interview interface. The warning box displays the country or region to which the prohibited behavior belongs, the category, the risk level, and recommended alternative words or actions to guide the interviewer to make immediate adjustments. The behavioral database records the frequency, time distribution, and emotional fluctuation values ​​of various taboo behaviors. After the interview, a cultural adaptation index is calculated based on the ratio of the number of taboo behaviors to the total number of behaviors to reflect the degree of control over cultural taboos during the interview. The cultural adaptation index, taboo behavior heatmap, and priority of improvement suggestions are combined to generate the interview debriefing report.

10. A cross-cultural recruitment interview intelligent perception and real-time interaction optimization system, characterized in that, The system includes: The rule base construction module is used to collect taboo data. Based on the taboo data, a regionalized cultural taboo rule base is constructed according to a three-level structure of national level, taboo category level, and taboo item level. The taboo data comes from one of the following sources: public documents, questionnaires, and expert knowledge. A multimodal data acquisition module is used to collect interview data, which is the video stream and audio stream of the interviewer simultaneously captured by a camera and microphone during the interview process; The modal channel distribution module is used to input the acquired video stream into the visual analysis channel and the acquired audio stream into the audio analysis channel. The limb taboo detection module is used to perform image preprocessing on the video stream based on the visual analysis channel, and call the gesture recognition model to extract the coordinates of key points of the interviewer's hands. Based on the spatial positional relationship between the key points of the hands, a multi-dimensional gesture feature vector is generated to represent the current gesture. The multi-dimensional gesture feature vector is compared with the taboo gesture template vector of the target country or region in the regionalized cultural taboo rule library to calculate the similarity. Based on the comparison result of the calculated gesture similarity with the preset first threshold and second threshold, the limb taboo detection result and the corresponding risk level are obtained. The language taboo detection module is used to perform noise reduction and speech endpoint detection on the speech stream based on the audio analysis channel, extract effective speech segments and input them into an automatic speech recognition engine to obtain transcribed text, perform case unification, stop word removal and synonym normalization on the transcribed text to obtain standardized text; based on the pre-constructed language taboo word list and tone taboo rules in the regional cultural taboo rule base, perform sensitive word scanning and semantic analysis on the standardized text, and combine the acoustic emotion features extracted from the speech stream to generate language taboo detection results and emotion risk levels; The fusion and early warning generation module is used to perform dual-channel collaborative fusion of the physical taboo detection results and the language taboo detection results, output the comprehensive risk level of the current interview round according to the preset decision rules, and generate an early warning signal and interaction optimization suggestions that include taboo type, trigger basis and recommended alternative behavior when the comprehensive risk level reaches the early warning threshold. The visualization module is used to present the warning signals and the interaction optimization suggestions in a visual manner in real time on the interview user interface, and write the corresponding taboo behavior occurrence time, risk level, multimodal characteristics and collection environment parameters into the behavior database, and dynamically update the regional cultural taboo rule base and generate the corresponding interview debriefing report.

Images