A multi-modal intelligent early warning system and method for off-campus conversation security
Patent Information
- Application Number
- CN202610742339.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-27
- Publication Date
- 2026-08-18
AI Technical Summary
[0003]但当前谈话环境调节多依赖主观经验，未结合被约谈人的历史生理反应、基础病史等个性化信息，无法精准匹配其生理耐受需求，易引发环境性生理应激，增加安全风险，同时现有预警多依赖定性判断，未建立生理和行为异常等量化指标，难以精准区分风险等级，导致处置措施针对性不足，既可能过度防控干扰谈话，也可能因处置滞后引发安全事件
本发明通过分析目标约谈对象的历史谈话生理数据与环境信息，精准提炼其最适环境参数，并自动调节谈话环境，减少环境因素引发的生理应激，降低被约谈人因环境不适产生的风险，既提升了谈话的人性化程度，也为后续状态监测建立了稳定的生理基线；
Smart Images
Figure CN122597141A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of conversation monitoring and early warning technology, and in particular to a multimodal intelligent early warning system and method for safe walking conversations. Background Technology
[0002] "Walking-through interviews" refers to interviews conducted by discipline inspection and supervision organs with individuals who have been preliminarily verified, those who have not been detained, and key personnel involved in cases during the process of supervision, inspection, review, and investigation.
[0003] However, current methods for adjusting the interview environment rely heavily on subjective experience and do not take into account the interviewee's historical physiological reactions, underlying medical history, and other personalized information. This makes it difficult to accurately match their physiological tolerance needs, which can easily trigger environmental physiological stress and increase safety risks. At the same time, existing early warning systems rely heavily on qualitative judgments and do not establish quantitative indicators such as physiological and behavioral abnormalities. This makes it difficult to accurately distinguish risk levels and results in insufficiently targeted measures. This may lead to over-control that interferes with the interview or to safety incidents due to delayed handling. Summary of the Invention
[0004] The purpose of this invention is to solve the problems in the background art by proposing a multimodal intelligent early warning system and method for safe walking conversations.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: A multimodal intelligent early warning system for safe walking conversations includes: The environment adjustment module is used to analyze historical conversation information, obtain environmental and physiological state information from the historical conversation information, determine the optimal environmental parameters of the target interviewee based on physiological state data at different time periods, adjust the conversation environment according to the optimal environmental parameters, and generate an adjustment completion signal. The status acquisition module is used to collect the status data of the target interviewee after detecting the adjustment completion signal. The status data includes real-time physiological data and behavioral images. The physiological analysis module is used to calculate the real-time abnormality coefficient based on real-time physiological data and corresponding baseline data, and then combine the real-time abnormality coefficient with the normal offset interval to determine the physiological abnormality rate. The behavior analysis module is used to identify unconscious actions in behavior images and count the real-time occurrence of unconscious actions. The real-time occurrence count is combined with the normal occurrence count for analysis to determine the behavior abnormality rate. The comprehensive analysis module is used to calculate the comprehensive abnormality rate based on the physiological abnormality rate and the behavioral abnormality rate, and to determine the state abnormality signal based on the comprehensive abnormality rate. The terminal warning module is used to identify abnormal status signals, generate a termination message, and display it in real time on the display terminal.
[0006] As a further aspect of the present invention, the method for determining the optimal environmental parameters includes: Historical conversation information is obtained from the associated information. Single consecutive conversation information is marked as individual information. Environmental information and physiological state information are obtained from each individual information. Physiological state data in time period t1 and physiological state data in time period t2 are selected in chronological order. Physiological state data in time period t1 is marked as baseline data and physiological state data in time period t2 is marked as control data. The baseline data and control data in the individual information are processed using the cosine similarity algorithm. If the data similarity value of all physiological state data in the individual information is greater than or equal to the similarity threshold X1, the environmental data in this individual information is marked as suitable data. If the data similarity value of physiological state data in the individual information is less than the similarity threshold X1, the environmental data in the corresponding individual information is marked as unsuitable data. All suitable data is acquired, the mode of each environmental parameter in the suitable data is identified, and the identified mode is marked as the optimal environmental parameter for the current target interviewee. Then, the environmental adjustment module adjusts the interview environment according to the optimal environmental parameter and generates an adjustment completion signal after the environmental adjustment is completed.
[0007] As a further aspect of the present invention, before processing the individual information, the number of individual information n is first counted. If n is less than the minimum sample size, a retrieval signal is generated. If n is greater than or equal to the minimum sample size, an analysis signal is generated. The minimum sample size is a threshold. When a retrieval signal is detected, the basic information of the target interviewee is obtained, keywords are selected from the basic information, and similar reference objects are retrieved from the database according to the keywords. The interview information of the reference objects is marked as individual information, and an analysis signal is generated. When the analysis signal is detected, the individual information is processed again, and the optimal environmental parameters are determined. When selecting keywords from the basic information, the selected keywords are general information of the target interviewee.
[0008] As a further aspect of the present invention, t1 is selected as the time period between 5 and 10 minutes after the start of the conversation, and t2 is selected as the time period between 30 and 35 minutes after the start of the conversation, and the selected physiological state data is data under a stable state. The environmental information includes temperature, humidity, and carbon dioxide concentration, while the physiological data includes heart rate, blood pressure, blood oxygen saturation, and respiratory rate.
[0009] As a further aspect of the present invention, a non-invasive contact sensor is used when collecting physiological data, and an image recognition device is used when collecting behavioral images.
[0010] As a further aspect of the present invention, the method for determining the physiological abnormality rate includes: Real-time physiological data is acquired and divided into unit time segments to obtain unit performance data. The unit performance data is compared with the normal data range. If the unit performance data is within the normal data range, a comparison signal is generated. If the unit performance data is not within the normal data range, a physiological abnormality signal is generated and transmitted to the terminal warning module. The terminal warning module generates a termination of conversation information based on the physiological abnormality signal. When a comparison signal is detected, the baseline data of the target interviewee is obtained, and the mean of the baseline data is calculated. The mean is then marked as the current baseline value of the target interviewee. Next, obtain real-time unit performance data, average the unit performance data to obtain a representative unit value, and then use the formula... The real-time anomaly coefficient Ys is obtained; Then obtain the normal offset interval [a1, a2] for each physiological parameter. The normal offset interval refers to the maximum fluctuation range of the physiological state of the target interviewee within the normal range. Using calculation formula Obtain the individual offset rate PL, acquire the individual offset rate PL of all physiological parameters, and select the maximum value among all individual offset rates PL, and set the maximum value as the physiological abnormality rate YL of the current target interviewee.
[0011] As a further aspect of the present invention, the method for determining the behavioral abnormality rate includes: The behavioral images of the target interviewee are obtained, and the YOLOv8 detection algorithm is used to identify unconscious movements in the behavioral images. Unconscious movements refer to the somatic micro-movements of the target interviewee that are not subject to subjective control under psychological pressure, emotional fluctuations, or subconscious resistance. When unconscious actions are detected in the target interviewee, the type of unconscious action is identified, and the number of times this unconscious action occurs normally within a unit of time is obtained. Next, the number of times the target interviewee's unconscious actions occurred in real time within a unit of time is counted. The number of times the real-time actions occurred in a unit of time is subtracted from the number of normal actions to obtain the difference. The absolute value of the difference is then processed, and the obtained absolute value is marked as the number of characteristic abnormalities. Divide the number of feature anomalies by the unit time to obtain the frequency of action occurrence, with the unit time as the threshold. Obtain the frequency of actions occurring over m consecutive time units, average the frequency of the m actions, and mark the average result as the current abnormal behavior rate YX of the target interviewee, where m is the threshold.
[0012] As a further aspect of the present invention, the method for determining the state anomaly signal includes: Obtain the physiological abnormality rate YL and the behavioral abnormality rate YX, and then use the formula The overall anomaly rate YZ is obtained. and All are weighting coefficients, and ; The overall anomaly rate YZ is compared with the anomaly threshold Yy. If YZ ≥ Yy, an anomaly signal is generated. If YZ < Yy, the target interviewee is monitored.
[0013] A multimodal intelligent early warning method for safe walking conversations includes: Step 1: Collect relevant information about the target interviewee. Relevant information includes basic information and historical conversation information. Basic information refers to the target interviewee's identity information and physical condition. Step 2: Based on the relevant information of the target interviewee, analyze the historical conversation information to obtain the environmental and physiological state information in the historical conversation information. Based on the physiological state data of different time periods, determine the optimal environmental parameters of the target interviewee, adjust the conversation environment according to the optimal environmental parameters, and generate an adjustment completion signal. Step 3: After the adjustment completion signal is detected, the status data of the target interviewee is collected, including real-time physiological data and behavioral images; Step 4: Analyze the real-time physiological data. Based on the real-time physiological data and the corresponding baseline data, calculate the real-time abnormality coefficient. Then, combine the real-time abnormality coefficient with the normal deviation interval to determine the physiological abnormality rate. Step 5: Analyze the behavioral images, identify unconscious actions in the images, and count the real-time occurrences of unconscious actions. Combine the real-time occurrences with the normal occurrences to determine the abnormal behavior rate. Step Six: Calculate the overall abnormality rate based on the physiological abnormality rate and the behavioral abnormality rate, determine the state abnormality signal based on the overall abnormality rate, generate the termination of conversation information based on the state abnormality signal, and display it in real time on the display terminal.
[0014] Compared with existing technologies, the advantages of this invention are: This invention analyzes the historical physiological data and environmental information of the target interviewee to accurately extract their optimal environmental parameters and automatically adjust the interview environment to reduce physiological stress caused by environmental factors and reduce the risk of the interviewee being uncomfortable with the environment. This not only improves the humanization of the interview but also establishes a stable physiological baseline for subsequent status monitoring. By collecting real-time physiological data and behavioral images, the physiological abnormality rate and behavioral abnormality rate are calculated separately, and then a comprehensive abnormality rate is generated through weighted fusion. The multi-dimensional verification mechanism effectively avoids the false alarm and missed alarm problems of single-modal monitoring. At the same time, the risk is transformed into a quantifiable numerical indicator with different warning levels. Relevant personnel can directly match the response measures based on the value of the comprehensive abnormality rate, avoiding the subjectivity of qualitative judgment and greatly improving the efficiency and accuracy of decision-making. Attached Figure Description
[0015] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a schematic diagram of the method flow structure of the present invention. Detailed Implementation
[0016] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0017] Reference Figure 1 and Figure 2 A multimodal intelligent early warning system for safe walking conversations includes an information input module, an environment adjustment module, a status acquisition module, a physiological analysis module, a behavior analysis module, a comprehensive analysis module, and a terminal early warning module. The information input module is used to identify the current interviewee and mark the interviewee as the target interviewee. Then, it collects the relevant information of the target interviewee, including basic information and historical interview information. The basic information refers to the target interviewee's identity information and physiological state, and the historical interview information refers to the target interviewee's historical interview file. Then, a one-way communication connection is established between the information input module and the environmental adjustment module, and the relevant information is transmitted to the environmental adjustment module. It should be further explained that the historical conversation information of the target interviewees was obtained through legal and compliant channels, and all of them strictly followed the confidentiality regulations and case-handling procedures of discipline inspection and supervision work. Moreover, the principle of illegal disclosure and dissemination was not violated in the process of obtaining the information. The environmental adjustment module receives relevant information about the target interviewee, analyzes this information, and determines the optimal environmental parameters for the target interviewee. Specifically, the methods for determining the optimal environmental parameters for the target interviewee include: The historical conversation information in the associated information is obtained, and the single continuous conversation information is marked as individual information. First, the number of individual information n is counted. If n is less than the minimum sample size, a retrieval signal is generated. If n is greater than or equal to the minimum sample size, an analysis signal is generated. The minimum sample size is a threshold, and the specific value of the minimum sample size is set by those skilled in the art based on big data experience. When a retrieval signal is detected, the basic information of the target interviewee is obtained, keywords are selected from the basic information, and similar reference objects are retrieved from the database according to the keywords. The interview information of the reference objects is marked as individual information, and an analysis signal is generated. Furthermore, when selecting keywords in the basic information, the selected keywords are general information for the target interview, such as gender, age, and physiological status. When the analysis signal is detected, the environmental information and physiological state information of each individual are obtained. Physiological state data in time period t1 and physiological state data in time period t2 are selected in chronological order. The physiological state data in time period t1 is marked as the baseline data and the physiological state data in time period t2 is marked as the control data. Furthermore, the specific time intervals of t1 and t2 are set by those skilled in the art based on big data experience. In this embodiment, t1 is selected as the time interval between 5 and 10 minutes after the start of the conversation, and t2 is selected as the time interval between 30 and 35 minutes after the start of the conversation. The selected physiological state data is data under a stable state. The environmental information includes temperature, humidity, and carbon dioxide concentration, while the physiological data includes heart rate, blood pressure, blood oxygen saturation, and respiratory rate. The baseline data and control data in the individual information are processed for similarity. If the data similarity value of all physiological state data in the individual information is greater than or equal to the similarity threshold X1, the environmental data in this individual information is marked as suitable data. If the data similarity value of physiological state data in the individual information is less than the similarity threshold X1, the environmental data in the corresponding individual information is marked as unsuitable data. The specific value of the similarity threshold X1 is obtained by those skilled in the art based on big data calculations, and the cosine similarity algorithm is used when performing similarity processing. Acquire all suitable data, identify the mode of each environmental parameter in the suitable data, mark the identified mode as the most suitable environmental parameter for the current target interviewee, then the environmental adjustment module adjusts the interview environment according to the most suitable environmental parameter, and generates an adjustment completion signal after the environmental adjustment is completed. At the same time, a one-way communication connection is established between the environmental adjustment module and the status acquisition module, and the adjustment completion signal is transmitted to the status acquisition module. The status acquisition module is used to receive the adjustment completion signal. When the adjustment completion signal is detected, the status acquisition module collects the status data of the target interviewee. The status data includes real-time physiological data and behavioral images. Then, the status acquisition module establishes a one-way communication connection with the physiological analysis module and the behavioral analysis module respectively, and transmits the real-time physiological data to the physiological analysis module and the behavioral images to the behavioral analysis module. Furthermore, when collecting physiological data, non-invasive contact sensors are used, such as wrist-worn non-invasive physiological bracelets; when collecting behavioral images, image recognition devices, such as high-definition infrared cameras, are used. The physiological analysis module receives real-time physiological data corresponding to different physiological parameters, analyzes the real-time physiological data, and determines the physiological abnormality rate. Physiological parameters include heart rate, blood pressure, blood oxygen saturation, and respiratory rate. Specific methods for determining the physiological abnormality rate include: Real-time physiological data is acquired and divided into unit time segments to obtain unit performance data. The unit performance data is compared with the normal data range. If the unit performance data falls within the normal data range, a comparison signal is generated. If the unit performance data does not fall within the normal data range, a physiological abnormality signal is generated and transmitted to the terminal warning module. The normal data range is set by those skilled in the art based on big data experience. When a comparison signal is detected, the baseline data of the target interviewee is obtained, and the mean of the baseline data is calculated. The mean is then marked as the current baseline value of the target interviewee. Next, obtain real-time unit performance data, average the unit performance data to obtain a representative unit value, and then use the formula... The real-time anomaly coefficient Ys is obtained; Then, the normal offset interval [a1, a2] of each physiological parameter is obtained. The normal offset interval refers to the maximum fluctuation range of the physiological state of the target interviewee within the normal range. The normal offset interval of each physiological parameter is set by a person skilled in the art based on big data experience. Using calculation formula Obtain the individual offset rate PL, acquire the individual offset rate PL of all physiological parameters, and select the maximum value among all individual offset rates PL, and set the maximum value as the physiological abnormality rate YL of the current target interviewee; Then, a one-way communication connection is established between the physiological analysis module and the comprehensive analysis module, and the physiological abnormality rate is transmitted to the comprehensive analysis module; The behavior analysis module is used to acquire behavioral images of the target interviewee, analyze these images, and determine the behavioral anomaly rate. Specific methods for determining the behavioral anomaly rate include: The process involves acquiring behavioral images of the target interviewee and using artificial intelligence algorithms to identify unconscious movements within these images. Unconscious movements refer to the subtle, uncontrolled bodily movements that the target interviewee makes without conscious control under psychological pressure, emotional fluctuations, or subconscious resistance. Examples include frequently adjusting posture, leaning forward or backward, repeatedly rubbing hands, blinking frequently, rapidly shifting gaze, and fixing one's gaze for extended periods. Furthermore, when using artificial intelligence algorithms to identify unconscious movements, the YOLOv8 detection algorithm is employed. YOLOv8 is an existing technology, and the specific detection and processing procedures will not be elaborated upon here. When unconscious actions are detected in the target interviewee, the type of unconscious action is identified, and the normal occurrence number of this unconscious action per unit time is obtained. The normal occurrence number is obtained by a person skilled in the art based on big data calculations. Next, the number of times the target interviewee's unconscious actions occurred in real time within a unit of time is counted. The number of times the real-time actions occurred in a unit of time is subtracted from the number of normal actions to obtain the difference. The absolute value of the difference is then processed, and the obtained absolute value is marked as the number of characteristic abnormalities. The frequency of action occurrence is obtained by dividing the number of characteristic anomalies by the unit time. The unit time is a threshold. In this embodiment, the unit time is set to 5 minutes. The frequency of actions occurring over m consecutive time units is obtained, and the average of the m frequency of actions is calculated. The average result is then marked as the current abnormal behavior rate YX of the target interviewee, where m is a threshold and the specific value is set by those skilled in the art based on big data experience. Then, a one-way communication connection is established between the behavior analysis module and the comprehensive analysis module, and the behavior anomaly rate YX is transmitted to the comprehensive analysis module; The comprehensive analysis module receives the physiological abnormality rate YL and the behavioral abnormality rate YX, and then uses the calculation formula... The overall anomaly rate YZ is obtained. and All are weighting coefficients, and In this embodiment, and The specific values were obtained by those skilled in the art based on big data calculations. The overall anomaly rate YZ is compared with the anomaly threshold Yy. If YZ≥Yy, an anomaly signal is generated. If YZ<Yy, the target interviewee is monitored. The specific value of the anomaly threshold Yy is obtained by those skilled in the art based on big data calculations. Then, a one-way communication connection is established between the comprehensive analysis module and the terminal early warning module, and the abnormal status signal is transmitted to the terminal early warning module; The terminal early warning module is used to receive physiological abnormality signals and status abnormality signals. When physiological abnormality signals and status abnormality signals are detected, a termination of conversation information is generated and displayed in real time on the display terminal. When the manager receives the termination of conversation information, the conversation process with the target interviewee is immediately terminated.
[0018] A multimodal intelligent early warning method for safe walking conversations, the method specifically includes the following steps: Step 1: Collect relevant information about the target interviewee. Relevant information includes basic information and historical conversation information. Basic information refers to the target interviewee's identity information and physical condition. Step 2: Based on the relevant information of the target interviewee, analyze the historical conversation information to obtain the environmental and physiological state information in the historical conversation information. Based on the physiological state data of different time periods, determine the optimal environmental parameters of the target interviewee, adjust the conversation environment according to the optimal environmental parameters, and generate an adjustment completion signal. Step 3: After the adjustment completion signal is detected, the status data of the target interviewee is collected, including real-time physiological data and behavioral images; Step 4: Analyze the real-time physiological data. Based on the real-time physiological data and the corresponding baseline data, calculate the real-time abnormality coefficient. Then, combine the real-time abnormality coefficient with the normal deviation interval to determine the physiological abnormality rate. Step 5: Analyze the behavioral images, identify unconscious actions in the images, and count the real-time occurrences of unconscious actions. Combine the real-time occurrences with the normal occurrences to determine the abnormal behavior rate. Step Six: Calculate the overall abnormality rate based on the physiological abnormality rate and the behavioral abnormality rate, determine the state abnormality signal based on the overall abnormality rate, generate the termination of conversation information based on the state abnormality signal, and display it in real time on the display terminal.
[0019] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A multimodal intelligent early warning system for safe walking-and-talking conversations, characterized in that, include: The information input module is used to collect relevant information about the target interviewee. The relevant information includes basic information and historical conversation information. The basic information refers to the target interviewee's identity information and physiological status. The environment adjustment module is used to analyze historical conversation information, obtain environmental and physiological state information from the historical conversation information, determine the optimal environmental parameters of the target interviewee based on physiological state data at different time periods, adjust the conversation environment according to the optimal environmental parameters, and generate an adjustment completion signal. The status acquisition module is used to collect the status data of the target interviewee after detecting the adjustment completion signal. The status data includes real-time physiological data and behavioral images. The physiological analysis module is used to calculate the real-time abnormality coefficient based on real-time physiological data and corresponding baseline data, and then combine the real-time abnormality coefficient with the normal offset interval to determine the physiological abnormality rate. The behavior analysis module is used to identify unconscious actions in behavior images and count the real-time occurrence of unconscious actions. The real-time occurrence count is combined with the normal occurrence count for analysis to determine the behavior abnormality rate. The comprehensive analysis module is used to calculate the comprehensive abnormality rate based on the physiological abnormality rate and the behavioral abnormality rate, and to determine the state abnormality signal based on the comprehensive abnormality rate. The terminal warning module is used to identify abnormal status signals, generate a termination message, and display it in real time on the display terminal.
2. The multimodal intelligent early warning system for safe walking-around conversations according to claim 1, characterized in that, Methods for determining optimal environmental parameters include: Historical conversation information is obtained from the associated information. Single consecutive conversation information is marked as individual information. Environmental information and physiological state information are obtained from each individual information. Physiological state data in time period t1 and physiological state data in time period t2 are selected in chronological order. Physiological state data in time period t1 is marked as baseline data and physiological state data in time period t2 is marked as control data. The baseline data and control data in the individual information are processed using the cosine similarity algorithm. If the data similarity value of all physiological state data in the individual information is greater than or equal to the similarity threshold X1, the environmental data in this individual information is marked as suitable data. If the data similarity value of physiological state data in the individual information is less than the similarity threshold X1, the environmental data in the corresponding individual information is marked as unsuitable data. All suitable data is acquired, the mode of each environmental parameter in the suitable data is identified, and the identified mode is marked as the optimal environmental parameter for the current target interviewee. Then, the environmental adjustment module adjusts the interview environment according to the optimal environmental parameter and generates an adjustment completion signal after the environmental adjustment is completed.
3. The multimodal intelligent early warning system for safe walking conversations according to claim 2, characterized in that, Before processing the individual information, the number of individual information n is counted. If n is less than the minimum sample size, a retrieval signal is generated. If n is greater than or equal to the minimum sample size, an analysis signal is generated. The minimum sample size is a threshold. When a retrieval signal is detected, the basic information of the target interviewee is obtained, keywords are selected from the basic information, and similar reference objects are retrieved from the database according to the keywords. The interview information of the reference objects is marked as individual information, and an analysis signal is generated. When the analysis signal is detected, the individual information is processed again, and the optimal environmental parameters are determined. When selecting keywords from the basic information, the selected keywords are general information of the target interviewee.
4. The multimodal intelligent early warning system for safe walking conversations according to claim 2, characterized in that, t1 is the time period between 5 and 10 minutes after the start of the conversation, and t2 is the time period between 30 and 35 minutes after the start of the conversation. The selected physiological data are data under stable conditions. The environmental information includes temperature, humidity, and carbon dioxide concentration, while the physiological data includes heart rate, blood pressure, blood oxygen saturation, and respiratory rate.
5. The multimodal intelligent early warning system for safe walking conversations according to claim 1, characterized in that, Non-invasive contact sensors are used when collecting physiological data, and image recognition equipment is used when collecting behavioral images.
6. The multimodal intelligent early warning system for safe walking-around conversations according to claim 1, characterized in that, Methods for determining the rate of physiological abnormalities include: Real-time physiological data is acquired and divided into unit time segments to obtain unit performance data. The unit performance data is compared with the normal data range. If the unit performance data is within the normal data range, a comparison signal is generated. If the unit performance data is not within the normal data range, a physiological abnormality signal is generated and transmitted to the terminal warning module. The terminal warning module generates a termination of conversation information based on the physiological abnormality signal. When a comparison signal is detected, the baseline data of the target interviewee is obtained, and the mean of the baseline data is calculated. The mean is then marked as the current baseline value of the target interviewee. Next, obtain real-time unit performance data, average the unit performance data to obtain a representative unit value, and then use the formula... The real-time anomaly coefficient Ys is obtained; Then obtain the normal offset interval [a1, a2] for each physiological parameter. The normal offset interval refers to the maximum fluctuation range of the physiological state of the target interviewee within the normal range. Using calculation formula Obtain the individual offset rate PL, acquire the individual offset rate PL of all physiological parameters, and select the maximum value among all individual offset rates PL, and set the maximum value as the physiological abnormality rate YL of the current target interviewee.
7. The multimodal intelligent early warning system for safe walking-around conversations according to claim 1, characterized in that, Methods for determining behavioral abnormality rates include: The behavioral images of the target interviewee are obtained, and the YOLOv8 detection algorithm is used to identify unconscious movements in the behavioral images. Unconscious movements refer to the somatic micro-movements of the target interviewee that are not subject to subjective control under psychological pressure, emotional fluctuations, or subconscious resistance. When unconscious actions are detected in the target interviewee, the type of unconscious action is identified, and the number of times this unconscious action occurs normally within a unit of time is obtained. Next, the number of times the target interviewee's unconscious actions occurred in real time within a unit of time is counted. The number of times the real-time actions occurred in a unit of time is subtracted from the number of normal actions to obtain the difference. The absolute value of the difference is then processed, and the obtained absolute value is marked as the number of characteristic abnormalities. Divide the number of feature anomalies by the unit time to obtain the frequency of action occurrence, with the unit time as the threshold. Obtain the frequency of actions occurring over m consecutive time units, average the frequency of the m actions, and mark the average result as the current abnormal behavior rate YX of the target interviewee, where m is the threshold.
8. The multimodal intelligent early warning system for safe walking conversations according to claim 1, characterized in that, Methods for determining abnormal status signals include: Obtain the physiological abnormality rate YL and the behavioral abnormality rate YX, and then use the formula The overall anomaly rate YZ is obtained. and All are weighting coefficients, and ; The overall anomaly rate YZ is compared with the anomaly threshold Yy. If YZ ≥ Yy, an anomaly signal is generated. If YZ < Yy, the target interviewee is monitored.
9. A multimodal intelligent early warning method for safe walking-around conversations, wherein the method is applied to a multimodal intelligent early warning system for safe walking-around conversations as described in any one of claims 1-8, characterized in that, include: Step 1: Collect relevant information about the target interviewee. Relevant information includes basic information and historical conversation information. Basic information refers to the target interviewee's identity information and physical condition. Step 2: Based on the relevant information of the target interviewee, analyze the historical conversation information to obtain the environmental and physiological state information in the historical conversation information. Based on the physiological state data of different time periods, determine the optimal environmental parameters of the target interviewee, adjust the conversation environment according to the optimal environmental parameters, and generate an adjustment completion signal. Step 3: After the adjustment completion signal is detected, the status data of the target interviewee is collected, including real-time physiological data and behavioral images; Step 4: Analyze the real-time physiological data. Based on the real-time physiological data and the corresponding baseline data, calculate the real-time abnormality coefficient. Then, combine the real-time abnormality coefficient with the normal deviation interval to determine the physiological abnormality rate. Step 5: Analyze the behavioral images, identify unconscious actions in the images, and count the real-time occurrences of unconscious actions. Combine the real-time occurrences with the normal occurrences to determine the abnormal behavior rate. Step Six: Calculate the overall abnormality rate based on the physiological abnormality rate and the behavioral abnormality rate, determine the state abnormality signal based on the overall abnormality rate, generate the termination of conversation information based on the state abnormality signal, and display it in real time on the display terminal.