System and method for supporting command decision-making based on commander emotional state
The system addresses the limitations of conventional decision support systems by analyzing a commander's emotional state through voice, facial expressions, and biosignals to dynamically adjust information complexity and format, improving decision-making speed and accuracy.
Patent Information
- Application Number
- KR1020260089470
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2026-05-18
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2046-05-18
AI Technical Summary
Conventional command decision support systems fail to consider a commander's emotional and physiological state, leading to decreased decision-making speed and accuracy due to information overload or reduced efficiency in high-stress situations.
A system that analyzes a commander's voice, facial expressions, and biosignals in real-time to determine their emotional state, dynamically adjusting the complexity, amount, and format of information provided based on the emotional state.
Improves decision-making efficiency and accuracy by providing customized countermeasures tailored to the commander's emotional state, reducing cognitive burden and enhancing operational response in stressful conditions.
Smart Images

Figure 112026059903615-PAT00001_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to command and control and artificial intelligence-based decision support technology, and more specifically, to a commander emotion-based command decision support system and method that analyzes a commander's voice, facial expressions, heart rate, and other biosignals in real time to determine the emotional state, and provides customized decision support information in which the type, complexity, and method of provision of measures are adjusted according to the determined emotional state. Background Technology
[0003] In fields requiring real-time situational awareness, such as military operations, disaster response, and national security, technologies based on various sensors and information systems are continuously advancing to support commanders' decision-making. In particular, driven by recent advancements in artificial intelligence, big data analysis, and situational awareness technologies, command decision support systems capable of rapidly analyzing battlefield situations and proposing multiple response strategies are being actively researched.
[0004] In general, rapid and accurate judgment by a commander is critical in environments with high time pressure and rapid situational changes, such as military operations, disaster response, and emergency control situations. However, in actual operational environments, commanders may experience increased cognitive load due to a continuous influx of information, battlefield stress, accumulated fatigue, and emotional tension, which can lead to a decrease in the speed and accuracy of decision-making.
[0005] Conventional command decision support systems focus on generating or recommending multiple candidate measures based on battlefield situation information, enemy and friendly deployment information, rules of engagement, and past operational data. However, most of these conventional technologies have the problem of failing to reflect the commander's current emotional or physiological state. In other words, even in the same battlefield situation, the appropriate method of providing information or presenting measures may vary depending on the commander's stress level, fatigue, or overload status; yet, existing systems have the limitation of providing only uniform and fixed forms of information.
[0006] For example, when a commander is in a stable state, it may be effective to provide multiple alternative measures and the results of a comparative analysis together; however, when a commander is under high stress or cognitive overload, it may be more appropriate to provide a single measure with higher priority or concise command information rather than complex comparative information. Nevertheless, conventional decision support systems fail to consider the commander's emotional and physiological state, which may instead cause information overload due to the provision of excessive information or reduce decision-making efficiency.
[0007] Furthermore, conventional emotion recognition technology primarily focuses on the technology itself for analyzing a user's emotional state using voice, facial expressions, or biosignals, while the technology for organically linking the analyzed emotional state with the logic for generating, recommending, and providing actual command decision-making strategies is currently insufficient.
[0008] Therefore, there is a need for a new type of command decision support technology that can analyze the commander's emotional state in real time using their voice, facial expressions, and biosignals, and dynamically adjust the complexity, amount of information, delivery format, and priority of measures according to the analyzed emotional state. The problem to be solved
[0010] The problem to be solved by the present invention is a commander emotion-based command decision support system and method that can improve the speed and accuracy of command decision-making by analyzing the commander's emotional state in real time based on voice, video, and biosignals, and dynamically adjusting the amount of information, the complexity of measures, and the format of provision according to the analyzed emotional state.
[0011] However, the problems that the present invention aims to solve are not limited to those described above, and other problems may exist. means of solving the problem
[0013] A method for supporting command decision-making based on the emotional state of a commander according to a first aspect of the present invention for solving the aforementioned problem comprises: collecting state data including at least one of the commander's voice, facial image, and biosignal; performing preprocessing and feature extraction on the collected state data; analyzing the commander's emotional state by fusing the extracted feature information; generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information; and outputting the generated countermeasure information to the commander.
[0014] In one embodiment of the present invention, the step of analyzing the emotional state of a commander by fusing the extracted feature information may include: a step of generating an integrated feature vector by fusing a plurality of feature information among voice features, image features, and biosignal features; and a step of classifying the emotional state of the commander into at least one state among stable, attention, stress, and overload based on the integrated feature vector.
[0015] In one embodiment of the present invention, the step of performing preprocessing and feature extraction on the collected state data may include: a step of extracting at least one of pitch, speech rate, intonation change, energy, and MFCC (Mel Frequency Cepstral Coefficient) from the voice; a step of extracting at least one of facial landmarks, expression change, gaze direction, and eye blink frequency from the face image; and a step of extracting at least one of heart rate, heart rate variability, skin conductivity, and breathing pattern from the biosignal.
[0016] In one embodiment of the present invention, the step of generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information may include: generating comparative countermeasure information including a plurality of candidate countermeasures and the advantages and disadvantages, expected effects, and risk factors of each candidate countermeasure when the emotional state is in a stable state; and generating a single countermeasure with high priority or a summarized command countermeasure when the emotional state is in a stressful state or an overloaded state.
[0017] In one embodiment of the present invention, the summarized command measures may include at least one of current position defense, reinforcement of reserve forces, intensive surveillance of reconnaissance assets, and emergency response instructions.
[0018] In one embodiment of the present invention, the step of generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information may include the step of adjusting at least one of the amount of information, number of alternatives, level of detailed explanation, inclusion of comparison information, and presentation order of the countermeasures according to the emotional state of the commander.
[0019] In one embodiment of the present invention, the step of generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information may generate countermeasure information including a recommendation to delegate at least one of detailed tactical analysis, surveillance mission, and supply judgment to a subordinate unit, deputy commander, or auxiliary system when the emotional state is determined to be in an overloaded state or a fatigued state.
[0020] In one embodiment of the present invention, the step of generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information may generate countermeasure information by referring to at least one of a previously stored doctrine database, tactical rule database, mission guideline database, or policy database, or generate countermeasure information using at least one of a rule-based inference engine, machine learning model, deep learning model, time series model, and large-scale language model.
[0021] In one embodiment of the present invention, the step of outputting the generated countermeasure information to a commander may provide the countermeasure information in at least one of voice output, screen display, map overlay display, alarm sound output, vibration output, and augmented reality-based visual output.
[0022] In one embodiment of the present invention, the step of performing preprocessing and feature extraction on the collected state data may perform at least one of noise removal, filtering, normalization, missing data correction, and time axis synchronization.
[0023] One embodiment of the present invention may further include the step of collecting a commander's reaction information, selection result, state change, or biosignal change after outputting the generated strategy information; and the step of updating the accuracy of subsequent emotional state analysis or strategy generation using the collected information.
[0024] In one embodiment of the present invention, the step of outputting the generated countermeasure information to a commander may be provided in the form of summary information including at least one of an enemy threat level, the status of friendly major assets, priority response areas, and high-urgency command items.
[0025] Additionally, a system for supporting command decision-making based on the emotional state of a commander according to a second aspect of the present invention comprises: a commander terminal that collects state data including at least one of the commander's voice, facial image, and biosignal; a data collection unit that performs preprocessing and feature extraction on the collected state data; an emotion analysis engine that analyzes the commander's emotional state by fusing the extracted feature information; a strategy generation unit that generates one or more strategy information based on the analyzed emotional state and battlefield situation information; and an output unit that outputs the generated strategy information to the commander.
[0026] In one embodiment of the present invention, the commander terminal may include: a voice collection module for collecting the commander's spoken voice; an image collection module for capturing the commander's face image or facial expression image; and a biosignal collection module for measuring at least one biosignal among the commander's heart rate, heart rate variability, skin conductivity, respiratory rate, and body temperature.
[0027] In one embodiment of the present invention, the data collection unit may include a synchronization module that aligns the time axes of different types of state data; a preprocessing module that performs at least one of noise removal, filtering, normalization, and missing data correction; and a feature extraction module that extracts feature information for sentiment analysis from the state data.
[0028] In one embodiment of the present invention, the emotion analysis engine may include: a multi-signal fusion module that fuses multiple feature information among voice features, image features, and biosignal features to generate an integrated feature vector; and an emotion state classifier that classifies the commander's emotional state into at least one of stable, attention, stress, and overload based on the integrated feature vector.
[0029] In one embodiment of the present invention, the emotional state classifier may include at least one of a rule-based algorithm, a machine learning model, a deep learning model, a time series model, and a multimodal neural network model.
[0030] In one embodiment of the present invention, the strategy generation unit includes a strategy generation module that generates or selects one or more candidate strategies based on the sentiment state and battlefield situation information, and the strategy generation module can adjust at least one of the information amount, complexity, level of detailed explanation, inclusion of comparison information, and presentation priority of the strategy according to the sentiment state.
[0031] In one embodiment of the present invention, the countermeasure generation module may first generate simple command-centered countermeasure information that can be executed immediately when the emotional state is a stress state or an overload state.
[0032] In one embodiment of the present invention, the simple command-centered countermeasure information may include at least one of current location defense, reinforcement of reserve forces, intensive surveillance of reconnaissance assets, and emergency response instructions.
[0033] In one embodiment of the present invention, the countermeasure generation module may generate countermeasure information including a recommendation to delegate at least one of detailed tactical analysis, surveillance mission, and supply judgment to a subordinate unit, deputy commander, or auxiliary system when the emotional state is determined to be an overloaded state or a fatigued state.
[0034] In one embodiment of the present invention, the strategy generation module may generate strategy information by referring to at least one of a previously stored doctrine database, tactical rule database, mission guideline database, and policy database, or by using at least one of a rule-based inference engine, machine learning model, deep learning model, time series model, and large-scale language model.
[0035] In one embodiment of the present invention, the output unit includes at least one of a voice output module, a screen display module, and an alarm / notification module, and can provide the countermeasure information in at least one of voice output, screen display, map overlay display, alarm sound output, vibration output, and augmented reality-based visual output.
[0036] In one embodiment of the present invention, the output unit may provide countermeasure information in the form of summary information including at least one of an enemy threat level, the status of friendly major assets, priority response areas, and high-urgency command items.
[0037] One embodiment of the present invention may further include a feedback processing unit that collects a commander's response information, selection result, state change, or biosignal change regarding the countermeasure information provided through the output unit, and updates the accuracy of subsequent emotional state analysis or countermeasure generation using the collected information.
[0038] A computer program according to another aspect of the present invention for solving the above-mentioned problem is combined with a computer, which is hardware, to execute the commander sentiment-based command decision support system and method, and is stored in a computer-readable recording medium.
[0039] Other specific details of the present invention are included in the detailed description and drawings. Effects of the invention
[0041] According to one embodiment of the present invention described above, by analyzing a commander's voice, facial image, and biosignals in real time to determine their emotional state and providing countermeasure information in a form suitable thereto, the efficiency of decision-making and the accuracy of response in battlefield situations can be improved. In particular, since it is possible to provide customized countermeasures that consider both the commander's current state and the battlefield situation, it is possible to support more rapid and stable command execution even in a state of stress or fatigue.
[0042] In addition, since the present invention comprehensively analyzes voice information, facial expression information, and biosignal information, it can more accurately determine the commander's emotional state.
[0043] In addition, the present invention can dynamically adjust the amount of information, complexity, and presentation method of the plan according to the emotional state, thereby reducing the provision of unnecessary information and decreasing the cognitive burden on the commander during stressful or overloaded situations. Accordingly, the commander can receive simple and clear plans focused on key information first, even in urgent situations, thereby improving the speed and accuracy of decision-making.
[0044] Furthermore, since the present invention generates strategies by combining emotional state analysis results with battlefield situation information, it enables decision support that goes beyond simple emotion recognition and can be practically utilized in actual operational execution. In addition, when the commander's reaction information and state changes are utilized as feedback, the accuracy and personalization of strategy generation can be further enhanced.
[0045] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing
[0047] The drawings attached below are intended to aid in understanding the embodiments thereof and provide embodiments together with a detailed description. However, the technical features of the embodiments thereof are not limited to specific drawings, and the features disclosed in each drawing may be combined with one another to form new embodiments. FIG. 1 is a block diagram of a commander sentiment-based command decision support system according to one embodiment of the present invention. FIG. 2 is a flowchart of a commander sentiment-based command decision-making method according to one embodiment of the present invention. Specific details for implementing the invention
[0048] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.
[0049] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.
[0050] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.
[0051] Hereinafter, a commander sentiment-based command decision support system (1) according to an embodiment of the present invention will be described with reference to FIG. 1.
[0052] FIG. 1 is a block diagram of a commander sentiment-based command decision support system (1) according to one embodiment of the present invention.
[0053] A commander sentiment-based command decision support system (1) according to one embodiment of the present invention comprises a commander terminal (100) for acquiring state data of a commander, a data collection unit (200) for collecting and synchronizing the state data and performing preprocessing and feature extraction, a sentiment analysis engine (300) for analyzing the commander's sentiment state by fusing the extracted feature information, a strategy generation unit (400) for generating strategy information based on the analyzed sentiment state and battlefield situation information, an output unit (500) for providing the generated strategy information to the commander, and a feedback processing unit (600) for updating system performance using subsequent response information.
[0054] A commander emotion-based command decision support system (1) according to one embodiment of the present invention can analyze the commander's current cognitive state and emotional state in real time by utilizing the commander's voice, facial image, and biosignals in combination, and can dynamically generate and provide strategy information in a form suitable for the current situation based on the analyzed results. In particular, one embodiment of the present invention can reduce information overload and improve the speed and accuracy of decision-making by adjusting the amount of information, complexity, presentation priority, and expression method of the strategy according to the emotional state.
[0055] In addition, one embodiment of the present invention goes beyond simple emotion analysis and can generate customized strategies suitable for actual operational execution by combining battlefield situation information and emotional state information. For example, when the commander is in a stable state, multiple candidate strategies and comparative analysis information of each strategy can be provided, and when the commander is in a stressed or overloaded state, simple command-centered strategies that can be executed immediately or minimized strategies focused on key information can be provided first.
[0056] Specifically, the commander terminal (100) acquires status data related to the commander's status. The commander terminal (100) can be implemented as a helmet, headset, wearable device, smart glasses, vehicle interior mounting device, or a combination thereof, and can be modified into various forms depending on the operational environment.
[0057] In one embodiment, the commander terminal (100) may include a plurality of sensors for acquiring at least one of the commander's voice, facial image, and biosignal. For example, the commander terminal (100) may include a voice collection module (110) for collecting the commander's spoken voice, an image collection module (120) for capturing the commander's facial image or facial expression image, and a biosignal collection module (130) for measuring at least one of the commander's heart rate, heart rate variability, skin conductivity, respiratory rate, and body temperature.
[0058] Here, the voice collection module (110) can acquire a voice signal of a commander using a microphone or an acoustic sensor, and can generate state data including information related to volume, speech rate, pitch change and intonation change from the acquired voice signal.
[0059] Additionally, the image acquisition module (120) can acquire a face image or an expression image using a camera or an image sensor, and can generate state data including information related to changes in expression, gaze direction, blinking frequency and facial muscle movements from the face image.
[0060] Additionally, the biosignal collection module (130) can measure biosignals such as heart rate, heart rate variability, skin conductivity, respiratory rate, and body temperature using a wearable sensor or a sensor inside the vehicle. The biosignal collection module (130) can obtain physiological indicators related to stress or fatigue, such as an increase in heart rate, a decrease in heart rate variability, and an increase in skin conductivity response.
[0061] In one embodiment, the commander terminal (100) can transmit the acquired status data to the data collection unit (200) via wired or wireless communication.
[0062] Additionally, the commander terminal (100) may be configured to perform some functions using only biosignals without voice data or video data depending on the situation, and the biosignals may additionally include at least one of body temperature, respiration, blood pressure, electromyography, and brainwaves in addition to heart rate.
[0063] Next, the data collection unit (200) collects status data input from the commander terminal (100) and sorts and processes it into a form suitable for sentiment analysis.
[0064] In one embodiment, the data collection unit (200) may include a synchronization module (210) that aligns the time axes of different types of state data, a preprocessing module (220) that performs at least one of noise removal, filtering, normalization, and missing data correction, and a feature extraction module (230) that extracts feature information for sentiment analysis from the state data.
[0065] Here, the synchronization module (210) can align voice, video, and biosignals based on a common time standard when they have different sampling rates and collection cycles. This allows for the linked analysis of state data collected from different sensors based on the same time point.
[0066] Additionally, the preprocessing module (220) can perform various preprocessing on the state data to improve analysis accuracy. For example, it can perform noise removal on voice signals, image frame correction, biosignal smoothing, outlier removal, missing data processing, and normalization.
[0067] Additionally, the feature extraction module (230) can extract feature information effective for determining an emotional state from state data. For example, from a voice signal, pitch, energy, speech rate, Mel Frequency Cepstral Coefficient (MFCC), and voice stability index can be extracted; from a video signal, facial landmarks, expression vectors, changes in gaze, and changes in muscles around the eyes / mouth can be extracted; and from a biosignal, average heart rate, heart rate variability, changes in skin conductivity, and breathing patterns can be calculated as feature values.
[0068] The feature information extracted by the above feature extraction module (230) can be transmitted to the sentiment analysis engine (300).
[0069] Next, the sentiment analysis engine (300) analyzes and classifies the commander's emotional state based on feature information transmitted from the data collection unit (200).
[0070] In one embodiment, the emotion analysis engine (300) may include a multi-signal fusion module (310) that fuses multiple feature information among voice features, image features and biosignal features to generate an integrated feature vector, and an emotion state classifier (320) that classifies the commander's emotional state into at least one of stable, attention, stress and overload based on the integrated feature vector.
[0071] Here, the multi-signal fusion module (310) can integrate voice features, image features, and biosignal features to generate a single integrated feature vector. At this time, the fusion method may be feature-level fusion, score-level fusion, decision-level fusion, or a combination thereof.
[0072] Additionally, the emotional state classifier (320) can determine the emotional state of the commander by receiving an integrated feature vector as input. The emotional state can be classified into stable, attentive, stressful, and overloaded states, but is not limited thereto and can also be expressed in the form of subdivided levels or continuous scores.
[0073] Additionally, the emotional state classifier (320) can be implemented using at least one of a rule-based algorithm, a machine learning model, a deep learning model, a time series model, and a multimodal neural network model.
[0074] For example, if an increase in average heart rate, a decrease in heart rate variability, an increase in tense voice pitch, and stiff facial expression features are detected simultaneously, the emotional state classifier (320) can determine this as a stress state. Additionally, if multiple physiological and behavioral indicators exceed a threshold value and behavioral patterns with a heavy information processing burden are detected together, it can be determined as an overload state.
[0075] The analysis results of the sentiment analysis engine (300) can be transmitted to the strategy generation unit (400).
[0076] The strategy generation unit (400) generates strategy information to be provided to the commander by comprehensively considering the commander's emotional state information received from the emotional analysis engine (300) and battlefield situation information, mission information, threat level, available assets, time constraints, communication status, and tactical priority input through the situation information input unit (410). At this time, the strategy generation unit (400) can generate customized strategy information suitable for the commander's emotional state even in the same battlefield situation by reflecting both the situation change and the commander's state together.
[0077] In one embodiment, the strategy generation unit (400) may include, for example, a situation information input unit (410) that receives battlefield situation information, and a strategy generation module (420) that generates or selects one or more candidate strategies based on the sentiment state information and situation information. The situation information input unit (410) may receive information regarding enemy and friendly forces locations, threat levels, mission objectives, available assets, time constraints, communication status, and tactical priorities.
[0078] In one embodiment, the strategy generation module (420) can generate strategy information suitable for a commander by synthesizing emotional state information and situational information, referring to a previously stored doctrine database, tactical rule database, mission guideline database, or policy database, or by using a rule-based reasoning engine, machine learning model, deep learning model, time series model, artificial intelligence model, or large-scale language model. Additionally, the strategy generation module (420) can selectively provide the strategy most suitable for the current situation and emotional state among a plurality of candidate strategies by prioritizing them.
[0079] In one embodiment, the strategy generation module (420) can adjust at least one of the content, amount of information, expression method, number of alternatives, level of detailed explanation, inclusion of comparative information, and presentation priority of the strategy according to the commander's emotional state. To this end, the strategy generation module (420) can perform at least one of a simple command suggestion function, an information summary function, a complexity adjustment function, and a delegation recommendation function.
[0080] For example, the countermeasure generation module (420) can generate countermeasure information in a summarized form by selecting only key information from situational information and countermeasure-related information to reduce the cognitive burden on the commander. At this time, the summarized information may be composed mainly of enemy threat levels, the status of key friendly assets, priority response areas, and high command items with an urgency level greater than a preset value.
[0081] Additionally, the strategy generation module (420) can adjust the complexity of the strategy according to the emotional state. For example, if the commander's emotional state is classified as stable, comparative or detailed strategy information including multiple candidate strategies, the advantages and disadvantages of each strategy, expected effects, risk factors, and required assets can be generated. On the other hand, if the commander's emotional state is classified as cautionary, only some strategies with a priority above a certain standard can be selected, and strategy information summarized with key information can be generated.
[0082] Additionally, if the commander's emotional state is determined to be a stressful state or classified as stressful, the above-mentioned strategy generation module (420) may prioritize generating simple, command-centered strategy information that can be executed immediately, instead of complex comparative information. For example, it may prioritize providing short and clear command-type strategies such as "maintain current position defensive posture," "reinforce the right flank with reserves," "intensive surveillance of the northern area with reconnaissance assets," or "execute emergency response instructions." Furthermore, if a stressful state is determined from the commander's voice tone and biosignals during a fierce battle, it may prioritize generating simple and clear strategies such as "maintain current position defensive posture for all units."
[0083] Meanwhile, if the commander's emotional state is determined to be overloaded or highly fatigued, or if it is classified as overloaded, the countermeasure generation module (420) can generate minimized countermeasure information that includes only essential commands and key warnings. Here, a highly fatigued state means a state of fatigue that is determined to be above a certain level according to a predetermined fatigue standard. Additionally, countermeasure information can be generated that includes recommendations to delegate some detailed judgments or auxiliary missions to subordinate units, deputy commanders, or auxiliary systems. For example, it may recommend delegating detailed tactical analysis to deputy commanders, delegating subsequent surveillance missions to reconnaissance units, and delegating supply redeployment judgments to support control units.
[0084] That is, the strategy generation unit (400) can generate customized strategy information that reflects at least one of simple command suggestions, information summaries, complexity adjustments, and delegation recommendations by taking into account the commander's emotional state and battlefield situation together.
[0085] Next, the output unit (500) transmits the wall information generated by the wall generation unit (400) to the commander.
[0086] In one embodiment, the output unit (500) may include at least one of a voice output module (510) that provides voice synthesis or recorded guidance messages, a screen display module (520) that visually provides barrier information through a display, and an alarm / notification module (530) that provides an alarm such as an alarm sound, flashing, or vibration.
[0087] Here, the voice output module (510) can output key commands in voice form in situations where it is difficult for the commander to watch the screen. The screen display module (520) can provide countermeasure information in various visual forms, including text, icons, map overlays, priority markings, and color warnings. Additionally, an augmented reality-based visual output method may be applied.
[0088] Additionally, the alarm / notification module (530) can immediately draw attention in an overloaded or emergency threat situation in a manner such as an alarm sound, flashing, and vibration.
[0089] Meanwhile, the feedback processing unit (600) can collect the commander's reaction information, selection results, state changes, or biosignal changes regarding the countermeasure information provided through the output unit (500). The feedback processing unit (600) can use the collected information to update the accuracy of subsequent emotional state analysis or countermeasure generation, thereby improving the adaptability and reliability of the system.
[0090] According to one embodiment of the present invention, when a tense pitch change and rapid speech rate are detected from the commander's voice in a situation where combat is intensified, an increase in heart rate and a decrease in heart rate variability are detected from the biosignal, and a rigid facial expression change is detected from the facial image, the emotion analysis engine (300) can determine the commander's state as a stress state. At this time, the emotion analysis engine (300) can classify the state as a stress state when multiple stress-related indicators are detected simultaneously by integrally analyzing voice features, image features, and biosignal features.
[0091] As described above, if the commander's emotional state is analyzed as stress, the strategy generation unit (420) may prioritize generating strategy information centered on simple commands that can be executed immediately, rather than listing multiple complex options. For example, it may prioritize providing short and clear command-type strategies such as "Maintain defensive posture at current positions for all units," "Concentrate surveillance of the right flank with reconnaissance assets," or "Reinforce the right flank with reserves." Additionally, to reduce the cognitive burden on the commander, the strategy generation unit (420) may reduce unnecessary comparative information or detailed explanations and organize strategy information primarily around core commands and emergency response information.
[0092] Additionally, if the commander's condition is analyzed as being overloaded or highly fatigued, the countermeasure generation unit (420) may generate minimized countermeasure information containing only essential commands and key warnings, and at the same time may include recommendations to delegate some detailed tactical judgments or incidental missions to subordinate units, deputy commanders, or rear control systems. For example, it may suggest delegating detailed tactical comparative analysis to deputy commanders and delegating subsequent surveillance missions to reconnaissance units.
[0093] Conversely, if the commander is analyzed to be in a stable state, the countermeasure generation unit (420) may present multiple candidate countermeasures together and provide comparative countermeasure information including the expected effect, risk level, required assets, response time, and advantages and disadvantages of each countermeasure. Through this, the commander can make more comprehensive decisions based on sufficient situation analysis information.
[0094] Meanwhile, the present invention is not limited to the above embodiments and can be implemented in various modified forms. For example, the classification step of emotional state is not limited to the four stages of stable, attention, stress, and overload, but can be implemented in three or five or more detailed stages. In addition, the emotional state may be calculated not only as a categorical state value but also in the form of a continuous score or a risk index.
[0095] In addition, the biosignals may additionally include at least one of body temperature, respiration, blood pressure, electromyography, and brain waves, in addition to heart rate and heart rate variability. In some embodiments, the system may be configured to analyze the emotional state using only biosignals without voice or video data. Conversely, in certain environments, it is also possible to implement the system to determine the emotional state using only voice and facial images without biosignals.
[0096] The method of analyzing the emotional state can also be varied in many ways. For example, the emotional analysis engine (300) can be implemented using a rule-based algorithm, a machine learning model, a deep learning model, a time series model, a multimodal neural network model, or a combination thereof. Additionally, the feature fusion method can be implemented as feature-level fusion, score-level fusion, decision-level fusion, or a combination thereof.
[0097] The strategy generation method may also be implemented using a rule-based database, a tactical doctrine database, a policy-based inference engine, a machine learning model, a deep learning model, or a large-scale language model, either individually or in combination. For example, in some embodiments, it may be configured to improve the accuracy of strategy recommendations by sentiment state by learning past operational history and commander response data.
[0098] The output method can also be modified into various forms. For example, the output unit (500) can provide countermeasure information in various ways, such as not only text-based screen display but also voice synthesis-based guidance, map overlay display, color-based warning display, augmented reality-based visual output, and vibration or tactile-based alarm. Additionally, the output method may be configured to change dynamically depending on the battlefield environment or the commander's status.
[0099] Hereinafter, with reference to FIG. 2, a method performed by a commander sentiment-based command decision support system (1) according to an embodiment of the present invention will be described.
[0100] FIG. 2 is a flowchart of a commander sentiment-based command decision-making method according to one embodiment of the present invention.
[0101] First, the commander terminal (100) collects status data including at least one of the commander's voice, facial image, and biosignal (S110). At this time, the status data may be acquired in real time through a microphone, camera, wearable sensor, in-vehicle sensor, or other sensor, and may include voice data, facial image data, and biosignal data such as heart rate, heart rate variability, and skin conductivity.
[0102] Next, the data collection unit (200) can perform preprocessing and feature extraction on the collected state data (S120). For example, the data collection unit (200) can perform preprocessing such as noise removal, filtering, normalization, missing data correction, and time axis synchronization, and can extract features such as pitch, speech rate, and intonation changes from voice data, extract facial landmarks, expression changes, and gaze direction from image data, and extract feature information such as heart rate, heart rate variability, skin conductivity, and breathing patterns from biosignal data.
[0103] Afterward, the emotion analysis engine (300) fuses the extracted feature information to analyze the commander's emotional state (S130). For example, the emotion analysis engine (300) can generate an integrated feature vector by integrating voice features, image features, and biosignal features, and based on this, can classify the commander's emotional state into at least one of stable, attention, stress, and overload.
[0104] Next, the strategy generation unit (400) generates one or more strategy information based on the analyzed emotional state and battlefield situation information (S140). At this time, the strategy generation unit (400) may consider situation information such as enemy and friendly forces locations, threat levels, mission objectives, available assets, and time constraints, and may adjust the amount of information, complexity, level of detailed explanation, and presentation priority of the strategy according to the emotional state. For example, in a stressful state, simple command-centered strategy information may be generated, and in a stable state, detailed strategy information including multiple candidate strategies and comparison information may be generated.
[0105] Finally, the output unit (500) outputs the generated defense information to the commander (S150). For example, the output unit (500) may provide the defense information in at least one of voice output, screen display, map overlay, alarm sound, vibration, or augmented reality-based visual output, and may also provide it in a summary form focusing on key information as needed.
[0106] Meanwhile, in the above description, steps S110 to S150 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Also, some steps may be omitted as necessary, and the order between steps may be changed. Furthermore, even if other omitted details are included, the details described in FIGS. 2 and 3 and the details described in FIG. 4 may be mutually applicable.
[0107] The commander sentiment-based command decision support system and method according to one embodiment of the present invention described above may be implemented as a program (or application) and stored on a medium to be executed in combination with a computer, which is hardware.
[0108] The aforementioned program may include code encoded in computer languages such as C, C++, JAVA, Ruby, and machine language, which can be read by the computer's processor (CPU) through the computer's device interface, in order for the computer to read the program and execute the methods implemented in the program. Such code may include functional code related to functions that define the necessary functions for executing the methods, and may include control code related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. Additionally, such code may further include memory reference code regarding where (address) additional information or media necessary for the computer's processor to execute the functions should be referenced in the computer's internal or external memory. In addition, if the processor of the computer needs to communicate with any other computer or server located remotely in order to execute the above functions, the code may further include communication-related code regarding how to communicate with any other computer or server located remotely using the communication module of the computer, and what information or media to transmit or receive during communication.
[0109] The above-mentioned storage medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, examples of the above-mentioned storage medium include, but are not limited to, ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. That is, the above-mentioned program may be stored on various recording media on various servers that the computer can access, or on various recording media on the user's computer. Additionally, the above-mentioned medium may be distributed across networked computer systems, and computer-readable code may be stored in a distributed manner.
[0110] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.
[0111] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols
[0112] 1: Commander Sentiment-Based Command Decision Support System 100: Commander Terminal 200: Data Collection Department 300: Sentiment Analysis Engine 400: Barrier generation section 500: Output section 600: Feedback processing unit
Claims
Claim 1 A method for supporting command decision-making based on the emotional state of a commander, comprising: collecting state data including at least one of the commander's voice, facial image, and biosignal; performing preprocessing and feature extraction on the collected state data; fusing the extracted feature information to analyze the commander's emotional state; generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information; and outputting the generated countermeasure information to the commander, wherein the step of fusing the extracted feature information to analyze the commander's emotional state comprises: fusing a plurality of feature information among voice features, image features, and biosignal features to generate an integrated feature vector; and classifying the commander's emotional state into at least one of stable, alert, stress, and overload based on the integrated feature vector. Claim 2 delete Claim 3 A commander emotion-based command decision support method according to claim 1, wherein the step of performing preprocessing and feature extraction on the collected state data comprises: a step of extracting at least one of pitch, speech rate, intonation change, energy, and MFCC (Mel Frequency Cepstral Coefficient) from the voice; a step of extracting at least one of facial landmarks, expression change, gaze direction, and blink frequency from the face image; and a step of extracting at least one of heart rate, heart rate variability, skin conductivity, and breathing pattern from the biosignal. Claim 4 A method for supporting command decision-making based on commander sentiment according to claim 1, wherein the step of generating one or more countermeasure information based on the analyzed sentiment state and battlefield situation information comprises: generating comparative countermeasure information including a plurality of candidate countermeasures and the advantages and disadvantages, expected effects, and risk factors of each candidate countermeasure when the sentiment state is a stable state; and generating a single countermeasure with high priority or a summarized command countermeasure when the sentiment state is a stress state or an overload state. Claim 5 A commander sentiment-based command decision support method according to paragraph 4, wherein the above-sum summarized command measures include at least one of current position defense, reserve reinforcement, concentrated surveillance of reconnaissance assets, and emergency response directives. Claim 6 A method for supporting commander's emotional decision-making based on the commander's emotional state, wherein the step of generating one or more countermeasure information based on the analyzed emotional state and battlefield situation information comprises the step of adjusting at least one of the amount of information, number of alternatives, level of detailed explanation, inclusion of comparative information, and presentation order of the countermeasures according to the commander's emotional state. Claim 7 A commander sentiment-based command decision support method according to claim 1, wherein the step of generating one or more countermeasure information based on the analyzed sentiment state and battlefield situation information comprises generating countermeasure information including a recommendation to delegate at least one of detailed tactical analysis, surveillance mission, and supply judgment to a subordinate unit, deputy commander, or auxiliary system when the sentiment state is determined to be in an overload state or a fatigue state. Claim 8 A method for supporting commander sentiment-based command decision-making according to claim 1, wherein the step of generating one or more countermeasure information based on the analyzed sentiment state and battlefield situation information comprises generating countermeasure information by referring to at least one of a previously stored doctrine database, tactical rule database, mission guideline database, or policy database, or generating countermeasure information using at least one of a rule-based inference engine, machine learning model, deep learning model, time series model, and large-scale language model. Claim 9 A commander sentiment-based command decision support method according to claim 1, wherein the step of outputting the generated countermeasure information to the commander is to provide the countermeasure information in at least one of voice output, screen display, map overlay display, alarm sound output, vibration output, and augmented reality-based visual output. Claim 10 A commander sentiment-based command decision support method according to claim 1, wherein the step of performing preprocessing and feature extraction on the collected state data is to perform at least one preprocessing among noise removal, filtering, normalization, missing data correction, and time axis synchronization. Claim 11 A commander emotion-based command decision support method according to claim 1, further comprising: a step of collecting commander reaction information, selection results, state changes, or biosignal changes after outputting the generated countermeasure information; and a step of updating subsequent emotional state analysis or countermeasure generation accuracy using the collected information. Claim 12 A method for supporting commander sentiment-based command decision-making, wherein, in claim 1, the step of outputting the generated countermeasure information to the commander is to provide it in the form of summary information including at least one of the enemy threat level, the status of friendly major assets, priority response areas, and high-urgency command items. Claim 13 A commander emotion-based command decision support system that supports command decision-making based on the emotional state of a commander, comprising: a commander terminal that collects state data including at least one of the commander's voice, facial image, and biosignal; a data collection unit that performs preprocessing and feature extraction on the collected state data; an emotion analysis engine that analyzes the commander's emotional state by fusing the extracted feature information; a strategy generation unit that generates one or more strategy information based on the analyzed emotional state and battlefield situation information; and an output unit that outputs the generated strategy information to the commander, wherein the emotion analysis engine comprises: a multi-signal fusion module that generates an integrated feature vector by fusing a plurality of feature information among voice features, image features, and biosignal features; and an emotion state classifier that classifies the commander's emotional state into at least one of stable, alert, stress, and overload based on the integrated feature vector. Claim 14 In paragraph 13, the commander terminal comprises: a voice collection module for collecting the commander’s spoken voice; an image collection module for capturing the commander’s face image or facial expression image; and a biosignal collection module for measuring at least one biosignal among the commander’s heart rate, heart rate variability, skin conductivity, respiratory rate, and body temperature, in a commander emotion-based command decision support system. Claim 15 In paragraph 13, the data collection unit comprises: a synchronization module that aligns the time axes of different types of state data; a preprocessing module that performs at least one of noise removal, filtering, normalization, and missing data correction; and a feature extraction module that extracts feature information for sentiment analysis from state data, thereby forming a commander sentiment-based command decision support system. Claim 16 delete Claim 17 A commander sentiment-based command decision support system according to claim 13, wherein the sentiment state classifier comprises at least one of a rule-based algorithm, a machine learning model, a deep learning model, a time series model, and a multimodal neural network model. Claim 18 A commander sentiment-based command decision support system according to claim 13, wherein the above-mentioned strategy generation unit includes a strategy generation module that generates or selects one or more candidate strategies based on the sentiment state and battlefield situation information, and the strategy generation module adjusts at least one of the information amount, complexity, level of detailed explanation, inclusion of comparison information, and presentation priority of the strategies according to the sentiment state. Claim 19 A commander emotion-based command decision support system according to claim 18, wherein the above-mentioned strategy generation module prioritizes generating immediately executable simple command-centered strategy information when the above-mentioned emotional state is a stress state or an overload state. Claim 20 A commander sentiment-based command decision support system in paragraph 19, wherein the above simple command-centered countermeasure information includes at least one of current position defense, reserve reinforcement, concentrated surveillance of reconnaissance assets, and emergency response instructions. Claim 21 A commander sentiment-based command decision support system according to claim 18, wherein the above-mentioned strategy generation module generates strategy information including a recommendation to delegate at least one of detailed tactical analysis, surveillance mission, and supply judgment to a subordinate unit, deputy commander, or auxiliary system when the above-mentioned sentiment state is determined to be an overloaded state or a fatigued state. Claim 22 A commander sentiment-based command decision support system, wherein the above-mentioned strategy generation module refers to at least one of a previously stored doctrine database, tactical rule database, mission guideline database, and policy database, or generates strategy information using at least one of a rule-based inference engine, machine learning model, deep learning model, time series model, and large-scale language model. Claim 23 A commander sentiment-based command decision support system according to claim 13, wherein the output unit comprises at least one of a voice output module, a screen display module, and an alarm / notification module, and provides the countermeasure information in at least one of voice output, screen display, map overlay display, alarm sound output, vibration output, and augmented reality-based visual output. Claim 24 A commander sentiment-based command decision support system according to paragraph 13, wherein the output unit provides countermeasure information in the form of summary information including at least one of enemy threat level, status of friendly major assets, priority response area, and high-urgency command item. Claim 25 A commander emotion-based command decision support system according to claim 13, further comprising a feedback processing unit that collects the commander's reaction information, selection result, state change, or biosignal change regarding the countermeasure information provided through the output unit, and updates the accuracy of subsequent emotional state analysis or countermeasure generation using the collected information.
Citation Information
Patent Citations
Method and apparatus for assisting playing with performance using music conducting recognition of conductor
KR1020130064569A
Decision support system for automatically recommending the optimal attack strategy and weapon combination, including the oc-gaws (optimal configuration of ground attack weapon system) and its control method
KR1020250135658A
Device, method and readable media for multimodal recognizing emotion based on artificial intelligence
KR1020190060630A
Integrated controlling apparatus for operating equipment and integrated controlling method same
KR1020240150693A
