VOICE ASSISTANT AND TASK MANAGEMENT SYSTEM

Abstract

The invention relates to AI-based crew-vehicle interface and mission management systems used in the defense industry and military land vehicle technologies, particularly in armored and military land vehicles. Specifically, the invention relates to an integrated voice assistant and mission management system that is configured to operate independently of external networks, especially in armored and military land vehicles; vehicle sensors (10) that collect internal status data and raw environmental data simultaneously; an isolated central embedded processing unit (40) that synthesizes data from the sensors (10) in real-time and converts it into risk assessment and decision support actions; and a natural language rule and bidirectional command bridge (50) that translates linguistic commands from the crew into machine language and verbally transmits the risk assessment and notifications generated by the central embedded processing unit (40) to the crew.

Description

1 TARIFF VOICE ASSISTANT AND TASK MANAGEMENT SYSTEM TECHNICAL FIELD The invention has implications for the defense industry and military land vehicle technologies, particularly in the field of armored and 5-cylinder vehicles. AI-based crew-vehicle interface and mission used in military land vehicles. It is related to management systems. The invention is particularly suitable for use in armored and military land vehicles, independently of external networks. configured to work; the vehicle's internal condition data and raw environmental data 10 vehicle sensors simultaneously collect data from the sensors in real-time. synthesizing isolated data and transforming it into risk assessment and decision support actions. central embedded processing unit and linguistic commands from the crew are converted into machine language. translator and risk assessment and notifications generated by the central embedded processing unit Integrated 15 that includes natural language rules and a two-way command bridge that communicates verbally to the crew. It is related to a voice assistant and task management system. STATE OF THE ART Today, defense industry and military land vehicle technologies have advanced to the 20s on the battlefield. To increase the crew's situational awareness of complex and dynamic threats. For this purpose, it is rapidly digitizing. Modern armored vehicles have numerous sensors and subsystems. The system has a high volume of data flow, which causes operational stress and challenges. This significantly increases the cognitive load on personnel working under these environmental conditions. artificial intelligence and 25 to overcome challenges and maximize task efficiency Natural language processing (NLP) technologies have begun to be integrated into military platforms; Voice assistants and voice command interfaces, in particular, have become a critical solution. Artificial intelligence... Intelligent voice assistants process data collected from in-vehicle systems in real time. By processing and interpreting the information, it provides personnel with rapid decision support, while enabling the crew to... By reducing reliance on manual screen or button controls, 'hands and eyes' 30 2 It provides a 'free' operating environment. The defense industry requires strict information security. In line with their requirements, these artificial intelligence and voice command systems can be integrated into external networks. Its ability to work on enclosed, in-vehicle embedded hardware, military vehicles the most important technology in modernization and increasing levels of autonomy It stands out as one of the trends. 5 In current technology, parameters such as engine temperature, fuel level, and speed are monitored in armored and military land vehicles. Critical status data is displayed via physical instrument panels or digital screens. This data is presented and access to it requires the crew to constantly look at the screen and This requires manually navigating between menus. This situation makes access to critical data difficult. slowing down the crew and keeping their hands and visual attention occupied. This causes a significant distraction during the operation. With current technology, vehicle status data is only provided to the crew as raw numerical or visual information. They are presented as values; the interpretation of the data, understanding what it means, and 15 Determining the course of action to be taken is entirely up to the crew in the moment. This is left to their assessment. This situation means that a potential risk or critical failure may be overlooked. This creates the risk of being detected, thus jeopardizing operational safety. In the current technique, task information is retrieved, summarized, prioritized, and 20 Operations such as management are carried out via radio communications, manually operated task computer interfaces, or through written / printed instructions, largely through the manual efforts of the crew. It is carried out entirely manually under intense stress, high noise and time pressure. And these methods, which are based on visual interaction, increase the likelihood of user error. It slows down operational processes and reduces overall task efficiency. 25 In current technology, the familiar voice assistant systems used in the automotive field are generally It is intended for comfort and infotainment purposes and only includes predefined basic features. These systems are able to execute commands. They interpret vehicle data and assess risk. making assessments, generating suggestions, or making sense of complex task information 30 3 Because they lacked the ability to summarize, they could not provide any decision support to the crew. It is unable to provide this. Current technology offers an integrated system that provides the crew with hands-free control. Because it is not present, both the vehicle's safety and control during the operation are necessary. 5 Furthermore, access to critical information cannot be achieved simultaneously. In short; Fast, secure, and hands-free access to critical data in an automatically interpreted format. The inability to reach these areas is the biggest technical problem for military vehicles in challenging conditions. It is emerging. 10 In conclusion, in order to solve the aforementioned problems that exist in the current technology, a new economical, user-friendly AI-based crew-vehicle interface and mission The need for management systems and the inadequacy of existing solutions are related to technical issues. This has made it necessary to make improvements in this field. 15 THE PURPOSE OF THE INVENTION The present invention aims to eliminate the aforementioned disadvantages and contribute to the relevant technical field. operational in armored and military land vehicles, developed to bring new advantages. An integrated voice assistant and task management system developed to increase efficiency 20 It is related to. The main purpose of the invention is to monitor engine temperature, speed, or fuel level in armored and military vehicles. The goal is to minimize the risk of potential failures by providing quick access to critical data such as this. The AI ​​model embedded in the vehicle processes data from the sensors in real time. Instead of providing comments and raw numbers to the crew, a direct risk assessment. It produces [product]. This allows critical situations to be avoided without the need for personnel to constantly monitor the screen. It is automatically detected without delay. 30 4 Another important objective of the invention is to autonomously evaluate incoming data and provide information to the crew. The goal is to provide proactive decision support. This involves understanding what data means in existing systems and... While the decision on which action to take is entirely left to the discretion of the personnel, this invention, Thanks to artificial intelligence, the crew is directly informed with "this risk exists, this action must be taken". It simplifies operational processes by generating clear recommendations in this format. 5 Another important objective of the invention is to enable the retrieval and management of task information via voice commands. The aim is to increase operational coordination on the battlefield by providing this. The system includes The field's natural language processing and speech recognition features understand the user's intent, handling complex tasks. It summarizes and prioritizes information. This allows staff to manually navigate through 10 menus. The need for navigation is eliminated. Another important objective of the invention is to allow the crew to operate completely hands-free during the process. The aim is to enhance navigation and mission safety by offering (hands-free) control capabilities. Voice commands. Thanks to the human-machine interface based on the machine, the crew's hands and visual attention are kept busy. This ensures that the personnel can both maintain control of the vehicle's driving and perform critical tasks. Access to information occurs simultaneously and securely. Another important objective of the invention is to protect against high stress, noise, and time pressure. The goal is to minimize user errors in demanding military missions. Hands-free 20 Voice interaction with AI-generated automated commenting and prioritization. Presenting these features together reduces the cognitive load (mental fatigue) of the staff. It reduces costs, speeds up processes, increases task efficiency, and saves vital time. provides. 25 Another important objective of the invention is to embed the artificial intelligence model directly within the vehicle (edge). Because it operates as AI, it does not require an external server or internet / network connection. It is the ability to work independently without hearing. This feature is ideal for environments where connectivity options are limited or In military operational conditions where external data transfer poses a security risk, the system It enables uninterrupted, fast, and secure operation against cyber threats. 30 The structural and characteristic features and all the advantages of the invention are given in the figures below. This becomes clearer thanks to the detailed explanation written with references to these figures. This will be understood, and therefore the evaluation should also take these forms and detailed explanations into consideration. It needs to be done by taking precautions. 5 FIGURES THAT WILL HELP UNDERSTAND THE INVENTION FIGURE 1; This image illustrates the voice assistant and task management system that is the subject of the invention. FIGURE 2; Schematic illustration of the voice assistant and task management system that is the subject of the invention. It is a picture. 10 REFERENCE NUMBERS 10. sensors 20. Multiple source collection and segmented inference unit 30. Persistent object tracking and multi-image memory unit 40. Central Embedded Processing and Operator Interface Unit 15 41. Mission computer with integrated embedded software. 42. Operator Interface 50. Natural language rule and bidirectional command bridge 60. Vehicle data bus and controlled function execution unit 20 6 DETAILED DESCRIPTION OF THE INVENTION This detailed description focuses on a voice assistant and task management system, which is the subject of the invention. the implemented methods are intended to facilitate a better understanding of the invention. It is explained. 5 The invention provides hands-free and voice-based command-based control to the crew in armored and military land vehicles. an interface that is embedded directly into the vehicle (edge) without the need for external networks an integrated voice assistant and task management system powered by an artificial intelligence processing unit It is a system. The basic working principle of the system is; 10 of the dense and complex raw vehicle data the collection of this data, processing of this data by artificial intelligence for risk assessment and decision-making. transforming them into support elements and crew-vehicle with natural language processing capabilities It is based on the principle of ensuring bidirectional interaction. The starting point of the system in question is the vehicle sensors (10). Vehicle 15 The sensors (10) are placed at different points of the vehicle and measure engine temperature, vehicle speed, In addition to critical internal status data such as fuel level and ammunition status; front, rear, right, Left or 360-degree cameras, thermal cameras, Lidar, radar, and positioning (GNSS / INS) It simultaneously generates and collects raw environmental data from its sensors. These sensors provide a continuous data stream under the challenging conditions of the battlefield, 20 It forms the basis of the system's environmental awareness. Large volumes of raw data collected from vehicle sensors (10) are directly from multiple sources. The data is transmitted to the summation and slice subtraction unit (20). This unit (20) uses different hardware and 25 independent data formats (visual, digital, signal-based) from the sensors It harmonizes the data in a single center. The heavy data load is embedded within the vehicle. In order to prevent the equipment from slowing down, the unit (20) uses "sliced ​​inference. It uses the "inference" method. Thanks to this method, the collected data is synchronous and 7 By breaking it down into smaller pieces (slices), the system's processing power is optimized. By reducing the requirements, it can be processed in real time and without delay. The processable structured by the multi-source collection and sliced ​​inference unit (20) data, then persistent object tracking and multi-image memory unit (30) 5 This unit (30) does not merely rely on instantaneous data flow, but also transmits the vehicle and its surroundings. By storing historical state data in memory, objects and states can be displayed on the timeline. It allows for permanent tracking. For example, a temperature detected seconds ago. if the source or moving target temporarily leaves the sensor's field of view The memory unit (30) protects this data. In this way, the central processing unit needs 10 The integrity of both current and historical data is fully preserved. The system's decision-making brain is the central embedded processing and operator interface unit. (40). Independent of external connections, isolated from cybersecurity threats. This working unit (40) basically consists of two main components: Large language models 15 (LLM) and integrated embedded systems incorporating artificial intelligence inference algorithms. mission computer (41) with software and crew can interact visually / physically operator interface (42). History from central unit (40), unit (20) and unit (30) and It synthesizes and interprets current data in real time. The vehicle's status is reported to the crew purely as raw data. Instead of being presented as numbers (e.g., engine temperature 115°C); mission computer (41) 20 Interpreted by: "Engine temperature is at a critical level, you need to reduce speed or..." a risk assessment and decision stating "it is recommended that you activate the cooling system" It is converted into a support action. At the same time, this unit (40) is involved in complex operational their tasks include analyzing and summarizing radio communications or combat reports It prioritizes task information internally. 25 Hands-free interaction between the central embedded processing unit (40) and the crew, This unit is provided via natural language rule and bidirectional command bridge (50). (50), advanced voice recognition and intention to interpret complex voice / linguistic commands from the crew 8 It translates into machine language using understanding algorithms. Only with predefined constraints. It allows communication to take place using natural human speech, rather than templates. The recommendations, risk assessments and feedback generated by the system are also made through this bridge (50) The text is transmitted to the crew audibly via text-to-speech technology. This allows... The crew must keep their eyes on the road / target 5 during combat or challenging off-road driving. without disconnecting and without taking their hands off the controls, bi-directional with the task computer and It can establish an interactive engagement. The voice assistant should be able to move beyond simply providing information and directly intervene in the vehicle. The structure that provides this is the vehicle data bus and the controlled function execution unit (60). This unit (60), 10 The vehicle's physical subsystems (headlights, etc.) can be connected via CAN and CAN-FD communication protocols. It communicates via horn, signals, actuators, wipers, etc. It receives natural language commands from the crew. A voice command such as "Turn off the headlights and switch to night mode" coming through the bridge (50), After being interpreted by the task computer (41), it is sent to the vehicle bus unit (60) It is transmitted. Unit (60) transfers the physical functions required by this command to subsystems 15 It ensures that they are operated in a controlled manner. The invention has an alternative structure, in addition to the basic components described above. It is also suitable for different structures that will increase operational flexibility. An alternative. In the configuration, the operator interface (42) is integrated into tactical helmets or ballistic goggles 20 It may include an enhanced Augmented Reality (AR) module. This allows for voice commands. The data produced by the bridge (50) is holographically displayed directly into the operator's field of vision. can be reflected. In another alternative configuration, the system would measure the crew's pulse and stress levels using 25 sensors. It may include biometric sensors (10). The central embedded processing unit (40) can include personnel stress sensors. by activating the "dynamic prioritization" algorithm when the level is high, It can only provide urgent vital warnings audibly and prioritize lower-priority information. It can keep it as text in the operator interface (42) until its level drops. Thus 9 personnel's cognitive load, autonomous adaptation to the stress factors of combat. provides. In addition, timestamped data collected in the persistent memory unit (30) is embedded Predictive maintenance algorithm after operation by the task computer (41) 5 can be processed and the crew can be informed before the next mission "which subsystem of the vehicle AI-based predictions can be offered indicating that the area is "prone to malfunction." Consequently, the invention is the result of the coordinated operation of all these hardware and software elements. Defense industry land vehicle personnel: a hands-free and eyes-free battlefield 10 It offers maximum efficiency because it can perform risk assessments and operates in isolation from external networks. A cybersecurity-enabled, lag-free, innovative, and intelligent task management and assistant. It provides the system. The scope of protection for this application is defined in the claims section above, and is explicitly mentioned in point 15. The examples given cannot be limited to those described; a person skilled in the technique can make a difference in the invention. the innovation presented can be achieved by using similar structures and / or This structure can also be applied to other similar fields using the same technology. It is clear that it can be implemented. Therefore, such structures foster innovation and especially technology. It is also obvious that it will lack the criterion of exceeding the known situation. 20

Claims

REQUESTS 1- An integrated system in armored and military land vehicles that operates independently of external networks. It relates to voice assistant and task management system, and its feature is; 5 - Simultaneously transmitting vehicle internal condition data and environmental raw data. collecting vehicle sensors (10), - by synthesizing the data from the sensors (10) in real time, risk isolated data that transforms into assessment and decision support actions central embedded processing unit (40), 10 - a central unit that translates linguistic commands from the crew into machine language. Risk assessment and notifications generated by the embedded processing unit (40) Natural language rules and a two-way command bridge that communicates verbally to the crew. (50) It is characterized by its inclusion. 15 2- A voice assistant and task management system compliant with Claim 1, featuring natural language. transmitted via the command bridge (50) and by the central processing unit (40) by transferring the physical functions required by the interpreted commands to the vehicle's subsystems Vehicle data bus and controlled function execution that enable them to be operated in a controlled manner 20 It is characterized by containing unit (60). 3- A voice assistant and task management system compliant with Request 1, whose feature is; different receiving data in independent formats from hardware and sensors (10) harmonizing, optimizing by simultaneously breaking down the collected data into smaller pieces. 25 It is characterized by containing a multi-source collection and sliced ​​extraction unit (20). It is done. 4- It is a voice assistant and task management system compliant with Claim 1, and its features are: Historical data stream of the vehicle and its surroundings, along with structured, processable real-time data. 30 It stores data in memory, and if the detected targets leave the sensor's field of view, the data is returned. 11 Persistent object tracking and multiple image memory that ensure integrity is maintained. It is characterized by containing unit (30). 5- It is a voice assistant and task management system compliant with Claim 1, and its features include; as mentioned above. central embedded processing unit (40), large language models and artificial intelligence inference 5 mission computer with integrated embedded software containing its algorithms (41) and includes an operator interface (42) through which the crew can visually / physically interact. It is characterized by... 6- It is a voice assistant and task management system compliant with Request 5, and its feature is; mentioned 10 The data generated by the operator interface (42) and the voice command bridge (50) are used by the operator. tactical helmets that allow holographic projection directly into the field of vision or by including an Augmented Reality module integrated into ballistic glasses It is the characterization of the situation. 15 7- It is a voice assistant and task management system compliant with Claim 1, and its features include; as mentioned above. vehicle sensors (10) biometric sensors measuring crew’s pulse and stress level and the central embedded processing unit (40) is only when the stress level is high. It delivers urgent vital warnings audibly and keeps low-priority information waiting in text format. It is characterized by its inclusion of a dynamic prioritization algorithm. 20 8- A voice assistant and task management system compliant with Claim 1, whose feature is; permanent. object tracking and timestamped data collected in the multi-image memory unit (30) by processing the vehicle's subsystems' susceptibility to failure using artificial intelligence. It is characterized by including a post-operative predictive maintenance algorithm that provides insights. 25 It is done. 30

Images