Cockpit information sharing system and method based on wearable device gaze tracking

Abstract

The application discloses a cockpit information sharing system and method based on a line-of-sight tracking of a wearable device, and relates to the field of human-computer interaction, comprising a wearable visual perception terminal, a vehicle-mounted central processing unit and a cockpit sharing display device; the vehicle-mounted central processing unit is respectively connected with the wearable visual perception terminal and the cockpit sharing display device in data; wherein the wearable visual perception terminal is internally provided with a camera, an eye movement tracking sensor and a wireless communication module, is used for continuously shooting a first visual angle picture of a user and accurately locking a focus point of a line-of-sight of the user on the first visual angle picture; synchronously judging whether the user has an information sharing intention; if yes, intercepting a key image or a short video comprising an external object corresponding to the focus point, encapsulating the key image or the short video and auxiliary information into a data packet, and sending the data packet through the wireless communication module.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and in particular to a cockpit information sharing system based on eye tracking of wearable devices, a cockpit information sharing method based on eye tracking of wearable devices, electronic devices, storage media, and vehicle cockpits. Background Technology

[0002] In the field of smart cockpits, the existing technologies that are closest to the core application scenarios and target requirements of this application are mainly divided into two categories. Although the two types of technologies have respectively realized some interactive or information transfer functions in the cockpit, they both have inherent defects and cannot meet the core needs of in-vehicle occupants to share information "instantly, intuitively, and around the real world".

[0003] I. Category 1 Existing Technology: Eye-tracking technology based on fixed in-vehicle cameras. This type of technology is a typical solution for driver monitoring and basic interaction in current smart cockpits, and its main design objectives focus on two points:

[0004] Driver Monitoring System (DMS): The system uses cameras fixed inside the vehicle to capture the driver's facial and eye movements to determine whether the driver is paying attention to the road (such as identifying whether the driver's gaze is off the road or whether the driver is closing their eyes). It also detects the driver's fatigue state (such as frequent blinking, abnormal posture such as head drooping) and issues warnings when risks are detected to ensure driving safety.

[0005] Limited UI control: Allows drivers to confirm selections by looking at specific controls on the in-vehicle central control screen (such as air conditioning adjustment buttons, music switching icons, etc.) and then using the steering wheel buttons, thereby reducing the frequency of drivers manually operating the central control screen and reducing the risk of driver distraction.

[0006] However, this type of technology has three inherent pain points and drawbacks:

[0007] The interaction range is extremely limited: due to the fixed installation position of the camera, its shooting angle is strictly limited to the front of the vehicle and a limited area inside the vehicle (such as the dashboard, central control screen, and rearview mirror). It can only capture the driver's gaze at the above-mentioned areas and cannot know the driver's or passengers' focus on the broad physical environment outside the window (such as roadside landmarks, shops, natural landscapes, etc.), and cannot achieve the connection between "external scene and internal information".

[0008] The interaction has a single purpose and is self-centered: the technical design focuses on "driver monitoring" or "driver's personal control of the in-vehicle UI", and does not have the attributes of "information sharing" and "cabin collaboration" - passengers cannot use the system, and even if the driver captures interesting in-vehicle information through eye movement, he cannot share it with other in-vehicle members, which cannot meet the information interaction needs in multi-person cabin scenarios.

[0009] Hardware deployment is costly and inflexible: dedicated cameras and processors need to be integrated during the vehicle design phase, making later installation extremely difficult (requiring adaptation to vehicle system protocols, modification of vehicle wiring, etc.). Furthermore, it is limited by the installation location and number of cameras, making it impossible to cover all occupants in the vehicle (such as rear passengers being completely outside the monitoring range), resulting in poor versatility and scalability.

[0010] II. Existing technologies in the second category: Screen projection technology based on mobile devices. This type of technology is currently the most widely used information migration solution in smart cockpits. Typical applications include Apple's CarPlay and the self-developed car-handling interconnection technology of various car brands. Its core function is to allow users to project the interface of their smartphone (such as the phone desktop and contacts) or specific application content (such as navigation maps, music playlists, and video software) onto the in-vehicle central control screen, so as to realize "the display and operation of mobile phone content on the in-vehicle large screen".

[0011] However, this type of technology also has two major drawbacks, making it difficult to meet the needs of efficient information sharing in driving scenarios:

[0012] The interaction chain is long and the cognitive load is high: users need to complete the entire process of "taking out the phone → unlocking the screen → finding the specific app → searching or locating the target content in the app → initiating a screen mirroring request". The whole process is fragmented and not real-time. Especially during driving, each operation requires users to divert their attention to the phone, which violates the core safety principle of "minimizing distraction in driving scenarios".

[0013] Lack of context awareness: The content projected is completely decoupled from the user's current physical environment. That is, the content that the user searches for and projects on their phone (such as an introduction to a tourist attraction or information about a restaurant) is not directly related to "the scene the user sees outside the car window at this moment". It cannot achieve the immediacy of "seeing and sharing". The user needs to manually match "what is seen outside the car" with "projected content", which lacks immersion and context relevance and cannot meet the need for "information sharing around the real world".

[0014] In summary, both of the aforementioned existing technologies have shortcomings in core functionality: one type, due to its fixed hardware, cannot capture the view outside the vehicle and cannot connect to the real physical environment; the other type, due to its cumbersome interaction process and lack of context awareness, cannot achieve real-time information sharing. Neither addresses the core pain point of "information sharing and collaboration among in-vehicle occupants in a real-world, intuitive manner" within a smart cockpit. Therefore, a novel cockpit information sharing technology solution is urgently needed to overcome the deficiencies of existing technologies. Summary of the Invention

[0015] The purpose of this invention is to provide a cockpit information sharing system based on wearable device eye tracking, a cockpit information sharing method based on wearable device eye tracking, an electronic device, a storage medium, and a vehicle cockpit, thereby solving at least one of a number of technical problems.

[0016] 1. Inherent limitations of existing eye-tracking technologies based on fixed in-vehicle cameras:

[0017] For example, the field of vision is limited: because the camera position is fixed, it can only capture the driver's gaze on a limited area inside the car (instrument panel, central control screen, etc.), and cannot perceive the driver's or passengers' focus on the wide physical environment outside the car (such as landmarks, shops, scenery).

[0018] For example, there is no information sharing and cabin collaboration attribute: the technology was originally designed as "driver monitoring (DMS)" or "driver's personal UI control", which passengers cannot use, nor can they share information that the user is interested in with other passengers in the vehicle;

[0019] For example, the hardware deployment is inflexible: dedicated cameras and processors need to be integrated during the vehicle design phase, which makes it difficult to install them later and cannot cover all occupants in the vehicle (such as rear passengers).

[0020] 2. Key shortcomings of existing mobile device-based screen mirroring technologies (such as CarPlay and car-to-vehicle connectivity):

[0021] For example, the interaction chain is long and the cognitive load is high: users need to complete multiple fragmented operations such as "taking out the phone → unlocking → finding the specific app → locating the content → initiating screen mirroring", which is not real-time and violates the safety principle of "reducing distraction" in driving scenarios.

[0022] For example, it lacks context awareness: the content projected is completely decoupled from the user's current physical environment, and it cannot directly associate "the scene outside the car that the user sees at this moment" with "the content shared on the screen", resulting in a lack of immersion and immediacy.

[0023] 3. Lack of an overall information sharing mechanism for the intelligent cockpit:

[0024] For example, existing technologies have not built an information sharing and collaboration mechanism that can be initiated by any member of the vehicle, based on real-world scenarios, and synchronized to all members in real time. As a result, smart cockpits can only serve as "personal driving tools" or "single information display terminals" and cannot meet the collaborative needs of multi-person travel scenarios (such as information synchronization and interaction when traveling with family).

[0025] This invention provides the following solution:

[0026] According to a first aspect of the present invention, a cockpit information sharing system based on wearable device eye tracking is provided, comprising:

[0027] Wearable visual sensing terminals, in-vehicle central processing units, and cockpit shared display devices;

[0028] The vehicle's central processing unit is connected to the wearable visual sensing terminal and the cockpit shared display device for data transmission.

[0029] Among them, the wearable visual perception terminal has a built-in camera, eye-tracking sensor and wireless communication module, which is used to continuously capture the user's first-person view and accurately lock the user's gaze on the first-person view.

[0030] Simultaneously determine whether the user intends to share information;

[0031] If it exists, then capture key images or short videos of the external objects corresponding to the focal point, encapsulate them with auxiliary information into a data packet, and send it through the wireless communication module;

[0032] The vehicle-mounted central processing unit is equipped with image recognition, data parsing, and multi-source information aggregation computing capabilities, as well as vehicle network connectivity, for receiving and parsing data packets.

[0033] External objects are identified through preset image recognition, data parsing, and multi-source information aggregation computing capabilities, and the recognition results are output.

[0034] Using the recognition results as keywords, the system queries the cloud information service platform or retrieves relevant information about external objects from the local database based on the vehicle network connectivity capability. The relevant information is then integrated into interactive content that is suitable for multi-person shared cockpit scenarios.

[0035] The cockpit-shared display device is used to receive and display interactive content and output it to the occupants of the vehicle.

[0036] Furthermore, including:

[0037] Wearable visual sensing terminals can be any of the following: smart glasses, smart headphones with cameras, AR contact lenses, smart bracelets with pointing functions, or smartphones.

[0038] Furthermore, including:

[0039] The auxiliary information includes at least one of the following: the time when the user's gaze is focused on an external object, the vehicle's current geographical location, and the user's identification information.

[0040] Furthermore, including:

[0041] The onboard central processing unit's local database is used to store basic information about commonly used external objects;

[0042] The local database is updated in real time or periodically by synchronizing data with the cloud information service platform via the vehicle network.

[0043] Furthermore, including:

[0044] The cockpit shared display equipment includes at least one of the following: in-vehicle central control screen, passenger entertainment screen, and rear entertainment screen;

[0045] This also includes multi-screen synchronous display of interactive content on shared cockpit display devices;

[0046] Interactive content consists of illustrated interactive cards or scrollable interactive pages.

[0047] Furthermore, including:

[0048] The wireless communication module uses Bluetooth, Wi-Fi, or automotive-grade proprietary wireless communication protocols.

[0049] The cockpit-shared display also supports touch or voice interaction, allowing occupants to trigger further actions on the interactive content.

[0050] Further steps include playing detailed videos from the relevant information and setting the location corresponding to the external object as the navigation target.

[0051] According to a second aspect of the present invention, a method for sharing cockpit information based on eye tracking using a wearable device is provided, comprising:

[0052] Step S1: Eye tracking and scene capture;

[0053] The wearable visual sensing terminal continuously captures the user's first-person perspective image through its built-in camera and uses a built-in eye-tracking sensor to lock the user's gaze on the first-person perspective image in real time.

[0054] Step S2: Determine user intent;

[0055] Continuously analyze user behavior data, and when a preset trigger condition is detected, determine that the user has the intention to share information;

[0056] The preset trigger conditions include at least one of the following: the user’s gaze lingers on the external object corresponding to the focal point for a longer period of time than a set threshold, the user issues a preset voice command while looking at the external object, or the user performs a preset touch operation on the wearable visual sensing terminal.

[0057] Step S3: Data packet encapsulation and transmission;

[0058] Capture key images or short videos containing external objects and encapsulate them with auxiliary information into a data packet;

[0059] Data packets are sent to the vehicle's central processing unit via the wireless communication module of the wearable visual sensing terminal.

[0060] Step S4: Information analysis and target recognition;

[0061] The vehicle-mounted central processing unit receives and parses data packets, and analyzes key images or short videos through its built-in image recognition function to identify the specific type and identification information of external objects.

[0062] Step S5: Information aggregation and content generation;

[0063] The vehicle-mounted central processing unit uses the identification information of external objects as keywords to query the cloud information service platform or search the local database through the vehicle network to obtain relevant information about the external objects, including at least two of the following: text descriptions, high-definition images, related videos, user reviews, and navigation coordinates.

[0064] Intelligently integrate relevant information about external objects into interactive content adapted to multi-person shared cockpit scenarios;

[0065] S6: Shared display and collaborative interaction;

[0066] The onboard central processing unit sends interactive content to the cockpit shared display device, which then displays the interactive content and outputs it to the occupants of the vehicle.

[0067] Furthermore, including:

[0068] The preset triggering conditions in step S2 also include at least one of the following: the user makes a preset head posture, the user executes a preset eye movement mode, or the wearable visual perception terminal's biosensor detects that the user's interest level has increased beyond a preset threshold.

[0069] Preset head postures include nodding in confirmation;

[0070] The preset eye-tracking patterns include blinking twice rapidly.

[0071] Furthermore, including:

[0072] The image recognition and acquisition of relevant information in steps S4 to S5 include:

[0073] The onboard central processing unit is the main execution entity to complete all tasks;

[0074] After image recognition and information aggregation are completed by the cloud-based information service platform, the results are sent to the vehicle-mounted central processing unit.

[0075] Alternatively, after simple image recognition is performed by a wearable visual sensing terminal, the preliminary results are sent to the vehicle's central processing unit for further information processing.

[0076] Furthermore, including:

[0077] Step S6 also includes: triggering further operations on the interactive content by touching the cockpit shared display device or issuing a voice command;

[0078] Further operations include controlling the cockpit shared display device to play detailed videos from relevant information;

[0079] It also includes instructing the onboard central processing unit to set the location corresponding to an external object as the navigation target.

[0080] Furthermore, including:

[0081] The wearable visual sensing terminal in step S1 can be any one of smart glasses, smart headphones with a camera, AR contact lenses, smart bracelets with pointing function, or smartphones.

[0082] The auxiliary information in step S3 includes at least one of the following: user gaze focus time information, vehicle current geographical location information, and user identification information.

[0083] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0084] The memory stores a computer program that, when executed by a processor, causes the processor to perform steps of a cockpit information sharing method based on eye tracking of a wearable device.

[0085] According to a fourth aspect of the present invention, a computer-readable storage medium is provided storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a cockpit information sharing method based on eye tracking of a wearable device.

[0086] According to a fifth aspect of the present invention, a vehicle cabin is provided, comprising:

[0087] Electronic devices, steps for implementing a cockpit information sharing method based on wearable device eye tracking;

[0088] The processor runs a program that, when running, executes steps of a cockpit information sharing method based on wearable device eye tracking from data output by the electronic device.

[0089] Storage medium for storing programs that, when running, execute steps of a cockpit information sharing method based on wearable device eye tracking on data output from electronic devices.

[0090] The above solution achieves the following beneficial technical effects:

[0091] This application extends the starting point of information interaction from the in-vehicle screen to the real physical world around the vehicle in 360°. By using wearable devices (such as smart glasses) to capture the user's first-person perspective and gaze focus, it achieves "interaction based on what you see," completely solving the problem that existing technologies cannot perceive the outside view and greatly expanding the boundaries of interaction.

[0092] This application enables users to initiate information acquisition and sharing by using the almost instinctive action of "looking at the target object" in conjunction with preset triggering conditions (such as gaze duration, voice commands, and touch). This eliminates the need for multiple manual operations, greatly shortens the interaction path, reduces operational complexity and cognitive load, meets the safety requirements of driving scenarios, and significantly improves interaction efficiency and intuitiveness.

[0093] This application enables any member wearing a designated wearable device (not limited to the driver) in the vehicle to become an information sharing initiator. The in-vehicle unit integrates information about objects of interest to users (such as landmark historical background and store reviews) into interactive content (such as text and image cards) suitable for multiple viewers, and displays it through the central control screen and the passenger / rear entertainment screen for all occupants to view simultaneously. This promotes the transformation of the smart cockpit from a "personal tool" to a "mobile family information center," achieving true cockpit collaboration.

[0094] This application enables real-time synchronization of external information (such as introductions to historical buildings, tourist attractions, and restaurant navigation coordinates) that users are interested in with all passengers in scenarios such as tourism, science and education, and daily life services. For example, when traveling with a family, the information about the clock tower that the passenger in the front seat is interested in can be instantly transformed into "history lessons" that are visible to everyone, enriching the fun and practicality of the trip and enhancing the contextual value of driving and the travel experience. Attached Figure Description

[0095] Figure 1 This is a structural diagram of a cockpit information sharing system based on wearable device eye tracking, provided by one or more embodiments of the present invention.

[0096] Figure 2 This is a flowchart of a cockpit information sharing method based on eye tracking of wearable devices, provided by one or more embodiments of the present invention.

[0097] Figure 3 This is a structural block diagram of an electronic device for a cockpit information sharing method based on wearable device eye tracking, provided by one or more embodiments of the present invention. Detailed Implementation

[0098] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0099] Figure 1 This is a structural diagram of a cockpit information sharing system based on wearable device eye tracking, provided by one or more embodiments of the present invention.

[0100] like Figure 1 The cockpit information sharing system based on wearable device eye tracking, as shown, includes:

[0101] Wearable visual sensing terminals, in-vehicle central processing units, and cockpit shared display devices;

[0102] The vehicle's central processing unit is connected to the wearable visual sensing terminal and the cockpit shared display device for data transmission.

[0103] Among them, the wearable visual perception terminal has a built-in camera, eye-tracking sensor and wireless communication module, which is used to continuously capture the user's first-person view and accurately lock the user's gaze on the first-person view.

[0104] Simultaneously determine whether the user intends to share information;

[0105] If it exists, then capture key images or short videos including the external objects corresponding to the focal point, encapsulate them with auxiliary information into a data packet, and send it through the wireless communication module;

[0106] The vehicle-mounted central processing unit is equipped with image recognition, data parsing, and multi-source information aggregation computing capabilities, as well as vehicle network connectivity, for receiving and parsing data packets.

[0107] External objects are identified through preset image recognition, data parsing, and multi-source information aggregation computing capabilities, and the recognition results are output.

[0108] Using the recognition results as keywords, the system queries the cloud information service platform or retrieves relevant information about external objects from the local database based on the vehicle network connectivity capability. The relevant information is then integrated into interactive content that is suitable for multi-person shared cockpit scenarios.

[0109] The cockpit-shared display device is used to receive and display interactive content and output it to the occupants of the vehicle.

[0110] Specifically, the technical features disclosed in this application include: a wearable visual perception terminal with a built-in camera, eye-tracking sensor, and wireless communication module. The technical problems solved by this feature include: 1. Existing eye-tracking technology based on fixed in-vehicle cameras cannot capture external visual information due to the fixed camera position; 2. Existing technology suffers from inflexible hardware deployment and cannot cover all occupants in the vehicle. The technical effects of this feature include: 1. Overcoming the viewing angle limitations of fixed cameras, allowing flexible capture of first-person perspective images depending on the user's wearing position; 2. The device is portable, allowing any occupant in the vehicle to wear and use it, extending coverage to all occupants.

[0111] The technical features disclosed in this application include: a wearable visual sensing terminal for continuously capturing the user's first-person perspective view and accurately locking the user's gaze focus point on the first-person perspective view. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking can only capture gaze in a limited area inside the vehicle, failing to determine the user's focus on the broader external environment; 2. Existing screen projection technology lacks context awareness and cannot correlate with "the real scene the user is currently seeing." The technical effects of this feature include: 1. Extending the gaze capture range from inside the vehicle to the 360° real physical world around the vehicle; 2. Achieving precise correlation between "gaze and real scene," providing a contextual basis for subsequent information sharing.

[0112] The technical features disclosed in this application include: when a wearable visual sensing terminal determines that a user intends to share information, it captures a key image or short video containing the external object corresponding to the focal point, encapsulates it and auxiliary information into a data packet, and then sends it through a wireless communication module. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking lacks an "information sharing" attribute and cannot transmit information about the user's attention to external objects; 2. Existing screen-projection technology has a long interaction chain (requiring multiple steps to initiate), and cannot instantly transmit user intent. The technical effects of this feature include: 1. It can actively capture and transmit the user's intent to pay attention to objects outside the vehicle, providing a foundation for information sharing; 2. It simplifies the information initiation process, achieving instant data transmission through "gaze + automatic capture and transmission," shortening the interaction path.

[0113] The technical features disclosed in this application include: the in-vehicle central processing unit possesses computing capabilities supporting image recognition, data parsing, and multi-source information aggregation, as well as in-vehicle network connectivity. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking technology lacks image recognition and multi-source information aggregation capabilities, and can only monitor or perform simple control; 2. Existing screen projection technology cannot mine information from real objects of user interest and lacks the ability to generate contextualized content. The technical effects of this feature include: 1. Providing computing power support for "identifying external objects of user interest + obtaining related information"; 2. Connecting to external resources via the in-vehicle network to achieve the transformation from "single image" to "rich information".

[0114] The technical features disclosed in this application include: an in-vehicle central processing unit for receiving and parsing data packets, and identifying external objects through a built-in image recognition function. The technical problems solved by this feature include: 1. Existing technologies cannot convert user-focused external "visual objects" into parseable "digital information"; 2. Existing projection technologies are decoupled from the real physical environment and cannot identify objects seen by the user. The technical effects of this feature include: 1. Accurately identifying external objects (such as landmarks and shops) that the user is looking at, establishing a connection between "visual scene" and "digital information"; 2. Providing core evidence for subsequent acquisition of object-related information, solving the problem of lack of contextual awareness.

[0115] The technical features disclosed in this application include: the vehicle-mounted central processing unit uses the recognition results as keywords to query a cloud information service platform or retrieve relevant information about external objects through the vehicle network. The technical problems solved by this feature include: 1. Existing technologies lack information retrieval capabilities and cannot provide additional information about objects of interest to users; 2. Existing screen projection technologies require users to prepare content in advance on their mobile phones, making it impossible to obtain object-related information in real time. The technical effects of this feature include: 1. Real-time acquisition of rich information about objects (such as historical background, user reviews, and navigation coordinates), overcoming the information limitations of existing technologies; 2. Eliminating the need for prior user operation, achieving "see and search instantly," reducing cognitive load.

[0116] The technical features disclosed in this application include: the in-vehicle central processing unit integrates relevant information into interactive content adapted to multi-person sharing scenarios in the cabin. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking technology is centered on "personal control / monitoring" and lacks a multi-person shared content format; 2. Existing screen projection technology mostly replicates mobile phone interfaces, which is not suitable for multi-person viewing and interaction in the cabin. The technical effects of this feature include: 1. Generating content that meets the needs of multi-person sharing (such as text and image cards), possessing "cabin collaboration" attributes; 2. The content format is adapted to the in-vehicle large screen, facilitating viewing by all occupants and promoting information synchronization among them.

[0117] The technical features disclosed in this application include: establishing a communication connection between the cockpit shared display device and the vehicle's central processing unit. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking technology lacks a dedicated shared display link, making it impossible to transmit information to all occupants; 2. Existing screen projection technology relies on a single central control screen display and does not clearly define dedicated communication compatibility with the vehicle's processing unit. The technical effects of this feature include: 1. Establishing a stable information transmission link between the processing unit and the display device, ensuring efficient output of shared content; 2. Providing a hardware connection foundation for subsequent viewing by all occupants, solving the problem of "information not being able to be transmitted to multiple people."

[0118] The technical features disclosed in this application include: a cockpit-shared display device for receiving and displaying interactive content for all occupants to view. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking technology lacks a function for displaying information to all occupants, serving only the driver; 2. Existing technology lacks an information presentation carrier that allows for "full participation," thus failing to achieve cockpit collaboration. The technical effects of this feature include: 1. Breaking the limitation of "information only visible to the individual," allowing all occupants to simultaneously obtain object-related information; 2. Promoting the transformation of the smart cockpit from a "personal tool" to a "mobile home information center," enhancing collaboration.

[0119] In this embodiment, it includes:

[0120] Wearable visual sensing terminals can be any of the following: smart glasses, smart headphones with cameras, AR contact lenses, smart bracelets with pointing functions, or smartphones.

[0121] Specifically, the technical features disclosed in this application include: wearable visual sensing terminals comprising smart glasses, smart headphones with cameras, AR contact lenses, smart bracelets with pointing functionality, or smartphones. The technical problems addressed by these features include: existing technologies do not address the diverse forms of wearable terminals, only defaulting to smart glasses, resulting in a single device form and insufficient adaptability. The technical effects of these features include: 1. Covering different user habits and scenarios (such as users who prefer headphones or bracelets), improving device versatility; 2. Reducing dependence on a single device, enhancing the system's adaptability among different cabin occupants.

[0122] In this embodiment, it includes:

[0123] The auxiliary information includes at least one of the following: the time when the user's gaze is focused on an external object, the vehicle's current geographical location, and the user's identity information.

[0124] Specifically, the technical features disclosed in this application include: auxiliary information containing user gaze focus time information. The technical problems solved by this feature include: existing technologies only mention "auxiliary information" without specifying its content, which may lead to the in-vehicle unit being unable to determine the temporal relevance of the object being viewed by the user, affecting the accuracy of the information. The technical effects of this feature include: 1. helping the in-vehicle unit accurately match "user gaze time" with "object state" (e.g., whether a store's opening hours match the gaze time); 2. improving the timeliness of information retrieval and avoiding the acquisition of outdated information.

[0125] The technical features disclosed in this application include: auxiliary information containing the vehicle's current geographical location information. The technical problems solved by this feature include: existing technologies lack geographical location association, which may lead to the in-vehicle unit being unable to distinguish objects with the same name (such as "clock towers" in different cities), posing a risk of information confusion. The technical effects of this feature include: 1. providing geographical coordinate constraints for object recognition, accurately locating "objects in specific locations" of interest to the user; 2. avoiding information confusion related to objects with the same name, improving the accuracy of information retrieval.

[0126] The technical features disclosed in this application include: auxiliary information containing user identification information. The technical problems solved by this feature include: existing technologies do not distinguish user identities, which may result in generated content that cannot adapt to the needs of different members (such as the difference in information complexity between children and adults). The technical effects of this feature include: 1. The information display format can be customized according to user identity (such as children or adults) (such as simplified text for children and detailed information for adults); 2. It enhances the personalization of content and meets the information needs of different members.

[0127] In this embodiment, a local database of the vehicle-mounted central processing unit is used to store basic information about commonly used external objects.

[0128] The local database is updated in real time or periodically by synchronizing data with the cloud information service platform via the vehicle network.

[0129] Specifically, the technical features disclosed in this application include: storing basic information about commonly used external objects in a local database. This technical feature solves the following technical problems: existing technologies only mention "retrieving from a local database" without specifying the stored content; if relying entirely on cloud queries, basic information may be unavailable when the network is poor. The technical effects of this feature include: 1. Quick retrieval of local basic information (such as commonly used landmark names and addresses) when the network is poor, avoiding information retrieval interruption; 2. Reducing the frequency of cloud queries, lowering network traffic consumption and information retrieval delay.

[0130] The technical features disclosed in this application include: a local database that is synchronized and updated in real time or periodically with a cloud information service platform via an in-vehicle network. The technical problems solved by this feature include: existing technologies do not mention a local database update mechanism, which may lead to outdated locally stored information (such as store closures or changes in landmark names). The technical effects of this feature include: 1. ensuring that local database information is synchronized with the cloud, avoiding the provision of outdated content; 2. balancing "fast local retrieval" and "information timeliness," achieving a balance between query efficiency and content accuracy.

[0131] In this embodiment, it includes:

[0132] The cockpit shared display equipment includes at least one of the following: in-vehicle central control screen, passenger entertainment screen, and rear entertainment screen;

[0133] This also includes multi-screen synchronous display of interactive content on shared cockpit display devices;

[0134] Interactive content consists of illustrated interactive cards or scrollable interactive pages.

[0135] Specifically, the technical features disclosed in this application include: the cockpit shared display device includes at least one of an in-vehicle central control screen, a front passenger entertainment screen, and a rear passenger entertainment screen. The technical problems solved by this feature include: existing technologies mention multiple screens but do not explicitly cover the front passenger / rear passenger entertainment screens, which may lead to inconvenience for rear passengers and prevent true sharing for all occupants. The technical effects of this feature include: 1. Covering display devices in different locations within the vehicle, ensuring clear viewing for both front and rear passengers; 2. Further strengthening the "all-occupant participation" sharing attribute and avoiding information lag for rear passengers.

[0136] The technical features disclosed in this application include: the cockpit shared display device supports multi-screen synchronous display of interactive content. The technical problems solved by this feature include: existing technologies do not mention multi-screen synchronization, which may lead to inconsistencies in content across different screens (e.g., the central control screen displays information A, while the rear screen displays information B), affecting the sharing experience. The technical effects of this feature include: 1. ensuring that all display devices synchronously present the same content, ensuring consistent information access for all vehicle occupants; 2. avoiding communication discrepancies caused by content asynchrony, thus improving collaboration efficiency.

[0137] The technical features disclosed in this application include: interactive content consisting of interactive cards with both text and images or scrollable interactive pages. The technical problems solved by this feature include: existing technologies only mention "interactive cards or pages" without specifying the attributes of "text and images" and "scrollable," which may lead to monotonous content and insufficient information capacity. The technical effects of this feature include: 1. "Text and images" enhances the intuitiveness of the content, facilitating quick understanding by members (e.g., landmark images + text descriptions); 2. "Scrollable" increases information capacity, allowing members to view more details as needed (e.g., user reviews, detailed historical background).

[0138] In this embodiment, it includes:

[0139] The wireless communication module uses Bluetooth, Wi-Fi, or automotive-grade proprietary wireless communication protocols.

[0140] The cockpit-shared display also supports touch or voice interaction, allowing occupants to trigger further actions on the interactive content.

[0141] Further steps include playing detailed videos from the relevant information and setting the location corresponding to the external object as the navigation target.

[0142] Specifically, the technical features disclosed in this application include: the wireless communication module adopts Bluetooth, Wi-Fi, or automotive-grade dedicated wireless communication protocols. The technical problems solved by this feature include: existing technologies only mention "wireless communication module" without specifying the communication protocol type, which may lead to poor communication stability (such as signal interruption and high latency). The technical effects of this feature include: 1. Using a mature and stable communication protocol ensures uninterrupted and low-latency data transmission between the wearable terminal and the vehicle unit; 2. Automotive-grade protocols are adapted to the vehicle environment (such as anti-interference), improving communication reliability.

[0143] The technical features disclosed in this application include: the cockpit shared display device supports touch interaction or voice interaction. The technical problems solved by this feature include: existing technologies do not mention interaction methods for display devices, allowing only passive viewing and failing to meet the further operational needs of occupants. The technical effects of this feature include: 1. Breaking the limitation of "only viewing without operation," allowing occupants to actively trigger operations through touch / voice, enhancing their sense of participation; 2. Adapting to driving scenarios (such as drivers using voice operation, avoiding manual distraction), balancing convenience and safety.

[0144] The technical features disclosed in this application include: further operations include playing detailed videos from relevant information. The technical problems solved by this feature include: existing technologies only mention "display content" without mentioning video playback functionality, which may result in a limited information presentation format (only text and images). The technical effects of this feature include: 1. Enriching information presentation formats, allowing members to understand objects more intuitively through videos (such as landmark introduction videos); 2. Enhancing the interest of information, especially improving the experience in tourism and science education scenarios.

[0145] The technical features disclosed in this application include: a further operation that sets the location corresponding to an external object as the navigation target. The technical problems solved by this feature include: existing technologies do not mention navigation association functions, requiring users to manually input locations into the navigation app, resulting in a long interaction chain. The technical effects of this feature include: 1. Achieving "navigation upon seeing," directly setting the location of the object of interest as the navigation target, shortening the operation path; 2. Avoiding the distraction risk of manually inputting navigation addresses while driving, improving driving safety.

[0146] Figure 2 This is a flowchart of a cockpit information sharing method based on eye tracking of wearable devices, provided by one or more embodiments of the present invention.

[0147] like Figure 2 The cockpit information sharing method based on wearable device eye tracking shown includes:

[0148] Step S1: Eye tracking and scene capture;

[0149] The wearable visual sensing terminal continuously captures the user's first-person perspective image through its built-in camera and uses a built-in eye-tracking sensor to lock the user's gaze on the first-person perspective image in real time.

[0150] Step S2: Determine user intent;

[0151] Continuously analyze user behavior data, and when a preset trigger condition is detected, determine that the user has the intention to share information;

[0152] The preset trigger conditions include at least one of the following: the user’s gaze lingers on the external object corresponding to the focal point for a longer period of time than a set threshold, the user issues a preset voice command while looking at the external object, or the user performs a preset touch operation on the wearable visual sensing terminal.

[0153] Step S3: Data packet encapsulation and transmission;

[0154] Capture key images or short videos containing external objects and encapsulate them with auxiliary information into a data packet;

[0155] Data packets are sent to the vehicle's central processing unit via the wireless communication module of the wearable visual sensing terminal.

[0156] Step S4: Information analysis and target recognition;

[0157] The vehicle-mounted central processing unit receives and parses data packets, and analyzes key images or short videos through its built-in image recognition function to identify the specific type and identification information of external objects.

[0158] Step S5: Information aggregation and content generation;

[0159] The vehicle-mounted central processing unit uses the identification information of external objects as keywords to query the cloud information service platform or search the local database through the vehicle network to obtain relevant information about the external objects, including at least two of the following: text descriptions, high-definition images, related videos, user reviews, and navigation coordinates.

[0160] Intelligently integrate relevant information about external objects into interactive content adapted to multi-person shared cockpit scenarios;

[0161] Step S6: Shared display and collaborative interaction;

[0162] The onboard central processing unit sends interactive content to the cockpit shared display device, which then displays the interactive content and outputs it to the occupants of the vehicle.

[0163] Specifically, the technical features disclosed in this application include: Step S1: Continuously capturing the user's first-person perspective image through the built-in camera of the wearable visual perception terminal, and using the built-in eye-tracking sensor to lock the user's gaze focus point on the first-person perspective image in real time. The technical problems solved by this feature include: 1. Existing fixed-camera eye-tracking cannot capture external gaze; 2. Existing screen projection technology cannot correlate with the user's actual perspective. The technical effects of this feature include: 1. Achieving continuous capture and precise gaze lock of external perspective; 2. Establishing a real-time correlation between "user's perspective and focus point," laying the foundation for contextualized information sharing.

[0164] The technical features disclosed in this application include: Step S2: Continuously analyze user behavior data, and when a preset trigger condition is detected, determine that the user has an intention to share information; the preset trigger conditions include at least one of the following: the user's gaze lingers on an external object corresponding to the focal point for a duration exceeding a set threshold, the user issues a preset voice command (such as "share this") while looking at the external object, or the user performs a preset touch operation on a wearable visual sensing terminal. The technical problems solved by this feature include: 1. Existing technologies lack a "user sharing intention judgment" mechanism, making it impossible to proactively initiate information sharing; 2. Existing screen projection technologies require multiple user steps to initiate, resulting in cumbersome interaction. The technical effects of this feature include: 1. Accurately judging the user's sharing intention through multiple trigger conditions, avoiding false triggers; 2. Simplifying the intention initiation process, allowing users to trigger via "gaze / voice / touch," reducing operational complexity.

[0165] The technical features disclosed in this application include: Step S3: Capturing key images or short videos containing external objects, encapsulating them and auxiliary information into a data packet, and sending it to the vehicle-mounted central processing unit via the wireless communication module of the wearable visual perception terminal. The technical problems solved by this feature include: 1. Existing technologies cannot transmit information about external objects of interest to the user; 2. Existing screen projection technologies have long and non-real-time data transmission links. The technical effects of this feature include: 1. Encapsulating key images / auxiliary information to ensure complete information transmission; 2. Real-time wireless transmission, shortening the data transmission time from the terminal to the vehicle-mounted unit.

[0166] The technical features disclosed in this application include: Step S4: The vehicle-mounted central processing unit receives and parses data packets, analyzes key images or short videos using its built-in image recognition function, and identifies the specific type and identification information of external objects. The technical problems solved by this feature include: 1. Existing technologies cannot identify specific information about external objects of interest to users; 2. Existing screen projection technologies lack image recognition capabilities and cannot associate object attributes. The technical effects of this feature include: 1. Accurately identifying object types (such as "clock tower" or "restaurant") and identification (such as "city historical clock tower"); 2. Providing clear keywords for subsequent information retrieval, solving the problem of "not knowing what the object is".

[0167] The technical features disclosed in this application include: Step S5: The vehicle-mounted central processing unit uses the identification information of external objects as keywords to query the cloud information service platform or search the local database through the vehicle network to obtain relevant information about the external objects (including at least two of the following: text description, high-definition images, related videos, user reviews, and navigation coordinates), and intelligently integrates the relevant information into interactive content adapted to the multi-person sharing scenario in the cockpit. The technical problems solved by this feature include: 1. Existing technologies cannot obtain object-related information; 2. Existing technologies lack the ability to integrate content adapted for multi-person sharing. The technical effects of this feature include: 1. Obtaining multi-dimensional object information, avoiding information monotony; 2. Integrating into interactive content shared by multiple people, ensuring that all passengers can understand and view it.

[0168] The technical features disclosed in this application include: Step S6: The onboard central processing unit sends interactive content to the cockpit shared display device, which then displays the interactive content for all occupants to view. The technical problems solved by this feature include: 1. Existing technologies cannot display information to all occupants; 2. Existing technologies lack a content transfer process between the processing unit and the display device. The technical effects of this feature include: 1. Completing the closed loop of "information generation-display," ensuring that all occupants simultaneously obtain information; 2. Enabling cockpit collaboration and avoiding the limitation of information being visible only to individuals.

[0169] In this embodiment, it includes:

[0170] The preset triggering conditions in step S2 also include at least one of the following: the user makes a preset head posture, the user executes a preset eye movement mode, or the wearable visual perception terminal's biosensor detects that the user's interest level has increased beyond a preset threshold.

[0171] Preset head postures include nodding in confirmation;

[0172] The preset eye-tracking patterns include blinking twice rapidly.

[0173] Specifically, the technical features disclosed in this application include: preset triggering conditions that include the user making a preset head posture (such as nodding to confirm). The technical problems solved by this feature include: existing triggering conditions only cover gaze, voice, and touch, lacking head posture triggering and having insufficient adaptability (e.g., inability to touch when hands are busy). The technical effects of this feature include: 1. providing users with a contactless triggering method, adapting to scenarios where hands are busy (e.g., holding objects); 2. increasing the diversity of triggering methods and improving operational convenience.

[0174] The technical features disclosed in this application include: preset triggering conditions involving the user executing a preset eye-tracking pattern (such as blinking twice rapidly). The technical problems solved by this feature include: existing technologies do not mention eye-tracking pattern triggering, thus failing to meet the needs of purely "eye-based operation" (such as when voice is inconvenient). The technical effects of this feature include: 1. Achieving pure eye-tracking triggering, requiring no hand or voice intervention, adapting to quiet scenarios (such as a sleeping baby); 2. Further simplifying operation, conforming to the core logic of "eye-tracking interaction".

[0175] The technical features disclosed in this application include: preset triggering conditions include the wearable visual sensing terminal's biosensors detecting a significant increase in user interest. The technical problems solved by this feature include: existing technologies require active user triggering, cannot passively identify potential user interests, and may miss information that the user is interested in but has not actively initiated. The technical effects of this feature include: 1. passively identifying user interests (such as changes in heart rate, pupil dilation reflecting increased interest), and actively initiating information sharing; 2. improving system intelligence, eliminating the need for manual user triggering, and enhancing the user experience.

[0176] In this embodiment, it includes:

[0177] The image recognition and acquisition of relevant information in steps S4 to S5 include:

[0178] The onboard central processing unit is the main execution entity to complete all tasks;

[0179] After image recognition and information aggregation are completed by the cloud-based information service platform, the results are sent to the vehicle-mounted central processing unit.

[0180] Alternatively, after simple image recognition is performed by a wearable visual sensing terminal, the preliminary results are sent to the vehicle's central processing unit for further information processing.

[0181] Specifically, the technical features disclosed in this application include: image recognition and information processing tasks are primarily performed by an onboard central processing unit. This technical feature addresses the following technical problems: existing technologies do not clearly define the task execution entity; if entirely reliant on the cloud, tasks may fail to be processed when the network is poor. The technical effects of this feature include: 1. Processing can be completed independently even with poor network conditions, ensuring uninterrupted task execution; 2. Reduced cloud reliance, lower data transmission volume, and reduced latency.

[0182] The technical features disclosed in this application include: image recognition and information processing tasks are primarily executed by a cloud-based information service platform, and the results are sent to the vehicle's central processing unit upon completion. This technical feature addresses the following technical problems: existing technologies do not mention a cloud-based processing method; if the vehicle's computing power is insufficient, it may be unable to efficiently process complex tasks (such as high-definition image recognition). The technical effects of this feature include: 1. Utilizing the powerful computing power of the cloud to process complex tasks, improving recognition accuracy and efficiency; 2. Reducing the computing burden on the vehicle's unit, adapting to vehicle models with weaker computing power.

[0183] The technical features disclosed in this application include: after simple image recognition is performed by a wearable visual sensing terminal, the preliminary results are sent to the vehicle-mounted central processing unit for subsequent processing. The technical problems solved by this feature include: existing technologies do not address terminal-vehicle collaborative processing; if the terminal has no processing capabilities, it may increase data transmission volume (e.g., sending raw high-definition images). The technical effects of this feature include: 1. Terminal preprocessing reduces the amount of transmitted data (e.g., sending preliminary recognition results instead of raw images), reducing latency; 2. Achieving collaboration between "simple terminal processing + complex vehicle processing," balancing efficiency and computing power.

[0184] In this embodiment, it includes:

[0185] Step S6 also includes: triggering further operations on the interactive content by touching the cockpit shared display device or issuing a voice command;

[0186] Further operations include controlling the cockpit shared display device to play detailed videos from relevant information;

[0187] It also includes instructing the onboard central processing unit to set the location corresponding to an external object as the navigation target.

[0188] Specifically, the technical features disclosed in this application include: vehicle occupants can trigger further operations on interactive content by touching the shared cockpit display device or issuing voice commands. This technical feature solves the following technical problems: existing technologies only mention "displayed content" without a subsequent interactive operation mechanism, preventing occupants from gaining in-depth information. The technical effects of this feature include: 1. Providing an active interaction entry point, allowing occupants to trigger further operations as needed, enhancing engagement; 2. Adapting to different scenarios (e.g., using voice while driving, and touch while parking), balancing safety and convenience.

[0189] The technical features disclosed in this application include: further operations include controlling the cockpit shared display device to play detailed videos of relevant information. The technical problems solved by this feature include: existing technologies lack video playback operations, have a limited content presentation format, and cannot meet the needs for in-depth information. The technical effects of this feature include: 1. Presenting richer details (such as dynamic introductions to landmarks) through video, improving information comprehension; 2. Enhancing the enjoyment of the scene (such as watching scenic spot videos while traveling), enriching the travel experience.

[0190] The technical features disclosed in this application include: a further operation that instructs the vehicle's central processing unit to set the location corresponding to an external object as the navigation target. The technical problems solved by this feature include: existing technologies lack navigation-related operations, requiring users to manually input navigation addresses, which is cumbersome and distracting. The technical effects of this feature include: 1. Achieving one-click triggering of "viewing-navigation," shortening the operation path; 2. Avoiding the safety risks of manually inputting navigation while driving, improving driving safety.

[0191] In this embodiment, it includes:

[0192] The wearable visual sensing terminal in step S1 can be any one of smart glasses, smart headphones with a camera, AR contact lenses, smart bracelets with pointing function, or smartphones.

[0193] The auxiliary information in step S3 includes at least one of the following: user gaze focus time information, vehicle current geographical location information, and user identification information.

[0194] Specifically, the technical features disclosed in this application include: the wearable visual perception terminal in step S1 is any one of smart glasses, smart headphones with a camera, AR contact lenses, a smart bracelet with pointing function, or a smartphone. The technical problems solved by this feature include: existing technologies only default to smart glasses, resulting in a single device form and inability to adapt to different user habits. The technical effects of this feature include: 1. Covering diverse device forms to meet different user preferences (such as using headphones or bracelets); 2. Improving system adaptability, allowing any occupant in the vehicle to choose a familiar device.

[0195] The technical features disclosed in this application include: the auxiliary information in step S3 includes user gaze focus time information. The technical problems solved by this feature include: the prior art does not clearly define the type of auxiliary information, which may lead to the vehicle unit being unable to determine the timeliness of the information. The technical effects of this feature include: 1. helping the vehicle unit obtain the "gaze time" and match the real-time status of the object (such as whether a store is open); 2. improving the timeliness of information retrieval and avoiding outdated content.

[0196] The technical features disclosed in this application include: the auxiliary information in step S3 includes the vehicle's current geographical location information. The technical problems solved by this feature include: the lack of geographical information in existing technologies, which may lead to confusion between objects with the same name (such as "parks" in different cities). The technical effects of this feature include: 1. accurately locating the object's position, avoiding information confusion; 2. providing a geographical coordinate basis for navigation associations (such as setting a destination).

[0197] The technical features disclosed in this application include: the auxiliary information in step S3 includes user identification information. The technical problems solved by this feature include: existing technologies do not distinguish user identities and cannot customize personalized content. The technical effects of this feature include: 1. Adjusting the content format (e.g., simplified / detailed) according to user identity (e.g., child, adult); 2. Enhancing the personalization of content and meeting the needs of different members.

[0198] It is worth noting that although this system / device only discloses the wearable visual perception terminal, the vehicle central processing unit, and the cockpit shared display device mentioned above, it does not mean that this system / device is limited to the above basic functional modules. On the contrary, what this invention intends to express is that, based on the above basic functional modules, those skilled in the art can add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. It should not be assumed that the scope of protection of the claims of this invention is limited to the above-disclosed basic functional modules just because this embodiment only discloses a few basic functional modules.

[0199] Figure 3 This is a structural block diagram of an electronic device for a cockpit information sharing method based on wearable device eye tracking, provided by one or more embodiments of the present invention.

[0200] like Figure 3 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0201] The memory stores a computer program that, when executed by a processor, causes the processor to perform steps of a cockpit information sharing method based on eye tracking of a wearable device.

[0202] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform steps of a cockpit information sharing method based on wearable device eye tracking.

[0203] This application also provides a vehicle cabin, including:

[0204] Electronic devices, steps for implementing a cockpit information sharing method based on wearable device eye tracking;

[0205] The processor runs a program that, when running, executes steps of a cockpit information sharing method based on wearable device eye tracking from data output by the electronic device.

[0206] Storage medium for storing programs that, when running, execute steps of a cockpit information sharing method based on wearable device eye tracking on data output from electronic devices.

[0207] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0208] The electronic device comprises a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory. The operating system can be any one or more computer operating systems that control the electronic device through processes, such as Linux, Unix, Android, iOS, or Windows. Furthermore, in this embodiment of the invention, the electronic device can be a smartphone, tablet computer, or other handheld device, or a desktop computer, portable computer, or other electronic device; there is no particular limitation in this embodiment.

[0209] In this embodiment of the invention, the executing entity for electronic device control can be an electronic device itself, or a functional module within an electronic device capable of calling and executing a program. The electronic device can obtain the firmware corresponding to the storage medium. This firmware is provided by the supplier, and different storage media may have the same or different firmware; no limitation is made here. After obtaining the firmware corresponding to the storage medium, the electronic device can write this firmware into the storage medium; specifically, it burns the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented using existing technology, and will not be elaborated upon in this embodiment of the invention.

[0210] Electronic devices can also obtain reset commands corresponding to the storage media. The reset commands corresponding to the storage media are provided by the supplier. The reset commands corresponding to different storage media can be the same or different, and no restrictions are imposed here.

[0211] At this time, the storage medium of the electronic device is a storage medium on which the corresponding firmware has been written. The electronic device can respond to the reset command corresponding to the storage medium on which the corresponding firmware has been written, thereby resetting the storage medium on which the corresponding firmware has been written according to the reset command. The process of resetting the storage medium according to the reset command can be implemented by existing technology and will not be described in detail in this embodiment of the invention.

[0212] For ease of description, the above devices are described separately by function as various units and modules. Of course, in implementing this application, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0213] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0214] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0215] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cockpit information sharing system based on wearable device eye tracking, characterized in that, The cockpit information sharing system based on wearable device eye tracking includes: Wearable visual sensing terminals, in-vehicle central processing units, and cockpit shared display devices; The vehicle's central processing unit is connected to the wearable visual sensing terminal and the cockpit shared display device for data transmission. Among them, the wearable visual perception terminal has a built-in camera, eye-tracking sensor and wireless communication module, which is used to continuously capture the user's first-person perspective image and accurately lock the user's gaze on the first-person perspective image. Simultaneously determine whether the user intends to share information; If so, then capture key images or short videos of the external objects corresponding to the focal point, encapsulate them with auxiliary information into a data packet, and send it through the wireless communication module; The vehicle-mounted central processing unit is equipped with image recognition, data parsing, and multi-source information aggregation computing capabilities, as well as vehicle network connectivity, and is used to receive and parse the data packets. The external object is identified using preset image recognition, data parsing, and multi-source information aggregation computing capabilities, and the recognition result is output. Using the recognition results as keywords, and leveraging the vehicle network connectivity, relevant information about the external object can be retrieved by querying a cloud-based information service platform or searching a local database. Then, integrate the relevant information into interactive content adapted for multi-person shared cockpit scenarios; The interactive content is an interactive card or a scrollable interactive page, which is compatible with multi-screen synchronous display on cockpit shared display devices and supports further operations triggered by in-vehicle occupants. The cockpit shared display device includes at least one of the following: in-vehicle central control screen, passenger entertainment screen, and rear entertainment screen; The cockpit-shared display device supports touch or voice interaction. The further operations include playing a detailed video from the relevant information and setting the location corresponding to the external object as a navigation target; A cockpit-shared display device is used to receive and display the interactive content and output it to the occupants of the vehicle.

2. The cockpit information sharing system based on wearable device eye tracking according to claim 1, characterized in that, include: The wearable visual sensing terminal can be any one of the following: smart glasses, smart headphones with a camera, AR contact lenses, smart bracelets with pointing function, or smartphones.

3. The cockpit information sharing system based on wearable device eye tracking according to claim 1, characterized in that, include: The auxiliary information includes at least one of the following: the time when the user's gaze is focused on the external object, the vehicle's current geographical location information, and the user's identity information.

4. The cockpit information sharing system based on wearable device eye tracking according to claim 1, characterized in that, include: The local database of the vehicle-mounted central processing unit is used to store basic information about commonly used external objects; The local database is updated in real time or periodically with the cloud information service platform via the vehicle network.

5. A cockpit information sharing method based on eye tracking using wearable devices, characterized in that, The cockpit information sharing method based on wearable device eye tracking includes: Step S1: Eye tracking and scene capture; The wearable visual sensing terminal continuously captures the user's first-person perspective image through its built-in camera and uses a built-in eye-tracking sensor to lock the user's gaze on the first-person perspective image in real time. Step S2: Determine user intent; Continuously analyze user behavior data, and when a preset trigger condition is detected, determine that the user has the intention to share information; The preset triggering conditions include at least one of the following: the user's gaze lingers on the external object corresponding to the focal point for a duration exceeding a set threshold; the user issues a preset voice command while looking at the external object; or the user performs a preset touch operation on the wearable visual sensing terminal. Step S3: Data packet encapsulation and transmission; Capture key images or short videos of the external objects and encapsulate them with auxiliary information into a data packet; Data packets are sent to the vehicle-mounted central processing unit via the wireless communication module of the wearable visual sensing terminal. Step S4: Information analysis and target recognition; The vehicle-mounted central processing unit receives and parses the data packet, and analyzes the key image or short video through its built-in image recognition function to identify the specific type and identification information of the external object. Step S5: Information aggregation and content generation; The vehicle-mounted central processing unit uses the identification information of the external object as keywords to query the cloud information service platform or search the local database through the vehicle network to obtain relevant information about the external object, including at least two of the following: text description, high-definition pictures, related videos, user reviews, and navigation coordinates. The relevant information of the external objects is intelligently integrated into interactive content that is adapted to the multi-person shared cockpit scenario; The interactive content is an interactive card or a scrollable interactive page, which is compatible with multi-screen synchronous display on cockpit shared display devices and supports further operations triggered by in-vehicle occupants. The cockpit shared display device includes at least one of the following: in-vehicle central control screen, passenger entertainment screen, and rear entertainment screen; The cockpit-shared display device supports touch or voice interaction. The further operations include playing a detailed video from the relevant information and setting the location corresponding to the external object as a navigation target; S6: Shared display and collaborative interaction; The vehicle-mounted central processing unit sends the interactive content to the cockpit shared display device, which then displays the interactive content and outputs it to the occupants of the vehicle.

6. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory stores a computer program that, when executed by a processor, causes the processor to perform the steps of the cockpit information sharing method based on wearable device eye tracking as described in claim 5.

7. A computer-readable storage medium, characterized in that, The device stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the cockpit information sharing method based on wearable device eye tracking as described in claim 5.

8. A vehicle cabin, characterized in that, include: An electronic device for implementing the steps of the cockpit information sharing method based on wearable device eye tracking as described in claim 5; The processor runs a program that, when running, executes the steps of the cockpit information sharing method based on wearable device eye tracking as described in claim 5 from data output by the electronic device. A storage medium for storing a program that, when running, performs the steps of the cockpit information sharing method based on wearable device eye tracking as described in claim 5 on data output from an electronic device.

Images