Sentinel mode system and method based on dual-domain cooperation of intelligent driving domain and cockpit domain

Abstract

The invention relates to a sentry mode system based on double-domain cooperation of an intelligent driving domain and a cockpit domain, and the system comprises an intelligent driving domain controller which is used for entering a dormant state to save power consumption, processing camera video data of a vehicle through a consistency algorithm, carrying out the fusion judgment through combining with the vibration detection of IMU data, recognizing an approaching or touch event, and sending the collision event to a cabin; meanwhile, video data in the time period of sending the approaching or touch event is used as an alarm video to be sent to a cockpit domain controller; the cabin domain controller is used for triggering an alarm strategy, determining a risk level, starting an alarm measure, storing the alarm video after receiving the alarm video, and prompting a user to check and display the alarm video; and the shared memory is used for storing the camera video data, the IMU data and the identification result written by the intelligent driving domain controller. The problems of resource waste, long response delay and single function in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, in particular to a sentinel mode system and method based on cooperation of a driving domain and a cabin domain. BACKGROUND

[0002] With the continuous improvement of the intelligent degree of automobiles, the sentinel mode, as an important function of vehicle safety monitoring, is increasingly valued by consumers. The sentinel mode is a vehicle protection mechanism that monitors the vehicle and the surrounding environment through various sensors and cameras on the vehicle body. When the vehicle is in sentinel mode, the vehicle owner can view the video information of the cameras around the vehicle body in real time through the mobile phone. If the vehicle is hit or moved, the external camera will record the surrounding situation and notify the owner through the mobile phone APP or SMS. For example, vehicles of brands such as Tesla, GAC Aion, and Geely Galaxy E5 are equipped with this function. Traditional sentinel mode usually adopts an independent system or is implemented based on a single domain controller, which has the limitations of low resource utilization, long response delay, and single function. The current main technical solutions have the following problems: First, the system separation architecture causes resources to be unable to be shared, such as the driving domain and the cabin domain being deployed in different ECUs, and data needing to be transmitted through a physical bus, which has low transmission rate and is susceptible to electromagnetic interference. Second, the power consumption control is not good, and the traditional solution needs to keep multiple system components running all the time, which is not conducive to vehicle endurance. Third, the event detection accuracy is insufficient, and only relying on a single sensor data source has a high false alarm rate. Finally, the user experience is not complete, and the event recording and playback functions are not intuitive and convenient. With the development of electronic and electrical architecture from distributed to centralized, the "cabin-driving integrated domain controller" has become an industry trend. This architecture can significantly improve system efficiency and performance through hardware resource sharing and software architecture simplification. However, there is no complete technical solution for how to use the cabin-driving integrated architecture to optimize the specific implementation of the sentinel mode. SUMMARY

[0003] In view of the above problems, the present application provides a sentinel mode system and method based on cooperation of a driving domain and a cabin domain, which solves the problems of resource waste, long response delay, and single function in the prior art.

[0004] The application provides a sentinel mode system based on cooperation of a driving intelligence domain and a cabin domain, applied to an intelligent driving vehicle, and the system comprises: a driving intelligence domain controller, which is used for entering a sleep state to save power consumption after activating the sentinel mode, processing camera video data of the vehicle through a consistency algorithm in a low-power consumption mode, and performing fusion judgment in combination with vibration detection of IMU data to identify a proximity or touch event, so that a cabin domain controller is woken up, and video data in a time period in which the proximity or touch event is sent is sent to the cabin domain controller as alarm video; the cabin domain controller is connected with the driving intelligence domain controller, and is used for triggering an alarm strategy after being woken up, determining a risk level according to an identification result, starting an alarm measure according to the risk level, storing the alarm video after receiving the alarm video, and prompting a user to view and display the alarm video after the user gets on the vehicle; and a shared memory is connected with the driving intelligence domain controller and the cabin domain controller respectively, and is used for storing camera video data, IMU data and an identification result written by the driving intelligence domain controller, so as to read the data by the cabin domain controller, and complete data exchange of the driving intelligence domain and the cabin domain.

[0005] Further, the camera is a four-way surround-view camera, and the resolution is not less than 5M.

[0006] The application further provides a sentinel mode method based on cooperation of a driving intelligence domain and a cabin domain, characterized in that the method comprises the following steps: step 1, after activating the sentinel mode, the cabin domain controller enters a sleep state to save power consumption; step 2, the driving intelligence domain controller processes camera video data of the vehicle through a consistency algorithm in a low-power consumption mode, performs fusion judgment in combination with vibration detection of IMU data, identifies a proximity or touch event, and writes the camera video data and an identification result into a shared memory; step 3, when the proximity or touch event is identified, the cabin domain controller is woken up, data in the shared memory is read, a multi-level alarm strategy is triggered, a risk level is determined, an alarm measure is started according to the risk level, and meanwhile, the driving intelligence domain controller sends video data in a time period in which the proximity or touch event occurs to the cabin domain controller as alarm video; and step 4, after a user gets on the vehicle, the cabin domain controller prompts the user to view and display the alarm video.

[0007] Further, the step 2 comprises: step 21, continuously analyzing four video streams by data obtained by the ring camera, identifying and tracking moving objects in the picture, and estimating the relative distance and motion trend of the objects with the vehicle by comparing the pixel size changes of the objects in consecutive frames; step 22, when the vehicle has physical contact, the IMU obtains the acceleration and angular velocity changes generated thereby, and preliminarily judges the strength and approximate direction of the touch by analyzing the frequency and amplitude of the vibration signals; step 23, in the sentinel mode, the ultrasonic radar measures the absolute distance from the surrounding obstacles by emitting and receiving ultrasonic waves, so as to monitor the visual blind area of the vehicle and verify whether the events triggered by vision or IMU are real at an extremely close distance; step 24, fusing the judgment results of various sensors, identifying the approaching or touching event, and writing the camera video data and identification results into the shared memory.

[0008] Further, the step 3 comprises: step 31, when the approaching or touching event is identified, the intelligent driving domain controller sends the alarm video to the cabin domain controller, and the gateway or the vehicle controller sends an interrupt signal to wake up the cabin domain controller; step 32, after the cabin domain controller is woken up, the data in the shared memory is read, and the risk level is determined according to the fusion judgment result; step 33, when the risk level is low, the vehicle only flashes the parking light or sends a remote notification to the owner; step 34, when the risk level is medium or high, the audible and visual alarm is started; step 35, when the risk level is high, the maximum strength audible and visual alarm is triggered, the alarm video is uploaded to the cloud through the cabin domain, and a remote emergency reminder is sent to the owner at the same time.

[0009] Further, the method further comprises: step 5, dynamically calculating a recommendation index according to the historical setting preferences of the user, the environment in which the vehicle is located, and historical event data, so as to intelligently suggest the user to start the sentinel mode with which sensitivity this time, wherein the historical setting preferences of the user, the environment in which the vehicle is located, and the historical event data are parameters for calculating the recommendation index.

[0010] Further, the environment in which the vehicle is located comprises geographical location information, time information, and environmental dynamic and static degree.

[0011] Further, the step 5 further comprises: before calculating the recommendation index, assigning weights to the parameters, normalizing different dimension parameters, after calculating the recommendation index, projecting the recommendation index to the corresponding mode suggestion.

[0012] Further, the step 1 further comprises: the intelligent driving domain enters a monitoring state, only maintains the minimum necessary computing power, and adopts power gating and clock gating technologies to reduce power consumption.

[0013] Further, the step 3 further includes: the cabin domain controller saves the alarm video by using H.265 encoding compression technology, so as to reduce the storage space occupation while ensuring the quality.

[0014] The application provides a sentinel mode system and method based on cooperation of intelligent driving and cabin domains, to solve the technical problems that the intelligent driving and cabin domain resources in the prior art cannot be shared, a plurality of system components need to be kept running all the time, which is not conducive to vehicle endurance, only a single sensor data source is relied on, the false alarm rate is high, the user experience is incomplete, the event recording and playback function is not intuitive and convenient, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 A method flow chart of the application is provided. Figure 2 A method flow chart of the application is provided. Figure 3 A method flow chart of the application is provided. Figure 4 A method flow chart of the application is provided. DETAILED DESCRIPTION

[0016] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0017] Embodiment one: The application provides a sentinel mode system and method based on cooperation of intelligent driving and cabin domains, which comprises an intelligent driving domain controller, a cabin domain controller and a shared memory. Figure 1 As shown, the method comprises the following steps.

[0018] Step 1: After activating the sentinel mode, the cabin domain controller enters a sleep state to save power consumption; Step 2: The intelligent driving domain controller processes the camera video data of the vehicle in a low-power mode by using a consistency algorithm, and combines vibration detection of IMU data for fusion judgment, to identify a proximity or touch event, and writes the camera video data and the identification result into the shared memory; The shared memory is connected with the intelligent driving domain controller and the cockpit domain controller respectively, and is used for storing camera video data, IMU data and identification results written by the intelligent driving domain controller, so as to be read by the cockpit domain controller to complete data exchange between the intelligent driving domain and the cockpit domain. The intelligent driving domain controller and the cockpit domain controller can be integrated in the same high-performance computing chip (such as NVIDIA Thor platform or Qualcomm Snapdragon Ride Flex platform, which provides 800-1000 TOPS of computing power, sufficient to support complex visual analysis and AI inference tasks), and data exchange is performed through shared memory, greatly improving communication efficiency. Data exchange between the two domains is performed through shared memory, replacing traditional physical bus communication. Specifically, the intelligent driving domain system writes the processing result data into the dynamic random access memory, and the cockpit domain system can directly read these data, realizing nanosecond-level data transmission speed, which is much faster than the millisecond-level speed of traditional bus communication. Since the driving domain and the cockpit domain share the same computing resources within the chip, the computing power can be dynamically allocated according to the task requirements. During the activation of the sentinel mode, the system prioritizes the visual processing needs of the intelligent driving domain; when an alarm is needed, resources are temporarily allocated to the alarm function of the cockpit domain.

[0019] Step 3, when the approach or touch event is identified, the cockpit domain controller is awakened, the data in the shared memory is read, the multi-level alarm strategy is triggered, the risk level is determined, the alarm measures are started according to the risk level, and at the same time, the intelligent driving domain controller sends the video data in the time period when the approach or touch event occurs as alarm video to the cockpit domain controller. The intelligent driving domain controller is used to enter a sleep state to save power consumption after the sentinel mode is activated, and in the low-power mode, the camera video data of the vehicle is processed through a consistency algorithm, and the vibration detection of the IMU data is combined for fusion judgment to identify the approach or touch event, so that the cockpit domain controller is awakened, and at the same time, the video data in the time period when the approach or touch event is sent is sent as alarm video to the cockpit domain controller. The camera is usually a four-way surround camera with a resolution of not less than 5M to meet the monitoring needs in low-illumination environments. The IMU is used to detect vehicle vibration and attitude change, and the visual analysis result is fused for judgment to improve the event detection accuracy.

[0020] Step 4, after the user gets on the vehicle, the cockpit domain controller prompts the user to view and display the alarm video.

[0021] The cockpit domain controller, connected to the intelligent driving domain controller, is used to trigger alarm strategies upon being woken up. It determines the risk level based on the identification results and initiates alarm measures accordingly. Upon receiving alarm video, it stores it and prompts the user to view and displays the alarm video after the user enters the vehicle. The cockpit domain controller is responsible for providing the user interface, alarm video storage, and alarm control. When the intelligent driving domain controller detects a potential threat, it is woken up via an interrupt signal, triggering alarm devices (such as horn blaring and flashing lights) and simultaneously storing the event video data in non-volatile memory.

[0022] This invention provides a Sentinel Mode system and method based on dual-domain collaboration between the intelligent driving domain and the cockpit domain, in order to solve the technical problems of existing technologies, such as the inability to share resources between the intelligent driving domain and the cockpit domain, the need to keep multiple system components running at all times, which is not conducive to vehicle range, the reliance on a single sensor data source, the high false alarm rate, the incomplete user experience, and the lack of intuitive and convenient event recording and playback functions.

[0023] Example 2: This invention provides a method for a sentinel mode system based on dual-domain collaboration between the intelligent driving domain and the cockpit domain, such as... Figure 1 As shown, the method includes the following steps.

[0024] Step 1: After activating Sentinel Mode, the cockpit domain controller enters sleep mode to save power. After the vehicle is powered off and locked, the user can activate Sentry Mode via the cockpit interface or mobile app. The system initializes the intelligent driving domain controller, configures the surround-view camera and IMU sensor, and the cockpit domain controller enters sleep mode to save power. The intelligent driving domain controller enters monitoring mode, maintaining only the minimum necessary computing power, and employs power gating and clock gating technologies to reduce power consumption. When an event is detected, the cockpit domain controller is woken up via a dedicated hardware interrupt line to ensure rapid response.

[0025] Step 2: In low-power mode, the intelligent driving domain controller processes the vehicle's camera video data through a consensus algorithm and combines it with vibration detection of IMU data for fusion judgment to identify approach or touch events, and writes the camera video data and recognition results into shared memory. Once Sentinel Mode is activated, the Intelligent Driving Domain Controller monitors the surrounding environment based on 4-channel surround-view video, 12-channel ultrasonic radar data, and IMU inertial navigation data. In low-power mode, the Intelligent Driving Domain Controller continuously analyzes the surround-view video and IMU data. Video analysis employs an improved consensus algorithm based on T-tests to monitor the vehicle's surroundings, combined with IMU vibration detection, to accurately identify proximity or touch events, such as... Figure 2 As shown, step 2 includes: Step 21, through the data obtained by the ring camera, continuously analyze the four-way video stream, identify and track the moving objects in the picture, estimate its relative distance and motion trend with the vehicle by comparing the pixel size changes of the object in the continuous frames; This step detects potential proximity events through visual analysis. The algorithm continuously analyzes the four-way video stream, identifies and tracks moving objects (such as pedestrians, vehicles) in the picture. By comparing the pixel size changes of the object in the continuous frames, the relative distance and motion trend of the object with the vehicle can be estimated. For example, when the pixel size of an object continues to increase, it usually means that it is approaching. The advantage of visual analysis is that it can identify object types and intentions, but it is susceptible to light and weather.

[0026] Step 22, when the vehicle is physically contacted, the IMU obtains the acceleration and angular velocity changes generated thereby, and estimates the strength and approximate direction of the touch by analyzing the frequency and amplitude of these vibration signals; This step detects touch events through vibration analysis. When the vehicle is physically contacted, such as being scratched, hit or slapped, the IMU can sensitively capture the tiny acceleration and angular velocity changes (i.e. vehicle vibration) generated thereby. By analyzing the frequency and amplitude of these vibration signals, the algorithm can preliminarily judge the strength and approximate direction of the touch. The advantage of IMU is that it is not affected by visual conditions, but it is powerless for non-contact proximity.

[0027] Step 23, in the sentry mode, the ultrasonic radar measures the absolute distance from the surrounding obstacles by emitting and receiving ultrasonic waves, in order to monitor the visual blind area of the vehicle, and verify whether the event triggered by vision or IMU is real at very close range.

[0028] This step performs precise ranging at very close range through distance analysis. It measures the absolute distance from the surrounding obstacles by emitting and receiving ultrasonic waves, with an accuracy of centimeters. In the sentry mode, it is mainly used to monitor the visual blind area of the vehicle, such as the side and corner, and to verify whether the event triggered by vision or IMU is real at very close range.

[0029] Step 24, fuse the judgment results of each sensor, identify the proximity or touch event, and write the camera video data and identification results into shared memory.

[0030] The hardware architecture of the system of the application further comprises an ultrasonic radar (USS), an inertial measurement unit (IMU) and a vehicle alarm device in addition to the cabin domain controller, the intelligent driving domain controller and the multi-lane look-around camera. Since each sensor works independently and performs preliminary anomaly identification based on its characteristics, a multi-sensor fusion algorithm based on the look-around camera, the IMU and the ultrasonic radar is needed to complement the advantages of different sensors through a hierarchical and progressive fusion judgment architecture to achieve high-precision and high-reliability vehicle surrounding environment monitoring. The essence of the algorithm is to process the data of different sensors in a hierarchical and phased manner to form a complete closed loop from preliminary judgment to comprehensive decision. The preliminary judgment results of all sensors are combined and a certain strategy is used to make high-level decisions, which is the "brain" of the fusion algorithm. The specific fusion decisions are shown in Table 1.

[0031]

[0032] Table 1 Through this multi-sensor fusion algorithm, the sentry mode has three advantages: accurate perception: fusion of visual, vibration and distance information, cross verification, significantly reducing false positives and false negatives. Context understanding: can distinguish between different scenarios such as "someone passing by", "minor scratching" and "malicious damage", and realize intelligent response. Full-time and full-area: combining the characteristics of different sensors, making up for the limitations of a single sensor, achieving 24-hour reliable monitoring in various lighting and weather conditions.

[0033] Step 3, when the approach or touch event is identified, the cabin domain controller is awakened to read the data in the shared memory, trigger the multi-level alarm strategy, determine the risk level, and start the alarm measures according to the risk level. At the same time, the intelligent driving domain controller sends the video data in the time period when the approach or touch event occurs to the cabin domain controller as alarm video; Through the multi-sensor fusion perception of step 2, the judgment result is obtained, and the system will start a progressive warning and response mechanism. That is, when the approach or touch event is detected, the intelligent driving domain controller wakes up the cabin domain controller for warning and horn alarm, and sends the event video to the cabin domain controller for storage, as shown in Figure 3 Step 3 includes: Step 31, when the approach or touch event is identified, the intelligent driving domain controller sends the alarm video to the cabin domain controller, and the gateway or vehicle controller sends an interrupt signal to wake up the cabin domain controller; When a potential threat is detected, the intelligent driving domain controller wakes up the cabin domain controller through an interrupt signal, triggering a multi-level alarm strategy. The intelligent driving domain controller marks and transmits the video data in a specific time period before and after the event, i.e. the alarm video, to the memory of the cabin domain controller. After receiving the alarm video, the cabin domain controller saves it using H.265 encoding compression technology, which reduces storage space occupation while ensuring quality.

[0034] Step 32, after the cabin domain controller is woken up, it reads the data in the shared memory and determines the risk level based on the fusion judgment result. According to the risk level, the system may start light warning, horn alarm and other measures in turn.

[0035] Step 33, when the risk level is low, the vehicle only flashes the parking light or sends a remote notification to the owner. Primary warning (low risk): When the fusion algorithm judges that it is a low-risk approaching event (such as someone staying for a short time at a certain distance), it may only warn by flashing the parking light or sending a remote notification to the owner's mobile phone APP, without triggering the horn, to avoid disturbing.

[0036] Step 34, when the risk level is medium or high, start sound and light alarm. Secondary alarm (medium or high risk): When it is confirmed that there is continuous approach or slight touch, the system will start sound and light alarm, such as flashing the headlight and accompanied by a short horn sound, to deter suspicious persons.

[0037] Step 35, when the risk level is high, trigger the maximum intensity sound and light alarm, upload the alarm video to the cloud through the cabin domain, and send a remote emergency reminder to the owner at the same time.

[0038] Top-level response (high risk): When strong impact or illegal intrusion is detected, the system will trigger the maximum intensity sound and light alarm, immediately upload the event video (including a period of time before and after the event) to the cloud through the cabin domain, and send an emergency reminder to the owner at the same time.

[0039] As can be seen, the software architecture of the method adopts a layered design: the hardware layer provides basic computing and transmission capabilities; the driver layer manages hardware resources; the operating system layer uses a real-time operating system (RTOS) combined with a general operating system; the middleware layer provides communication, data management and AI inference functions; and the application layer implements specific sentinel mode business logic.

[0040] Step 4, after the user gets on the vehicle, the cabin domain controller prompts the user to view and display the alarm video.

[0041] The application provides a sentinel mode system and method based on the cooperation of the intelligent driving domain and the cockpit domain, to solve the technical problems that the resources of the intelligent driving domain and the cockpit domain cannot be shared in the prior art, that multiple system components need to be kept running at all times, which is not conducive to vehicle endurance, that only a single sensor data source is relied on, that the false positive rate is high, that the user experience is incomplete, and that the event recording and playback functions are not intuitive and convenient.

[0042] Embodiment three: The application provides a method for a sentinel mode system based on the cooperation of the intelligent driving domain and the cockpit domain, as shown in Figure 4 The method comprises the following steps: Step 1: After activating the sentinel mode, the cockpit domain controller enters a sleep state to save power consumption; Step 2: The intelligent driving domain controller processes the camera video data of the vehicle in a low-power mode through a consistency algorithm, and combines vibration detection of IMU data for fusion judgment to identify a proximity or touch event, and writes the camera video data and the identification result into shared memory; Step 3: When the proximity or touch event is identified, the cockpit domain controller is awakened, the data in the shared memory is read, a multi-level alarm strategy is triggered, the risk level is determined, the alarm measures are started according to the risk level, and at the same time, the intelligent driving domain controller sends the video data in the time period when the proximity or touch event occurs to the cockpit domain controller as alarm video; Step 4: After the user gets on the vehicle, the cockpit domain controller prompts the user to view and display the alarm video.

[0043] Step 5: According to the historical setting preferences, the environment of the vehicle, and the historical event data, a recommended index is dynamically calculated to intelligently suggest the user to start the sentinel mode with which sensitivity this time, wherein the historical setting preferences, the environment of the vehicle, and the historical event data are parameters for calculating the recommended index.

[0044] According to the historical setting preferences of the user, the environment in which the vehicle is located, and historical event data, a "recommendation index" is dynamically calculated, thereby intelligently suggesting to the user which sensitivity level of the sentinel mode should be turned on this time, which is the core of the "personalized mode recommendation" function. The historical setting preferences are user habits, for example, whether the user often manually turns on or off the sentinel mode, and which sensitivity level of high, medium, or low is preferred. The environment in which the vehicle is located includes geographic location information, time information, and environmental activity level, which is the key to dynamic adjustment. Among them, the geographic location information: whether the vehicle is in a usually safe area (such as a home or company parking lot) or an unfamiliar public parking lot is determined through the geographic fence technology; the time information: whether it is daytime or nighttime, and a higher monitoring level is usually recommended at night; the environmental activity level: the degree of noise in the surrounding environment is determined through the baseline data of the sensor. The historical event data is the frequency of past alarm events triggered in a specific location or similar environment. The high-frequency triggering area will suggest a more cautious mode. The generation of the recommendation index is not a simple addition, but a fusion calculation through a dynamic weighting calculation model. The calculation of the comprehensive recommendation index can be abstracted as the following formula concept: recommendation index = F (user preference weight, environmental risk weight, historical event weight), where F represents the fusion function. In actual application, this may be manifested as: weight allocation: the model allocates dynamic weights to different types of parameters. For example, in a strange parking lot at night, the "environmental risk weight" will be significantly increased; and for a user who always manually turns on the highest mode, the "user preference weight" will dominate. Normalization processing: different dimension parameters (such as distance, times, preference level, etc.) are processed into comparable standardized values through algorithms. Comprehensive calculation and output: the model performs comprehensive calculation on the weighted parameters to finally generate a quantitative recommendation index. This index is mapped to a specific mode suggestion, for example: high recommendation index → strongly recommend turning on "high sensitivity mode"; medium recommendation index → recommend "standard mode" or "energy-saving mode"; low recommendation index → prompt "in a safe area, you can turn off or use a low-power mode". The system generates a sentinel mode recommendation index according to the user habit tag, the environment tag, and the group user habit tag, realizes intelligent mode recommendation, and improves user experience.

[0045] The application provides a sentinel mode system and method based on the cooperation of the intelligent driving domain and the cockpit domain, to solve the technical problems that the intelligent driving domain and the cockpit domain resources in the prior art cannot be shared, multiple system components need to be kept running all the time, which is not conducive to vehicle endurance, only a single sensor data source is relied on, the false positive rate is high, the user experience is incomplete, the event recording and playback function is not intuitive and convenient, and the like.

[0046] In summary, the application provides a sentinel mode system and method based on the cooperation of the intelligent driving domain and the cockpit domain, which realizes the dynamic allocation and reuse of hardware resources through the cooperation of the two domains, avoids repeated construction, reduces the system cost, and the intelligent driving domain runs with low power consumption, only wakes up the cockpit domain when an event occurs, and significantly reduces the overall power consumption of the system. At the same time, the on-chip shared memory communication (ns level) is used instead of physical bus communication (ms level), which greatly improves the event response speed, and can also combine visual analysis (surrounding video) and physical vibration detection (IMU) to reduce the false positive rate and the false negative rate, and provide complete event recording, intelligent alarm and intuitive playback function, and enhance the user's sense of security.

[0047] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A sentinel mode system based on dual-domain collaboration of intelligent driving domain and cockpit domain, applied to intelligent driving vehicles, characterized in that, The system includes: The intelligent driving domain controller is used to enter a sleep state to save power after the sentry mode is activated. In low power mode, it processes the vehicle's camera video data through a consensus algorithm and combines it with vibration detection of IMU data for fusion judgment to identify proximity or touch events, which wakes up the cockpit domain controller. At the same time, it sends the video data within the time period of the proximity or touch event to the cockpit domain controller as alarm video. The cockpit domain controller, connected to the intelligent driving domain controller, is used to trigger alarm strategies after being woken up, determine the risk level based on the identification results, and activate alarm measures according to the risk level. After receiving the alarm video, it is stored and the user is prompted to view and display the alarm video after getting into the vehicle. The shared memory is connected to both the intelligent driving domain controller and the cockpit domain controller. It is used to store the camera video data, IMU data and recognition results written by the intelligent driving domain controller, so that the cockpit domain controller can read them and complete the data exchange between the intelligent driving domain and the cockpit domain.

2. The Sentinel Mode system based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 1, characterized in that, The camera is a four-way surround-view camera with a resolution of no less than 5M.

3. A method for a sentry mode system based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in any one of claims 1-2, characterized in that, The method includes: Step 1: After activating Sentinel Mode, the cockpit domain controller enters sleep mode to save power. Step 2: In low-power mode, the intelligent driving domain controller processes the vehicle's camera video data through a consensus algorithm and combines it with vibration detection of IMU data for fusion judgment to identify approach or touch events, and writes the camera video data and recognition results into shared memory. Step 3: When an approach or touch event is detected, the cockpit domain controller is woken up, reads the data in the shared memory, triggers a multi-level alarm strategy, determines the risk level, and starts alarm measures according to the risk level. At the same time, the intelligent driving domain controller sends the video data during the time period of the approach or touch event to the cockpit domain controller as alarm video. Step 4: After the user gets into the vehicle, the cockpit domain controller prompts the user to view and displays the alarm video.

4. The sentinel mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 3, characterized in that, Step 2 includes: Step 21: Analyze the data obtained from the surround-view camera to continuously analyze the four video streams, identify and track moving objects in the scene, and estimate the relative distance and motion trend of the object to the vehicle by comparing the pixel size changes of the object in consecutive frames. Step 22: When the vehicles make physical contact, the IMU acquires the resulting changes in acceleration and angular velocity. By analyzing the frequency and amplitude of these vibration signals, the intensity and approximate location of the contact are preliminarily determined. Step 23: In Sentry Mode, the ultrasonic radar measures the absolute distance to surrounding obstacles by transmitting and receiving ultrasonic waves in order to monitor the vehicle's blind spots and verify whether visual or IMU-triggered events actually exist at extremely close range. Step 24: Combine the judgment results of each sensor to identify the approach or touch event, and write the camera video data and recognition results into the shared memory.

5. The sentinel mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 3 or 4, characterized in that, Step 3 includes: Step 31: When an approach or touch event is detected, the intelligent driving domain controller sends an alarm video to the cockpit domain controller, and at the same time, the gateway or vehicle controller sends an interrupt signal to wake up the cockpit domain controller. Step 32: After the cockpit domain controller is woken up, it reads the data in the shared memory and determines the risk level based on the result of the fusion judgment. Step 33: When the risk level is low, the vehicle will only flash its parking lights or send a remote notification to the owner. Step 34: When the risk level is medium to high, activate the audible and visual alarm; Step 35: When the risk level is high, trigger the maximum intensity audible and visual alarm, simultaneously upload the alarm video to the cloud via the cockpit domain, and send a remote emergency reminder to the vehicle owner.

6. The sentinel mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 3, characterized in that, The method further includes: Step 5: Based on the user's historical settings preferences, the vehicle's environment, and historical event data, a recommendation index is dynamically calculated to intelligently suggest to the user which sensitivity of Sentinel mode should be activated this time. The historical settings preferences, the vehicle's environment, and historical event data are the parameters for calculating the recommendation index.

7. The Sentinel Mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 6, characterized in that, The vehicle's environment includes geographical location information, time information, and the level of activity in the environment.

8. The Sentinel Mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 6, characterized in that, Step 5 further includes: before calculating the recommendation index, assigning weights to the parameters, normalizing the parameters of different dimensions, and after calculating the recommendation index, mapping the recommendation index to the corresponding pattern suggestions.

9. The sentinel mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 3, characterized in that, Step 1 further includes: the intelligent driving domain enters a monitoring state, maintaining only the minimum necessary computing power, and using power gating and clock gating technologies to reduce power consumption.

10. The sentinel mode method based on dual-domain collaboration of intelligent driving domain and cockpit domain as described in claim 3, characterized in that, Step 3 further includes: the cockpit domain controller uses H.265 encoding compression technology to save alarm videos, reducing storage space usage while ensuring quality.

Images