Intelligent cabin low-power-consumption sentry monitoring method and intelligent cabin

Abstract

The invention discloses a low-power-consumption sentry monitoring method for an intelligent cabin and the intelligent cabin, and relates to the technical field of intelligent automobiles. A system-on-chip of the intelligent cabin comprises a first core and a second core, and the operation power consumption of the second core is lower than that of the first core. The method comprises the steps that when a vehicle flames out, operation of the hard disk video recorder is stopped, a second core takes over the use permission of a camera outside the vehicle to collect images of the surrounding environment regularly, whether the surrounding environment changes or not is monitored, and after it is monitored that the surrounding environment changes, the second core is switched back to a first core to operate the hard disk video recorder; after the hard disk video recorder records the video of the preset duration, the hard disk video recorder is turned off, the first core is switched to the second core, whether the surrounding environment changes or not is monitored again, and power consumption of sentry mode operation after vehicle flameout is reduced.

Description

Technical Field

[0001] The present application relates to the field of smart car technology, and in particular to a method for low-power sentinel monitoring of a smart cockpit and a smart cockpit. Background Art

[0002] With the continuous advancement of smart car technology, cars can now use advanced technologies such as driving recorders and vehicle sensing systems to accurately sense and record potential dangerous factors around the vehicle and accident details such as collisions, providing traceable evidence for accidents. This function is vividly called "Sentinel Mode" in the field of smart cars.

[0003] Currently, in actual application, the continuous operation of Sentry Mode consumes a significant amount of the car's battery power, thereby reducing the car's range, and users tend to turn off this function.

[0004] However, turning off Sentry Mode because it consumes too much power may result in damage to the vehicle after it is parked, and you cannot be held accountable because Sentry Mode was not turned on, which may lead to economic losses. Summary of the invention

[0005] The present application provides a method for low-power sentinel monitoring of a smart cockpit and a smart cockpit, which are used to save energy when the sentinel mode is turned on.

[0006] In a first aspect, the present application provides a method for low-power sentinel monitoring of a smart cockpit, which is applied to a smart cockpit and is characterized in that the system-level chip of the smart cockpit includes a first core and a second core, and the operating power consumption of the second core is lower than that of the first core. The method comprises: after the vehicle is started, the hard disk recorder is operated by the first core so that the video captured by the external camera is stored; when the vehicle is turned off, the operation of the hard disk recorder is stopped, and the first core is switched to the second core to take over the use authority of the external camera, and the surrounding environment images are regularly collected; through the collected surrounding environment images, whether the surrounding environment has changed; when the surrounding environment is monitored to change, the second core is switched back to the first core to run the hard disk recorder; after the hard disk recorder records a video of a preset length, the steps of stopping the operation of the hard disk recorder when the vehicle is turned off, switching from the first core to the second core to take over the use authority of the external camera, and regularly collecting surrounding environment images are executed.

[0007] In the above embodiment, when the vehicle is turned off, when the surrounding environment has not changed, only the second core is used to take over the external camera to monitor the surrounding environment. After the surrounding environment changes are detected, the second core is switched to the second core and the hard disk recorder is woken up. After recording for a period of time, the hard disk recorder is turned off and switched back to the first core. This reduces the power consumption of the sentinel mode after the vehicle is turned off, and can also respond quickly when changes in the surrounding environment are detected.

[0008] In combination with some embodiments of the first aspect, in some embodiments, the surrounding environment is monitored for changes by collecting the surrounding environment images, specifically including: establishing an image queue, collecting surrounding environment images at a preset frequency, and storing them in the image queue; comparing each newly stored image with the previous image, and determining whether the surrounding environment has changed by whether the similarity between the previous and subsequent images is lower than a preset similarity threshold; if the similarity is lower than the preset similarity threshold, determining that the surrounding environment has changed; if the similarity is higher than or equal to the preset similarity threshold, determining that the surrounding environment has not changed.

[0009] In the above embodiment, each time an image of the surrounding environment is collected, the newly collected image is compared with the previous image, and changes in the surrounding environment are judged by the similarity between the previous and next images. When the surrounding environment changes, a timely response can be made, while avoiding the situation where the sentinel mode operation has high power consumption due to frequent responses due to slight changes in the surrounding environment.

[0010] In combination with some embodiments of the first aspect, in some embodiments, the surrounding environment is monitored for changes by collecting the surrounding environment images, specifically including: establishing an image queue, collecting surrounding environment images at a preset frequency, and storing them in the image queue; performing feature extraction on each newly stored image, and comparing them with the features of the previous image, if a new feature appears in the latter image, extracting the new feature separately to determine whether the new feature is a person, animal or vehicle; if the new feature is not a person, animal or vehicle, determining that the surrounding environment has not changed; if the new feature is a person or an animal, determining that the surrounding environment has changed; if the new feature is a vehicle, detecting whether the vehicle distance is less than a set safety threshold; if the vehicle distance is less than the set safety threshold, determining that the surrounding environment has changed.

[0011] In the above embodiment, by judging whether the newly added feature is the person, animal or vehicle that causes most accidents when the surrounding environment changes, the number of meaningless recordings of surrounding environment changes is effectively reduced, and the power consumption of sentinel mode operation after the vehicle is turned off is reduced.

[0012] In combination with some embodiments of the first aspect, in some embodiments, when a change in the surrounding environment is monitored, the second core switches back to the first core to run the hard disk recorder, specifically including: when a change in the surrounding environment is detected, the last picture in the image queue is retrieved to obtain the current environmental information of the vehicle, and determine whether it is currently in an outdoor environment; if it is not currently in an outdoor environment, whether the change in the surrounding environment is a dangerous change is determined by whether the changed object is close to the door, window or sunroof; if it is currently in an outdoor environment, whether the change in the surrounding environment is a dangerous change is determined by whether the change is close to the door, window or sunroof and whether the residence time of the changed object exceeds a set threshold; if the change in the surrounding environment is a dangerous change, the second core is immediately switched back to the first core to run the hard disk recorder.

[0013] In the above embodiment, by judging whether the changing object is close to the door, window or sunroof where accidents are prone to occur when the surrounding environment changes, and at the same time limiting the stay time of the changing object in the outdoor environment where there may be a large flow of people which may cause misjudgment, the number of meaningless responses to changes in the surrounding environment is effectively reduced, and the power consumption of the sentinel mode operation after the vehicle is turned off is reduced.

[0014] In combination with some embodiments of the first aspect, in some embodiments, the method further includes: responding to the vehicle glass sensor signal through the second core, monitoring whether the vehicle glass is broken when no changes in the surrounding environment are detected; if the glass is broken, immediately turning on the in-vehicle camera; when the in-vehicle camera is turned on, confirming whether there are safety hazards inside the vehicle by monitoring whether there are moving objects in the picture; if there are no safety hazards inside the vehicle, sending a prompt message to the owner's mobile terminal, the prompt message is used to remind the owner that the current vehicle glass is broken. If there are safety hazards inside the vehicle, the vehicle is controlled to turn on the harmless smoke release device, and the harmless smoke release device interferes with the intruder's sight and attracts the attention of people around by releasing harmless smoke, prompting the intruder to leave.

[0015] In the above embodiment, by monitoring the breakage of vehicle glass and detecting whether there are moving objects in the vehicle through the camera inside the vehicle, it is avoided that an intruder breaks the window and enters the vehicle to steal during the interval between two shots of the surrounding environment by the external camera at a certain frequency. While reducing power consumption, various abnormal situations that may occur after the vehicle is turned off are monitored, and after monitoring the existence of safety hazards inside the vehicle, the vehicle is promptly controlled to turn on the harmless smoke release device to prompt the intruder to leave.

[0016] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: when a change in the surrounding environment is monitored, sending an abnormal prompt information to the vehicle owner's mobile device, the abnormal prompt information is used to prompt the vehicle owner on the mobile device that the vehicle's surrounding environment is abnormal and display whether to check the abnormal environment controls around the vehicle; when a viewing instruction is returned by the mobile device, the video recorded by the hard disk recorder is transmitted to the mobile terminal in real time.

[0017] In the above embodiment, after an abnormal situation is detected, the car owner is notified in time and can view the abnormal situation of the vehicle's surrounding environment in real time on the mobile terminal. In case of an emergency, the car owner can be notified as soon as possible and be informed of the situation in time, thereby improving the user experience of the smart cockpit sentry mode.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the method also includes: determining the real-time remaining power of the vehicle, and if the remaining power is lower than a preset monitoring power threshold, sending a prompt message to the owner's mobile terminal, wherein the prompt message is used to prompt the owner that the current power is low and display whether to turn on the low-power monitoring mode control; when receiving an instruction to turn on low-power monitoring returned by the mobile terminal, stop monitoring whether the surrounding environment has changed, reduce the frequency of periodically collecting surrounding environment images, and store the collected surrounding environment images.

[0019] In the above embodiment, when the battery power is low, the car owner can be reminded to turn on the low power mode. By reducing the recording frequency while continuously recording the surrounding environment and no longer monitoring whether the surrounding environment has changed, the operating power consumption can be reduced as much as possible to save power.

[0020] In a second aspect, an embodiment of the present application provides a smart cockpit, which includes: one or more system-level chips and a memory; the system-level chip includes one or more processors; the memory is coupled to the system-level chip, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the mobile terminal to execute the method described in the first aspect and any possible implementation method of the first aspect.

[0021] In a third aspect, an embodiment of the present application provides a computer program product comprising instructions, and when the above-mentioned computer program product is run on a smart cockpit, the above-mentioned smart cockpit executes the method described in the first aspect and any possible implementation method of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] It can be understood that the smart cockpit provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the method for low-power sentinel monitoring of the smart cockpit provided in the embodiment of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, which will not be repeated here.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: 1. When the vehicle is turned off, the present application uses only the second core to take over the external camera to monitor the surrounding environment when the surrounding environment has not changed. After detecting changes in the surrounding environment, the application switches to the second core and wakes up the hard disk recorder. After recording for a period of time, the hard disk recorder is turned off and switched back to the first core. This reduces the power consumption of the sentinel mode after the vehicle is turned off, and can also respond quickly when changes in the surrounding environment are detected.

[0025] 2. This application determines whether the changing object is close to the door, window or sunroof where accidents are prone to occur when the surrounding environment changes. At the same time, it limits the stay time of the changing object in the outdoor environment where there may be a large flow of people, which may cause misjudgment. This effectively reduces the number of meaningless responses to changes in the surrounding environment and reduces the power consumption of the sentinel mode after the vehicle is turned off.

[0026] 3. In the above embodiments of the present application, by monitoring the breakage of vehicle glass and detecting whether there are moving objects in the vehicle through the camera inside the vehicle, it avoids the situation where an intruder breaks the window and enters the vehicle to steal during the interval between two shots of the surrounding environment taken by the external camera at a certain frequency. While reducing power consumption, it monitors various abnormal situations that may occur after the vehicle is turned off. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a system architecture diagram of the low-power sentinel monitoring method for the intelligent cockpit of the present application.

[0028] Figure 2 It is a flow chart of the method for low-power sentinel monitoring of a smart cockpit provided in an embodiment of the present application.

[0029] Figure 3 It is another flow chart of the method for low-power sentinel monitoring of the smart cockpit provided in an embodiment of the present application.

[0030] Figure 4 It is a schematic diagram of the structure of a smart cockpit provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0033] The following is a description of the system architecture of the low-power sentinel monitoring of the intelligent cockpit involved in this application: like Figure 1 , which is a schematic diagram of a system architecture of the low-power sentinel monitoring method for a smart cockpit of the present application. The system includes a smart cockpit system-level chip 110, an external camera 101 and a hard disk recorder 103; wherein the smart cockpit system-level chip 110 includes a first core 111 and a second core 112.

[0034] The system-level chip 110 is a high-end chip that provides computing and processing capabilities for the smart cockpit. It has multiple high-performance cores built in, including a first core 111 and a second core 112. The first core 111 has high computing power and rich functional support, but its operating power consumption is relatively high; the second core 112 is designed to be in low-power mode, which can significantly reduce energy consumption while ensuring basic functions.

[0035] The vehicle exterior camera 101 is a camera installed outside the vehicle for photographing the surrounding environment.

[0036] The hard disk video recorder 103 is a digital video recording device that uses digital video recording technology and a video recording / playback device that uses a hard disk as a storage medium.

[0037] Both the first core 111 and the second core 112 are connected to the external camera 101 via a bus or interface inside the system-level chip; the first core 111 is connected to the hard disk recorder 103 via a bus or interface inside the system-level chip; when the first core 111 is turned on, the second core 112 is in charge of the use rights of the external camera 101 and the internal camera 102, and turns on the hard disk recorder 103; when the first core 111 is turned off, the second core 112 takes over the external camera 101, and the hard disk recorder 103 is turned off.

[0038] In some embodiments, the system further includes an in-car camera 102 and a harmless smoke release device 104. The in-car camera 102 is a camera installed inside the car to capture the interior environment of the car; the harmless smoke release device 104 is a device installed inside the car to release harmless gas; the first core 111 and the second core 112 are connected to the in-car camera 102 and the harmless smoke release device 104 through a bus or interface inside the system-level chip.

[0039] This application provides a method for low-power sentinel monitoring of a smart cockpit. After the vehicle is turned off, the smart cockpit only needs to run in a low-power mode to monitor whether the vehicle's surrounding environment has changed. If changes are detected in the surrounding environment, the high-power mode is switched to record, and after a certain period of time, the low-power mode is switched back to monitor the surrounding environment. This can reduce the power consumption of the sentinel mode after the vehicle is turned off, and can also respond quickly when changes in the surrounding environment are detected.

[0040] The following describes a method for low-power sentinel monitoring of a smart cockpit in one embodiment of the present application: like Figure 2 As shown, it is a flow chart of a method for low-power sentinel monitoring of an intelligent cockpit provided in an embodiment of the present application, which can be used for Figure 1 The system architecture shown, for a smart cockpit, includes the following steps: S201, after the vehicle is started, the hard disk recorder is operated by the first core so that the video captured by the camera outside the vehicle is stored; After the vehicle is started, the hard disk recorder must be run through the first core to monitor and record the dynamics outside the vehicle in real time to ensure driving safety.

[0041] In some embodiments of the present application, the external camera may be a 360° panoramic camera that can capture a 360° panoramic image around the vehicle body; in some embodiments, multiple external cameras may be installed to combine the images of multiple external cameras to create a panoramic image around the vehicle body, etc., which is not limited here.

[0042] In some embodiments, the first core is an ARM core. In a system-level chip, the ARM core serves as a core processor responsible for coordinating system operations, supporting the operating system, and executing applications. At the same time, it becomes a key component for the system-level chip to achieve high performance, low power consumption, and safe operation by virtue of its high energy efficiency, good scalability, and built-in security features. In some embodiments, the first core may also be an A7 core, which is a Cortex-A7 processor core and has good performance in energy efficiency. It is suitable for low-power devices and can be used as an independent processor, which is not limited here.

[0043] S202, when the vehicle is turned off, the operation of the hard disk video recorder is stopped, the first core is switched to the second core to take over the use authority of the external camera of the vehicle, and the surrounding environment images are collected regularly; Regularly collecting surrounding environment images means using an external camera to take pictures of the surrounding environment at regular intervals. The interval between pictures can be set by the vehicle owner or by the system default setting, and is not limited here.

[0044] In some embodiments, the second core is an R5 core, which refers to the ARM Cortex-R5 processor core, which is mainly used in system-level chips, emphasizes high performance and low latency, and is suitable for applications that require fast response and high reliability; in some embodiments, the second core can also be an A53 core, which is a Cortex-A53 processor core. It is widely used in mobile devices and Internet of Things terminals due to its high performance and low power consumption, and can provide excellent performance and energy efficiency, which is not limited here.

[0045] S203, monitoring whether the surrounding environment changes by collecting the surrounding environment image; If the surrounding environment changes, the following step S204 is executed; If no changes in the surrounding environment are detected, the above step S202 is executed; After the surrounding environment images are collected and stored in the image queue, each newly stored image is compared with the previous image to determine whether the surrounding environment has changed by checking whether the similarity between the previous and next images is lower than a preset similarity threshold.

[0046] In some embodiments of the present application, the similarity between the previous and next pictures is determined by extracting picture feature points, and the preset threshold is the number of different feature points set in advance. If the number of different feature points between the previous and next pictures is lower than the number of different feature points set in advance, it is determined that the surrounding environment has not changed; if the number of different feature points between the previous and next pictures is higher than or equal to the number of different feature points set in advance, it is determined that the surrounding environment has changed; in some embodiments, the similarity between the previous and next pictures is determined by the mean square error, the average of the square of the difference between each pixel value of the two pictures is calculated, and an average value is set in advance as the similarity threshold to determine the similarity between the previous and next pictures, etc., which is not limited here.

[0047] S204, when a change in the surrounding environment is detected, the second core switches back to the first core to run the hard disk recorder; When changes in the surrounding environment are detected, the second core switches back to the first core to run the hard disk recorder, respond to and record changes in the surrounding environment in the first place.

[0048] S205, the hard disk video recorder records a video of a preset duration; After the recording is completed, the above step S202 is executed.

[0049] After the hard disk camera records a video of a preset length, the first core switches to the second core to control the external camera, turn off the hard disk recorder, and monitor whether the surrounding environment has changed. In this embodiment, the hard disk recorder records the changes in the surrounding environment and then turns off the hard disk recorder in time. In the case of being able to record abnormal events, the operating power consumption of the sentinel mode is reduced as much as possible.

[0050] In the above embodiment, when the vehicle is turned off, by only using the second core to take over the external camera to monitor the surrounding environment when the surrounding environment has not changed, switching to the second core and waking up the hard disk recorder after the surrounding environment changes, and turning off the hard disk recorder after recording for a period of time, switching back to the first core, the power consumption of the sentinel mode after the vehicle is turned off is reduced, and a quick response can also be made when changes in the surrounding environment are detected. By only using the second core to take over the external camera to monitor the surrounding environment when the surrounding environment has not changed, the power consumption of the sentinel mode after the vehicle is turned off is reduced, and a quick response can also be made when changes in the surrounding environment are detected.

[0051] In some embodiments, when the smart cockpit uses the second core to control the external camera of the vehicle and collects surrounding images at a preset frequency, an emergency event may occur in the time interval between two collected images, such as objects thrown from high places smashing the vehicle sunroof, and the smart cockpit can record the emergency event by responding to the emergency event method. Figure 3 As shown, another flow chart of the method for low-power sentinel monitoring of the intelligent cockpit provided in the embodiment of the present application is provided. The method can be used Figure 1 The system architecture shown, for a smart cockpit, includes the following steps: S301, after the vehicle is started, the hard disk recorder is run by the first core to store the video captured by the camera outside the vehicle; S302, when the vehicle is turned off, the operation of the hard disk video recorder is stopped, the first core is switched to the second core to take over the use authority of the external camera of the vehicle, and the surrounding environment images are collected regularly; S303, establishing an image queue, collecting surrounding environment images at a preset frequency, and storing them in the image queue; S304, extracting features from each newly stored image, and comparing the features with those of the previous image to confirm whether new features appear; If it is confirmed that a new feature has been added, the following step S305 is executed; If it is confirmed that no new features appear, then execute the above step S302; In some embodiments of the present application, image feature points are extracted to determine whether new features appear in the previous and next images, and a threshold for the number of different feature points is set in advance. If the number of different feature points in the previous and next images is lower than the threshold for the number of different feature points, it is determined that no new features appear in the surrounding environment; if the number of different feature points in the previous and next images is higher than or equal to the threshold for the number of different feature points, it is determined that new features appear in the surrounding environment; in some embodiments, grayscale matching and other methods can also be used to determine whether new features appear in the image, which is not limited here.

[0052] S305, extracting the newly added feature separately to determine whether the newly added feature is a person, an animal or a vehicle; If the newly added feature is not a person, an animal or a vehicle, then the above step S302 is executed; If the newly added feature is a person or an animal, the following step S307 is executed; If the newly added feature is a vehicle, the following step S306 is executed; The newly added features detected in step S304 are extracted separately. Since most accidents when the vehicle is parked are caused by people, animals or vehicles, in order to reduce the number of useless switches to the first core and the number of times the hard disk recorder is turned on, it is determined whether the newly added features are people, animals or vehicles; if the newly added features are not people, animals or vehicles, return to the above-mentioned step of regularly collecting surrounding images; if the newly added features are people or animals, perform the following detection steps; if the newly added features are vehicles, further determine the vehicle distance.

[0053] In some embodiments of the present application, different feature points of the previous step are extracted from the front and back pictures as new features, and the target detection model determines whether the new features are people, animals or vehicles; in some embodiments, deep learning algorithms and other methods can also be used to determine whether the new features are people, animals or vehicles, which is not limited here.

[0054] S306, confirming whether the distance of the newly added feature vehicle is less than the safety distance threshold; If it is less than the safety distance threshold, the following step S310 is executed; If it is greater than or equal to the safety distance threshold, the above step S302 is executed.

[0055] When it is determined that the newly added feature is a vehicle, the distance of the newly added feature vehicle is confirmed by retrieving the image last stored in the image queue and detecting the distance between the newly added feature vehicle and the camera in the picture. The distance is compared with the safety distance threshold set in advance. If it is less than the safety distance threshold, it is confirmed that the surrounding environment has changed; if it is greater than or equal to the safety distance threshold, it is determined that the surrounding environment has not changed, and the process returns to the step of collecting surrounding environment images according to the frequency.

[0056] In some embodiments of the present application, monocular vision ranging is used to confirm the distance of the newly added feature vehicle. The actual distance is calculated by obtaining camera parameters and combining them with feature points in the photo. In some embodiments, it is also possible to train a deep learning model to achieve methods such as measuring the distance of objects in the image, which is not limited here.

[0057] S307, determining whether the current environment is outdoor; If the current environment is not outdoor, execute the following step S308; If the current environment is outdoor, execute the following step S309; When a change in the surrounding environment is detected, the last picture in the image queue is retrieved to determine whether the current environment is an outdoor environment. Since there are more factors causing changes in the surrounding environment in an outdoor environment than in a non-outdoor environment, after determining whether the environment is an outdoor environment, different detection steps are performed.

[0058] In some embodiments of the present application, by retrieving the last picture acquired in the image queue, it is determined whether the current environment is an outdoor environment by extracting information in the picture, such as surrounding objects; in some embodiments, other methods such as identifying the light intensity in the picture can also be used to retrieve the picture to determine whether the current environment is an outdoor environment, which is not limited here.

[0059] S308, when the surrounding environment is detected to have changed, the area where the front and rear images have changed is a changed object, and monitoring whether the changed area is close to a vehicle door, a vehicle window or a sunroof; If the changed object is not close to the door, window or sunroof, then execute the above step S302; If the changed object is close to the door, window or sunroof, the following step S3010 is executed; When it is determined that the current environment is not outdoor, the distance between the changing object and the door, window or sunroof is detected in the image to confirm the distance of the changing object, and compared with the door and window safety distance threshold set in advance; if the distance of the changing object is less than the door and window safety distance threshold, it is confirmed that the changing object is close to the door, window or sunroof; if the distance of the changing object is greater than or equal to the door and window safety distance threshold, it is confirmed that the changing object is not close to the door, window or sunroof, and the process returns to execute the step of collecting surrounding environment images according to the frequency.

[0060] In some embodiments of the present application, monocular vision ranging is used to confirm the distance between the changing object and the door, window or sunroof. The actual distance is inferred by obtaining camera parameters and combining them with feature points in the photo. In some embodiments, the distance measurement of objects in the image can also be achieved by training a deep learning model, which is not limited here.

[0061] In this embodiment, since the places with greater safety hazards when the vehicle is parked are the doors, windows or sunroofs of the vehicle, the decision on whether to switch the core and start the hard disk recorder is made by detecting whether the changing objects in the surrounding environment are close to the doors, windows or sunroofs. Only the places with a high accident rate are monitored, which effectively reduces the number of useless switches to the second core and the number of times the hard disk recorder is turned on, and reduces the power consumption of the sentinel mode after the vehicle is turned off.

[0062] S309, determining whether the changed object is close to a door, a window, or a sunroof and whether the residence time of the changed object exceeds a set threshold; If the changed object is not close to the door, window or sunroof or the residence time of the changed object does not exceed the set threshold, the above step S302 is executed; If the changed object is close to the door, window or sunroof and the residence time of the changed object exceeds the set threshold, the following step S310 is executed; When it is determined that the current environment is outdoor, the distance between the changing object and the door, window or sunroof in the picture is detected to confirm the distance of the changing object, and compared with the door and window safety distance threshold set in advance. If it is less than the door and window safety distance threshold, multiple photos are detected to determine whether the moving object within the safety distance threshold has a residence time greater than the safety time threshold; if the residence time is greater than the safety threshold time, it is determined that the changing object is close to the door, window or sunroof and the residence time of the changing object exceeds the set threshold; if the residence time is less than or equal to the safety threshold time, or the distance of the changing object is greater than or equal to the door and window safety distance threshold, then return to the step of collecting surrounding environment images according to the frequency.

[0063] In some embodiments of the present application, template matching and timetable notation are used to confirm the residence time of a changing object. The residence time is calculated by searching for the time point at which the area matching the template of the moving object is successfully matched in multiple photos. In some embodiments, inter-frame difference analysis methods, etc. can also be used, which are not limited here.

[0064] In this embodiment, when the surrounding environment is outdoors, whether the surrounding environment has changed is determined by whether there is an environmental change near the vehicle door, window or sunroof, and whether the changed moving object stays for more than a certain time. This avoids the situation where the vehicle frequently responds to changes in the surrounding environment due to someone approaching when the vehicle is parked in a place with a large flow of people, such as a parking lot in a scenic spot, etc., reduces the number of useless switches to the second core and the number of times the hard disk recorder is turned on, and reduces the power consumption of the sentinel mode after the vehicle is turned off.

[0065] S310, immediately switching from the second core back to the first core to run the hard disk video recorder; When the surrounding environment is not outdoor, if the changed object is close to the car door, window or sunroof, the second core will immediately switch back to the first core to run the hard disk recorder; when the surrounding environment is outdoor, if the changed object is close to the car door, window or sunroof and the residence time of the changed object exceeds the set threshold, the second core will immediately switch back to the first core to run the hard disk recorder.

[0066] S311, after the hard disk video recorder records the video of preset length, the above step S302 is performed.

[0067] S312: When a change in the surrounding environment is detected, the above step S308 is executed and an abnormal prompt message is sent to the vehicle owner's mobile device, and a user is prompted to check the abnormal environment control around the vehicle. When a change in the surrounding environment is detected and step S308 is executed, the owner is prompted that an abnormality has occurred around the current vehicle and is asked whether to view the abnormal environment control around the vehicle. The owner can click the control to view the vehicle's surrounding environment in real time.

[0068] S313: When receiving the viewing instruction returned by the mobile device, the video recorded by the hard disk recorder is transmitted to the mobile terminal in real time.

[0069] After receiving the viewing command, the smart cockpit retrieves the images recorded by the hard disk recorder and transmits them to the user's mobile terminal so that the user can view abnormal conditions in the vehicle's surrounding environment in real time.

[0070] S314, when no change in the surrounding environment is detected, monitoring whether the glass of the vehicle is broken; If no glass breakage occurs, continue to step S314; If glass breakage occurs, the following step S315 is executed; When no changes in the surrounding environment are detected, the second core responds to the vehicle glass pressure sensor signal to monitor whether the vehicle glass is broken. If no glass breakage occurs, continuous monitoring is performed; if glass breakage occurs, the following steps are executed.

[0071] S315, turning on the in-vehicle camera to monitor whether there is a moving object in the picture; If there is no moving object in the picture, then execute the following step S316; If there is a moving object in the picture, the following step S317 is executed; When glass breakage is detected, the second core responds immediately, turns on the in-car camera, and detects whether there is a moving object in the picture by monitoring the picture taken by the in-car camera; if there is no moving object in the picture or there is no moving object in the picture, the following different steps are performed respectively.

[0072] S316, sending a glass breakage reminder message to the car owner; When only the vehicle glass is detected to be broken but no moving objects are detected inside the vehicle, it may be an unexpected incident such as an object thrown from a high altitude hitting the vehicle sunroof. In this case, it is sufficient to remind the car owner that the vehicle glass is broken by sending a glass breakage prompt message to the car owner's mobile terminal. This prompt message is used to inform the car owner that the vehicle glass is currently detected to be broken.

[0073] S317, control the vehicle to start the harmless smoke release device; When it is detected that the vehicle glass is broken and there are moving objects inside the vehicle, it is possible that an intruder has entered the vehicle by smashing the window, posing a hidden danger to the safety of property inside the vehicle. The intelligent cockpit controls the vehicle to turn on the harmless smoke release device, releasing a small amount of harmless smoke inside the vehicle, interfering with the intruder's line of sight and attracting the attention of people around him, increasing the intruder's psychological fear and prompting the intruder to leave.

[0074] In some embodiments of the present application, after the harmless smoke release device of the vehicle is turned on, a small amount of colored smoke will be released in the vehicle. The smoke is a mixture of food-grade pigments, water vapor and other harmless gases, which can effectively interfere with the intruder's vision without causing any negative impact. In some embodiments, after the harmless smoke release device of the vehicle is turned on, inert gas with added harmless dyes can also be released, which is not limited here.

[0075] S318, determining the real-time remaining power of the vehicle, and if the remaining power is lower than a preset monitoring power threshold, sending a prompt message to the vehicle owner's mobile terminal and displaying whether to turn on the low power monitoring mode control; Set the monitoring power threshold in advance. If the current power is detected to be lower than the monitoring power threshold set in advance, the current vehicle power is low. The smart cockpit sends a prompt message to the owner's mobile terminal to remind the owner that the current power is low; display whether to turn on the low power monitoring mode control, and the owner can turn on the low power monitoring mode by clicking the control.

[0076] S319: Upon receiving the instruction for starting low-battery monitoring returned by the mobile terminal, start the low-battery monitoring mode.

[0077] In some embodiments of the present application, the low-battery monitoring mode stops monitoring whether the surrounding environment has changed, reduces the frequency of periodically capturing images of the surrounding environment, and stores the captured images of the surrounding environment; in some embodiments, the low-battery monitoring mode can also be enabled by changing a preset image similarity threshold, etc., which is not limited here.

[0078] In the above embodiment, by judging whether the changing object is close to the door, window or sunroof where accidents are frequent when the surrounding environment changes, and at the same time limiting the residence time of the changing object and judging whether the new feature is the person, animal or vehicle that causes most accidents when the surrounding environment changes, the number of meaningless responses to changes in the surrounding environment is effectively reduced, and the power consumption of the sentinel mode after the vehicle is turned off is reduced. In addition, after the abnormal situation is detected, the owner is notified in time and the owner can view the abnormal situation of the vehicle's surrounding environment in real time on the mobile terminal. For emergencies, the owner can be notified as soon as possible and the owner can be informed of the situation in time, which improves the user experience of the smart cockpit sentinel mode. In addition, by monitoring the glass breakage of the vehicle and detecting whether there is a moving object in the vehicle through the camera inside the vehicle, in the interval between two shots of the surrounding environment image taken by the external camera at a certain frequency, the intruder smashes the window and other similar situations, releases harmless smoke to prompt the intruder to leave, while reducing power consumption, monitoring various abnormal situations that may occur after the vehicle is turned off and taking measures. In addition, when the vehicle's battery is low, the owner can be reminded to turn on the low-power mode. By continuously recording the surrounding environment, reducing the recording frequency, and no longer monitoring whether the surrounding environment has changed, the operating power consumption can be reduced as much as possible to save power.

[0079] Please refer to Figure 4 , which shows a schematic diagram of the structure of a smart cockpit provided in an embodiment of the present application. The smart cockpit is used to implement the method provided in the above embodiment. Specifically: The smart cockpit 400 may include an information input device 410, a cockpit domain controller 420, and an information output device 430. Those skilled in the art will appreciate that Figure 4 The smart cockpit structure shown in the figure does not constitute a limitation on the smart cockpit, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently. Among them, the information input device 410 is used to collect information such as voice, gestures, and sight of the driver and passengers, as well as environmental data inside and outside the car; the cockpit domain controller 420 is usually composed of multiple chips to support the realization of various functions of the smart cockpit; the information output device 430 is used to provide sensory information such as vision and hearing to the driver or passengers or transmit information to the mobile terminal.

[0080] The information input device 410 of the smart cockpit mainly includes: sensor 411, camera 412 and microphone 413. Sensor 410 is used to sense environmental parameters inside and outside the vehicle, such as temperature, humidity, light intensity and the movement state of the vehicle, or specific part sensors, such as vehicle glass pressure sensors, which can be used to sense whether the vehicle glass is broken. Camera 412 is used to collect visual information. The camera outside the vehicle is used to capture the road conditions, pedestrians and vehicle dynamics outside the vehicle; the camera inside the vehicle is used to monitor the interior of the vehicle, such as driver fatigue monitoring, gesture recognition, etc. Microphone 413 is used to collect sound signals. Through voice recognition technology, the microphone array can capture and recognize voice commands in the car, thereby realizing the control of functions such as navigation, phone calls, and voice assistants. In addition, the information input device 410 also includes a variety of devices working together to provide data support for the smart cockpit, which will not be repeated here.

[0081] The cockpit domain controller 420 (CDC) of the smart cockpit is the core processing unit of the entire smart cockpit system. It integrates multiple processors to work together to support the efficient processing of various complex data and instructions in the smart cockpit.

[0082] In terms of hardware composition, the cockpit domain controller 420 usually includes key components such as a system-level chip 421, a functional safety chip 422, a peripheral auxiliary supporting chip 423, and a variety of communication interfaces. Among them, the system-level chip 421, as the core component of the cockpit domain controller 420, can support high-resolution display, 3D graphics rendering, smooth user interface, and complex artificial intelligence algorithms by integrating high-performance processors and sufficient storage space. The functional safety chip 422 is mainly responsible for providing encryption and security authentication services to ensure the security of data transmission and storage. The peripheral auxiliary supporting chip 423 includes a communication module, a high-side switch, an A / D conversion module, etc., which is responsible for functions such as data transmission, circuit driving, and signal conversion to support the normal operation of the system-level chip. In addition, the cockpit domain controller 420 also includes a variety of components that work together to process various types of data and instructions to achieve intelligent control and adjustment of various functions of the vehicle, which will not be repeated here.

[0083] The information output device 430 of the smart cockpit is responsible for presenting various types of information processed by the system, so as to realize various functions such as real-time monitoring of the vehicle status. It mainly includes: display screen 431, audio system 432 and vehicle information system 433. Display screen 431 is the core information output device in the smart cockpit. Display screen 431 can display basic vehicle information such as speed, rotation speed, fuel consumption, etc., and can also display navigation maps, entertainment information, vehicle settings and other contents in real time. In addition, with the development of technology, new display technologies such as curved screens and OLED screens are also applied to the display screens of smart cockpits. The audio system 432 can play audio content such as music and navigation prompts, so that the driver can obtain information more conveniently during driving. The vehicle information system 433 can connect to mobile devices and transmit information through mobile device mapping functions such as Bluetooth or CarPlay (Apple CarPlay, the car version of Apple system), CarLife (BaiduCarLife, Baidu car networking system), etc. In addition, the information output device 430 also includes multiple devices working together to process display information and enhance the interactive experience, thereby realizing intelligent control and adjustment of various functions of the vehicle, which will not be elaborated here.

[0084] Although not shown, the smart cockpit 400 may also include an information processing module, etc., which will not be described in detail here.

[0085] Specifically in this embodiment, the smart cockpit 400 further includes a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors. The one or more programs include instructions for executing the above method.

[0086] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0087] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0088] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0089] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for low-power sentinel monitoring of a smart cockpit, applied to a smart cockpit, characterized in that: The system-on-chip of the smart cockpit includes a first core and a second core, the operating power consumption of the second core is lower than the operating power consumption of the first core, and the method includes: After the vehicle is started, the hard disk recorder is run through the first core so that the video captured by the camera outside the vehicle is stored; When the vehicle is turned off, the operation of the hard disk video recorder is stopped, the first core is switched to the second core to take over the use authority of the external camera of the vehicle, and the surrounding environment images are collected regularly; Monitoring whether the surrounding environment changes by collecting the surrounding environment image; When a change in the surrounding environment is detected, the second core switches back to the first core to run the hard disk video recorder; After the hard disk video recorder records a video of a preset length, the steps of stopping the operation of the hard disk video recorder when the vehicle is turned off, switching from the first core to the second core to take over the use authority of the external camera, and regularly collecting images of the surrounding environment are executed.

2. The method according to claim 1, characterized in that The monitoring of whether the surrounding environment changes by collecting the surrounding environment image specifically includes: Establish an image queue, collect surrounding environment images according to a preset frequency, and store them in the image queue; Compare each newly stored image with the previous image, and determine whether the surrounding environment has changed based on whether the similarity between the previous and the previous images is lower than a preset similarity threshold; When the similarity is lower than the preset similarity threshold, determining that the surrounding environment has changed; When the similarity is greater than or equal to the preset similarity threshold, it is determined that the surrounding environment has not changed.

3. The method according to claim 1, characterized in that The monitoring of whether the surrounding environment changes by collecting the surrounding environment image specifically includes: Establish an image queue, collect surrounding environment images according to a preset frequency, and store them in the image queue; Extract features from each newly stored image and compare them with the features of the previous image. If new features appear in the latter image, extract the new features separately to determine whether the new features are people, animals or vehicles. If the newly added feature is not a person, animal or vehicle, it is determined that the surrounding environment has not changed; If the newly added feature is a person or an animal, it is determined that the surrounding environment has changed; If the newly added feature is a vehicle, detecting whether the distance of the vehicle is less than a set safety threshold; If the vehicle distance is less than a set safety threshold, it is determined that the surrounding environment has changed.

4. The method according to claim 1, characterized in that: When a change in the surrounding environment is detected, the second core switches back to the first core to run the hard disk video recorder, specifically including: When a change in the surrounding environment is detected, the vehicle's current environment information is obtained by retrieving the last image in the image queue to determine whether it is currently in an outdoor environment; If the vehicle is not currently in an outdoor environment, the change in the surrounding environment is determined to be a dangerous change based on whether the object is close to the vehicle door, window or sunroof. If the current environment is outdoors, determine whether the change in the surrounding environment is a dangerous change based on whether the change is close to the door, window or sunroof and whether the residence time of the changed object exceeds the set threshold; If the change in the surrounding environment is a dangerous change, the second core is immediately switched back to the first core to run the hard disk recorder.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Responding to the vehicle glass sensor signal through the second core, monitoring whether the vehicle glass is broken when no changes in the surrounding environment are detected; If the glass breaks, the in-car camera will be activated immediately; When the in-car camera is turned on, check whether there are any moving objects in the monitoring image to confirm whether there are any safety hazards inside the vehicle; If there is no hidden danger in the car, a prompt message is sent to the car owner's mobile terminal, and the prompt message is used to remind the car owner that the current car glass is broken; If there is a potential safety hazard in the vehicle, the vehicle is controlled to activate a harmless smoke release device, which releases harmless smoke to interfere with the intruder's sight and prompt the intruder to leave.

6. The method according to claim 1, characterized in that The method further comprises: When a change in the surrounding environment is detected, an abnormal prompt message is sent to the vehicle owner's mobile device, where the abnormal prompt message is used to prompt the vehicle owner on the mobile device that the vehicle's surrounding environment is abnormal and to display whether to check the abnormal environment control around the vehicle; When the viewing instruction returned by the mobile device is received, the video recorded by the hard disk video recorder is transmitted to the mobile terminal in real time.

7. The method according to claim 1 or 6, characterized in that: The method further comprises: Determine the real-time remaining power of the vehicle. If the remaining power is lower than the preset monitoring power threshold, send a prompt message to the owner's mobile terminal, which is used to remind the owner that the current power is low and display whether to turn on the low power monitoring mode control; When receiving the instruction to start low-battery monitoring returned by the mobile terminal, stop monitoring whether the surrounding environment has changed, reduce the frequency of regularly collecting surrounding environment images, and store the collected surrounding environment images.

8. A smart cockpit, characterized in that: The smart cockpit includes: one or more system-level chips and a memory; the system-level chip includes one or more processors; the memory is coupled to the system-level chip, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the mobile terminal to execute the method described in any one of claims 1-7.

9. A computer program product comprising instructions, characterized in that When the computer program product runs on the smart cockpit, the mobile terminal executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium comprising instructions, characterized in that: When the instruction is executed on a mobile terminal, the mobile terminal is caused to execute the method according to any one of claims 1 to 8.

Images