A vehicle monitoring method, apparatus and vehicle

Abstract

The embodiment of the application provides a vehicle monitoring method, device and vehicle. When the vehicle is in an off state, at least one sensor is selected from a plurality of sensors installed on the vehicle according to at least one of a scene type of a surrounding environment, a moving object in the surrounding environment, and a barrier condition of the surrounding environment, and then the surrounding environment is monitored based on the at least one sensor. In the method, the vehicle can dynamically select a sensor according to the surrounding environment, event recognition can be realized, the monitoring accuracy can be improved, and meanwhile, all sensors do not need to be used to monitor the surrounding environment, so that the service life of the sensor can be improved and the energy consumption of the sensor can be reduced while the safety of the vehicle in the off state is considered.

Description

Technical Field

[0001] This application relates to the field of smart / inteligent cars, and more particularly to a vehicle monitoring method, device, and vehicle. Background Technology

[0002] With rapid economic and technological development and rising living standards, vehicles have become necessities in people's lives. While providing convenience for car owners, vehicles also bring some problems and inconveniences. For example, car owners often leave their vehicles unattended after turning off the engine.

[0003] To ensure vehicle safety when the engine is off, existing technologies use sensors, such as cameras, lidar, or millimeter-wave radar, to monitor the vehicle's surrounding environment. However, a single sensor can only detect a limited number of threats and cannot cope with the complex and ever-changing real-world scenarios; furthermore, prolonged monitoring causes significant wear and tear on the sensor, reducing its lifespan and consuming a large amount of energy.

[0004] Therefore, how to balance the safety of the vehicle when the engine is off with the lifespan and energy consumption of the sensors has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a vehicle monitoring method, device, and vehicle that can balance the safety of the vehicle when the engine is off with the lifespan and energy consumption of the sensors.

[0006] Firstly, a vehicle monitoring method is provided, applicable to a vehicle in a turned-off state. The method includes: selecting at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment; and then monitoring the vehicle's surrounding environment based on the at least one sensor.

[0007] In this embodiment, the vehicle monitors threat factors in the surrounding environment by combining different sensors based on at least one of the following: scene type of the surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment. This optimizes the traditional monitoring mechanism, enables the perception and identification of multiple types of events, and can improve monitoring accuracy while reducing sensor wear and extending sensor lifespan.

[0008] The following describes a specific implementation method for a vehicle to select at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barriers in the surrounding environment.

[0009] In one possible design, the vehicle can first identify the scene type of the surrounding environment, and then select, from a variety of sensors installed on the vehicle, a sensor that corresponds to the scene type of the surrounding environment as the at least one sensor, based on the correspondence between the scene type and the sensor.

[0010] In this design, the sensors selected by the vehicle are more compatible with the scene types of the surrounding environment, which can improve the vehicle's monitoring accuracy of the surrounding environment.

[0011] In one possible design, the vehicle can first use a camera mounted on the vehicle to capture images of its surrounding environment. When the camera's images indicate the presence of a moving object, it then selects a camera and other types of sensors from a variety of sensors mounted on the vehicle as at least one of these sensors. In other words, if there are no moving objects around the vehicle, a camera is sufficient for monitoring; if moving objects are present, other sensors are selected to work in conjunction with the camera to enhance monitoring effectiveness.

[0012] In this design, the vehicle can minimize the number of activated sensors while maintaining monitoring accuracy, thereby reducing sensor wear and extending sensor lifespan.

[0013] In one possible design, the vehicle determines that a moving object meets preset conditions before selecting a camera and other types of sensors (excluding the camera) from a variety of sensors installed on the vehicle as at least one sensor. For example, the preset conditions may include any one or more of the following: the moving object moves towards the vehicle; the frequency of the moving object's appearance exceeds a preset frequency; the duration of the moving object's appearance exceeds a preset duration; or the moving object is within a preset range of the vehicle.

[0014] In this design, the vehicle can further reduce wear and tear on the sensors and extend their lifespan while maintaining monitoring accuracy.

[0015] In one possible design, the vehicle can first obtain information about the barriers around the vehicle. If a barrier exists in a first area, the first area is in the first orientation of the vehicle, and the distance between the first area and the vehicle is less than a first threshold, it indicates that the first area poses a very low security threat to the vehicle. In this case, at least one sensor selected by the vehicle may not include a sensor used for monitoring in the first orientation.

[0016] In this design, the vehicle can make good use of the existence of barriers by sensing whether there are barriers around the vehicle body, avoid potential threats on the side of the barrier, improve vehicle safety, reduce sensor wear and tear, extend sensor lifespan, and improve monitoring effectiveness.

[0017] In one possible design, the barrier includes a wall or other vehicles.

[0018] In this design, vehicles can utilize the presence of walls or other vehicles to avoid potential threats, reduce sensor wear and tear, extend sensor lifespan, and improve monitoring effectiveness.

[0019] It should be understood that the above-mentioned sensor selection schemes can be implemented in combination.

[0020] The following describes a specific method for a vehicle to monitor its surrounding environment based on at least one sensor.

[0021] In one possible design, the vehicle can monitor at least one factor in the surrounding environment that poses a threat to the vehicle's safety based on at least one sensor; then, based on the at least one factor, determine the threat level of the surrounding environment to the vehicle; and then, execute a response event corresponding to the threat level.

[0022] This design allows the vehicle to promptly eliminate threats, improving its safety when the engine is off.

[0023] In one possible design, the threat level of the surrounding environment to the vehicle is determined based on at least one of the following: the type of at least one factor, the value of at least one factor, the duration of at least one factor, the number of changes in the surrounding environment, or the vehicle's speed. For example, the more types of factors, the higher the corresponding threat level; or, for example, the more values ​​of the factors, the higher the threat level.

[0024] In this design, the vehicle can be subdivided into different levels of threat posed by the surrounding environment, further improving the vehicle's safety when the engine is off.

[0025] In one possible design, the threat levels are arranged from low to high as Level 1, Level 2, and Level 3. The response events corresponding to Level 1 include any one or more of the following: flashing headlights, honking the horn, or flashing the central control screen. The response events corresponding to Level 2 include the response events corresponding to Level 1, as well as recording and saving video using a camera. The response events corresponding to Level 3 include the response events corresponding to Level 2, sending an alert to the user device, and uploading the video to the cloud for download by the user device.

[0026] In this design, different threat levels correspond to different response events, which can eliminate threats in a targeted and timely manner, and further improve the safety of the vehicle when the engine is off.

[0027] Secondly, embodiments of this application provide a vehicle monitoring device, such as a vehicle in a turned-off state, or a component or processing chip located in a vehicle in a turned-off state. The device includes functions, modules, units, or means for performing the methods described in the first aspect or any possible design of the first aspect.

[0028] For example, the device may include: a processing unit for selecting at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment; and a monitoring unit for monitoring the vehicle's surrounding environment based on at least one sensor.

[0029] The specific implementation of the methods and steps executed by each unit can be found in the specific implementation of the corresponding methods and steps in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0030] Thirdly, embodiments of this application provide a vehicle monitoring device for use in a vehicle in a turned-off state. The vehicle monitoring device includes a processor and a memory. The memory stores computer program instructions, and the processor executes the computer program instructions to implement the method described in the first aspect or any possible design of the first aspect.

[0031] For example, the processor can be used to: select at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment; and monitor the vehicle's surrounding environment based on at least one sensor.

[0032] The specific implementation of the method steps executed by the processor can be found in the specific implementation of the corresponding method steps in the first aspect or any possible design of the first aspect, and will not be repeated here.

[0033] Fourthly, embodiments of this application provide a vehicle, including: multiple sensors; and a vehicle monitoring device as described in the second aspect or as described in the third aspect.

[0034] Fifthly, embodiments of this application provide a computer-readable storage medium for storing instructions that, when executed, cause the method described in the first aspect or any possible design of the first aspect to be implemented.

[0035] Sixthly, embodiments of this application provide a computer program product storing instructions that, when executed on a processor, cause the method described in the first aspect or any possible design of the first aspect to be implemented.

[0036] The beneficial effects of each design in aspects two through six above are the same as the beneficial effects of the corresponding design in aspect one, and will not be repeated here. Attached Figure Description

[0037] Figure 1 This application provides a schematic diagram of the architecture of a possible vehicle system.

[0038] Figure 2 This is a schematic diagram of a possible sensor layout provided in an embodiment of this application;

[0039] Figure 3 This application provides a schematic diagram of the architecture of a possible ECU system.

[0040] Figure 4 A flowchart illustrating a vehicle monitoring method provided in this application embodiment;

[0041] Figure 5 A flowchart illustrating a scene recognition-based sensor selection method provided in this application embodiment;

[0042] Figure 6 A flowchart illustrating a sensor selection method based on a moving object, provided for an embodiment of this application;

[0043] Figure 7 A flowchart illustrating a sensor selection method based on environmental barrier conditions, provided for an embodiment of this application;

[0044] Figure 8 This is a diagram illustrating a possible threat level and its corresponding response events.

[0045] Figure 9 A schematic diagram illustrating one possible method for executing a response event;

[0046] Figure 10 This is a schematic diagram of the structure of a vehicle monitoring device 1000 provided in an embodiment of this application;

[0047] Figure 11 This is a schematic diagram of the structure of a vehicle-mounted device 1100 provided in an embodiment of this application. Detailed Implementation

[0048] This application's embodiments are applicable to in-vehicle systems, which can be deployed in vehicles. It should be understood that this application's embodiments primarily use vehicles in a powered-off (or parked) state as an example, but these embodiments can also be applied to vehicles in other states, such as vehicles moving slowly, or vehicles that have stopped but are not turned off; this application makes no limitation. Here, "powered-off state" refers to the vehicle's engine being off and the vehicle being stationary.

[0049] See Figure 1 This is a schematic diagram of a possible vehicle system architecture provided in an embodiment of this application. The vehicle system architecture includes at least a sensor system and an Electronic Control Unit (ECU) system. The sensor system can collect data on the vehicle's surrounding environment and input the collected data into the ECU system for processing.

[0050] Sensor systems include a variety of sensors, such as, but not limited to, the following: ultrasonic radar (USS), camera, inertial navigation system (INS), and global positioning system (GPS).

[0051] 1) Ultrasonic radar refers to radar that uses ultrasonic waves for detection. The working principle of ultrasonic radar is to calculate distance by measuring the time difference between when an ultrasonic transmitter emits ultrasonic waves and when the transmitted ultrasonic waves are received by a receiver. Ultrasonic waves are vibrations with a frequency greater than 20,000 Hz. Their frequency (number of vibrations per second) is extremely high, exceeding the general upper limit of human hearing (20,000 Hz). These inaudible sound waves are called ultrasonic waves.

[0052] Ultrasonic radar includes, but is not limited to, the following two types: The first type is mounted on the front and rear bumpers of a vehicle, used for measuring obstacles in front of and behind the vehicle; this type is known in the industry as UPA. The second type is mounted on the side of the vehicle, used for measuring the distance to side obstacles; this type is known in the industry as APA. UPA is a short-range ultrasonic sensor, mainly installed at the front and rear of the vehicle body, with a detection range of 25cm to 2.5m. Due to its long detection distance, it has less Doppler effect and temperature backlash interference, resulting in more accurate detection. APA is a long-range ultrasonic sensor, mainly used on the side of the vehicle body, with a detection range of 35cm to 5m, covering a parking space. It has strong directionality, better propagation performance than UPA, and is less susceptible to interference from other APA and UPA sensors.

[0053] For example, Figure 2 This diagram illustrates the layout of multiple sensors on a vehicle. Figure 2 In the example shown, ultrasonic radars a, b, g, h, i, and j are short-range ultrasonic radars, installed at the front and rear of the vehicle, while ultrasonic radars c, d, e, and f are long-range ultrasonic radars, installed on the left and right sides of the vehicle.

[0054] 2) Camera, also known as a camera sensor. The camera in this application embodiment may include any camera used to acquire images of the environment in which the vehicle is located, such as including but not limited to: infrared cameras, visible light cameras, etc.

[0055] For example, in Figure 2 In the example shown, camera 1 is located at the front of the vehicle and can capture images of the front of the vehicle; camera 2 is located at the rear of the vehicle and can capture images of the rear of the vehicle; cameras 3 and 4 are located on the left and right sides of the vehicle, respectively, and can capture images of the left and right sides of the vehicle.

[0056] 3) An inertial navigation system is a navigation parameter calculation system that uses gyroscopes and accelerometers as sensing devices. The system establishes a navigation coordinate system based on the output of the gyroscope and calculates the speed and position of the vehicle (such as a vehicle) in the navigation coordinate system based on the output of the accelerometer.

[0057] 4) The Global Positioning System, also known as the Global Satellite Positioning System or simply "Global Positioning System", is a medium-range circular orbit satellite navigation system that combines satellite and communication technologies to use navigation satellites for timing and ranging.

[0058] It should be understood that Figure 2 As an example only, the placement of various sensors in practical applications can be similar to... Figure 2 It may vary and may include more or fewer sensors, or may include other types of sensors; this application does not limit this.

[0059] The ECU system can process data collected by various sensors in the sensor system. For example, the ECU system processes image data captured by a camera to identify objects (such as obstacles) in the image. Based on the processing results, the ECU system can also make decisions and drive controlled components to operate. These controlled components include, but are not limited to, sensors, speakers, headlights, and the central control screen.

[0060] In the embodiments of this application, the ECU system consists of multiple ECUs, which can communicate with each other to exchange data. For example, each ECU is connected to a Controller Area Network (CAN) bus, and the ECUs exchange data based on the CAN bus.

[0061] An ECU can be implemented as any device or module with processing capabilities. For example, an ECU can be a Central Processing Unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0062] See Figure 3 Based on the functional division of each ECU, the ECUs in this application embodiment include, but are not limited to, the following types: mobile data center (MDC), body control management (BCM), cockpit domain controller (CDC), and telematics box (TBOX).

[0063] 1) The MDC is the core ECU of the vehicle. The MDC has the functions of calculation and control. It can calculate the data collected by various sensors and convert the calculation results into control commands. The control commands control the operation of the controlled components. For example, the MDC sends the control commands to the ECU corresponding to the controlled component (such as BCM, CDC, TBOX, etc.), and the ECU corresponding to the controlled component drives the controlled component to work according to the control commands.

[0064] MDC can also control memory (ROM / FLASH / EEPROM, RAM), input / output interfaces (I / O), and other external circuits; memory can store programs.

[0065] The vehicle monitoring method provided in this application embodiment can be controlled by MDC or called by other components to complete, such as calling the processing program of this application embodiment stored in the memory to perform calculations on the data collected by each sensor and control the operation of the controlled element.

[0066] 2) BCM, also known as body computer, is an ECU used to control the vehicle's electrical systems. Components controlled by the BCM include, but are not limited to: power windows, power mirrors, air conditioning, lights (such as headlights and turn signals), anti-theft locking systems, central locking, and defrosting devices. The BCM can connect to other onboard ECUs via the CAN bus.

[0067] 3) The CDC (Control Unit) is the ECU used to control various components in the smart cockpit. Components in the smart cockpit include, but are not limited to, the following: instrument panel, central control panel (referred to as the central control screen), head-up display, microphone, camera, speaker (i.e., horn), or Bluetooth module, etc. The smart cockpit can control the operating status and trajectory of the autonomous vehicle through human-machine interaction according to the needs of the passengers, thus enabling both human-machine interaction and remote control within the smart cockpit to transmit the same commands to control the vehicle's operation.

[0068] 4) The TBOX is primarily used to communicate with backend systems or user devices' applications (APPs) to enable APP-related vehicle information display and control. The TBOX can use 3G cellular communication, such as Code Division Multiple Access (CDMA), EVDO, Global System for Mobile Communications (GSM) / General Packet Radio Service (GPRS), or 4G cellular communication, such as Long Term Evolution (LTE), or 5G cellular communication. The TBOX can also communicate via WiFi and Wireless Local Area Network (WLAN). In some embodiments, the TBOX can communicate directly with devices using infrared links, Bluetooth, or ZigBee. The TBOX can also communicate based on other wireless protocols, such as the Vehicle Dedicated Short Range Communications (DSRC) protocol, for example, to communicate directly with other vehicles and / or roadside stations.

[0069] It should be noted that, Figure 3 This is merely an example; in practical applications, the number and layout of ECUs can be implemented in other ways, and this application does not impose specific limitations here. Furthermore, Figure 3 Each ECU in the system can be deployed independently or integrated with each other; this application does not limit this.

[0070] Based on the above description, embodiments of this application provide a vehicle monitoring method, which is applied to... Figure 1 For example, see the vehicle-mounted system shown. Figure 4 The method includes the following steps:

[0071] S401. When the vehicle is turned off, the vehicle selects at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment.

[0072] Specifically, the ECU system in the vehicle determines the type of event to be monitored based on at least one of the following: scene type of the surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment. Then, it selects at least one sensor from a variety of sensors installed on the vehicle that corresponds to the type of event to be monitored, meaning that the selected at least one sensor can effectively monitor the event of that type.

[0073] Optionally, selecting at least one sensor from a plurality of sensors installed on the vehicle can be implemented by selecting and activating at least one sensor from the plurality of sensors installed on the vehicle. It should be understood that if some of the at least one sensor has already been selected and activated, only the sensors that have not been activated need to be activated. Optionally, if other sensors besides the selected at least one sensor have been selected or activated, then those other sensors are deselected or turned off.

[0074] It should be noted that, in the specific implementation process, the multiple elements used by the vehicle to select at least one sensor from the multiple sensors (i.e., the scene type of the surrounding environment, the moving objects in the surrounding environment, and the barrier conditions of the surrounding environment) can be implemented separately or in combination, and this application does not limit them.

[0075] The following sections will introduce the scenarios for different types of surrounding environments, moving objects in the surrounding environment, and barriers in the surrounding environment, and will be implemented separately.

[0076] I. The vehicle selects at least one sensor based on the type of scene in the surrounding environment.

[0077] The scene type of the surrounding environment can be characterized by: classification of the surrounding environment based on its formation method, function and purpose, geographical location, time period, facilities, natural environmental elements, characteristics of human activities, building type or privacy, etc.

[0078] This application does not limit the specific classification method of scene types. For example, according to the formation of the surrounding environment, the scene types of the surrounding environment can be divided into natural environment, artificial environment, etc.; according to the function of the surrounding environment, the scene types of the surrounding environment can be divided into living environment, ecological environment, etc.; according to different elements in the surrounding environment, the scene types of the surrounding environment can be divided into atmospheric environment, water environment, soil environment, biological environment, geological environment, etc.; according to the way humans gather in the surrounding environment, it can be divided into rural environment, urban environment, etc.; according to the privacy of the surrounding environment, the scene types of the surrounding environment can be divided into private environment, public environment, etc.; according to the type of buildings in the surrounding environment, the scene types of the surrounding environment can be divided into residential community environment, open-air / underground parking lot environment, street roadside environment, highway roadside environment, wilderness environment, etc.

[0079] In this embodiment, the vehicle can pre-set the correspondence between scene types and sensors. Before using sensors to monitor the vehicle's surrounding environment, the scene type can be identified first. Then, based on the correspondence between scene types and sensors, a sensor corresponding to the scene type of the surrounding environment can be selected from a variety of sensors installed on the vehicle. In this way, the vehicle does not need to constantly activate all sensors to monitor the surrounding environment; instead, it selects the appropriate sensors to monitor the surrounding environment as needed. This improves monitoring accuracy while reducing wear and tear on other sensors, extending sensor lifespan, and reducing the overall energy consumption of the vehicle's sensors.

[0080] For example, Figure 5 A flowchart of a scene recognition-based sensor selection method is shown, which can be applied to... Figure 1 The vehicle-mounted system shown can be specifically executed by the ECU system within that system. The method includes:

[0081] S501. The scenario type for the vehicle to identify the surrounding environment when the engine is off.

[0082] Specifically, the MDC (Modular Data Center) in the vehicle's ECU system identifies the scene type of the vehicle's surrounding environment. There can be various identification methods, and this application does not impose any limitations. For example, the MDC acquires the vehicle's historical records during driving (e.g., image data captured by cameras during driving, location data from navigation systems, etc.), and then determines the scene type of the vehicle's surrounding environment based on these historical records. Alternatively, the MDC first collects data about the surrounding environment based on one or more sensors on the vehicle (e.g., cameras), and then determines the scene type of the vehicle's surrounding environment based on this data.

[0083] S502. Based on the correspondence between scene type and sensor, the vehicle selects a sensor from a variety of sensors installed on the vehicle that corresponds to the scene type of the surrounding environment.

[0084] Specifically, the MDC can pre-set a first correspondence between scene types and sensors, for example, by storing the first correspondence between scene types and sensors in memory. After the MDC determines the scene type of the surrounding environment, it selects the sensor corresponding to that scene type from a variety of sensors installed on the vehicle based on this first correspondence.

[0085] The sensors corresponding to each scene type can be determined based on the types of events that need to be monitored under that scene type. This application does not specifically limit the correspondence between scene types and sensors. The following are some possible examples:

[0086] Example 1: Inside a residential community: In this scenario, there are few pedestrians and the road conditions are simple. Generally, only minor vehicle collisions are possible. Therefore, choosing only cameras and ultrasonic radar can ensure vehicle safety.

[0087] Example 2, Streetside: In this scenario, there is a large flow of people and vehicles, and the situation is complex and changeable. Various security threats may occur, such as towing, vehicle scratches, and theft. Therefore, cameras, ultrasonic radar, inertial navigation systems, and global positioning systems can be selected to ensure vehicle safety.

[0088] Example 3, Open-air / underground parking lot: The situation is simple, but there is a risk of vehicle scratches and theft, so inertial navigation systems, cameras, and ultrasonic radar can be used to detect objects approaching.

[0089] Example 4: Unfamiliar outdoor environment: When a vehicle is parked in an unfamiliar outdoor environment, the risk of theft is extremely high. Therefore, cameras and INS can be used to detect changes in the vibration amplitude and direction of the vehicle.

[0090] It should be understood that before step S502, some sensors may have already been selected to monitor the vehicle's surrounding environment. Therefore, when executing step S502, if a sensor corresponding to the scene type of the surrounding environment is not selected, then the sensor corresponding to the scene type of the surrounding environment is selected. If a sensor corresponding to the scene type of the surrounding environment has already been selected to monitor the vehicle's surrounding environment, then that sensor will continue to monitor the vehicle's surrounding environment. Optionally, if other sensors that do not correspond to the scene type of the surrounding environment are selected to monitor the vehicle's surrounding environment, then the monitoring of the surrounding environment by these non-corresponding sensors can be cancelled.

[0091] This application's embodiments propose different monitoring mechanisms for different scenarios of the vehicle's surrounding environment. Specifically, it selects a sensor from a variety of sensors installed on the vehicle that corresponds to the scenario type of the surrounding environment for monitoring. This improves monitoring accuracy while reducing wear and tear on other sensors, extending sensor lifespan, and decreasing the overall energy consumption of the vehicle's sensors.

[0092] 2. The vehicle selects at least one sensor from a variety of sensors installed on the vehicle based on moving objects in the surrounding environment.

[0093] In this application's embodiments, "moving object" refers to any object capable of movement. This includes any living object that can move (such as a person, cat, dog, rabbit, snake, butterfly, wolf, bird, etc.) and non-living objects (such as vehicles, drones, landslide rocks, fallen leaves, etc.). It should be understood that the movement of a moving object can be autonomous (e.g., a person walking, the movement of surrounding vehicles, the flight of birds, the running of animals, etc.) or passive (e.g., leaves falling in the wind, landslides, etc.), and this application does not impose any limitations.

[0094] In this embodiment, the vehicle can select a sensor from a variety of sensors installed on the vehicle by sensing the presence of moving objects around the vehicle body. This can improve monitoring accuracy while reducing wear and tear on other sensors, extending sensor lifespan, and reducing overall energy consumption of the vehicle's sensors.

[0095] Here are some typical application scenarios: Scenario 1: Late at night in residential areas / streetsides: Pedestrians or vehicles pass by only occasionally, and there are almost no moving objects. Scenario 2: Outdoor scenarios: There are few pedestrians, and few moving objects appear around vehicles. Scenario 3: Underground parking lots: Areas not used for vehicle entry and exit, and few moving objects appear around vehicles.

[0096] In these scenarios, before using sensors to monitor the vehicle's surroundings, the vehicle can first select a small number of sensors (such as a first sensor) from a variety of sensors installed on the vehicle. The first sensor is then used to detect whether any moving objects are present in the surrounding environment. Once the vehicle determines that a moving object is present, it then selects the first sensor and other types of sensors from the variety of sensors installed on the vehicle to monitor the surrounding environment. These other types of sensors include ultrasonic radar, inertial navigation systems, etc.

[0097] Optionally, to improve monitoring accuracy, the vehicle may select other types of sensors besides the first sensor only after determining that a moving object exists in the surrounding environment and that the moving object meets preset conditions. Further optionally, the preset conditions include, but are not limited to, any one or more of the following: the moving object is moving towards the vehicle; the frequency of the moving object's appearance exceeds a preset frequency; the duration of the moving object's appearance exceeds a preset duration; the moving object is within a preset range of the vehicle.

[0098] It should be understood that this application does not limit the type of the first sensor; for example, the first sensor can be a camera or ultrasonic radar, etc. Taking a camera as an example, Figure 6 A flowchart of a sensor selection method based on a moving object is shown, which can be applied to... Figure 1 The vehicle-mounted system shown, the method includes:

[0099] S601. When the engine is off, the vehicle's camera is activated and enters monitoring mode, capturing images of the surrounding environment.

[0100] S602. The vehicle uses images captured by a camera to detect whether there are moving objects around the vehicle.

[0101] Specifically, MDC identifies images captured by cameras and detects whether there are moving objects in the surrounding environment.

[0102] S603. If a moving object is detected, the vehicle continues to monitor the frequency of the object's appearance based on images captured by the camera; if no moving object is detected, the camera simply needs to remain on.

[0103] S604. When the frequency of moving objects is very high (such as exceeding the set frequency threshold), the vehicle's safety is threatened. The vehicle selects other types of sensors (such as ultrasonic radar, inertial navigation system, etc.) from the various sensors installed on the vehicle to work with the camera to monitor the surrounding environment. If the frequency is low, it may just be pedestrians or animals passing by without any intention to approach the vehicle. In this case, it is sufficient to keep the camera on, i.e., detection is based on the camera.

[0104] Optionally, the vehicle may determine other types of sensors based on at least one of the following: the direction of movement of the moving object, the frequency of occurrence of the moving object, the duration of occurrence of the moving object, or the distance between the moving object and the vehicle.

[0105] Example 1: The vehicle's MDC (Moving Data Center) pre-sets a second correspondence between the moving direction of a moving object and a sensor. After obtaining the moving direction of the moving object, the MDC selects a second sensor from a variety of sensors installed on the vehicle according to the second correspondence.

[0106] Taking a camera as the primary sensor, if a moving object moves in a curved path toward the vehicle, it indicates that the object will not approach the vehicle soon (it may just be a pedestrian passing by), and ultrasonic radar can be used in conjunction with the camera for monitoring. If a moving object moves in a straight path toward the vehicle, it indicates that the moving object will approach the vehicle soon, and ultrasonic radar, inertial navigation system, global positioning system, etc. can be used in conjunction with the camera for monitoring to quickly improve the vehicle's monitoring capabilities.

[0107] Example 2: The vehicle's MDC pre-sets a third correspondence between the frequency of occurrence of moving objects and the sensors. After obtaining the frequency of occurrence of moving objects, the MDC determines the third sensor to be selected from the various sensors installed on the vehicle based on the third correspondence.

[0108] Taking a camera as the primary sensor, if the frequency of a moving object's appearance is below a first frequency threshold, ultrasonic radar is used in conjunction with the camera for monitoring. If the frequency of a moving object's appearance is above the first frequency threshold but below a second frequency threshold, ultrasonic radar and a global positioning system (GPS) are used in conjunction with the camera for monitoring. If the frequency of a moving object's appearance is above the second frequency threshold, ultrasonic radar, GPS, and an inertial navigation system (INS) are used in conjunction with the camera for monitoring. The first frequency threshold is lower than the second frequency threshold, and both the first and second frequency thresholds are greater than 0.

[0109] The frequency of occurrence of a moving object can be the number of times it appears within a preset time range, such as the number of times it appears in one minute. MDC can sample whether a moving object appears once per second; if it does, the count is incremented by 1. Of course, this is merely an example, and this application does not limit the specific implementation method of MDC for counting the frequency of occurrence of moving objects.

[0110] Example 3: The vehicle's MDC is pre-set with a fourth correspondence between the duration of the appearance of a moving object and the sensor. After obtaining the duration of the appearance of the moving object, the MDC determines the fourth sensor to be selected from the various sensors installed on the vehicle based on the fourth correspondence.

[0111] Taking the camera as the primary sensor, if the moving object appears for 5 seconds, ultrasonic radar is selected in conjunction with the camera for monitoring; if the moving object appears for 30 seconds, ultrasonic radar and GPS are selected in conjunction with the camera for monitoring; if the moving object appears for 1 minute, ultrasonic radar, GPS, and inertial navigation system are selected in conjunction with the camera for monitoring.

[0112] Example 4: The vehicle's MDC is pre-set with a fifth correspondence between the distance between the moving object and the vehicle and the sensor. After obtaining the distance between the moving object and the vehicle, the MDC determines the fifth sensor to be selected from the various sensors installed on the vehicle based on the fifth correspondence.

[0113] Taking the camera as the primary sensor, if the distance between the moving object and the vehicle is 5 to 10 meters, ultrasonic radar is selected in conjunction with the camera for monitoring; if the distance between the moving object and the vehicle is 2 to 5 meters, ultrasonic radar and GPS are selected in conjunction with the camera for monitoring; if the distance between the moving object and the vehicle is within 2 meters, ultrasonic radar, GPS, and inertial navigation system are selected in conjunction with the camera for monitoring.

[0114] Optionally, after the vehicle selects other types of sensors to work with cameras to monitor the surrounding environment, if the frequency of the moving object appears decreases or the moving object disappears, other types of sensors can be reduced or turned off.

[0115] This application's embodiments establish different monitoring mechanisms by sensing whether there are moving objects around the vehicle. If there are no moving objects around the vehicle, a camera is sufficient to meet the monitoring requirements. If there are moving objects around the vehicle, other sensors are selected to work in conjunction with the camera to increase monitoring intensity. This can improve monitoring accuracy while reducing wear and tear on other sensors, extending sensor lifespan, and reducing the overall energy consumption of the vehicle's sensors.

[0116] Third, the vehicle selects at least one sensor from a variety of sensors installed on the vehicle based on the barrier conditions of the surrounding environment.

[0117] The barrier situation of the surrounding environment includes whether barriers exist in the surrounding environment, the type of barriers, and the degree of barrier protection. A barrier refers to other objects that can protect the safety of a vehicle, such as those that prevent other moving objects from approaching the vehicle. This application does not limit the specific type of barrier; for example, barriers include, but are not limited to, walls, other vehicles, trees, or fences. The degree of barrier protection in the surrounding environment can refer to the extent or ability of barriers in the surrounding environment to prevent other moving objects from approaching or damaging the vehicle.

[0118] Optionally, vehicles can determine the degree of barrier required by the openness of the surrounding environment. For example, lower openness corresponds to a stronger barrier, and vice versa. Optionally, the degree of barrier required by the surrounding environment is related to the type of scene, the degree of spatial enclosure, or the size of the surrounding space. For example, on a street, the space is poorly enclosed, and there is a high flow of pedestrians and other vehicles, so the surrounding environment is relatively open and therefore has a weak barrier. In a residential area, access is restricted, so the space is generally enclosed, and there is relatively little flow of pedestrians and other vehicles, resulting in a moderate degree of openness and a moderate barrier. In a private garage, the space is small and highly enclosed, with very little flow of pedestrians and other vehicles, resulting in low openness and a strong barrier.

[0119] In this embodiment of the application, the vehicle can monitor the barrier status of the surrounding environment in various ways. For example, the barrier status of the surrounding environment can be determined based on the scene type of the surrounding environment, or based on the size of the surrounding space, or based on the degree of enclosure of the surrounding space.

[0120] Taking the judgment of the barrier status of the surrounding environment based on the degree of enclosure of the surrounding space as an example: In one possible implementation, the vehicle can monitor whether there is a safety barrier in each direction of the vehicle; if there is no safety barrier in a certain direction of the vehicle, it means that the openness of that direction is high, the barrier degree is light, and the security is poor. The sensor that can monitor that direction can be selected from the various sensors installed on the vehicle. If there is a safety barrier in another direction of the vehicle, it means that the openness of that direction is low, the barrier degree is high, and the security is high. The number of sensors used to monitor that direction can be reduced or turned off.

[0121] Optionally, the barrier is a safety barrier when the distance between the area where the barrier is located and the vehicle is less than a first threshold.

[0122] For example, Figure 7 A flowchart illustrating a sensor selection method based on environmental barrier conditions is shown, which can be applied to... Figure 1 The vehicle-mounted system shown, the method includes:

[0123] S701. When the engine is off, the vehicle's monitoring function is activated, and the vehicle enters monitoring mode.

[0124] Optionally, when the vehicle's monitoring function is enabled, all sensors may be enabled, or only some sensors may be enabled (e.g., only the camera may be enabled). This application does not impose any restrictions.

[0125] S702, Vehicle monitoring: Is there a safety barrier around the vehicle body?

[0126] For example, a vehicle can determine whether there are walls or other vehicles around it based on images captured by a camera.

[0127] Optionally, the barrier is considered a safety barrier when the distance between the barrier and the vehicle is less than a preset distance. This preset distance can be, for example, 1 meter, 1.5 meters, or 2 meters, and is not limited in this application. Furthermore, the preset distance value can differ for different types of barriers. For example, the preset distance is 1.5 meters for walls and 1 meter for other vehicles.

[0128] S703A: If there is a safety barrier on one side, the sensor on that side of the barrier will not be turned on / off.

[0129] Specifically, if the vehicle's sensor on the barrier side is not yet turned on, then the sensor on that side of the vehicle remains off; if the vehicle's sensor on that side is already turned on, then the sensor on that side of the vehicle is turned off.

[0130] Optionally, the vehicle can turn off all sensors on the barrier side to minimize sensor power consumption and extend sensor lifespan.

[0131] Optionally, the vehicle can turn off only some of the sensors on the barrier side to further improve safety performance while appropriately saving sensor power consumption.

[0132] S703B: For other sides where there is no safety barrier, the sensor on that side is turned on normally.

[0133] Here are a few more typical application scenarios:

[0134] Scenario 1: When a vehicle is parked in a parking space, with vehicles parked on one or both sides: On the side where vehicles are parked, the distance between them is narrow, making it inconvenient for pedestrians or other vehicles to pass through. Therefore, the threat or damage to that side of the vehicle is negligible. In this case, activating the sensors on that side for monitoring is not very meaningful, so the sensors on that side (such as cameras and ultrasonic radar) can be turned off or off.

[0135] Scenario 2: When a vehicle is parked on the side of the road, there is an obstacle such as a wall on one side: The presence of an obstacle such as a wall on one side and the narrow distance between the vehicle and the wall can avoid risks such as vehicle scratches, abnormal movement, and theft. The obstacle can protect the safety of the vehicle on that side, so the sensors on that side (such as cameras and ultrasonic radar) can be turned off or on.

[0136] In this embodiment, the vehicle senses whether a safety barrier exists around the vehicle body, makes reasonable use of the value of the barrier, avoids potential threats on the side of the vehicle body, avoids unnecessary wear and tear on the sensor, extends the sensor's lifespan, and improves the effectiveness of monitoring.

[0137] The above describes the implementation of each of the three elements (scene type of the surrounding environment, moving objects in the surrounding environment, and barrier conditions of the surrounding environment) used in this application for sensor selection. In practice, the above three sensor selection schemes can also be combined.

[0138] The following are some examples of possible combinations.

[0139] Fourth, the vehicle selects at least one sensor from a variety of sensors installed on the vehicle, based on the scene type of the surrounding environment and the moving objects in the surrounding environment.

[0140] For example, after the vehicle switches from driving to off state, the scene type of the vehicle's surrounding environment is first determined based on the vehicle's historical driving history, and at least one sensor corresponding to the scene type of the surrounding environment is selected; then, based on the at least one sensor, it is detected whether there are moving objects in the surrounding environment. If there are moving objects in the surrounding environment or the frequency of moving objects exceeds a preset frequency, other types or more numbers of sensors are further selected.

[0141] V. The vehicle selects at least one sensor from a variety of sensors installed on the vehicle, based on the scene type of the surrounding environment and the barriers in the surrounding environment.

[0142] For example, after the vehicle switches from driving to off state, the scene type of the vehicle's surrounding environment is first determined based on the vehicle's historical driving history, and at least one sensor corresponding to the scene type of the surrounding environment is selected; then, based on the selected sensors, it is detected whether there is a safety barrier around the vehicle. For the side without a safety barrier, some or all of the sensors on that side are selected, and for the side with a safety barrier, all sensors on that side are deselected or turned off.

[0143] VI. The vehicle selects at least one sensor from a variety of sensors installed on the vehicle based on moving objects in the surrounding environment and the barriers in the surrounding environment.

[0144] For example, after the vehicle switches from driving to off state, it first selects the cameras on the front, rear, left, and right sides. Based on these four cameras, it detects whether there are any safety barriers around the vehicle. For the side without a safety barrier, some or all sensors on that side are selected; for the side with a safety barrier, all sensors on that side are turned off. The vehicle then continues to detect whether there are moving objects in the surrounding environment based on the selected sensors. If moving objects appear in the surrounding environment or the frequency of moving objects exceeds a preset frequency, other types or more numbers of sensors are further selected.

[0145] 7. The vehicle selects at least one sensor from a variety of sensors installed on the vehicle based on the scene type of the surrounding environment, moving objects in the surrounding environment, and the barriers in the surrounding environment.

[0146] For example, after the vehicle switches from driving to off state, it first selects all four cameras on the front, rear, left, and right sides. Based on these cameras, it detects whether there are safety barriers around the vehicle. For the side without a safety barrier, some or all sensors on that side are selected; for the side with a safety barrier, all sensors on that side are turned off. Next, the vehicle detects the scene type of the surrounding environment based on the selected cameras and selects sensors corresponding to the current scene type. Then, the vehicle continues to continuously detect whether there are moving objects in the surrounding environment based on all selected sensors. If moving objects appear in the surrounding environment or the frequency of moving objects exceeds a preset frequency, other types or more numbers of sensors are further selected.

[0147] It should be understood that the above-mentioned sections four through seven are merely examples of some of the combined implementation methods. In actual implementation, there may be other combined implementation methods.

[0148] S402, The vehicle monitors the surrounding environment based on at least one of the sensors.

[0149] Specifically, the system identifies at least one factor that poses a threat to the safety of the vehicle based on at least one sensor. A specific implementation process includes, for example, the MDC (Multi-Controller) in the vehicle's ECU system controlling selected sensors to collect data about the vehicle's surrounding environment; each sensor transmitting its collected data to the MDC; and the MDC analyzing the data to determine the factors in the surrounding environment that threaten the vehicle's safety.

[0150] The following are some specific examples to illustrate this:

[0151] Example 1: Taking a camera as an example: MDC can monitor the surrounding environment based on the data collected by the camera, including whether there are obstacles, the types of obstacles (such as pedestrians, bicycles, electric vehicles, and vehicles), the distance between obstacles and vehicles, and the movement trend of obstacles relative to vehicles (such as approaching, moving away, or remaining stationary).

[0152] Example 2, taking ultrasonic radar as an example: MDC can monitor whether there are obstacles in the surrounding environment and the distance between obstacles and vehicles based on the data collected by ultrasonic radar.

[0153] Example 3, taking an inertial navigation system as an example: MDC can monitor the vibration value, movement value (or position change value), duration of vehicle vibration, duration of vehicle movement, etc., based on the data collected by the inertial navigation system.

[0154] Example 4: Taking the Global Positioning System as an example: MDC can perform vehicle location tracking, vehicle status monitoring, and vehicle track recording based on data collected by the Global Positioning System.

[0155] It should be understood that MDC relies on the ability of at least one sensor to monitor at least one factor in the surrounding environment that poses a threat to vehicle safety. Whether MDC can obtain the corresponding factor after analyzing the data collected by the at least one sensor depends on whether the corresponding factor actually exists in the surrounding environment. If the corresponding factor exists in the surrounding environment, MDC can obtain the corresponding factor by analyzing the data collected by the at least one sensor; if the corresponding factor does not exist in the surrounding environment, MDC will not obtain the corresponding factor by analyzing the data collected by the at least one sensor.

[0156] Furthermore, after obtaining information about factors in the surrounding environment that pose a threat to the vehicle's safety, the vehicle can determine the level of threat posed by these factors.

[0157] MDC can determine the threat level of the surrounding environment to a vehicle based on one or more of the following: 1) the type of factors detected by the multiple sensors; 2) the values ​​of the factors detected by the multiple sensors; 3) the duration of each factor; 4) changes in the surrounding environment; 5) the number of changes in the surrounding environment; and 6) the vehicle's speed. It should be understood that some of the above items can be obtained based on other items. For example, the vehicle's speed can be obtained through statistical analysis of the factors "vehicle travel time" and "vehicle travel distance."

[0158] This application does not limit the specific method by which vehicles classify threat levels. Two possible methods are listed below: Method 1: The MDC classifies the threat level based on the number of types of factors detected by the sensors, where at a low threat level, the MDC classifies the threat level based on the number of types of factors detected by the multiple sensors being less than at a high threat level. Method 2: The MDC classifies the threat level based on the value of the factors detected by the sensors, where at a low threat level, the MDC classifies the threat level based on the value of any one factor detected by the multiple sensors being less than at a high threat level. It should be understood that Methods 1 and 2 can be implemented individually or in combination, and this is not limited here.

[0159] This application does not limit the total number of threat levels. For example, there may be one threat level: "threat exists"; two threat levels: level 1 is "low threat" and level 2 is "high threat"; or three threat levels: level 1 is "low threat," level 2 is "high threat," and level 3 is "dangerous." Optionally, "no threat exists" can also be categorized into a single level, for example, level 0 when there is no threat.

[0160] The following example uses four threat levels, with level 0 being "no threat", level 1 being "low threat", level 2 being "high threat", and level 3 being "dangerous":

[0161] Let MDC monitor the following factors based on the aforementioned sensors: detected obstacles, obstacle distance (m), vehicle vibration (g), vehicle position change (m), duration (ms), number of parking environment changes (N), and vehicle speed (m / s). Specifically: detected obstacles: MDC detects obstacles around the vehicle based on sensors; obstacle distance: the distance from the obstacle to the vehicle based on sensors; vehicle vibration: the vibration value of the vehicle detected by MDC based on sensors; vehicle position change: the change in vehicle position detected by MDC based on sensors (possibly due to vehicle theft or inclement weather); duration: the duration of a factor monitored by MDC based on sensors, such as the duration of vehicle movement or vibration; number of parking environment changes: the number of times the parking environment changes detected by MDC based on sensors; and vehicle speed: the speed at which the vehicle moves, monitored by MDC based on sensors.

[0162] Example 1: When an animal or person walks past a vehicle, the factors measured by MDC include: detection of an obstacle. MDC can determine that the surrounding environment does not pose a threat to the vehicle, and the threat level is 0.

[0163] Example 2: When an animal or person approaches the vehicle, the factors measured by MDC include: obstacles and obstacle distance values, where the obstacle distance value is small (e.g., 0.5m). MDC can determine that the surrounding environment poses a low threat to the vehicle, with a threat level of 1.

[0164] Example 3: When an animal or person touches a vehicle, the factors measured by MDC include: detected obstacle, vehicle vibration value, and obstacle distance value. The vehicle vibration value is relatively small (e.g., 0.1g), and the obstacle distance value is very small (e.g., 0.01m). MDC can determine that the surrounding environment poses a high threat to the vehicle, with a threat level of 2.

[0165] Example 4: When an animal or person attempts to force open a car door, the factors measured by the MDC include: detected obstacle, vehicle vibration level, obstacle distance, and duration. The vehicle vibration level is relatively high (e.g., 0.5g), the obstacle distance is very low (e.g., 0.01m), and the duration is relatively long (e.g., 3s). The MDC can determine that the surrounding environment is extremely dangerous relative to the vehicle, with a threat level of 3.

[0166] Furthermore, once the threat level is determined, the vehicle can execute response events corresponding to that threat level.

[0167] Specifically, MDC can pre-set the mapping between threat levels and response events, for example, by storing the mapping in memory. Once MDC determines the threat level of the surrounding environment to the vehicle, it executes the response event corresponding to that threat level based on the mapping.

[0168] See Figure 8 Taking the four threat levels (Level 0: no threat, Level 1: low threat, Level 2: high threat, and Level 3: dangerous) as an example, the response events for each level are as follows:

[0169] After the vehicle is turned off, the sensors are activated (see the specific implementation method of S401 for the selection of the sensors to be activated), and the vehicle enters "monitoring state", that is, the sensors are used to collect data of the surrounding environment. MDC analyzes the data collected by the sensors to monitor whether there is a threat.

[0170] 1) When the threat level is "no threat", the vehicle may not execute any response events, or the response event is that the MDC controls the vehicle to continue to maintain "monitoring status", that is, to use sensors to monitor the vehicle's surrounding environment;

[0171] 2) When the threat level is "low threat", the MDC controls the vehicle to enter "warning state" and the vehicle outputs warning information, such as flashing headlights, honking the horn, and flashing the central control screen.

[0172] 3) When the threat level is "high threat", MDC controls the vehicle to enter "event recording state" to record events that occur in the surrounding environment, such as saving video images captured by cameras;

[0173] 4) When the threat level is "dangerous", the MDC controls the vehicle to enter "alarm state". The vehicle sends alarm information to the user devices associated with the vehicle (such as mobile phones, smartwatches, etc.), such as sending text messages to mobile apps, uploading video recorded by the camera to the cloud to support the user device to download, etc.

[0174] Optionally, after the "alarm state" ends for a period of time, for example... Figure 8 As shown in the 30 seconds, the MDC can control the vehicle to return to its initial "monitoring state," that is, stop sending alarm information to the user equipment associated with the vehicle and continue to use sensors to collect data on the surrounding environment. This saves power consumption.

[0175] Optionally, the vehicle can remain in a "monitoring state" throughout the execution of a response event corresponding to any threat level, that is, it can continuously use sensors to monitor the vehicle's surrounding environment and thus update the threat level in real time.

[0176] It should be understood that the process by which the MDC monitors the threat level of the surrounding environment to the vehicle can be a sequential process of traversing each threat level from low to high, that is, first switching from "no threat" to "low threat", then from "low threat" to "high threat", and then to "dangerous". The threat level of the surrounding environment monitored by the MDC to the vehicle can also directly enter one of the higher levels, such as directly entering "high threat" or "dangerous". This application does not restrict this.

[0177] Optionally, response events corresponding to high threat levels may include response events corresponding to low threat levels to further improve the vehicle's responsiveness. For example, when the threat level is "high threat," the MDC controls the vehicle to enter "event logging state," where the vehicle records events occurring in the surrounding environment while outputting warning information. Similarly, when the threat level is "dangerous," the MDC controls the vehicle to enter "alarm state," where the vehicle outputs warning information, records events occurring in the surrounding environment, and sends alarm information to the user equipment associated with the vehicle.

[0178] Optionally, the MDC executes a response event corresponding to the threat level. Specifically, the MDC sends a control command to the ECU corresponding to the controlled element, causing the ECU to drive the controlled element to execute the corresponding response event.

[0179] For example, see Figure 9 This is an example of how MDC controls each ECU to drive the corresponding controlled element to execute response events under each threat level.

[0180] 1) When the engine is off, the vehicle automatically enters monitoring mode: MDC selects ultrasonic radar and camera, and monitors the environment around the vehicle based on the data collected by ultrasonic radar and camera.

[0181] 2) When the MDC detects an object approaching the vehicle based on data collected by ultrasonic radar and cameras, it determines that the surrounding environment poses a "low threat" to the vehicle and automatically switches to warning mode: the MDC wakes up the BCM, and the BCM controls the headlights to flash and the horn to sound according to the instructions of the MDC. At the same time, the MDC wakes up the CDC, and the CDC controls the central control screen to flash according to the instructions of the MDC to warn that the camera is recording and monitoring the approaching object.

[0182] 3) When the MDC detects an object touching the vehicle based on data collected by the ultrasonic radar and camera, it determines that the surrounding environment poses a "high threat" to the vehicle and automatically switches to event recording mode: the MDC selects the inertial navigation system (i.e., the inertial navigation system, camera and ultrasonic radar monitor simultaneously), the MDC wakes up the CDC to execute the central control flashing, the camera records video and stores the video to the CDC, the external USB flash disk saves the video, and supports importing the video to a personal computer (PC) for the user to view in the car.

[0183] 4) When MDC detects a more serious threat based on data collected by ultrasonic radar, cameras, inertial navigation systems, etc. (e.g., BCM is triggered by unauthorized vehicle entry or abnormal tire pressure, or INS is triggered by collision, prying, window breaking, abnormal vibration, or movement), MDC determines that the surrounding environment poses a "danger" to the vehicle and automatically switches to alarm mode: MDC wakes up CDC, CDC increases the display brightness of the central control screen, CDC turns the speaker volume to maximum to support voice communication, CDC uploads previously recorded videos to the cloud via TBOX, pushes SMS or APP reminders to the user's mobile phone, and supports the user's mobile phone to download videos from the cloud.

[0184] Based on the above, the vehicle in this embodiment can monitor potential threats in the surrounding environment by combining different sensors, based on at least one of the following: scene type, moving objects in the surrounding environment, and barrier conditions. This optimizes traditional monitoring mechanisms, enabling the perception and identification of multiple types of events. It can improve monitoring accuracy while reducing sensor wear and extending sensor lifespan. Furthermore, the vehicle in this embodiment can identify threat levels based on the monitored factors and execute response events corresponding to those levels, thereby promptly eliminating threats and improving vehicle safety when the engine is off.

[0185] Based on the same technical concept, this application also provides a vehicle monitoring device 1000, which has the capability to implement... Figures 4-9 The function of the method steps shown, for example, the device 1000 includes functions for performing the above. Figures 4-9 The function, module, unit, or means of the method steps shown can be implemented by software, hardware, or by hardware executing the corresponding software.

[0186] For example, see Figure 10 The device 1000 may include:

[0187] The processing unit 1001 is used to select at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions of the surrounding environment.

[0188] The monitoring unit 1002 is used to monitor the surrounding environment of the vehicle based on at least one sensor.

[0189] For details on the specific implementation of the methods and steps performed by each of the above units, please refer to the above. Figures 4-9 The specific implementation of the corresponding method steps performed by the vehicle in the illustrated embodiment will not be described in detail here.

[0190] Based on the same technical concept, embodiments of this application also provide a vehicle-mounted device 1100. See also Figure 11 The on-board device includes at least one processor 1101, which is used to perform... Figures 4-9 The steps are shown.

[0191] Optionally, the on-board device 1100 may also include a memory 1102, in Figure 11 The dashed box in the figure indicates that memory 1102 is optional for vehicle equipment 1100.

[0192] Optionally, the memory 1102 and the processor 1101 are connected via a bus for communication. Figure 11 The bus is represented by a thick black line.

[0193] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0194] For example, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0195] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0196] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0197] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0198] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium, which is used to store instructions that, when executed, cause... Figures 4-9 The method shown is implemented.

[0199] Based on the same technical concept, this application also provides a computer program product that stores instructions that, when run on a computer, cause the computer to perform actions such as... Figures 4-9 The method shown.

[0200] It should be understood that the above embodiments can be combined with each other.

[0201] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0202] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0203] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0204] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0205] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A vehicle monitoring method, characterized in that, The method, applied to vehicles in a powered-off state, includes: Based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment, at least one sensor is selected from a variety of sensors installed on the vehicle. The vehicle's surrounding environment is monitored based on at least one of the sensors; The method further includes, before selecting at least one sensor from a plurality of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment: Based on the first sensor on the vehicle, it is determined that a moving object has appeared in the surrounding environment; The moving object is determined to move toward the vehicle, and at least one of the following conditions is met: the frequency of occurrence of the moving object exceeds a preset frequency, the duration of occurrence of the moving object exceeds a preset duration, or the moving object is within a preset range of the vehicle. Based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment, at least one sensor is selected from a plurality of sensors installed on the vehicle, including: Select the first sensor and other types of sensors besides the first sensor; The selection of the first sensor and other types of sensors besides the first sensor includes: The other types of sensors are determined based on the correspondence between at least one of the moving object's direction of movement, frequency of occurrence, duration of occurrence, or distance from the vehicle and the sensor type.

2. The method according to claim 1, characterized in that, Before selecting at least one sensor from a plurality of sensors mounted on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment, the method further includes: Identify the scene type of the surrounding environment; Based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment, at least one sensor is selected from a plurality of sensors installed on the vehicle, including: Based on the correspondence between scene type and sensor, a sensor corresponding to the scene type of the surrounding environment is selected from a variety of sensors installed on the vehicle.

3. The method according to claim 1 or 2, characterized in that, The first sensor is a camera; Before selecting at least one sensor from a plurality of sensors mounted on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions in the surrounding environment, the method further includes: Images of the vehicle's surrounding environment are captured using the camera installed on it; Determining the presence of a moving object in the surrounding environment based on a first sensor on the vehicle includes: Based on the images captured by the camera, it is determined that there are moving objects in the surrounding environment.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The barrier situation is obtained, which includes the presence of a barrier in a first region, the first region being located at a first position of the vehicle and the distance between the first region and the vehicle being less than a first threshold. The at least one sensor does not include a sensor used for monitoring in the first orientation.

5. The method according to claim 3, characterized in that, The method further includes: The barrier situation is obtained, which includes the presence of a barrier in a first region, the first region being located at a first position of the vehicle and the distance between the first region and the vehicle being less than a first threshold. The at least one sensor does not include a sensor used for monitoring in the first orientation.

6. The method according to claim 4, characterized in that, The barrier includes walls or other vehicles.

7. The method according to claim 5, characterized in that, The barrier includes walls or other vehicles.

8. The method according to any one of claims 1-7, characterized in that, Monitoring the surrounding environment of the vehicle based on at least one of the sensors includes: Based on the at least one sensor, at least one factor in the surrounding environment that poses a threat to the safety of the vehicle is monitored; Based on at least one of the factors, determine the threat level of the surrounding environment to the vehicle; Execute the response event corresponding to the threat level.

9. The method according to claim 8, characterized in that, Determining the threat level of the surrounding environment to the vehicle based on at least one of the factors includes: The threat level of the surrounding environment to the vehicle is determined based on at least one of the following: the type of the at least one factor, the value of the at least one factor, the duration of the at least one factor, the number of changes in the surrounding environment, or the vehicle speed.

10. The method according to claim 8, characterized in that, The threat levels, from lowest to highest, include Level 1, Level 2, and Level 3. The response events corresponding to the first level include any one or more of the following: headlights flashing, horn honking, or central control screen flashing; The response events corresponding to the second level include: response events corresponding to the first level, as well as recording and saving video using a camera; The response events corresponding to the third level include: the response events corresponding to the second level, sending a reminder to the user device, and uploading the video to the cloud to support downloading by the user device.

11. The method according to claim 9, characterized in that, The threat levels, from lowest to highest, include Level 1, Level 2, and Level 3. The response events corresponding to the first level include any one or more of the following: headlights flashing, horn honking, or central control screen flashing; The response events corresponding to the second level include: response events corresponding to the first level, as well as recording and saving video using a camera; The response events corresponding to the third level include: the response events corresponding to the second level, sending a reminder to the user device, and uploading the video to the cloud to support downloading by the user device.

12. A vehicle monitoring device, characterized in that, For use in vehicles in a powered-off state, the device includes: The processing unit is configured to select at least one sensor from a variety of sensors installed on the vehicle based on at least one of the following: scene type of the vehicle's surrounding environment, moving objects in the surrounding environment, and barrier conditions of the surrounding environment. A monitoring unit is used to monitor the surrounding environment of the vehicle based on the at least one sensor; The processing unit is further configured to: determine, based on the first sensor on the vehicle, that a moving object has appeared in the surrounding environment; The moving object is determined to move toward the vehicle, and at least one of the following conditions is met: the frequency of occurrence of the moving object exceeds a preset frequency, the duration of occurrence of the moving object exceeds a preset duration, or the moving object is within a preset range of the vehicle. Select the first sensor and other types of sensors besides the first sensor; Specifically, when selecting the first sensor and other types of sensors besides the first sensor, the processing unit is used to: determine the other types of sensors based on the correspondence between at least one of the moving object's direction of movement, frequency of occurrence, duration of occurrence, or distance from the vehicle and the sensor type.

13. The apparatus according to claim 12, characterized in that, The processing unit is used for: Identify the scene type of the surrounding environment; Based on the correspondence between scene type and sensor, a sensor corresponding to the scene type of the surrounding environment is selected from a variety of sensors installed on the vehicle.

14. The apparatus according to claim 12 or 13, characterized in that, The first sensor is a camera; The processing unit is used for: The camera is used to capture images of the vehicle's surrounding environment. Based on the images captured by the camera, it is determined that there are moving objects in the surrounding environment.

15. The apparatus according to claim 12 or 13, characterized in that, The processing unit is also used for: The barrier situation is obtained, which includes the presence of a barrier in a first region, the first region being located at a first position of the vehicle and the distance between the first region and the vehicle being less than a first threshold. The at least one sensor does not include a sensor used for monitoring in the first orientation.

16. The apparatus according to claim 14, characterized in that, The processing unit is also used for: The barrier situation is obtained, which includes the presence of a barrier in a first region, the first region being located at a first position of the vehicle and the distance between the first region and the vehicle being less than a first threshold. The at least one sensor does not include a sensor used for monitoring in the first orientation.

17. The apparatus according to claim 15, characterized in that, The barrier includes walls or other vehicles.

18. The apparatus according to claim 16, characterized in that, The barrier includes walls or other vehicles.

19. The apparatus according to any one of claims 12-18, characterized in that, The monitoring unit is specifically used for: Based on the at least one sensor, at least one factor in the surrounding environment that poses a threat to the safety of the vehicle is monitored; Based on at least one of the factors, determine the threat level of the surrounding environment to the vehicle; Execute the response event corresponding to the threat level.

20. The apparatus according to claim 19, characterized in that, When determining the threat level of the surrounding environment to the vehicle based on at least one factor, the monitoring unit is specifically used for: The threat level of the surrounding environment to the vehicle is determined based on at least one of the following: the type of the at least one factor, the value of the at least one factor, the duration of the at least one factor, the number of changes in the surrounding environment, or the vehicle speed.

21. The apparatus according to claim 19, characterized in that, The threat levels, from lowest to highest, include Level 1, Level 2, and Level 3. The response events corresponding to the first level include any one or more of the following: headlights flashing, horn honking, or central control screen flashing; The response events corresponding to the second level include: response events corresponding to the first level, as well as recording and saving video using a camera; The response events corresponding to the third level include: the response events corresponding to the second level, sending a reminder to the user device, and uploading the video to the cloud to support downloading by the user device.

22. The apparatus according to claim 20, characterized in that, The threat levels, from lowest to highest, include Level 1, Level 2, and Level 3. The response events corresponding to the first level include any one or more of the following: headlights flashing, horn honking, or central control screen flashing; The response events corresponding to the second level include: response events corresponding to the first level, as well as recording and saving video using a camera; The response events corresponding to the third level include: the response events corresponding to the second level, sending a reminder to the user device, and uploading the video to the cloud to support downloading by the user device.

23. A vehicle monitoring device, characterized in that, For use in a vehicle in a powered-off state, the device includes a memory and a processor, the memory storing computer program instructions, and the processor executing the computer program instructions to perform the method as described in any one of claims 1-11.

24. A vehicle, characterized in that, include: Multiple sensors; as well as The vehicle monitoring device as described in any one of claims 12-22.

25. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-11 to be implemented.

26. A computer program product, characterized in that, The computer program product stores instructions that, when executed on a processor, cause the method described in any one of claims 1-11 to be implemented.

Images