Vehicle control method and related device

Abstract

The present disclosure provides a vehicle control method and related equipment, which belongs to the technical field of vehicles. In the present disclosure, when a target object such as a person in a target scene is in an abnormal state such as falling, a target reminding message is provided to the vehicle, and then a suitable reminding mode is selected in combination with vehicle state data to remind the driver and other people in the vehicle of the abnormality, so that the people in the vehicle know that the target object is in an abnormal state, and an implementation mode capable of notifying the abnormal state of a certain object in a certain scene outside the vehicle to the inside of the vehicle is provided.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle technology, and in particular to a vehicle control method and related equipment. Background Technology

[0002] In related technologies, the interconnection between vehicles and smart home products in residences often focuses on controlling smart home products through the vehicle. However, while the driver is driving, there is often a need to be aware of abnormal states of people inside the residence. For example, if an elderly person living alone falls, this information needs to be obtained through the vehicle immediately. Therefore, there is an urgent need to provide a technical solution that can notify the vehicle's interior of abnormal states of an object in a certain scenario outside the vehicle. Summary of the Invention

[0003] This disclosure provides a vehicle control method and related equipment, which can solve the problem of the urgent need for a technical solution that can provide anomaly alerts to drivers via a vehicle regarding abnormal states of occupants in a residence. The technical solution is as follows: On one hand, a vehicle control method is provided, the method comprising: upon receiving a target reminder message, acquiring vehicle status data, wherein the target reminder message indicates that a target object in a target scene is in an abnormal state, the vehicle status data is used to indicate the presence of occupants in the vehicle and / or the vehicle's driving state, and the presence of occupants in the vehicle is used to indicate whether there are any non-reminder objects in the vehicle other than the reminder object; determining a reminder method corresponding to the target reminder message based on the vehicle status data, and controlling the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

[0004] In one possible implementation, the reminder method includes at least one of voice reminder, text reminder, video playback, icon reminder, and indicator light reminder.

[0005] In one possible implementation, determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: when the presence of occupants in the vehicle is limited to the reminder target and the vehicle is not parked, determining a voice reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content; or, when the presence of occupants in the vehicle is limited to the reminder target and the vehicle is not parked, determining a voice reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content, and displaying a video playback pop-up window including a playback control, and in response to a trigger operation on the playback control, acquiring and playing the monitoring screen of the target object in the target scene.

[0006] In one possible implementation, determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: when the presence of occupants in the vehicle is limited to the reminder target and the vehicle is in a parked state, determining voice reminder and video playback as the reminder methods corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or the reminder content of the voice reminder includes preset voice content, and the playback content of the video playback includes the monitoring screen of the target object in the target scene.

[0007] In one possible implementation, determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: when the presence of occupants in the vehicle is either a reminder target or a non-reminder target, determining at least one of text reminders, icon reminders, and indicator light reminders as the reminder method corresponding to the target reminder message, wherein the reminder content of the text reminder is generated based on the target reminder message or includes preset text content; or, when the presence of occupants in the vehicle is either a reminder target or a non-reminder target, determining at least one of text reminders, icon reminders, and indicator light reminders as the reminder method corresponding to the target reminder message, wherein the reminder content of the text reminder is generated based on the target reminder message or includes preset text content, and displaying a video playback pop-up window including a playback control, and in response to a trigger operation on the playback control, acquiring and playing the monitoring screen of the target object in the target scene.

[0008] In one possible implementation, determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: when the vehicle is in a parked state, determining at least one of voice reminder, text reminder, video playback, icon reminder, and indicator light reminder as the reminder method corresponding to the target reminder message; the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content; the reminder content of the text reminder is generated based on the target reminder message or includes preset text content; and the playback content of the video playback includes monitoring footage of the target object in the target scene. Alternatively, when the vehicle is in a non-parked state, determining at least one of voice reminder, text reminder, icon reminder, and indicator light reminder as the reminder method corresponding to the target reminder message; the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content; and the reminder content of the text reminder is generated based on the target reminder message or includes preset text content.

[0009] In one possible implementation, the target alert message is generated and sent to the vehicle by the detection device after performing object detection and behavior recognition on the scene image of the target scene and determining that the behavior recognition result of the target object includes specified abnormal behavior; or, the target alert message is generated and sent to the vehicle by the detection device after detecting that the target object in the target scene has an abnormal breathing frequency, wherein the breathing frequency of the target object is determined based on the Fresnel effect of electromagnetic waves of WiFi.

[0010] In one possible implementation, the reminder target includes the driver, a preset user, or the user indicated by the target reminder message.

[0011] On the other hand, a vehicle control device is provided, the device comprising: a vehicle status data acquisition module, configured to acquire vehicle status data of the vehicle upon receiving a target reminder message, wherein the target reminder message indicates that a target object in a target scene is in an abnormal state, the vehicle status data is used to indicate the presence of occupants in the vehicle and / or the vehicle's driving state, and the presence of occupants in the vehicle is used to indicate whether there are any non-reminder objects in the vehicle other than the reminder object; and a reminder module, configured to determine the reminder method corresponding to the target reminder message based on the vehicle status data, and control the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

[0012] On the other hand, a vehicle control system is provided, comprising a state detection device and a vehicle, wherein: the state detection device is used to determine whether a target object in a target scene is in an abnormal state, and sends a target reminder message to the vehicle when the target object is in an abnormal state; the vehicle is used to acquire vehicle state data upon receiving the target reminder message, wherein the target reminder message indicates that the target object in the target scene is in an abnormal state, the vehicle state data indicates the presence of occupants and / or the vehicle's driving state, the presence of occupants indicates whether there are any non-reminder objects in the vehicle besides the reminder object, and determines the reminder method corresponding to the target reminder message based on the vehicle state data, and controls the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

[0013] On the other hand, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the method provided in any of the above embodiments.

[0014] On the other hand, a non-transitory computer-readable storage medium is provided, which stores computer instructions for causing a computer to perform the method provided in any of the above embodiments.

[0015] On the other hand, a computer program product is provided, including computer program instructions that, when executed on a computer, cause the computer to perform the method provided in any of the above embodiments.

[0016] The beneficial effects of the technical solution provided in this disclosure are as follows: When people or other objects in the target scene are in an abnormal state such as falling, a target reminder message is provided to the vehicle. Then, by combining the vehicle status data, an appropriate reminder method is selected to remind the driver and other people in the vehicle of the abnormal state, so that the people in the vehicle are aware that the target object is in an abnormal state. This provides an implementation method that can notify the inside of the vehicle of the abnormal state of an object in a certain scene outside the vehicle. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of this disclosure; Figure 2This is a schematic flowchart of a vehicle control method provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of the vehicle control device provided in the embodiments of this disclosure; Figure 4 This is a schematic diagram of a vehicle control system provided in an embodiment of this disclosure; Figure 5 This is a specific structural schematic diagram of the vehicle control system provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this disclosure. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0020] In this disclosure, the terms "first" and "second" are used to distinguish between identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on the quantity or execution order.

[0021] In the field of vehicle-home connectivity, the rapid development of the Internet of Things (IoT) enables interconnection between vehicles and smart home products, allowing drivers to control smart homes from inside the vehicle. Much of the research and development focuses on vehicle-to-home control, where vehicles issue commands to smart home devices. However, there are no solutions for home-to-vehicle control or proactive information reporting from the vehicle to "remind" others. For example, how can we promptly obtain information about elderly people living alone who have fallen while the driver is driving?

[0022] To address the aforementioned issues, the solution provided in this disclosure enables timely alerts for abnormal feedback from smart home devices during driving, facilitating prompt action. For example, home monitoring and WiFi (Wireless Fidelity) router terminals can proactively report the status of elderly individuals and children. Combined with vehicle-home interconnection terminal alerts, pop-up displays can be displayed on the vehicle's infotainment system without compromising driving safety. By embedding a vehicle-home interconnection app into the vehicle platform and linking it to the smart home monitoring system, the home monitoring system can detect falls and actions, while the WiFi router can detect abnormal breathing. Based on cloud-to-cloud interaction (interaction between IoT cloud services and vehicle-to-vehicle cloud services), alerts are sent to the vehicle's infotainment system. The system then integrates this with the driver's current behavior to provide voice prompts, pop-up windows, and other alerts and displays. This data-driven intelligent vehicle-home interconnection system moves from "single-point intelligence" to "integrated intelligence," moving beyond simple vehicle-to-home control and enhancing security by adding protective features.

[0023] Before providing a detailed explanation of the vehicle control method provided in the embodiments of this disclosure, the application scenarios provided in the embodiments of this disclosure will be introduced first. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided by an embodiment of this disclosure, such as... Figure 1 As shown, the vehicle-mounted cloud service 120 can send a target reminder message to the vehicle 110. After receiving the target reminder message, the vehicle 110 can combine its current vehicle status data to determine the presence of people in the vehicle and / or the vehicle's driving status, and then determine the reminder method corresponding to the target reminder message, so that the vehicle can remind the reminder object of the abnormality through the reminder method.

[0024] Optionally, vehicle 110 can be at least one of the following: a gasoline-powered vehicle, an electric vehicle, a hybrid vehicle, a fuel cell vehicle, and a solar-powered vehicle. A hybrid vehicle refers to a combination of a gasoline-powered vehicle and an electric vehicle. In this embodiment, vehicle 110 is a vehicle equipped with an intelligent system, and the specific type of vehicle is not specifically limited.

[0025] It should be noted that the data disclosed herein (including but not limited to surveillance videos used by the target entity) has been authorized by the user or by all parties, and the collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant regions. For example, the surveillance videos disclosed herein were obtained with full authorization.

[0026] In this embodiment of the disclosure, when a person or other object in the target scene is in an abnormal state such as falling, a target reminder message is provided to the vehicle. Then, based on the vehicle status data, an appropriate reminder method is selected to remind the driver and other people in the vehicle of the abnormal state, so that the people in the vehicle are aware that the target object is in an abnormal state. This provides a way to remind the driver and other people in the vehicle of the abnormal state of the people in the residence through the vehicle.

[0027] Based on the above, the vehicle control method provided in the embodiments of this disclosure will be described. Figure 2 This is a flowchart of a vehicle control method provided in an exemplary embodiment of this disclosure. In this embodiment, as an example, the method can be executed by an in-vehicle controller and a vehicle alert device (such as an indicator light, vehicle screen, vehicle audio equipment, etc.). Figure 2 As shown, the method includes the following steps: Step S210: If the target reminder message is received, obtain the vehicle status data.

[0028] Among them, the target reminder message indicates that the target object in the target scene is in an abnormal state, the vehicle status data is used to indicate the presence of people in the vehicle and / or the vehicle driving status, and the presence of people in the vehicle is used to indicate whether there are non-reminder objects in the vehicle other than the reminder object.

[0029] The target scenario can be a scenario defined by those skilled in the art, and the target object can also be an object defined by those skilled in the art. Target scenarios include, but are not limited to, interiors of residences, interiors of shops, courtyards, etc.; target objects include, but are not limited to, elderly people, children, and other people in residences; target objects can also be pets in the home, such as pet cats and dogs. An abnormal state can be caused by a person falling, abnormal breathing rate, or by actions such as a pet dog chewing on a sofa; it can also be any other state defined by those skilled in the art as an abnormal state.

[0030] In some embodiments, a target alert message is generated and sent to a vehicle by a detection device after performing object detection and behavior recognition on scene images of a target scene and determining that the behavior recognition result of the target object includes a specified abnormal behavior. The object detection and behavior recognition methods can be implemented using existing technologies known to those skilled in the art, and will not be elaborated here. The detection device can be a device such as a camera, which captures monitoring footage of the target scene and then performs object recognition (determining whether a target object exists in the target scene) and behavior recognition (determining whether the identified target object exhibits a specified abnormal behavior, such as falling) on ​​the monitoring footage locally on the detection device or on an IoT server. If the object detection result includes the target object, and the behavior detection result includes the target object performing the specified abnormal behavior, then the target object in the target scene can be considered to be in an abnormal state, triggering the generation of a target alert message. The target alert message may carry a message identifier, which can be used to match which vehicle the target alert message should be sent to, and also to match the alert object corresponding to the target alert message. As an example, the target alert message may include at least one of the following: a specific target object identifier, the content of the abnormal behavior (such as falling), the time of the abnormal behavior, and an abnormal state identifier. As another example, the target alert message could also only include an abnormal status indicator. When the vehicle receives this target alert message, it notifies the corresponding target object in the target scenario that a specified abnormal action has occurred. The target object in the target scenario can be one or more, and can be specifically set by those skilled in the art as needed. This method can promptly alert occupants of the vehicle to events such as elderly people falling in their homes.

[0031] As an example, it is possible to determine whether a person has fallen by using human posture recognition based on video surveillance. The specific determination method can be implemented using existing technologies known to those skilled in the art, and will not be elaborated here.

[0032] In other embodiments, the target alert message is generated and sent to the vehicle by the detection device when it detects an abnormal breathing frequency of a target object in the target scene. The breathing frequency of the target object is determined based on the Fresnel effect of electromagnetic waves in WiFi. For example, dual-beam coherent detection can be used to measure human breathing frequency. The detection signal is focused onto two beams using the beamforming function of a WiFi 6 access point (AP). The AP adjusts the power of each subcarrier of the two beams based on the power difference measured by the receiver, ensuring that the amplitude of each subcarrier of the two beams is equal at the receiver. The AP compensates for the fixed phase difference caused by the path difference between the two beams, transforming the periodic phase difference caused by human breathing into a periodic change in the coherent signal strength, thus extracting the weak human breathing signal and obtaining the human breathing frequency. The method of determining breathing frequency using the Fresnel effect of electromagnetic waves in WiFi can be implemented using existing technologies, which will not be elaborated here.

[0033] As another example, based on changes in electromagnetic waves sensed by WiFi, it can determine whether a person has abnormal breathing (rapid, difficult, stopped, etc.). The principle is that based on the Fresnel effect of electromagnetic waves of WiFi, it targets the CSI (channel state information) parameters (phase and amplitude changes) caused by the amplitude changes of the human chest cavity. At the same time, it establishes regular training data and performs normalization processing. Then, it can compare the real-time respiratory rate detected in real time with the baseline respiratory rate obtained from the regular training data established under normal conditions. If the deviation between the real-time respiratory rate and the baseline is greater than the preset deviation level, then the respiratory rate can be considered to be abnormal.

[0034] In some embodiments, the status of people in the home can be comprehensively determined by combining video and WiFi sensing results (as blind spot compensation). If the above-mentioned abnormality occurs, the abnormality information is reported and a target alert message is received.

[0035] In some embodiments, the determination of whether the target object in the target scene is in an abnormal state can also be achieved by using wearable devices, mobile phones, or other smart devices worn by the target object. If it is in an abnormal state, a target reminder message is generated.

[0036] In some embodiments, the targeted alert message can also be generated after cross-validation using at least two methods for determining abnormal states. For example, object detection and behavior recognition of images obtained through a camera might indicate the presence of a specific abnormal behavior. However, in cases like falls, the elderly person might get up on their own without serious consequences. Reporting every incident could cause unnecessary excessive worry for the recipient. In such cases, data such as respiratory rate determined by the Fresnel effect of electromagnetic waves based on WiFi, heart rate, and blood pressure from smart wearable devices can be used to comprehensively assess whether a targeted alert message needs to be generated. Using the above example, if the elderly person's heart rate, respiratory rate, or blood pressure are detected to be abnormal after a fall, then triggering the generation of a targeted alert message can make the alert more targeted and avoid many unnecessary alerts.

[0037] In some embodiments, the target alert message can also be generated after cross-validation based on the duration of the abnormal state. For example, issuing an alert as soon as a cat scratches the sofa might mean the cat only scratches briefly and then leaves; in this case, even if the alert is issued to the target, there's no need to chase the cat away. Similarly, an elderly person who has fallen might immediately get up. To avoid such "ineffective alerts," the longest interval between multiple received target alert messages can be recorded as the duration, or the duration of the abnormal state can be used to determine whether to trigger an abnormal alert. If the duration is less than a preset duration threshold, no alert is issued; otherwise, an abnormal alert is issued. A corresponding preset duration threshold can be set based on at least one of the following: different abnormal behaviors, target objects, and target scenarios.

[0038] In some embodiments, the alert target includes the driver, a preset user, or the user indicated by the target alert message. The driver can be identified by determining the occupant's seating position. The preset user can be any person in the vehicle other than the driver. The preset user can be designated by someone skilled in the art as needed; for example, a member in any seating position can be designated as the preset user, or an authenticated occupant can be designated as the preset user through identity verification methods such as facial recognition. An exemplary method for confirming the user indicated by the target alert message is as follows: the target alert message may include an identity identifier of the alert target. This identity identifier can be used to match the facial features of the alert target. Facial images of occupants in the vehicle are captured, and the image features of the captured images are matched with the matched facial features. If a match is successful, then the occupant is designated as the preset user. The above description uses facial recognition technology as an example of identity verification. Those skilled in the art can also use other known identity verification technologies to confirm the preset user, which will not be elaborated upon here.

[0039] As an example, vehicle status data includes, but is not limited to, at least one of the following: vehicle speed, in-vehicle app keep-alive status, and in-vehicle occupant data (obtainable through DMS (Driver Monitoring System) data, etc.). The in-vehicle app keep-alive status refers to the app that performs the anomaly alert, or any apps related to the anomaly alert function. The keep-alive status of the in-vehicle app determines whether an alert can be issued normally. If the in-vehicle app is online, an alert can be issued normally; otherwise, if it is offline, an alert cannot be issued normally, and the process ends. Vehicle speed determines whether the vehicle is parked. In-vehicle occupant data, such as DMS data, determines whether there are other non-alert-related individuals present.

[0040] Vehicle speed can determine the vehicle's driving status, whether it is parked or not. Passenger data can determine the presence of people inside the vehicle, such as whether a driver is present, whether a preset user is present and their location, whether the user indicated by the target alert message is present and their location, and whether there are any non-target individuals not included in the alert. This allows for the selection of appropriate alert methods based on different situations when issuing subsequent alerts.

[0041] Vehicle status data can be collected either triggered upon receiving a target alert message, or the collection frequency can be pre-set, with the most recently collected vehicle status data being extracted as the current vehicle status data upon receiving the target alert message. The specific timing of vehicle status data collection can be set according to the needs of those skilled in the art.

[0042] Step S220: Determine the reminder method corresponding to the target reminder message based on the vehicle status data, and control the vehicle to remind the reminder object of the abnormality through the reminder method.

[0043] In some embodiments, the reminder method includes at least one of voice reminder, text reminder, video playback, icon reminder, and indicator light reminder.

[0044] Voice reminders can be audio content played through the vehicle's audio system. For example, the voice reminder content can be generated based on a target reminder message. For instance, if the target reminder message contains detailed information such as the identifier of the target object and the time of the abnormal state, the voice content is generated and played based on this information. The recipient can quickly understand the current situation by listening to the broadcast voice content. The voice reminder content can be set to be brief or detailed as needed. Brief content contains less information than detailed content; for example, brief content only states that abnormal behavior occurred, while detailed content includes when, where, which target object, and what kind of abnormal behavior occurred. The above is just one example; the specific voice reminder content can be configured by those skilled in the art as needed. As another example, the voice reminder content can directly use preset voice content, which can be set by those skilled in the art as needed, such as "Person XX has fallen." In some embodiments, the target reminder message can also be configured with corresponding reminder levels, and then corresponding preset voice content can be determined according to different reminder levels. Each reminder level is configured with corresponding preset voice content, so that the recipient can intuitively assess the urgency of the event based on the preset voice content after hearing the voice reminder. The rules for setting alert levels can be set by those skilled in the art as needed, such as setting the alert level for certain abnormal behaviors.

[0045] The determination of the reminder content for text reminders is similar to that for voice reminders. Text reminder content is generated based on the target reminder message or includes preset text content. The specific determination method can be found in the previous embodiments regarding the determination method for voice reminder content, and will not be elaborated upon here. The difference is that voice reminders are broadcast through the vehicle's audio system to remind the corresponding object, while text reminders are displayed on the vehicle's infotainment screen or head-up display (HUD).

[0046] As an example, the display location of voice and text reminders can be determined based on the location of the recipient. For instance, if the recipient is the driver, a text reminder can be displayed on the vehicle's head-up display or main control screen, while a voice reminder can be played privately through speakers located on the driver's headrest or other similar locations. Alternatively, if the recipient is a rear passenger, a text reminder can be displayed on the rear screen (if one is provided), and a voice reminder can be played through speakers near that passenger, thus enhancing the privacy of the reminder.

[0047] Video playback can be achieved by playing surveillance video of the target scene.

[0048] For icon-based alerts, any number of alert icons can be preset by the object being alerted or by the system. These icons can be similar to using a wrench icon to indicate that a vehicle needs maintenance, defining one or more icons to represent an abnormal state of the target object in a specific scenario. Icons can also be pre-defined images, displayed on the vehicle's infotainment screen or head-up display for alerting users. When there are at least two alert icons, different icons can be configured to correspond to different target objects, different target scenarios, or different abnormal behaviors.

[0049] Indicator lights can provide alerts about malfunctions through their color, flashing frequency, or constant illumination. For example, reusing existing vehicle indicator lights can save on vehicle modification costs. When reusing existing indicator lights, the color or flashing frequency (e.g., alternating between long and short flashes) can be adjusted to provide an alert.

[0050] Voice alerts, icon alerts, and indicator light alerts offer greater privacy compared to voice alerts and video playback. When the recipient is the driver, these methods have minimal impact on driving safety. Text alerts, due to their typically short length and adjustable display position, also enhance privacy and reduce their impact on driving safety. Since video playback requires significant attention to the monitoring screen, the decision to play the video can be made based on the recipient's request.

[0051] In some embodiments, determining the reminder method corresponding to the target reminder message based on vehicle status data includes: when the presence of occupants in the vehicle is limited to the reminder target and the vehicle is not parked, determining a voice reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content; or, when the presence of occupants in the vehicle is limited to the reminder target and the vehicle is not parked, determining a voice reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content, and displaying a video playback pop-up window including a playback control, and in response to a trigger operation on the playback control, acquiring and playing the monitoring screen of the target object in the target scene.

[0052] "Not parked" can refer to any state other than parking. When the vehicle speed is 0, the vehicle's driving state is considered parked. As an example, it could also be required that the vehicle be in Park (P) gear, in which case the speed is naturally 0, and the vehicle's driving state is considered parked. Defining the vehicle's driving state as "not parked" when the speed is 0 but the vehicle is not in Park (e.g., in Neutral (N) gear, or when the speed is maintained at 0 by applying the brakes) further enhances driving safety. Of course, those skilled in the art can also determine whether the vehicle is parked solely based on speed; for example, a speed of 0 indicates parking, and a speed not of 0 indicates non-parking.

[0053] The presence of occupants inside the vehicle can be determined using in-cabin video surveillance equipment. This process can be reused from existing DMS systems. The in-cabin images reveal the current distribution of occupants within the vehicle. If necessary, this can be combined with identity verification to identify which individuals or groups are to be alerted. If the presence of occupants is limited to those requiring alerts, it means that regardless of the alert method used, it will not be known to outsiders. Therefore, the alert method can be selected based on the needs of the individuals being alerted or pre-set rules.

[0054] The presence of occupants in the vehicle can also be detected by reusing the vehicle's existing UWB (Ultra-Wideband) antenna for liveness detection and occupant location. The specific location method can be implemented in ways known to those skilled in the art, such as using AoA (Angle of Arrival) technology to locate occupants in the vehicle, which will not be elaborated here.

[0055] As an example, during the driving phase, to avoid driver distraction, abnormal alerts can be issued via voice prompts. Furthermore, when the recipient requests permission, they can obtain permission to play the monitoring footage through voice interaction, or by triggering a playback control on a pop-up window. This allows the recipient to view the monitoring footage only after confirming their own driving safety, satisfying both vehicle safety requirements and providing a timely and intuitive display of any abnormal behavior and the current status of the target.

[0056] The above method takes into account both the privacy of the reminder content and the safety requirements of driving.

[0057] In some embodiments, determining the reminder method corresponding to the target reminder message based on vehicle status data includes: when only the reminder target exists in the vehicle and the vehicle is parked, determining voice reminder and video playback as the reminder methods corresponding to the target reminder message. The reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content, and the playback content of the video playback includes monitoring footage of the target object in the target scene. When only the reminder target exists and no other non-reminder targets are present, the privacy of the reminder content is relatively better, and since the vehicle is not in motion and is in a parked state, watching the video will not disturb the driver's normal driving, thus avoiding any impact on vehicle driving safety.

[0058] In some embodiments, determining the reminder method corresponding to the target reminder message based on vehicle status data includes: when the presence of occupants in the vehicle is either a reminder target or a non-reminder target, determining at least one of text reminders, icon reminders, and indicator light reminders as the reminder method corresponding to the target reminder message, wherein the reminder content of the text reminder is generated based on the target reminder message or includes preset text content; or, when the presence of occupants in the vehicle is either a reminder target or a non-reminder target, determining at least one of text reminders, icon reminders, and indicator light reminders as the reminder method corresponding to the target reminder message, wherein the reminder content of the text reminder is generated based on the target reminder message or includes preset text content, and displaying a video playback pop-up window including a playback control, and in response to a trigger operation on the playback control, acquiring and playing the monitoring screen of the target object in the target scene.

[0059] When a non-identified person (i.e., an "outsider") is present in the vehicle, the surveillance footage is relatively private. Therefore, a second confirmation is required before deciding whether to play the footage. The above embodiment uses a pop-up window to indicate whether to play the footage. Voice interaction or other methods can also be used to assist in determining permission. This further enhances the privacy of the alert and improves the user experience. Under the above conditions, only the presence of a non-identified person is considered; whether the vehicle is parked or not is not taken into account. Since the playback decision is based on interaction, the person being alerted can decide whether watching the footage will affect driving safety by choosing whether to play it.

[0060] In some embodiments, determining the reminder method corresponding to the target reminder message based on vehicle status data includes: when the vehicle is in a parked driving state, determining at least one of voice reminder, text reminder, video playback, icon reminder, and indicator light reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content, the reminder content of the text reminder is generated based on the target reminder message or includes preset text content, and the playback content of the video playback includes the monitoring screen of the target object in the target scene; or, when the vehicle is in a non-parked driving state, determining at least one of voice reminder, text reminder, icon reminder, and indicator light reminder as the reminder method corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content, and the reminder content of the text reminder is generated based on the target reminder message or includes preset text content. The above embodiments provide a strategy for determining the reminder method solely based on the vehicle's driving status. As mentioned in the previous embodiments, video playback offers stronger privacy protection for the monitoring screen compared to other reminder methods, and may affect vehicle driving. Therefore, in non-parking scenarios (such as waiting at traffic lights), it is possible to confirm whether playback is required through pop-up windows or other means.

[0061] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0062] The vehicle control method provided in the above embodiments provides a target alert message to the vehicle when people or other objects in the target scene are in an abnormal state, such as falling. Then, based on vehicle status data, an appropriate alert method is selected to alert the driver and other occupants, making them aware that the target object is in an abnormal state. This provides a way to alert drivers and other occupants of abnormal states within a residence through the vehicle. By appropriately selecting the alert method, the negative impact of the alert on driving safety can be avoided. When the vehicle is not parked, video playback only occurs in response to a trigger operation of the playback control, avoiding sudden video playback that could interfere with the driver and ensuring driving safety. Furthermore, the appropriate selection of the alert method can protect user privacy. When there is a non-requisite object in the vehicle, a relatively privacy-preserving alert method, such as a text alert, can be selected.

[0063] Based on the same inventive concept, corresponding to the vehicle control method provided in the embodiments of this disclosure, this disclosure also provides a vehicle control device.

[0064] refer to Figure 3The vehicle control device 300 includes: a vehicle status data acquisition module 310, used to acquire vehicle status data when a target reminder message is received, wherein the target reminder message indicates that a target object in the target scene is in an abnormal state, the vehicle status data is used to indicate the presence of occupants in the vehicle and / or the vehicle's driving state, and the presence of occupants in the vehicle indicates whether there are any non-reminder objects in the vehicle other than the reminder object; and a reminder module 320, used to determine the reminder method corresponding to the target reminder message based on the vehicle status data, and control the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

[0065] In some embodiments, the reminder module is further configured to determine the voice reminder as the reminder method corresponding to the target reminder message when the presence of people in the vehicle is such that only the reminder target exists and the vehicle is not parked. The reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content.

[0066] In some embodiments, the reminder module is further configured to determine the voice reminder as the reminder method corresponding to the target reminder message when the presence of people in the vehicle is only the reminder object and the vehicle is not parked. The reminder content of the voice reminder is generated according to the target reminder message or includes preset voice content. The module also displays a video playback pop-up window including a playback control. In response to the triggering operation of the playback control, the module acquires and plays the monitoring screen of the target object in the target scene.

[0067] In some embodiments, the reminder module is further configured to determine voice reminder and video playback as the reminder methods corresponding to the target reminder message when the presence of people in the vehicle is only the reminder object and the vehicle is in a parked state. The reminder content of the voice reminder is generated according to the target reminder message or includes preset voice content, and the playback content of the video playback includes the monitoring screen of the target object in the target scene.

[0068] In some embodiments, the reminder module is further configured to determine at least one of text reminders, icon reminders, and indicator light reminders as the reminder method corresponding to the target reminder message when there are people in the vehicle and there are no reminder objects or no reminder objects. The reminder content of the text reminder is generated based on the target reminder message or includes preset text content.

[0069] In some embodiments, the reminder module is further configured to determine at least one of text reminder, icon reminder, and indicator light reminder as the reminder method corresponding to the target reminder message when the presence of people in the vehicle is either a reminder object or a non-reminder object. The reminder content of the text reminder is generated based on the target reminder message or includes preset text content. The module also displays a video playback pop-up window including a playback control. In response to the triggering operation of the playback control, the module acquires and plays the monitoring screen of the target object in the target scene.

[0070] It should be noted that the vehicle control device provided in the above embodiments is only illustrated by the division of the above functional modules when controlling vehicle abnormality alerts. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control device and the vehicle control method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0071] Based on the same inventive concept, corresponding to the vehicle control method provided in the embodiments of this disclosure, this disclosure also provides a vehicle control system.

[0072] refer to Figure 4 The vehicle control system 400 includes: a state detection device 410 for determining whether a target object in a target scene is in an abnormal state, and sending a target reminder message to a vehicle 420 if the target object is in an abnormal state; the vehicle 420 for acquiring vehicle state data upon receiving the target reminder message, wherein the target reminder message indicates that the target object in the target scene is in an abnormal state, the vehicle state data indicates the presence of occupants and / or the vehicle's driving state, the presence of occupants indicates whether there are any non-reminder objects in the vehicle other than the reminder object, and determining the reminder method corresponding to the target reminder message based on the vehicle state data, and controlling the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

[0073] As an example, the target reminder message sent by the status detection device can first be sent to the IoT cloud service. The target reminder message reported by the IoT cloud service can be obtained through the access cloud module. The corresponding vehicle identifier can be obtained by matching the status detection device identifier of the source detection device of the target reminder message. Based on the vehicle identifier, the target reminder message is sent to the vehicle cloud service. The vehicle cloud service then sends the target reminder message to the vehicle based on the vehicle identifier, so that the vehicle can obtain the target reminder message.

[0074] refer to Figure 5 An exemplary vehicle control system is provided, such as Figure 5As shown, the system utilizes two parallel and independent data links, with unified external connectivity achieved through a top-level access / cloud module. This module provides capabilities for device ID mapping, device status subscription, device alarm subscription, and device change subscription, acting as a unified gateway for command downlink and message uplink. On the left is the vehicle-mounted system (V2S) business link, where the V2S app interacts with the V2S cloud service to report vehicle device control, device status, and alarm information. Simultaneously, the V2S uses the in-vehicle app's keep-alive function, vehicle speed, and DMS data to determine the driving status, whether the vehicle is parked, and whether there are any non-alert objects besides the target being alerted, thus displaying the in-vehicle status and selecting appropriate alert methods for driving reminders. On the right is the service provider's IoT (Internet of Things) peripheral link, which relies on the user center for login, authorization, device configuration, and subscription management. It connects to video monitoring devices and WiFi sensing devices to determine device status, and communicates bidirectionally with the top-level gateway via the service provider's IoT cloud service to transmit control commands from external IoT devices and report device status and alarms.

[0075] Through the user center, the recipient can log in, authorize, and subscribe to their account, enabling the abnormal state of the target object in the target scenario to be communicated to the vehicle's app via interaction between the service provider's IoT cloud service and the vehicle's cloud service. Configuration in the user center allows for pre-configuration of alert methods for different driving states, as well as specific alert content and presentation for each method. For example, it allows for configuration of whether text alerts are detailed or simple, and the rules for determining the alert content. Similarly, if indicator light alerts are used, the position, color, flashing frequency, or constant-on rule of the indicator lights can be configured. By collecting monitoring footage of the target scenario, including but not limited to video surveillance, it is possible to identify specified abnormal behaviors and, when necessary, transmit monitoring footage and / or real-time monitoring footage back to the vehicle, allowing the recipient to understand the current status of the target scenario and assist in deciding on appropriate handling strategies. For example, monitoring an elderly person in their living room to see if they have fallen. When a fall is detected via video monitoring, the device status assessment uploads the device status information to the service provider's IoT cloud service. This IoT cloud service receives the device status information through the access / exit module's device status subscription and forwards it to the corresponding vehicle-mounted cloud service. The vehicle-mounted cloud service, based on a pre-configured subscription relationship, sends the corresponding target alert message to the vehicle's app, then assesses the driving status and determines the alert display method and how to issue it. The device ID mapping in the access / exit module allows for matching the device status information with the corresponding vehicle-mounted app. For example, the vehicle-mounted app can subscribe to the device status information corresponding to the vehicle's device status from the access / exit module. WiFi sensing can be used to detect the breathing rate of the target (person). If an anomaly is detected, the subsequent handling is similar to that described above for an elderly person falling, and will not be elaborated upon here. When the vehicle needs to display monitoring footage, it can issue device control commands via the in-vehicle app. These commands are then transmitted to the corresponding video detection devices and other status detection devices via the in-vehicle cloud service and the service provider's IoT cloud service, thereby executing commands such as monitoring footage capture. When a status detection device malfunctions, it can generate a device alarm message, which is uploaded to the service provider's IoT cloud service. This message is then forwarded to the in-vehicle cloud service via the access / exit module and subsequently sent to the corresponding in-vehicle app. Similar to the notification method for target alerts, the appropriate notification method is selected to alert the target of the device malfunction.

[0076] The device ID mapping in the cloud access module can associate acquired device alarm messages, device status messages, and device control messages with the issuing objects. Device ID mapping represents the correspondence between the identifier of the reminder object (or vehicle identifier) ​​and the identifier of the status detection device. Device status subscription can obtain device status messages (target reminder messages), device alarm subscription can obtain device alarm messages, and device change subscription can be used to obtain replacement data of status detection devices.

[0077] An exemplary vehicle control process is as follows: First, the operator's monitoring system is authenticated and accessed, and corresponding configuration operations are performed through the user center. Then, based on human posture recognition from video surveillance, it is determined whether someone has fallen. Based on electromagnetic wave changes sensed by WiFi, it is determined whether someone has abnormal breathing (rapid, difficult, stopped, etc.). Based on the Fresnel effect of WiFi electromagnetic waves, CSI parameters (phase and amplitude changes) caused by amplitude changes in the human chest cavity are established and normalized. It is determined whether there are abnormal heart rate changes. Combining the results of video and WiFi perception (as blind spot compensation), the status of people at home is comprehensively judged. If the above abnormalities occur, abnormal information (target reminder message) is reported. After receiving the abnormal information, the vehicle's APP determines whether the driver is currently driving, whether there are other people around (for privacy protection), and whether there are additional people in the car (non-reminder targets) through DMS. If so, a pop-up window prompts, "There is an abnormality in the home monitoring, do you want to show the video screen?" If there are no non-reminder targets, a voice broadcast says, "Elderly person / child at home has fallen." If the current vehicle speed is "0", the video screen is pushed simultaneously and immediately. If the vehicle speed is not "0", only a voice announcement will be made. The vehicle owner will then need to review the surveillance footage.

[0078] It should be noted that the vehicle control system provided in the above embodiments is only illustrated by the division of the above functional modules when controlling vehicle abnormality alerts. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the vehicle control system and the vehicle control method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0079] Based on the same inventive concept, corresponding to the vehicle control method provided in the embodiments of this disclosure, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle control method described in the above embodiments.

[0080] Figure 6This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 610, a memory 620, an input / output interface 630, a communication interface 640, and a bus 650. The processor 610, memory 620, input / output interface 630, and communication interface 640 are interconnected internally via the bus 650.

[0081] The processor 610 can be implemented using a general-purpose CPU (central processing unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0082] The memory 620 can be implemented in the form of ROM (read-only memory), RAM (random access memory), static storage device, dynamic storage device, etc. The memory 620 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 620 and is called and executed by the processor 610.

[0083] The input / output interface 630 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0084] The communication interface 640 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0085] Bus 650 includes a pathway for transmitting information between various components of the device, such as processor 610, memory 620, input / output interface 630, and communication interface 640.

[0086] It should be noted that although the above-described device only shows the processor 610, memory 620, input / output interface 630, communication interface 640, and bus 650, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0087] The electronic devices described above are used to implement the corresponding vehicle control methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0088] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to perform the vehicle control method described above. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (read-only memory), RAM (random access memory), CD-ROM (compact disc read-only memory), magnetic tape, floppy disk, and optical data storage devices, etc.

[0089] In an exemplary embodiment, a computer program product is also provided, including computer program instructions that, when executed on a computer, cause the computer to perform the vehicle control method described above.

[0090] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0091] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0092] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This disclosure does not limit this.

[0093] The above description is merely an optional embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A vehicle control method, characterized in that, The method includes: Upon receiving a target alert message, vehicle status data is obtained. The target alert message indicates that the target object in the target scene is in an abnormal state. The vehicle status data is used to indicate the presence of occupants in the vehicle and / or the vehicle's driving status. The presence of occupants in the vehicle is used to indicate whether there are any non-alert objects in the vehicle other than the alerted object. Based on the vehicle status data, determine the reminder method corresponding to the target reminder message, and control the vehicle to send an abnormal reminder to the reminder object through the reminder method.

2. The vehicle control method according to claim 1, characterized in that, The reminder methods include at least one of the following: voice reminder, text reminder, video playback, icon reminder, and indicator light reminder.

3. The vehicle control method according to claim 1, characterized in that, Determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: When the presence of people in the vehicle is such that only the target being reminded exists and the vehicle is not parked, the voice reminder is determined as the reminder method corresponding to the target reminder message. The reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content. or, When the presence of people inside the vehicle is limited to the target being reminded and the vehicle is not parked, a voice reminder is determined as the reminder method corresponding to the target reminder message. The reminder content of the voice reminder is generated based on the target reminder message or includes preset voice content. A video playback pop-up window including a playback control is displayed. In response to the triggering operation of the playback control, the monitoring screen of the target object in the target scene is acquired and played.

4. The vehicle control method according to claim 1, characterized in that, Determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: when the presence of people in the vehicle is only the reminder target and the vehicle is in a parked state, determining voice reminder and video playback as the reminder methods corresponding to the target reminder message, wherein the reminder content of the voice reminder is generated based on the target reminder message or the reminder content of the voice reminder includes preset voice content, and the playback content of the video playback includes the monitoring screen of the target object in the target scene.

5. The vehicle control method according to claim 1, characterized in that, Determining the reminder method corresponding to the target reminder message based on the vehicle status data includes: When the presence of people in the vehicle is divided into a reminder target and a non-reminder target, at least one of text reminder, icon reminder and indicator light reminder is determined as the reminder method corresponding to the target reminder message. The reminder content of the text reminder is generated according to the target reminder message or includes preset text content. or, When the presence of people inside the vehicle is divided into a reminder target and a non-reminder target, at least one of text reminder, icon reminder, and indicator light reminder is determined as the reminder method corresponding to the target reminder message. The reminder content of the text reminder is generated based on the target reminder message or includes preset text content. A video playback pop-up window including a playback control is displayed. In response to the trigger operation of the playback control, the monitoring screen of the target object in the target scene is acquired and played.

6. The vehicle control method according to any one of claims 1-5, characterized in that, The target alert message is generated and sent to the vehicle by the detection device after performing object detection and behavior recognition on the scene image of the target scene and determining that the behavior recognition result of the target object includes the specified abnormal behavior; or, The target alert message is generated by the detection device when it detects an abnormal breathing frequency of a target object in the target scene and sends it to the vehicle. The breathing frequency of the target object is determined based on the Fresnel effect of electromagnetic waves in WiFi.

7. The vehicle control method according to any one of claims 1-5, characterized in that, The target of the reminder includes the driver, the preset user, or the user indicated by the target reminder message.

8. A vehicle control device, characterized in that, The vehicle control device includes: The vehicle status data acquisition module is used to acquire vehicle status data when a target reminder message is received. The target reminder message indicates that the target object in the target scene is in an abnormal state. The vehicle status data is used to indicate the presence of people in the vehicle and / or the vehicle's driving status. The presence of people in the vehicle is used to indicate whether there are any non-reminder objects in the vehicle other than the reminder object. The reminder module is used to determine the reminder method corresponding to the target reminder message based on the vehicle status data, and control the vehicle to remind the reminder object of the abnormality through the reminder method.

9. A vehicle control system, characterized in that, The vehicle control system includes condition detection equipment and a vehicle, wherein: The state detection device is used to determine whether the target object in the target scene is in an abnormal state, and to send a target reminder message to the vehicle if the target object is in an abnormal state. The vehicle is used to acquire vehicle status data upon receiving a target reminder message. The target reminder message indicates that a target object in the target scene is in an abnormal state. The vehicle status data is used to indicate the presence of occupants and / or the vehicle's driving status. The presence of occupants indicates whether there are any non-reminder objects in the vehicle besides the reminder object. Based on the vehicle status data, the vehicle determines the reminder method corresponding to the target reminder message and controls the vehicle to provide an abnormal reminder to the reminder object through the reminder method.

10. An electronic device, comprising a memory, a processor, and components stored in the memory and operated on the processor. A computer program that performs a certain action, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

Images