Method and device for triggering vigilance state in sentry mode and vehicle

Abstract

The embodiment of the invention provides an alert state triggering method and device in a sentry mode and a vehicle, and relates to the technical field of vehicle monitoring, and the method comprises the steps: continuously obtaining various behavior data, at least including distance and hands, of surrounding people and current environment factor information in a monitoring state of a sentry system, wherein the behavior state detection of the personnel is used as a main calculation factor of the occurrence probability of the risk event, the environmental factor considers that the behavior of the personnel is associated with the influence of the environment, and the personnel factor is corrected through the environmental factor in order to reduce the prediction result of the possible risk of damaging the vehicle by misjudgment; therefore, whether the sentry system is triggered to enter the vigilance state or not can be decided according to more accurate risk probability prediction. Compared with an existing sentry state triggering mode, the method can enter the vigilance state more reliably and accurately, and the vigilance mechanism of a sentry system achieves a practical effect while it is guaranteed that meaningless resource consumption is reduced.

Description

[Technical field]

[0001] The embodiments of the present application relate to the field of vehicle monitoring technology, and in particular to a method, device and vehicle for triggering a vigilance state of a sentinel mode. [Background technology]

[0002] The sentry mode of a vehicle usually has three states: monitoring state, alert state and alarm state. Among them, the monitoring state means that the sentry system continues to work and monitors the environment around the vehicle, but does not trigger an alarm; the alert state means that when the sentry system detects that the vehicle is at risk of being damaged or injured through sensors such as surround-view cameras and ultrasonic radars, it will output a first-level alarm, such as lighting up the central control screen in time to warn people who intend to damage the vehicle that the vehicle is currently in a state of recording surveillance video; the alarm state means that the sentry system detects that the vehicle has been damaged and immediately issues a second-level alarm, such as an audible and visual alarm or an online alarm.

[0003] Since the sentry mode needs to continue to work after the vehicle is powered off, the high power consumption is a key issue of concern in the industry. It has been verified in practice that the power consumption is relatively large when in the aforementioned alert state and alarm state, especially compared with the triggering conditions of the alarm state, the triggering judgment of the alert state is relatively unclear and complex, so it is often mistakenly triggered to enter the alert state, resulting in unnecessary waste of resources. [Summary of the invention]

[0004] The embodiments of the present application propose a sentinel mode alert state triggering method, device and vehicle, which relate to the field of vehicle monitoring technology. The method can combine various behavioral states of people outside the vehicle and current environmental factors to comprehensively calculate the risk probability of causing damage to the vehicle, and decide whether to trigger the sentinel alert state to remind high-risk people. Compared with the existing sentinel state triggering form, the method can enter the alert state more reliably and accurately, and at the same time ensure that unnecessary resource consumption is reduced, so that the alert mechanism of the sentinel system can be effective.

[0005] In a first aspect, an embodiment of the present application provides a method for triggering an alert state of a sentinel mode, the method comprising:

[0006] After entering the sentry mode, the vehicle detects various behavior states of people around the vehicle and the current environmental state, wherein the behavior states at least include: the distance between the person and the vehicle, and the state of the person's hands;

[0007] Utilizing the detected behavior state data, calculating the probability value of a preset risk event corresponding to each of the behavior states;

[0008] Based on the preset first weight corresponding to each of the behavior states, the probability values ​​of all the behavior states are integrated to obtain a total probability value;

[0009] The total probability value is adjusted in combination with the detected environmental state, and a decision is made based on the adjusted total probability value whether to trigger entering the alert state.

[0010] The technical role of this solution can be referred to as follows: under the monitoring state of the sentinel system, it continuously obtains various behavioral data of surrounding personnel, including at least distance and hands, as well as current environmental factor information. The behavioral state detection of personnel is used as the main calculation factor for the probability of risk events, and environmental factors take into account the correlation between human behavior and environmental influences. In order to reduce the prediction results of the risk of misjudgment of possible damage to the vehicle, the human factors are corrected by environmental factors, so that it can be based on more accurate risk probability predictions to decide whether to trigger the sentinel system to enter the alert state. Compared with the existing sentinel state triggering form, this application can enter the alert state more reliably and accurately, while ensuring the reduction of unnecessary resource consumption, so that the alert mechanism of the sentinel system can play a practical effect.

[0011] In at least one possible implementation manner, the triggering method further includes: fusing the plurality of environmental states based on a preset second weight corresponding to each of the environmental states to obtain an adjustment parameter for adjusting the total probability value.

[0012] In this embodiment, for various environmental factors that may affect behavior, it is proposed to use weight fusion to integrate different environmental states into a unified parameter for correcting the total probability value, so as to facilitate the adjustment of the total probability value.

[0013] In at least one possible implementation manner, the environmental status includes at least one of the following: a parking location, a parking area monitoring location, a light intensity of the parking area, a date and time, and historical data of non-safety events in the parking area.

[0014] In this embodiment, environmental factors are expanded to propose external conditions that may affect human behavior from multiple dimensions such as space, time, lighting, external facilities and even the history of risk events, thereby improving the accuracy of predicting the risk level of human behavior.

[0015] In at least one possible implementation manner, the triggering method further includes: after the sentry mode decides to enter the alert state, outputting a preset warning audio to alert people outside the vehicle.

[0016] In this embodiment, it is emphasized that the preset audio is used as the execution result of the sentinel alert state, which not only saves energy, but also plays a better warning and reminder role.

[0017] In at least one possible implementation manner, the warning audio includes at least one of the following: audio simulating engine starting, and a warning voice.

[0018] In this embodiment, a specific warning audio type is used to ensure that the purpose of effectively avoiding risks is achieved in non-alarm situations.

[0019] In at least one possible implementation manner, the behavior state further includes at least one of the following: a person's moving direction, a person's moving speed, a person's facial orientation, a person's line of sight orientation, and a person's stay time.

[0020] In addition to the two main behavioral states mentioned above, this embodiment also provides detection of other behavioral states that may be associated with risky behaviors, increasing the reference dimension of risk probability estimation, helping to improve the reliability of subsequent triggering to enter the alert state, and reducing the chance of false triggering.

[0021] In a second aspect, an embodiment of the present application provides a sentinel mode alert state triggering device, the device comprising:

[0022] A behavior environment detection module is used to detect various behavior states of people around the vehicle and the current environment state after entering the sentry mode, wherein the behavior state at least includes: the distance between the person and the vehicle, and the state of the person's hand;

[0023] A risk event probability calculation module, used to calculate the probability value of a preset risk event corresponding to each of the behavior states using the detected behavior state data;

[0024] A probability fusion module, used for fusing the probability values ​​of all the behavior states based on a preset first weight corresponding to each of the behavior states to obtain a total probability value;

[0025] The state trigger decision module is used to adjust the total probability value in combination with the detected environmental state, and decide whether to trigger entering the alert state according to the adjusted total probability value.

[0026] In a third aspect, an embodiment of the present application provides an electronic device, comprising: one or more processors, a memory, and one or more computer programs, wherein the memory may adopt a non-volatile storage medium, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the device, the electronic device performs the method as described in the first aspect or any possible implementation manner of the first aspect.

[0027] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, the computer executes the method as described in the first aspect or any possible implementation of the first aspect.

[0028] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising the electronic device in the embodiment of the third aspect and the computer-readable storage medium in the embodiment of the fourth aspect.

[0029] It should be understood that the second to fifth aspects of the embodiments of the present application are consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here.

Brief Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 A flow chart of a method for triggering the alert state of a sentinel mode provided in an embodiment of the present application;

[0032] Figure 2 A structural schematic diagram of a sentinel mode alert state triggering device provided in an embodiment of the present application. [Specific implementation method]

[0033] In order to better understand the technical solution of this specification, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] It should be clear that the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0035] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit this specification. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0036] At present, the triggering conditions for the alert state of the sentry mode are relatively simple, which often leads to the vehicle entering the alert state by mistake and lighting up the screen inside the vehicle. This not only wastes unnecessary electricity, but also fails to reliably and accurately warn of risks. Especially in working scenarios where there are people coming and going around the vehicle, false triggering often occurs.

[0037] In view of this, the embodiment of the present application provides a method for triggering the alert state of the sentinel mode. The main design concept is that, under the monitoring state of the sentinel system, a variety of behavioral data of surrounding personnel, including at least distance and hands, and current environmental factor information are continuously obtained, among which the behavioral state detection of personnel is used as the main calculation factor for the probability of risk events, and the environmental factors take into account the correlation between human behavior and the impact of the environment. In order to reduce the prediction results of the risk of misjudgment of possible damage to the vehicle, the human factors are corrected by environmental factors, so that it can be based on a more accurate risk probability prediction to decide whether to trigger the sentinel system to enter the alert state. Compared with the existing sentinel state triggering form, the present application can enter the alert state more reliably and accurately, while ensuring the reduction of unnecessary resource consumption, so that the alert mechanism of the sentinel system can play a practical effect.

[0038] The technical solution protected by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0039] See also Figure 1 , is a flow chart of a method for triggering the alert state of a sentinel mode provided in an embodiment of the present application. The flow of the method is described as follows:

[0040] Step 101: After entering the sentry mode, detect various behavior states of people around the vehicle and the current environmental state, wherein the behavior state at least includes: the distance between the person and the vehicle, and the state of the person's hands.

[0041] Considering that in real situations, when an external person damages the vehicle, there are mainly two key factors. First, the people around the vehicle (pedestrians or people driving other vehicles, etc.) need to be within a certain distance from the vehicle; second, the behavior of damaging the vehicle is often strongly correlated with the state of the person's hands (for example, whether the hand holds an object, the hand is in a posture, etc.). Therefore, in this embodiment, it is mainly proposed to examine these two key factors, and the way of examination in actual operation is not limited to using existing machine learning and other algorithm models, and the images and other information obtained by the monitoring state of the sentinel mode. Those skilled in the art can understand that the technical implementation means of detecting the behavior state of people around the vehicle itself is not the focus of this application. There are many mature algorithm models that can be used for reference, and this embodiment will not be repeated or limited to this.

[0042] It is worth noting that, in addition to proposing to comprehensively predict whether there is a risk by combining multiple behavioral states of a person, this embodiment also mentions the investigation of environmental factors. Generally speaking, the object targeted by the sentinel mode is "people", but the present invention believes through analysis that environmental factors also play an important role in affecting the behavioral risks of "people", and thus proposes to add environmental states to the detection and perception objects of the sentinel mode. Specifically, in some preferred embodiments, the environmental state includes at least one of the following: parking location (such as underground or indoor parking lot, outdoor parking lot, etc.), parking area monitoring site (the number, location, and coverage of external monitoring of the current parking area can be obtained through image acquisition or data networking or navigation map, etc.), light intensity of the parking area (such as judging whether there is light or it is daytime), date and time (which can also involve daytime judgment, and can also include factors such as holidays and seasons); of course, in other embodiments, the environmental state can also include: historical data of non-safety events involving vehicle damage in the current parking area obtained through networked data, for example, the number, incidence, and trend of events of human-caused vehicle property loss in this area in the past period of time can be obtained from data information published by traffic management platforms, police platforms, etc.

[0043] It can be added here that in addition to the above-mentioned behavioral states, in other embodiments of the present application, it is further considered that in real situations, the behaviors that may cause damage to or destruction of the vehicle may also include at least one of: the direction of movement of the person, the speed of movement of the person, the direction of the person's face, the direction of the person's line of sight, the time the person stays, etc.

[0044] Step 102: Utilizing the detected behavior state data, calculate the probability value of a preset risk event corresponding to each of the behavior states.

[0045] It can be pointed out here that since the present application focuses on the possibility of damage, destruction and other behaviors of people outside the vehicle, the risk events corresponding to each of the above-mentioned behavior states can be set in advance based on different behavior states combined with established behavior thresholds, such as the risk event corresponding to the distance between the person and the vehicle is that the person is close to the vehicle (the distance can be defined according to the distance threshold), and the risk event corresponding to the state of the person's hand can include holding an item in the hand (which can be determined according to the target recognition algorithm) or the hand is dynamically approaching the vehicle (which can be determined according to the motion detection algorithm), and the risk event corresponding to the direction of the person's line of sight can be set to (which can be determined according to the number and duration of the line of sight looking at the vehicle within a given period of time), and so on, corresponding risk events can be set for each of the above-mentioned or not-mentioned behavior states. After that, algorithms in the field of probability statistics such as Bayesian networks can be used to calculate the probability value of the above-mentioned risk events leading to entering the alert state and triggering the first-level risk alarm. Some examples of calculation schemes for increasing probabilities are introduced later, which will not be repeated here.

[0046] Step 103: Based on the preset first weight corresponding to each of the behavior states, the probability values ​​of all the behavior states are integrated to obtain a total probability value.

[0047] The first weight mentioned in this link can be used to set the specific weight of each behavior state according to the criticality of the event factor that triggers the alert state as mentioned above; and the method of integrating various probability values ​​can be achieved through but not limited to weighted summation.

[0048] Step 104: The total probability value is adjusted in combination with the detected environmental state, and a decision is made whether to enter the alert state based on the adjusted total probability value.

[0049] In other words, since this embodiment not only detects the behavior of people outside the vehicle in the sentinel mode, but also detects the environmental data that affects the behavior of people, after obtaining the total probability score based on the multi-dimensional behavior state, it is proposed to use the current environmental state detection result to correct the total score value, that is, it is not only dependent on human behavior, as the decision basis for entering the sentinel alert state, but also integrates environmental factors such as the aforementioned parking place, light intensity, external monitoring facilities, date and time, and even historical data of high-risk events, which is very helpful to improve the accuracy of predicting the possibility of risk. Based on this concept, it can be supplemented that if more than one environmental state is detected, all environmental states detected can be integrated based on the preset second weight corresponding to each of the environmental states to obtain the summarized adjustment parameters. That is, the influence degree of different environmental factors can be integrated to facilitate the adjustment of the aforementioned total probability value; and the specific adjustment method can be reflected in the form of coefficients, or can also be reflected in the form of positive and negative compensation of probability values, which is not limited in this embodiment. In addition, it can be understood that the basis for decision making in this step is the comparison between the total probability value adjusted by environmental factors and the established probability threshold, and there is no limitation on this.

[0050] Here, in combination with the above embodiments and Bayesian network, the following examples are provided for reference. It should be understood that the following examples only use the four human behavior states mentioned above as schematic illustrations of the probability calculation process (other behavior considerations and environmental state adjustment mechanisms are not involved):

[0051] Set the A1 event as a person close to the vehicle, the A2 event as a person moving towards the vehicle, the A3 event as a person moving quickly, and the A4 event as a person facing the vehicle continuously; B represents the occurrence of a level 1 risk, which is the alert state of entering sentinel mode.

[0052] Probability of event A1: P(A1) = 0, if L > 2 meters; P(A1) = 1, if L < 0.2 meters; P(A1) = (L-0.2) / (2-0.2), if 2 meters ≥ L ≥ 0.2 meters, where L is the distance between the pedestrian and the vehicle.

[0053] Probability of event A2: P(A2) = 0, the pedestrian has not moved towards the vehicle in the past 5 seconds; P(A2) = 1, the pedestrian has always moved towards the vehicle in the past 5 seconds; P(A2) = t / 5. In the past 5 seconds, it takes t seconds for the pedestrian to move towards the vehicle.

[0054] A3 event probability: P(A3) = 0, in the past 5 seconds, the average speed of pedestrians v < 0.1 m / s; P(A3) = 1, in the past 5 seconds, the average speed of pedestrians v > 1.5 m / s; P(A3) = v / (1.5_0.1). In the past 5 seconds, the average speed of pedestrians v is between 0.1 and 1.5.

[0055] Probability of event A4: P(A4) = 0, the pedestrian’s face was not facing the vehicle in the past 5 seconds; P(A4) = 1, the pedestrian’s face was always facing the vehicle in the past 5 seconds; P(A4) = t / 5. In the past 5 seconds, the pedestrian’s face was facing the vehicle for t seconds.

[0056] It should be additionally pointed out that the above is only an example of probability calculation, and other increasing probability schemes can be adopted.

[0057] Next, according to the pre-set directed edge weights, that is, according to the importance of the factors of the vehicle being damaged or destroyed by external personnel to the first-level risk level, the weights of A1→B, A2→B, A3→B and A4→B are designed respectively. Those skilled in the art can understand that the weight values ​​can be calibrated values ​​and can be adjusted as needed. In the above example, α, β, γ, and δ are represented respectively, and the probabilities of multiple behavior states are combined to calculate P(B): P(B) = α*P(A1)+β*P(A2)+Y*P(A3)+δ*P(A4).

[0058] Then, a judgment is made based on the threshold: if P(B)>θ, it is determined that a first-level risk event has occurred (entering a vigilant state). θ can also be a calibration value that can be adjusted as needed.

[0059] Continuing from the previous text, after the sentry mode decision enters the alert state, at least a preset warning audio is output to remind people outside the car who may have high-risk behaviors. This embodiment proposes to use the preset warning audio as the execution content of the alert state, mainly considering two aspects: first, compared with the conventional screen lighting reminder method, audio output consumes less energy; second, for the scene concerned by this application, the sound reminder effect is better than lighting the screen. For example, in some other better embodiments, the warning audio includes at least one of the following: audio of simulating the engine (engine or motor) starting, voice output according to the preset warning template (for reference, "the vehicle is in video surveillance state", "the vehicle has been connected to the police platform", etc.), this kind of reminder information output in audio form, compared with lighting the screen in the car, can better achieve the warning effect of making people outside the car stop risky behaviors (such as people who are too close to the car will have a greater chance of staying away from the car when they hear the above audio), and even deter some illegal behaviors (people who approach the car from a distance with sharp weapons and focus their sight on the door, and commit sabotage).

[0060] Finally, it can be added that the aforementioned warning audio is different from the secondary alarm sound output by the alarm state of the sentry mode, but is an audio used for reminders and warnings. This is because the judgment of the alert stage is not based on a clear prediction, so only reminders and warnings are needed.

[0061] See also Figure 2 Based on the same inventive concept, the embodiment of the present application further provides a sentinel mode alert state triggering device, the device comprising:

[0062] The behavior environment detection module 201 is used to detect various behavior states of people around the vehicle and the current environment state after entering the sentry mode, wherein the behavior state at least includes: the distance between the person and the vehicle, and the hand state of the person;

[0063] The risk event probability calculation module 202 is used to calculate the probability value of the preset risk event corresponding to each of the behavior states using the detected behavior state data;

[0064] The probability fusion module 203 is used to fuse the probability values ​​of all the behavior states based on the preset first weights corresponding to the behavior states to obtain a total probability value;

[0065] The state triggering decision module 204 is used to adjust the total probability value in combination with the detected environmental state, and decide whether to trigger entering the alert state according to the adjusted total probability value.

[0066] Based on the same inventive concept, an embodiment of the present application further provides an electronic device, comprising at least one processor, the processor being used to execute a computer program stored in a memory to implement the flowchart steps of the above-mentioned sentinel mode alert state triggering method provided in an embodiment of the present application.

[0067] Optionally, the processor may specifically be a central processing unit, a specific ASIC, or one or more integrated circuits for controlling program execution.

[0068] Optionally, the electronic device may further include a memory connected to at least one processor, and the memory may include ROM, RAM, and disk storage. The memory is used to store data required by the processor when it is running, that is, it stores instructions that can be executed by at least one processor, and at least one processor executes the methods mentioned in the above embodiments by executing the instructions stored in the memory. Among them, the number of memories is one or more. Among them, the number of memories is one or more.

[0069] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores computer instructions. When the computer instructions are executed on a computer, the computer executes the methods mentioned in the above embodiments.

[0070] Based on the same inventive concept, the embodiment of the present application further provides a vehicle, comprising at least the above electronic device and / or computer-readable storage medium. The selection of the vehicle can be determined based on the correct understanding and reasonable implementation of the above solution by those skilled in the art.

[0071] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for triggering the alert state of a sentinel mode, characterized in that: The method comprises: After entering the sentry mode, the vehicle detects various behavior states of people around the vehicle and the current environmental state, wherein the behavior states at least include: the distance between the person and the vehicle, and the state of the person's hands; Utilizing the detected behavior state data, calculating the probability value of a preset risk event corresponding to each of the behavior states; Based on the preset first weight corresponding to each of the behavior states, the probability values ​​of all the behavior states are integrated to obtain a total probability value; The total probability value is adjusted in combination with the detected environmental state, and a decision is made based on the adjusted total probability value whether to trigger entering the alert state.

2. The method for triggering the alert state of the sentinel mode according to claim 1, characterized in that: The triggering method further includes: fusing a plurality of the environmental states based on a preset second weight corresponding to each of the environmental states to obtain an adjustment parameter for adjusting the total probability value.

3. The method for triggering the alert state of the sentinel mode according to claim 1, characterized in that: The environmental status includes at least one of the following: parking location, parking area monitoring location, light intensity of the parking area, date and time, and historical data of non-safety events in the parking area.

4. The method for triggering the alert state of the sentinel mode according to claim 1, characterized in that: The triggering method further includes: after the sentry mode decides to enter the alert state, outputting a preset warning audio to remind people outside the vehicle.

5. The method for triggering the alert state of the sentinel mode according to claim 4, characterized in that: The warning audio includes at least one of the following: audio simulating engine starting, and a warning voice.

6. The method for triggering the alert state of the sentinel mode according to any one of claims 1 to 5, characterized in that: The behavior status also includes at least one of the following: the direction of movement of the person, the speed of movement of the person, the direction of the face of the person, the direction of the sight of the person, and the stay time of the person.

7. A sentinel mode alert state triggering device, characterized in that: The device comprises: A behavior environment detection module is used to detect various behavior states of people around the vehicle and the current environment state after entering the sentry mode, wherein the behavior state at least includes: the distance between the person and the vehicle, and the state of the person's hand; A risk event probability calculation module, used to calculate the probability value of a preset risk event corresponding to each of the behavior states using the detected behavior state data; A probability fusion module, used for fusing the probability values ​​of all the behavior states based on a preset first weight corresponding to each of the behavior states to obtain a total probability value; The state trigger decision module is used to adjust the total probability value in combination with the detected environmental state, and decide whether to trigger entering the alert state according to the adjusted total probability value.

8. An electronic device, characterized in that: include: One or more processors, a memory and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions, which, when executed by the electronic device, enable the electronic device to execute the alert state triggering method of the sentry mode as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the alert state triggering method of the sentinel mode described in any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: The vehicle is provided with the electronic device of claim 8 or the computer-readable storage medium of claim 9.

Images