Safety monitoring method and device based on vehicle, computer equipment and storage medium

Abstract

The invention relates to the technical field of vehicles, and discloses a safety monitoring method and device based on a vehicle, computer equipment and a storage medium, and the method comprises the steps: obtaining a cabin image obtained by carrying out the image collection of a vehicle cabin through image collection equipment in a target vehicle; identifying a key area corresponding to the driver in the cockpit image to obtain facial features and gesture features of the driver; determining whether the driver has an alarm demand or not by using the facial features and the gesture features; and if it is determined that the driver has the alarm demand, executing an alarm operation based on a vehicle end alarm mechanism corresponding to the alarm demand. The vehicle-mounted alarm system solves the problem that the existing vehicle-mounted alarm system is lack of special design for personal safety threats faced by online hailing vehicle owners.

Description

Technical Field

[0001] The present invention relates to the field of vehicle technology, and in particular to a vehicle-based safety monitoring method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of human-computer interaction technology, gesture recognition and face recognition, as key branches of human-computer interaction technology, have shown wide application potential. Gesture recognition understands actions by analyzing human hand or body postures, and is widely used in virtual reality, smart home, medical health, smart cockpit and other fields. Face recognition performs identity authentication or recognition by analyzing facial features, and can be extended to extract biometric features such as mouth shape and eyeballs, and is used in scenarios such as speech recognition, line of sight analysis, and emotion inspection.

[0003] Existing vehicle alarm technology mainly focuses on monitoring vehicle operating conditions and driver information. It is not specifically designed for the personal safety threats faced by online car-hailing owners. In addition, the monitoring method is single and the technology is backward, which can easily lead to false alarms or alarm failures. At the same time, it lacks concealment and safety considerations, and cannot meet the needs of online car-hailing owners for highly accurate and reliable alarm systems. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a vehicle-based security monitoring method, apparatus, computer equipment and storage medium to solve the problem that existing vehicle-mounted alarm systems lack special designs for the personal safety threats faced by online car-hailing owners.

[0005] In a first aspect, an embodiment of the present invention provides a vehicle-based safety monitoring method, the method comprising:

[0006] Acquire a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit;

[0007] Identifying a key area corresponding to the driver in the cockpit image to obtain facial features and gesture features of the driver;

[0008] Determining whether the driver currently has an alarm demand by using the facial features and the gesture features;

[0009] If it is determined that the driver has an alarm demand, an alarm operation is performed based on the vehicle-side alarm mechanism corresponding to the alarm demand.

[0010] In an optional implementation of the present application, the identifying a key area corresponding to the driver in the cockpit image to obtain facial features and gesture features of the driver includes:

[0011] Positioning key areas in the cockpit image to obtain a facial area and a gesture area;

[0012] Extracting initial facial features in the facial area, and extracting initial gesture features in the gesture area;

[0013] Obtaining a timestamp associated with each of the initial facial features and the initial gesture features;

[0014] Corresponding image frame numbers are allocated to the initial facial features and the initial gesture features based on the timestamp to obtain facial features and gesture features.

[0015] In an optional implementation of the present application, the determining whether the driver currently has an alarm demand by using the facial features and the gesture features includes:

[0016] Identify the image frame sequence number of each of the facial features and the gesture features;

[0017] Based on the image frame sequence number, combining the facial features and the gesture features into a plurality of action feature sets;

[0018] Matching the action feature set with a plurality of preset feature sets to obtain a matching result, wherein the preset feature sets include preset facial information and preset gesture information;

[0019] Determine whether the driver has an alarm demand based on the matching result.

[0020] In an optional implementation manner of the present application, before identifying the image frame sequence number of each of the facial features and the gesture features, the method further includes:

[0021] Acquire a plurality of pre-recorded preset feature sets, and assign a corresponding identifier to each set of the preset feature sets, wherein the preset feature sets include preset facial information and preset gesture information;

[0022] Acquire multiple alarm strategies corresponding to the identifier input by the user;

[0023] A binding relationship between a preset feature set corresponding to the identifier and a plurality of alarm strategies is established, and the binding relationship is stored.

[0024] In an optional implementation of the present application, determining whether the driver has an alarm demand according to the matching result includes:

[0025] If the matching result is that the action feature set hits any preset feature set, then the duration of the simultaneous existence of the facial feature and the gesture feature is obtained, and the duration is compared with a preset time threshold. If the duration is greater than or equal to the time threshold, it is determined that there is an alarm requirement;

[0026] If the matching result is that the action feature set does not hit the preset feature set, or the duration is less than the preset time threshold, it is determined that there is no alarm demand.

[0027] In an optional implementation of the present application, the vehicle-side alarm mechanism corresponding to the alarm requirement performs an alarm operation, including:

[0028] Taking the preset feature set hit by the action feature set as the target feature set, and acquiring multiple candidate alarm strategies corresponding to the target feature set based on the binding relationship;

[0029] Detecting whether a target component of a target vehicle is in a locked state;

[0030] If the target component is in a locked state, controlling the target component to perform an unlocking operation and obtaining the current driving state of the target vehicle;

[0031] According to the driving state, the facial features and the gesture features, a target alarm strategy is determined from a plurality of candidate alarm strategies, and a corresponding alarm operation is performed according to the target alarm strategy.

[0032] In an optional implementation manner of the present application, after executing the corresponding alarm operation according to the target alarm strategy, the method further includes:

[0033] Receiving an alarm termination instruction input by a user;

[0034] According to the alarm termination instruction, verifying the identity information of the user;

[0035] If the identity information verification is successful, the currently executed alarm operation is canceled and the target component is restored to its initial state;

[0036] If the identity information verification fails, or the alarm termination instruction is not received, the current alarm operation continues to be executed.

[0037] In a second aspect, an embodiment of the present invention provides a vehicle-based safety monitoring device, the device comprising:

[0038] An acquisition module, used to acquire a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit;

[0039] A recognition module, used to recognize a key area corresponding to the driver in the cockpit image, and obtain facial features and gesture features of the driver;

[0040] A determination module, used to determine whether the driver currently has an alarm demand by using the facial features and the gesture features;

[0041] The execution module is used to execute an alarm operation based on the vehicle-side alarm mechanism corresponding to the alarm demand if it is determined that the driver has an alarm demand.

[0042] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0044] The method provided in the embodiment of the present application has the following beneficial effects:

[0045] The method provided in the embodiment of the present application acquires cockpit images in real time to provide basic data for subsequent gesture and facial feature recognition, thereby ensuring the timely response of the alarm system. The facial and gesture features of the driver are identified to improve the accuracy and reliability of the alarm system. The time sequence and consistency of the feature data are ensured by managing the timestamp and the image frame sequence number, which is convenient for subsequent analysis and processing. The features collected in real time are matched with the preset features to quickly determine whether the driver has an alarm demand. By comparing the feature duration and the preset time threshold, the authenticity of the alarm demand is further confirmed to avoid false alarms. The user is allowed to customize the binding relationship between the alarm strategy and the preset feature set to improve the flexibility and user-friendliness of the system. After confirming the alarm demand, the corresponding alarm operation can be quickly executed to ensure the safety of the driver. By considering the vehicle status and user identity authentication, the rationality and safety of the alarm operation are ensured. An alarm termination mechanism is provided to ensure that the alarm operation can be canceled in time and the vehicle is restored to normal state when a false alarm or emergency situation is resolved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a flow chart of a vehicle-based safety monitoring method according to some embodiments of the present invention;

[0048] Figure 2is a flow chart of another vehicle-based safety monitoring method according to some embodiments of the present invention;

[0049] Figure 3 is a structural block diagram of a vehicle-based safety monitoring system according to an embodiment of the present invention;

[0050] Figure 4 A schematic diagram of a vehicle-based safety monitoring system workflow according to an embodiment of the present invention;

[0051] Figure 5 is a structural block diagram of a vehicle-based safety monitoring device according to an embodiment of the present invention;

[0052] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0054] According to an embodiment of the present invention, a vehicle-based safety monitoring method, apparatus, computer device and storage medium are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0055] In this embodiment, a vehicle-based safety monitoring method is provided. Figure 1 is a flow chart of a vehicle-based safety monitoring method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0056] Step S11, obtaining a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit.

[0057] In an embodiment of the present application, the image acquisition device may refer to a vehicle-mounted camera, which is used to capture and record image information of the driver and his surroundings in real time. The vehicle-mounted camera can be installed in a position where the driver's face and gestures can be clearly captured, such as near the rearview mirror inside the vehicle, above the steering wheel, in the instrument center control screen area or the A-pillar, etc. The area for vehicle cabin image acquisition specifically refers to the cab area of ​​the main driving cabin where the driver is located. Since the focus of the present invention is to monitor and protect the safety of the driver, the main area of ​​image acquisition is the driver's face and gestures, which are used for subsequent feature recognition and analysis.

[0058] Step S12, identifying the key area corresponding to the driver in the cockpit image, and obtaining the driver's facial features and gesture features.

[0059] In the embodiment of the present application, the driver's face and gestures in the cockpit image are mainly recognized. The key area is located by image recognition technology, the features of the key area are extracted, the corresponding timestamps are associated, and the corresponding image frame numbers are assigned, providing an accurate data basis for subsequent processing.

[0060] In the embodiment of the present application, step S12 includes the following steps A1-A4:

[0061] Step A1, locate the key areas in the cockpit image to obtain the facial area and the gesture area.

[0062] Specifically, for the location of the facial region, a face recognition algorithm can be used to detect the face in the image and return the location information of the face, such as the bounding box of the face. In this way, the facial region can be cropped out from the cockpit image for subsequent facial feature extraction. For the location of the gesture region, a gesture recognition algorithm can be used to detect the gesture in the image and return the location information of the gesture. Similarly, the gesture region can be cropped out from the cockpit image for subsequent gesture feature extraction.

[0063] Step A2: extracting initial facial features in the facial area and extracting initial gesture features in the gesture area.

[0064] Specifically, for facial feature extraction, face recognition technology can be used to analyze the pixel information of the facial area, extract key features such as facial contour, eyes, nose, mouth, etc., and express them in the form of vectors. Similarly, gesture feature extraction uses gesture recognition technology to analyze the pixel information of the gesture area, extract the shape, contour, texture and other features of the gesture, and also express them in the form of vectors.

[0065] Step A3, obtaining the timestamp associated with each initial facial feature and initial gesture feature.

[0066] Specifically, the acquisition of timestamps depends on the frame synchronization signal of the image acquisition module. Whenever the camera captures a new frame of the cockpit image, the image acquisition module generates a timestamp and passes it to the face recognition and gesture recognition modules to mark the extracted features. When matching features, only time-synchronized features are considered valid matching pairs. If the timestamp difference is too large, it is considered that the features do not belong to the same action or expression and should not be considered a match.

[0067] Step A4, assigning corresponding image frame numbers to the initial facial features and initial gesture features based on the timestamp to obtain the facial features and gesture features.

[0068] Specifically, since facial and gesture recognition may process images at different frequencies, the output feature data does not correspond in time. For this reason, the image frame number is used as the timestamp to ensure that the facial and gesture features can be matched in the order in which they actually appear, thereby ensuring their synchronization. A unique image frame number is assigned to each feature to facilitate feature tracking and management, and to ensure that in subsequent processing and analysis, the facial features and gesture features are from the same time point. According to the order of the timestamp, the corresponding image frame number is assigned to each feature to ensure that the order of the features is consistent with the order of the image. After the assignment is completed, the facial and gesture features with the same frame number are integrated to form a feature pair containing information at the same time point for subsequent recognition and judgment.

[0069] Step S13, using facial features and gesture features to determine whether the driver currently has an alarm demand.

[0070] In the embodiment of the present application, the image frame numbers of the facial and gesture features are first identified and combined into action feature sets. Secondly, these sets are matched with the preset feature sets to obtain matching results. Finally, based on the comparison between the matching results and the duration of the simultaneous existence of the features and the preset threshold, it is determined whether the driver has an alarm demand. If the action feature set is matched and the duration meets the standard, it is determined that there is an alarm demand; otherwise, it is determined that there is no alarm demand.

[0071] In the embodiment of the present application, step S13 includes the following steps B1-B3:

[0072] Step B1, identifying the image frame number of each facial feature and gesture feature.

[0073] Specifically, when the image acquisition module continuously acquires images from the camera, these images are fed into the face recognition and gesture recognition modules. The face recognition module detects and extracts facial features, such as the shape and position of the eyes, nose, and mouth; the gesture recognition module detects and extracts gesture features, such as the shape of fingers and the contour of gestures. At the same time, each detected feature is assigned a sequence number determined according to the order of the image frames, such as the feature sequence number of the first frame is 1, the second frame is 2, and so on, in order to mark the time point when the feature appears.

[0074] Step B2, based on the image frame sequence number, combining facial features and gesture features into multiple action feature sets.

[0075] Specifically, the arbitration module will sort these features according to the frame number to ensure that they are arranged in the order of their actual appearance. Then, the arbitration module will combine the facial features and gesture features to form an action feature set. Each set contains a facial feature and a corresponding gesture feature, which have the same frame number, indicating that these features appear at the same time point. In this way, the changes in the face and gesture can be analyzed and judged as a whole action.

[0076] Step B3, matching the action feature set with multiple preset feature sets to obtain a matching result, wherein the preset feature set includes preset facial information and preset gesture information.

[0077] Specifically, the preset feature set is a combination of a set of facial and gesture features pre-entered by the driver, representing a specific alarm signal. During matching, the system compares whether the facial and gesture feature IDs in the real-time action feature set are completely consistent with the IDs in the preset feature set. If the real-time action feature set completely matches any preset feature set, it means that the driver needs to alarm; otherwise, it means that the driver currently has no alarm demand.

[0078] Step B4, determining whether the driver has an alarm demand based on the matching result.

[0079] In an embodiment of the present application, step B4 also includes: if the matching result is that the action feature set hits any preset feature set, then obtaining the duration of the simultaneous existence of the facial features and gesture features, and comparing the duration with the preset time threshold. If the duration is greater than or equal to the time threshold, it is determined that there is an alarm requirement; if the matching result is that the action feature set does not hit the preset feature set, or the duration is less than the preset time threshold, it is determined that there is no alarm requirement.

[0080] Specifically, first, determine whether the real-time action feature set completely matches any preset feature set. If it hits, further analysis is performed; if it does not hit, it is determined that the driver has no need to alarm. When the action feature set hits, the duration of the simultaneous existence of facial and gesture features is obtained, that is, the time period from the first hit to no longer matching. Secondly, compare this duration with the preset threshold. If it is equal to or exceeds the threshold, it is determined that the driver has an alarm demand, indicating that the driver deliberately made the preset combination action. If the duration is less than the threshold, it is determined that the driver has no need to alarm, because the shorter time may be an unintentional or accidental action of the driver during driving.

[0081] In the embodiments of the present application, Figure 2 As shown, before step B1, the following steps C1-C3 are also included:

[0082] Step C1, obtaining a plurality of pre-recorded preset feature sets, and assigning a corresponding identifier to each set of preset feature sets, wherein the preset feature sets include preset facial information and preset gesture information.

[0083] Specifically, in order to manage the preset feature sets, the system assigns a unique identifier (ID) to each set to identify and distinguish different feature sets. Support multiple drivers to enter their own facial and gesture features to form different preset feature sets, each of which has a unique identifier. Provide a user interface for drivers to enter features and set identifiers, which are then saved in the database for subsequent retrieval and matching. For example, the preset feature set includes combinations such as "eyes turn left and the gesture is a fist", "blink twice and the gesture is a wave", and "open mouth and the gesture is pointing", representing a specific emergency alarm signal.

[0084] Step C2, obtaining multiple alarm strategies corresponding to the identifier input by the user.

[0085] Specifically, in addition to facial and gesture feature matching, the alarm strategy can also include conditions such as vehicle speed, parking status, and specific time periods. The alarm method can be selected as a sound prompt on the vehicle side, a cloud text message notification to the emergency contact, or a combination of both. To prevent false alarms, a delay time can be set to ensure that the alarm is not triggered until the features continue to match. At the same time, the driver is allowed to cancel the triggered alarm under specific circumstances, such as through specific gestures or facial expressions. Automatically contact emergency contacts after the alarm, record and save video evidence before and after the alarm, etc. The driver can input or select the alarm strategy through the interactive interface or interface, and associate it with the identifier of the preset feature set. During actual operation, it can decide whether and how to alarm based on the bound alarm strategy.

[0086] Step C3, establishing a binding relationship between a preset feature set corresponding to the identifier and multiple alarm strategies, and storing the binding relationship.

[0087] Specifically, multiple alarm strategies are associated with corresponding preset feature set identifiers to form a binding relationship. This relationship determines how to judge and trigger an alarm according to the bound alarm strategy when a feature matching a preset feature set is detected in actual operation. In order to store this relationship, a database or other data storage method is used so that the corresponding alarm strategy can be quickly found and applied in actual operation without complex matching and judgment each time.

[0088] As an example, assume there are two drivers, Driver 1 and Driver 2, each of whom has recorded unique facial and gesture features, forming two preset feature sets, and each set is assigned a unique identifier. Driver 1's alarm strategy is set as follows: if a feature matching its feature set is detected, and the vehicle is parked and the match lasts for more than 3 seconds, the alarm is activated, and the alarm method includes a sound prompt on the vehicle side and a cloud text message to notify the emergency contact. Driver 2's alarm strategy is: once a feature matching its feature set is detected, regardless of the vehicle status, as long as the match lasts for more than 5 seconds, the alarm is triggered, and the emergency contact is notified only through a cloud text message.

[0089] Step S14: If it is determined that the driver has an alarm demand, an alarm operation is performed based on the vehicle-side alarm mechanism corresponding to the alarm demand.

[0090] In the embodiment of the present application, if it is determined that the driver has an alarm requirement, the corresponding vehicle-side alarm mechanism is executed. The specific steps include: matching the preset feature set to obtain candidate alarm strategies; detecting the locked state of vehicle components, unlocking and obtaining the driving state if locked; determining and executing the most appropriate alarm strategy based on the driving state, facial and gesture features.

[0091] In the embodiment of the present application, the vehicle-side alarm mechanism corresponding to the alarm demand performs an alarm operation, including the following steps D1-D4:

[0092] Step D1, taking the preset feature set hit by the action feature set as the target feature set, and obtaining multiple candidate alarm strategies corresponding to the target feature set based on the binding relationship.

[0093] Specifically, first, the driver's facial and gesture features are identified and matched with preset feature sets. These preset feature sets contain specific facial and gesture feature combinations, each of which corresponds to a different alarm strategy. When a matching feature combination is detected, it is regarded as a target feature set, and based on the binding relationship with the target feature set, the corresponding multiple candidate alarm strategies are obtained. These strategies are pre-defined and are used to perform corresponding alarm actions in different alarm scenarios, such as sending alarm information to emergency contacts, uploading to the cloud alarm system, triggering the vehicle alarm, etc.

[0094] Step D2, detecting whether the target component of the target vehicle is in a locked state.

[0095] Specifically, before performing any alarm operation, it is necessary to ensure that the vehicle is in a relatively safe state to prevent external interference or potential danger. By detecting the locking state of the target component, it can be determined whether it is necessary to unlock it first to ensure the safe evacuation of the occupants.

[0096] Step D3: If the target component is in a locked state, the target component is controlled to perform an unlocking operation and the current driving state of the target vehicle is obtained.

[0097] Specifically, if it detects that the target components are indeed locked, it will send instructions to unlock these components. This is to ensure that the occupants can quickly and safely leave the vehicle when the alarm occurs. At the same time, the current driving status of the vehicle is obtained, such as whether it is driving, parked, or in automatic driving state. This information is crucial for determining the most appropriate alarm strategy later.

[0098] Step D4, determining a target alarm strategy from a plurality of candidate alarm strategies according to the driving status, facial features and gesture features, and performing a corresponding alarm operation according to the target alarm strategy.

[0099] Specifically, the most appropriate strategy is selected from multiple pre-defined candidate alarm strategies by comprehensively considering the vehicle's driving state, the driver's facial features, and gesture features. For example, if the vehicle is parked and the driver's facial and gesture features match the preset emergency alarm features, an alarm message is sent to the emergency contact and the siren in the vehicle is triggered at the same time. Once the target alarm strategy is determined, the corresponding alarm operation is immediately executed to ensure the safety of the driver and the vehicle.

[0100] In an embodiment of the present application, after step D4, the method also includes the following steps: receiving an alarm termination instruction input by the user; verifying the user's identity information according to the alarm termination instruction; if the identity information verification is successful, canceling the currently executed alarm operation and restoring the initial state of the target component; if the identity information verification fails, or the alarm termination instruction is not received, continuing to execute the current alarm operation.

[0101] Specifically, the commands input by the user are continuously monitored, especially the termination commands for the alarm operation. The termination commands can be input through buttons in the vehicle, touch screens, voice commands or other interactive methods. When an alarm termination command is received, for security reasons, the identity information of the user who issued the command will be verified. Identity verification can include passwords, biometrics (such as fingerprints, facial recognition), PIN codes or other security verification methods. If the user's identity information is verified, the currently executing alarm operation is canceled immediately. At the same time, the state of the target component that was previously changed by the alarm operation is restored, such as re-locking the door. If the user's identity information verification fails, or the alarm termination command is not received at all, the current alarm operation will continue to be executed. This ensures that the alarm operation will only be canceled when the user explicitly and securely instructs to terminate the alarm.

[0102] Figure 3 is a structural block diagram of a vehicle-based safety monitoring system according to an embodiment of the present invention. Figure 3 As shown, the system includes: an image acquisition module 301, a face recognition module 302, a gesture recognition module 303, an arbitration module 304, a vehicle-side alarm module 305, a cloud-side alarm module 306, and a GPS positioning module 307;

[0103] An image acquisition module 301 is used to photograph a vehicle cabin in a target vehicle to obtain a cabin image;

[0104] A face recognition module 302 is used to identify the facial area corresponding to the driver in the cockpit image, extract facial features, and associate timestamps to form a facial feature sequence;

[0105] The gesture recognition module 303 is used to identify the driver's gesture area in the cockpit image, extract gesture features, and associate timestamps to form a gesture feature sequence;

[0106] The arbitration module 304 is used to receive the outputs of the face recognition module 302 and the gesture recognition module 303, combine the facial features and gesture features to form an action feature set, and match it with a preset feature set to determine whether the driver has an alarm demand;

[0107] The vehicle-side alarm module 305 is used to perform an alarm operation, such as controlling vehicle components, sounding an alarm, etc., based on the vehicle-side alarm mechanism corresponding to the alarm demand when the arbitration module 304 determines that there is an alarm demand;

[0108] The cloud alarm module 306 is used to upload the alarm information to the cloud for remote monitoring and processing when necessary;

[0109] The GPS positioning module 307 is used to obtain the current location information of the target vehicle so as to provide the accurate vehicle location when an alarm is issued.

[0110] Figure 4 Schematic diagram of the working process of the vehicle-based security monitoring system according to an embodiment of the present invention. Figure 4 As shown, the process includes:

[0111] First, the vehicle's safety monitoring system is started, and the face recognition module is used to identify the driver's facial area in the cockpit image, and determine whether the driver is a pre-stored driver. If the driver is a pre-stored driver, the vehicle alarm module is turned on, and the cockpit image is continuously acquired through the image acquisition module. Secondly, the system uses the face recognition and gesture recognition modules to extract the driver's facial and gesture features, and associates these features with the timestamp. Thirdly, the system determines whether the vehicle is in a safe driving state. If so, the arbitration module combines the facial and gesture features into an action feature set and matches them with the preset feature set. If the duration of the simultaneous existence of facial and gesture features exceeds the preset threshold, the vehicle-side alarm module is triggered, and the pre-stored information is transmitted to the cloud alarm module. Finally, the user determines whether there is no false triggering at present. If not, the positioning information collected by the GPS module is continuously transmitted to the cloud alarm module. Otherwise, the alarm operation is canceled to complete the entire safety monitoring process.

[0112] In this embodiment, a vehicle-based safety monitoring device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0113] This embodiment provides a vehicle-based safety monitoring device, such as Figure 5 As shown, including:

[0114] An acquisition module 51 is used to acquire a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit;

[0115] The recognition module 52 is used to recognize the key area corresponding to the driver in the cockpit image and obtain the driver's facial features and gesture features;

[0116] A determination module 53, used to determine whether the driver currently has an alarm demand by using facial features and gesture features;

[0117] The execution module 54 is used to execute an alarm operation based on the vehicle-side alarm mechanism corresponding to the alarm demand if it is determined that the driver has an alarm demand.

[0118] In an optional embodiment of the present application, the recognition module 52 is used to locate key areas in the cockpit image to obtain facial areas and gesture areas; extract initial facial features in the facial areas and extract initial gesture features in the gesture areas; obtain a timestamp associated with each initial facial feature and initial gesture feature; and assign corresponding image frame numbers to the initial facial features and initial gesture features based on the timestamp to obtain facial features and gesture features.

[0119] In an optional embodiment of the present application, a determination module 53 is used to identify the image frame number of each facial feature and gesture feature; based on the image frame number, the facial features and gesture features are combined into multiple action feature sets; the action feature sets are matched with multiple preset feature sets to obtain matching results, wherein the preset feature sets include preset facial information and preset gesture information; and whether the driver has an alarm requirement is determined based on the matching results.

[0120] In an optional embodiment of the present application, the device also includes: an input module, used to obtain multiple pre-entered preset feature sets and assign a corresponding identifier to each set of preset feature sets, wherein the preset feature sets include preset facial information and preset gesture information; obtain multiple alarm strategies corresponding to the identifier input by the user; establish a binding relationship between the preset feature set corresponding to the identifier and the multiple alarm strategies, and store the binding relationship.

[0121] In an optional embodiment of the present application, the determination module 53 is used to obtain the duration of the simultaneous existence of facial features and gesture features if the matching result is that the action feature set hits any preset feature set, and compare the duration with the preset time threshold. If the duration is greater than or equal to the time threshold, it is determined that there is an alarm requirement; if the matching result is that the action feature set does not hit the preset feature set, or the duration is less than the preset time threshold, it is determined that there is no alarm requirement.

[0122] In an optional embodiment of the present application, the execution module 54 is used to take the preset feature set hit by the action feature set as the target feature set, and obtain multiple candidate alarm strategies corresponding to the target feature set based on the binding relationship; detect whether the target component of the target vehicle is in a locked state; if the target component is in a locked state, control the target component to perform an unlocking operation, and obtain the current driving state of the target vehicle; determine the target alarm strategy from multiple candidate alarm strategies according to the driving state, facial features and gesture features, and perform the corresponding alarm operation according to the target alarm strategy.

[0123] In an optional embodiment of the present application, the device also includes: a verification module, used to receive an alarm termination instruction input by a user; verify the user's identity information according to the alarm termination instruction; if the identity information verification is successful, cancel the currently executed alarm operation and restore the initial state of the target component; if the identity information verification fails, or the alarm termination instruction is not received, continue to execute the current alarm operation.

[0124] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0125] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0126] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0127] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0128] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0129] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0130] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0131] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vehicle-based safety monitoring method, characterized in that: The method comprises: Acquire a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit; Identifying a key area corresponding to the driver in the cockpit image to obtain facial features and gesture features of the driver; Determining whether the driver currently has an alarm demand by using the facial features and the gesture features; If it is determined that the driver has an alarm demand, an alarm operation is performed based on the vehicle-side alarm mechanism corresponding to the alarm demand.

2. The method according to claim 1, characterized in that: The identifying the key area corresponding to the driver in the cockpit image to obtain the facial features and gesture features of the driver includes: Positioning key areas in the cockpit image to obtain a facial area and a gesture area; Extracting initial facial features in the facial area, and extracting initial gesture features in the gesture area; Obtaining a timestamp associated with each of the initial facial features and the initial gesture features; Corresponding image frame numbers are allocated to the initial facial features and the initial gesture features based on the timestamp to obtain facial features and gesture features.

3. The method according to claim 1, characterized in that The determining whether the driver currently has an alarm demand by using the facial features and the gesture features includes: Identify the image frame sequence number of each of the facial features and the gesture features; Based on the image frame sequence number, combining the facial features and the gesture features into a plurality of action feature sets; Matching the action feature set with a plurality of preset feature sets to obtain a matching result, wherein the preset feature sets include preset facial information and preset gesture information; Determine whether the driver has an alarm demand based on the matching result.

4. The method according to claim 3, characterized in that Before identifying the image frame sequence number of each of the facial features and the gesture features, the method further includes: Acquire a plurality of pre-recorded preset feature sets, and assign a corresponding identifier to each set of the preset feature sets, wherein the preset feature sets include preset facial information and preset gesture information; Acquire multiple alarm strategies corresponding to the identifier input by the user; A binding relationship between a preset feature set corresponding to the identifier and a plurality of alarm strategies is established, and the binding relationship is stored.

5. The method according to claim 3, characterized in that: Determining whether the driver has an alarm demand according to the matching result includes: If the matching result is that the action feature set hits any preset feature set, then the duration of the simultaneous existence of the facial feature and the gesture feature is obtained, and the duration is compared with a preset time threshold. If the duration is greater than or equal to the time threshold, it is determined that there is an alarm requirement; If the matching result is that the action feature set does not hit the preset feature set, or the duration is less than the preset time threshold, it is determined that there is no alarm demand.

6. The method according to claim 4, characterized in that The vehicle-side alarm mechanism corresponding to the alarm requirement performs an alarm operation, including: Taking the preset feature set hit by the action feature set as the target feature set, and acquiring multiple candidate alarm strategies corresponding to the target feature set based on the binding relationship; Detecting whether a target component of a target vehicle is in a locked state; If the target component is in a locked state, controlling the target component to perform an unlocking operation and obtaining the current driving state of the target vehicle; According to the driving state, the facial features and the gesture features, a target alarm strategy is determined from a plurality of candidate alarm strategies, and a corresponding alarm operation is performed according to the target alarm strategy.

7. The method according to claim 6, characterized in that After executing the corresponding alarm operation according to the target alarm strategy, the method further includes: Receiving an alarm termination instruction input by a user; According to the alarm termination instruction, verifying the identity information of the user; If the identity information verification is successful, the currently executed alarm operation is canceled and the target component is restored to its initial state; If the identity information verification fails, or the alarm termination instruction is not received, the current alarm operation continues to be executed.

8. A vehicle-based safety monitoring device, characterized in that: The device comprises: An acquisition module, used to acquire a cockpit image obtained by an image acquisition device in a target vehicle acquiring an image of the vehicle cockpit; A recognition module, used to recognize a key area corresponding to the driver in the cockpit image, and obtain facial features and gesture features of the driver; A determination module, used to determine whether the driver currently has an alarm demand by using the facial features and the gesture features; The execution module is used to execute an alarm operation based on the vehicle-side alarm mechanism corresponding to the alarm demand if it is determined that the driver has an alarm demand.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images