Vehicle early warning control method and system and electronic equipment

Abstract

The invention provides a vehicle early warning control method and system and electronic equipment, and relates to the field of vehicle early warning control, the method fully utilizes various sensors in a vehicle to accurately analyze child behaviors, and combines a corresponding early warning control strategy to control a target vehicle to carry out early warning. Therefore, accurate early warning of the behavior of operating the vehicle by the child is realized, and the probability of occurrence of such accidents is greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle early warning control, and in particular to a vehicle early warning control method, system and electronic device. BACKGROUND

[0002] In recent years, due to children touching the gear shifting mechanism or steering wheel of the vehicle, the vehicle moves unexpectedly, and such accidents occur frequently. According to statistics from relevant agencies, the number of such accidents has exceeded 200 per year. The child safety lock device currently used in vehicles is mainly limited to preventing children from opening the door inside, and has no effect on the child's misoperation behavior in the vehicle. The existing driver monitoring system (DMS, Driver Monitoring System) mainly identifies the driving behavior of the driver, such as whether the driver is dozing off, whether the driver's hand is away from the steering wheel for a long time, etc. However, there are obvious deficiencies in identifying such child-specific behavior patterns, and the false positive rate is high, making it difficult to effectively warn children operating the vehicle. SUMMARY

[0003] Therefore, the purpose of the present application is to provide a vehicle early warning control method, system and electronic device, which makes full use of various sensors in the vehicle to accurately analyze the behavior of children, and combines the corresponding early warning control strategy to control the target vehicle to perform early warning, thereby accurately warning the behavior of children operating the vehicle and greatly reducing the probability of such accidents.

[0004] In a first aspect, the present application provides a vehicle early warning control method, which comprises: acquiring a preset driver state monitoring sensor, a gear state sensor and a sensing sensor in a cabin area in a target vehicle, and controlling the driver state monitoring sensor, the gear state sensor and the sensing sensor to be in a working state after the vehicle is powered on; collecting early warning feature data in the cabin area in real time by using the driver state monitoring sensor and the sensing sensor, and determining age data and behavior data of a person in the cabin area based on the early warning feature data; determining a risk determination strategy and a corresponding early warning control strategy of the person based on the age data and gear data collected by the gear state sensor, and determining a risk level corresponding to the behavior data by using the risk determination strategy; controlling the target vehicle to perform a warning behavior corresponding to the early warning control strategy according to the early warning control strategy corresponding to the risk level.

[0005] Optionally, acquiring the preset driver state monitoring sensor, the gear state sensor and the sensing sensor in the cabin area in the target vehicle comprises: The driver state monitoring sensor is acquired based on a color camera, an infrared camera, a depth camera and a millimeter wave radar sensor deployed in a center console of the target vehicle; The gear state sensor is acquired based on a position sensor deployed in a gear lever of the target vehicle; The sensing sensor is acquired based on a first pressure sensor deployed in a gear lever of the target vehicle, a second pressure sensor deployed in a seat of the target vehicle, an acceleration sensor deployed in a center console of the target vehicle and a depth sensor deployed in a pedal of the target vehicle.

[0006] Optionally, the pre-warning feature data in the cockpit area is collected in real time by using the driver state monitoring sensor and the sensing sensor, including: The digital image corresponding to the cockpit area is acquired in real time by using the color camera and the infrared camera, the personnel in the cockpit area is identified through the digital image, and the image data corresponding to the personnel is acquired; The distance data corresponding to the personnel in the cockpit area is acquired in real time by using the depth camera and the millimeter wave radar; The first pressure data corresponding to the personnel holding the gear lever is acquired in real time by using the first pressure sensor; The second pressure data corresponding to the personnel touching the seat is acquired in real time by using the second pressure sensor; The acceleration data corresponding to the personnel touching the center console is acquired in real time by using the acceleration sensor; The pedal depth data corresponding to the personnel controlling the accelerator pedal and the brake pedal of the vehicle is acquired in real time by using the depth sensor; The pre-warning feature data is determined based on the image data, the distance data, the first pressure data, the second pressure data, the acceleration data and the pedal depth data.

[0007] Optionally, the age data and the behavior data of the personnel in the cockpit area are determined based on the pre-warning feature data, including: The personnel in the cockpit area within a preset range is acquired based on the distance data; The age data and the key point data of the personnel are determined by using the image data; The action data of the personnel is determined according to the key point data; The behavior data corresponding to the personnel is determined by using the action data, the first pressure data, the second pressure data, the acceleration data and the pedal depth data.

[0008] Optionally, the step of determining the age data and the key point data of the personnel by using the image data includes: The human body region corresponding to the personnel is acquired based on the image data, and the face region of the personnel is determined based on the human body region; calculate the interpupillary distance, face width-height ratio, and chin curvature of the person in the face region, and determine age data corresponding to the person according to the calculation results of the interpupillary distance, face width-height ratio, and chin curvature; identify and acquire a head region, a torso region, an upper limb region, and a lower limb region corresponding to the person in the human body region; identify and acquire a nose tip region or a head top center region corresponding to the person in the head region, a neck region, a left shoulder region, and a right shoulder region corresponding to the person in the torso region, an elbow region, a wrist region, and a finger root region corresponding to the person in the upper limb region, and a hip region, a knee region, an ankle region, and a toe tip region corresponding to the person in the lower limb region; determine key point data based on the nose tip region, the head top center region, the neck region, the left shoulder region, the right shoulder region, the elbow region, the wrist region, the finger root region, the hip region, the knee region, the ankle region, and the toe tip region.

[0009] Optionally, determine a risk judgment strategy and a corresponding early warning control strategy of the person based on the age data and gear data collected by the gear state sensor, including: determine an age value of the person based on the age data, and determine a gear state corresponding to the target vehicle based on the gear data collected by the gear state sensor; when the age value is not greater than a preset age threshold, determine a risk judgment strategy and an early warning control strategy corresponding to the person in the gear state according to the behavior data of the person; when the age value is greater than the age threshold, determine the risk judgment strategy and the early warning control strategy according to the gear state.

[0010] Optionally, determine a risk level of the behavior data by using the risk judgment strategy, including: determine action data, first pressure data, second pressure data, acceleration data, and pedal depth data corresponding to the person based on the behavior data; determine limb position data of the person by using key point data corresponding to the action data; determine gear holding data corresponding to the person by using the first pressure data; determine seat climbing data corresponding to the person by using the second pressure data; determine instrument force data corresponding to the person by using the acceleration data; determine driving control data corresponding to the target vehicle by using the pedal depth data; obtain a risk level corresponding to the limb position data, the gear holding data, the seat climbing data, the instrument force data, and the driving control data according to the risk judgment strategy.

[0011] Optionally, control the target vehicle to perform a warning behavior corresponding to the early warning control strategy according to the risk level corresponding to the early warning control strategy, including: acquire a first risk level corresponding to the limb position data, and control the target vehicle to perform one or more of the following warning behaviors: sound and light warning, central locking, window lifting, and double flash opening, according to a first pre-warning control strategy corresponding to the first risk level; acquire a second risk level corresponding to the gear holding data, and control the target vehicle to perform one or more of the following warning behaviors: sound and light warning, central locking, speed reduction and parking, double flash opening, and starting parking, according to a second pre-warning control strategy corresponding to the second risk level; acquire a third risk level corresponding to the seat climbing data, and control the target vehicle to perform one or more of the following warning behaviors: sound and light warning and central locking, according to a third pre-warning control strategy corresponding to the third risk level; acquire a fourth risk level corresponding to the instrument stress data, and control the target vehicle to perform a sound and light warning behavior, according to a fourth pre-warning control strategy corresponding to the fourth risk level; acquire a fifth risk level corresponding to the driving control data, and control the target vehicle to perform one or more of the following warning behaviors: sound and light warning, central locking, speed reduction and parking, double flash opening, starting parking, and calling for help, according to a fifth pre-warning control strategy corresponding to the fifth risk level.

[0012] In a second aspect, the present application provides a vehicle pre-warning control system, which comprises: an initialization module, configured to acquire a preset driver state monitoring sensor, a gear state sensor, and a sensing sensor in a cockpit area of a target vehicle, and control the driver state monitoring sensor, the gear state sensor, and the sensing sensor to be in a working state after the vehicle is powered on; a perception module, configured to collect pre-warning feature data in the cockpit area in real time by using the driver state monitoring sensor and the sensing sensor, and determine age data and behavior data of a person in the cockpit area based on the pre-warning feature data; a decision module, configured to determine a risk judgment strategy of the person and a corresponding pre-warning control strategy based on the age data and gear data collected by the gear state sensor, and determine a risk level corresponding to the behavior data by using the risk judgment strategy; an execution module, configured to control the target vehicle to perform a pre-warning behavior corresponding to the pre-warning control strategy according to the pre-warning control strategy corresponding to the risk level.

[0013] In a third aspect, the present application further provides an electronic device, which comprises a processor and a memory, the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the steps of the vehicle pre-warning control method provided in the first aspect.

[0014] In a fourth aspect, the embodiments of the present application further provide a storage medium, which stores computer executable instructions, and the computer executable instructions, when called and executed by a processor, cause the processor to implement the steps of the vehicle early warning control method provided in the first aspect.

[0015] The vehicle early warning control method, system and electronic device provided by the embodiments of the present application make full use of various sensors in the vehicle to accurately analyze the behavior of children, and combine corresponding early warning control strategies to control the target vehicle to perform early warning, so that the behavior of children operating the vehicle can be accurately warned, and the probability of such accidents can be greatly reduced.

[0016] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by the structure particularly pointed out in the description, claims and drawings.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 The flowchart of the vehicle early warning control method provided by the embodiments of the present application is shown in the figure; Figure 2 In step S101 of the vehicle early warning control method provided by the embodiments of the present application, the flowchart of acquiring the preset driver state monitoring sensor, gear state sensor and sensing sensor in the cockpit area of the target vehicle is shown in the figure; Figure 3 In step S102 of the vehicle early warning control method provided by the embodiments of the present application, the flowchart of collecting early warning feature data in the cockpit area by using the driver state monitoring sensor and the sensing sensor is shown in the figure; Figure 4 In step S102 of the vehicle early warning control method provided by the embodiments of the present application, the flowchart of determining the age data and behavior data of the person in the cockpit area based on the early warning feature data is shown in the figure; Figure 5A flow chart of step S402 in a vehicle early warning control method provided by an embodiment of the present application is shown in the figure; Figure 6 In step S103 of a vehicle early warning control method provided by an embodiment of the present application, a flow chart of determining a risk judgment strategy and a corresponding early warning control strategy of a person based on age data and gear data collected by a gear state sensor is shown in the figure. Figure 7 In step S103 of a vehicle early warning control method provided by an embodiment of the present application, a flow chart of determining a risk level corresponding to behavior data by using a risk judgment strategy is shown in the figure. Figure 8 A flow chart of step S104 in a vehicle early warning control method provided by an embodiment of the present application is shown in the figure. Figure 9 A flow chart of another vehicle early warning control method provided by an embodiment of the present application is shown in the figure. Figure 10 A structural schematic diagram of a vehicle early warning control system provided by an embodiment of the present application is shown in the figure. Figure 11 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in the figure.

[0020] Icon: 1010-initialization module; 1020-perception module; 1030-decision module; 1040-execution module; 101-processor; 102-memory; 103-bus; 104-communication interface. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in connection with the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0022] To make the embodiments of the present application clearer, a vehicle early warning control method disclosed by an embodiment of the present application will be introduced as follows. Figure 1 As shown in the figure, the method comprises: Step S101: obtaining a preset driver state monitoring sensor, a gear state sensor and a sensing sensor in a cockpit area in a target vehicle, and controlling the driver state monitoring sensor, the gear state sensor and the sensing sensor to be in a working state after the vehicle is powered on.

[0023] This step mainly involves sensor deployment and power-on activation. Its core function is to prepare the necessary sensing hardware for early warning, specifically including: Sensor list confirmation: Identify the three types of core sensors pre-installed in the target vehicle: driver status monitoring sensors (such as high-definition cameras and millimeter-wave radars for capturing the posture and position of the driver); gear status sensors (displacement and voltage sensors integrated into the transmission for real-time monitoring of whether the gear is in P / R / N / D and the shifting action); and cockpit area sensing sensors (such as seat pressure sensors and ultrasonic sensors on the center console, covering the front, rear, and center console areas of the cockpit to avoid blind spots).

[0024] Activation process after power-on: After the vehicle is powered on by key, keyless entry or remote start, it automatically triggers sensor self-test (such as camera focus test, radar signal strength detection). If all sensors meet the working threshold, it enters continuous working state; if there is a fault (such as camera obstruction), the driver will be immediately notified through the instrument panel pop-up window to ensure that there is no break in the perception link.

[0025] Step S102: Use driver status monitoring sensors and sensing sensors to collect early warning feature data in the cockpit area in real time, and determine the age and behavior data of the personnel in the cockpit area based on the early warning feature data.

[0026] This step mainly involves collecting early warning feature data and analyzing personnel information. The core is converting sensor signals into identifiable personnel data, which is performed in two steps: Real-time data acquisition: The driver status monitoring sensor collects visual data (such as facial contours and limb movement trajectories) and posture data (such as whether the person is bending over and the height of their hands raised) of the people in the cockpit at a frequency of 30 frames per second; the cockpit sensing sensor simultaneously collects auxiliary data (such as seat pressure distribution to determine the range of the person's weight and infrared sensors to capture the distance between the person and the central control). Both types of data are transmitted to the vehicle controller in real time via the CAN bus.

[0027] Data Analysis and Information Determination: Age data determination is mainly based on visual data. Facial features (such as eye distance, nose bridge height, and facial proportions) are extracted through pre-trained AI models (such as convolutional neural networks) and cross-validated with weight data to classify people into children and adults. When determining behavioral data, action trajectory analysis is used to identify whether people are approaching the gear shift mechanism, touching the steering wheel / gear lever, or pressing the central control buttons, and the duration of the behavior (such as whether touching the gear lever exceeds 2 seconds) and the force of the action (such as the pressure value of pressing the gear lever).

[0028] Step S103: Determine the risk judgment strategy of the person and the corresponding early warning control strategy based on the age data and the gear data collected by the gear state sensor, and determine the risk level corresponding to the behavior data using the risk judgment strategy.

[0029] This step mainly realizes risk judgment strategy matching and risk level division. The core is to develop differentiated judgment rules combined with personnel attributes and vehicle state, which specifically includes: Risk judgment strategy determination: First, lock the core risk object based on age data, and only start special risk judgment for children (adults operate by default in accordance with the normal driving logic, and do not trigger the child misoperation warning); Then, combined with the gear data collected by the gear state sensor, the scene is subdivided: Scenario 1: The gear is in P (vehicle stationary), and the risk judgment focuses on whether the child tries to move the gear lever; Scenario 2: The gear is in R / D (vehicle driving or waiting to drive), and the risk judgment focuses on whether the child touching the gear lever causes the gear to switch, and whether the brake pedal is pressed; Scenario 3: The gear is in N (neutral), and the risk judgment focuses on whether the child touches the gear lever.

[0030] The risk level determination process can be as follows: Low risk: Children enter the driver's cabin and approach the central control area but do not touch the gear lever / direction wheel (such as hands away from the gear lever > 10 cm); Medium risk: The child touches the gear lever / direction wheel, but does not produce substantial operation (such as touching the gear lever but not causing gear displacement, holding the steering wheel but not turning); High risk: The child's operation causes substantial risk (such as moving the gear lever to switch the gear from P to R / D, pressing the accelerator pedal to make the vehicle move).

[0031] Early warning control strategy matching: For each risk level, a corresponding response scheme is preset (such as low risk - mild reminder, high risk - emergency intervention), forming a mapping of risk level-control strategy.

[0032] Step S104: According to the early warning control strategy corresponding to the risk level, control the target vehicle to execute the early warning behavior corresponding to the early warning control strategy.

[0033] This step mainly realizes the process of executing graded early warning behavior, and the core is to trigger gradient early warning response according to the risk level, ensuring that the warning neither misses the risk nor avoids excessive intervention.

[0034] Low-risk warning execution: start the driver's seat buzzer (frequency 500 Hz, volume ≤60 decibels), display the yellow child proximity icon on the dashboard, and avoid startling the child; if the vehicle is bound to the driver's mobile phone APP, push the notification of the driver's cabin with child activity (low risk) to the APP, with a real-time driver's cabin picture (authorization required).

[0035] Medium-risk warning execution: strengthen the in-vehicle reminder measures, such as increasing the buzzer frequency to 800 Hz and the volume to 80 decibels, flashing the red light on the dashboard and displaying the "child touches operating components" text prompt, and triggering the driver's seat vibration (vibration frequency 2 times / second); the initial intervention can temporarily lock the gear shifting function (such as through an electronic gear lever locking mechanism to prevent gear displacement), but does not cut off the power to avoid affecting normal driving (if the driver is in the vehicle).

[0036] High-risk warning execution: emergency safety intervention measures, such as immediately cutting off the vehicle's power output (such as closing the throttle and disconnecting the motor drive circuit), automatically switching the gear to P and activating the hand brake, and turning on the double-flash warning light to prevent the vehicle from moving unexpectedly; in addition, multiple alarm triggers can be executed, including the highest level of sound and light warning in the vehicle (such as flashing the emergency light on the roof and playing the voice "child misoperation detected, emergency braking activated"), and synchronously pushing the high-risk warning (including positioning and accident type) to the owner's APP, and can be linked to the emergency warning system of the vehicle insurance company.

[0037] Optionally, the target vehicle is equipped with a driver state monitoring sensor, a gear state sensor, and an in-cabin sensing sensor, as shown in FIG. Figure 2 The driver state monitoring sensor includes: Step S201: Obtain the driver state monitoring sensor based on the color camera, infrared camera, depth camera, and millimeter wave radar sensor deployed in the instrument panel and roof console of the target vehicle.

[0038] Based on the hardware deployment of the target vehicle, the sensor is extracted from two key areas to form the driver state monitoring sensor.

[0039] Instrument panel area: obtain a pre-installed 2 million pixel color camera (used to capture the facial features and body movements of the person in the driver's cabin) and an 8 μm pixel pitch infrared camera (adapted to low light environments such as night and tunnel to ensure personnel identification in dark light); Roof console area (below the rearview mirror): VGA resolution depth camera (measures the distance between the person and the gear, steering wheel) and 24 GHz millimeter wave radar sensor can be obtained, and the four types of sensors work together to achieve comprehensive sensing of the driver's cabin personnel state.

[0040] Step S202: Obtain the gear state sensor based on the position sensor deployed in the gear of the target vehicle.

[0041] This step can focus on the hardware configuration of the target vehicle gear system. The preset position sensor (commonly used Hall or photoelectric, with anti-oil and anti-vibration characteristics) is obtained from the gear linkage key position (such as the inside of the transmission case or the transmission assembly at the bottom of the gear lever). The sensor can capture the gear displacement signal in real time, accurately identify the P (parking), R (reverse), N (neutral), D (forward) gear states, and provide vehicle gear basic data for subsequent risk judgment.

[0042] Step S203: Obtain the sensing sensor based on the first pressure sensor deployed in the target vehicle in the middle of the gear handle, the second pressure sensor deployed in the target vehicle in the seat, the acceleration sensor deployed in the target vehicle in the instrument panel, and the depth sensor deployed in the target vehicle in the pedal.

[0043] According to the sensor deployment of different components of the target vehicle, four types of sensors are integrated to form a sensing sensor. The first pressure sensor (detecting the force and contact area of the hand touching the gear handle) is obtained from the silicone layer on the surface of the gear handle; the second pressure sensor (judging the weight range of the person through pressure distribution, and distinguishing children from adults) is obtained from the seat cushion and backrest of the driver's seat, the front passenger's seat and the rear seats; the acceleration sensor (monitoring whether the vehicle produces abnormal vibration due to children climbing, etc.) is obtained from the position close to the central control inside the instrument panel (away from the engine to reduce vibration interference); and the depth sensor (detecting the depth of the pedal being stepped on, distinguishing children from adults) is obtained from the connecting rod mechanism of the accelerator and brake pedals.

[0044] Optionally, the driver state monitoring sensor and the sensing sensor are used to collect real-time warning feature data in the cockpit area, such as Figure 3 As shown, including: Step S301: Real-time acquisition of digital images corresponding to the cockpit area by using color cameras and infrared cameras, identifying the personnel in the cockpit area through digital image recognition, and obtaining image data corresponding to the personnel.

[0045] This step mainly collects personnel image data in the cockpit. Specifically, a 2 million pixel color camera (30 frames / second acquisition) and an 8μm infrared camera (adapted to dark light scenes) of the instrument panel are used to capture the cockpit image in real time. Through image recognition algorithm, the personnel in the picture are located (excluding non-personnel objects such as seats and central control), and the image data corresponding to the personnel are extracted, including face outline, limb movement trajectory (such as hand lifting direction), body posture (such as whether to bend down to approach the central control), providing visual basis for subsequent age determination and behavior analysis.

[0046] Step S302: Real-time acquisition of distance data corresponding to the personnel in the cockpit area by using depth cameras and millimeter wave radars.

[0047] This step mainly collects the distance data of the cockpit personnel, specifically, the VGA resolution depth camera (distance measurement accuracy ±2 cm) and 24 GHz millimeter wave radar (penetrating non-metallic obstruction) of the roof console are started to capture the distance data of the personnel and key components in real time: the distance between the hands and the gear lever, the distance between the body and the steering wheel, and the distance between the head and the instrument table are measured, and the distance change trend (such as whether the hands are continuously close to the gear lever) is recorded to avoid misjudgment as a risk behavior due to close distance.

[0048] For example, the depth camera (such as 3D TOF structured light) can capture the distance data of the personnel and key components in real time by The depth value is calculated (c is the speed of light, f is the modulation frequency, d is the phase difference, and z is the depth value); the effective detection distance is 0.5-5 meters; the accuracy error is 2 cm (within 1 m).

[0049] Step S303: Real-time acquisition of first pressure data corresponding to the holding of the gear lever by the personnel by using the first pressure sensor.

[0050] This step mainly collects the holding pressure data of the gear lever, specifically, the first pressure sensor (range 0-50N, response time ≤10ms) in the silicone layer on the surface of the gear lever can be used to collect the first pressure data when the personnel hold the gear lever in real time: including the pressure peak value (such as 1-5N of a child's fingertip pressing and 10-20N of an adult holding), the pressure duration (such as whether the pressing lasts more than 2 seconds), to distinguish between “temporary touch” and “continuous holding”, and reduce false positives of non-risk operations.

[0051] Step S304: Real-time acquisition of second pressure data corresponding to the touch of the seat by the personnel by using the second pressure sensor.

[0052] This step mainly collects the seat touch pressure data, specifically, the second pressure sensor on the seat cushion (4 collection points) and the backrest (2 collection points) can be used to obtain the second pressure data when the personnel touch the seat in real time: through the pressure distribution pattern (such as “local concentrated pressure” of a child and “uniformly dispersed pressure” of an adult) and the total pressure value (such as 10-40kg corresponding pressure of a child and >40kg corresponding pressure of an adult), to assist in judging the weight range and sitting posture of the personnel (such as whether to curl up in the driver's seat).

[0053] Step S305: Real-time acquisition of acceleration data corresponding to the touch of the instrument table by the personnel by using the acceleration sensor.

[0054] This step mainly collects instrument table touch acceleration data. Specifically, the acceleration sensor (sampling rate 1000 Hz, range ±16 g) inside the instrument table can be activated to capture the vibration acceleration data generated when a person touches the instrument table in real time: distinguish between "light touch" (acceleration <0.1 g, such as a child's fingertip touch) and "forceful impact" (acceleration >0.5 g, such as a child's slap), while filtering normal engine vibrations (preset vibration threshold) to ensure that the data only reflects the person's active touch behavior.

[0055] Step S306: Real-time acquisition of pedal depth data corresponding to the control of the accelerator pedal and brake pedal of the vehicle by the person using the depth sensor.

[0056] This step mainly collects pedal control depth data. Specifically, the depth sensor (range 0-100 mm, resolution 0.1 mm) of the accelerator / brake pedal linkage mechanism can be used to collect pedal depth data in real time when a person operates the pedal: record the depth values of the pedal from "completely released" (0 mm) to "partially stepped" (such as 2-5 mm of a child's accidental touch), "completely stepped down" (such as 60-80 mm of an adult's operation), to clearly indicate the operation force, and avoid misjudging a foot accidentally touching the pedal as a risk control behavior.

[0057] Step S307: Determine the warning feature data based on the image data, distance data, first pressure data, second pressure data, acceleration data, and pedal depth data.

[0058] This step mainly integrates and determines the warning feature data. Specifically, the above-mentioned image data, distance data, first / second pressure data, acceleration data, and pedal depth data are transmitted synchronously to the controller through the vehicle CAN bus in a time stamp alignment (to ensure that each data corresponds to the same time point behavior) manner: eliminate invalid data (such as blurred images, pressure values beyond the range), retain valid data and label the data type, and finally form complete warning feature data, providing unified data support for subsequent personnel age and behavior determination.

[0059] Optionally, based on the warning feature data, the age data and the behavior data of the person in the cockpit area are determined, as shown in Figure 4 , including: Step S401: Obtain the person contained in the preset range in the cockpit area based on the distance data.

[0060] This step mainly screens target personnel based on distance data. Specifically, in combination with the distance data in the early warning feature data, a "risk concern preset range" is first defined, taking the gear lever, steering wheel, and accelerator / brake pedal as the core and setting a circular area with a radius of 30 cm (the area within which personnel operation can directly affect the vehicle state); then, the objects with a distance of ≤30 cm between the personnel and the core components are screened out, and personnel with a distance beyond the range (such as children in the back row not close to the center control) are excluded, focusing on target personnel that may produce misoperation and reducing the amount of invalid determination.

[0061] Step S402: Determine the age data and key point data of the personnel using image data.

[0062] This step mainly determines the age and key point data based on image data. Specifically, for the screened target personnel, the image data in the early warning feature data is called.

[0063] Age data determination: Through a pre-trained face feature recognition model (such as a lightweight CNN model), the face key features (eye distance, nose bridge height, facial fat ratio) of the personnel in the image are extracted, and the personnel are divided into "children" (≤12 years old, with a more rounded face and a relatively wider eye distance) or "adults" (>12 years old, with a clearer facial contour) by comparing with a child face feature library, and the determination accuracy needs to be ≥95%.

[0064] Key point data extraction: Through a human body posture recognition algorithm, 17 core key points of the personnel (including head center point, double hand palm center point, double elbow joint point, double knee joint point, etc.) are located, and the real-time coordinates of each key point are recorded (a coordinate system is established with the driving cabin ground as the origin), providing a basis for subsequent action judgment.

[0065] Step S403: Determine the action data of the personnel based on the key point data.

[0066] This step mainly determines the action data based on the key point data. Specifically, the coordinate change trend of the key point data is analyzed to determine the real-time action of the personnel: if the double hand palm key point coordinates move towards the gear lever coordinates (such as the X-axis deviation reducing from 15 cm to 5 cm), it is determined that "the hands are close to the gear lever"; if the head center point coordinates deviate towards the steering wheel direction (such as the Z-axis height decreasing by 5 cm), it is determined that "the head is leaning forward and close to the steering wheel"; if the double knee joint point coordinates move towards the pedal area (such as the Y-axis distance shortening by 20 cm), it is determined that "the legs are close to the pedal", and finally the action data of "hand / head / leg action direction + action amplitude" is formed.

[0067] Step S404: Determine the behavior data corresponding to the personnel using the action data, first pressure data, second pressure data, acceleration data, and pedal depth data.

[0068] The step determines the behavior data through multi-type data, and specifically determines the specific behavior by associating the action data with the first pressure data, the second pressure data, the acceleration data and the pedal depth data in the early warning feature data.

[0069] If the action data is "hand close to gear lever", and the first pressure data detects a pressure value of 5-10N (child holding force range), it is determined that the behavior is "touching and holding the gear lever"; if the action data is "leg close to the pedal", and the pedal depth data shows a depth value of 2-5mm (child accidental touch force), it is determined that the behavior is "accidentally touching the pedal with the foot"; if the action data is "hand close to the instrument desk", and the acceleration data detects a vibration value of 0.3-0.5g (child hitting force), it is determined that the behavior is "hitting the instrument desk control"; if only the action data shows "close" but there is no corresponding pressure / acceleration data, it is determined as "no contact close" (non-risk behavior), ensuring that the behavior determination does not depend on single data and reducing misjudgment.

[0070] Optionally, the step S402 of determining the age data and the key point data of the person by using the image data comprises the following steps. Figure 5 As shown in the figure, it comprises: Step S501: obtaining a human body region corresponding to the person based on the image data, and determining a face region of the person based on the human body region.

[0071] Based on the digital image (color / infrared image fusion, ensuring effectiveness in dark light scene) in the early warning feature data, the core region is located in two steps.

[0072] First, the target detection algorithm (such as lightweight YOLO model) is used to scan the image, identify and frame the human body region in the cockpit, and exclude non-human objects such as seats and central controls, and output the rectangular coordinates of the human body region (such as the upper left corner (x1, y1) and the lower right corner (x2, y2)).

[0073] Then, in the human body region, the face detection model (such as MTCNN) is used to locate the face region, and the "complete face region containing eyes, nose bridge and mouth" is used as the standard to exclude regions with more than 30% face occlusion (to avoid feature extraction errors caused by occlusion), and finally determine the pixel range of the face region to provide a focused area for subsequent age determination.

[0074] Step S502: calculating the interpupillary distance, face width-height ratio and chin curvature of the person in the face region, and determining the age data corresponding to the person according to the calculation results of the interpupillary distance, face width-height ratio and chin curvature.

[0075] For the located face region, the age is determined by feature quantization and comparison. The feature calculation is as follows: Pupil distance is measured by the straight-line distance between the centers of the pupils of both eyes in image pixel coordinates, and then converted into the actual distance by combining the focal length of the camera (preset as 50mm) (e.g., the pupil distance of children is usually 45-55mm, and that of adults is 55-70mm); The face width-height ratio is calculated by the ratio of the distance between the left and right widest parts of the face (the outer sides of the ears) to the longest part from the top of the forehead to the bottom of the chin (e.g., the face width-height ratio of children is about 1:1.2, and that of adults is about 1:1.5, and the face of children is more rounded); The chin curvature is calculated by the curvature radius of the chin contour curve extracted by the curve fitting algorithm (e.g., least squares method) (e.g., the chin curvature radius of children is smaller, and the contour is more rounded; the curvature radius of adults is larger, and the contour is more sharp); The age determination is performed by comparing the calculation results of the above three features with the pre-trained "child-adult face feature library" (containing more than 100,000 samples) to calculate the matching degree by cosine similarity, and if the matching degree is greater than or equal to 90%, it is determined as the corresponding age category (≤12 years old as children, >12 years old as adults), and the determination accuracy is ensured to be greater than or equal to 95%.

[0076] Step S503: identifying and obtaining the head region, torso region, upper limb region and lower limb region corresponding to the personnel in the human body region.

[0077] Based on the positioning of the human body region, the four sub-regions are accurately divided by the semantic segmentation algorithm (such as the U-Net model), and the division standard is adapted to the cockpit scene.

[0078] The head region is bounded by the top of the head to the bottom of the mandibular line, and contains the complete head contour (excluding the boundary blur caused by hair obstruction, and the scalp contour is located by infrared image auxiliary); the torso region is bounded by the bottom of the mandibular line to the top of the hip (the top of the hip is taken as the reference of the narrowest part of the waist), and contains the chest and abdominal regions; the upper limb region is bounded by the outer side of the shoulder to the end of the fingertips, and is divided into left upper limb and right upper limb, and contains the upper arm, forearm and hand; the lower limb region is bounded by the top of the hip to the end of the toes, and is divided into left lower limb and right lower limb, and contains the thigh, calf and foot; after division, the pixel range of each sub-region is output, providing a basis for subsequent sub-region feature extraction.

[0079] Step S504: identifying and obtaining the nose tip region or the top of the head region in the head region, the neck region, the left shoulder region and the right shoulder region in the torso region, the elbow region, the wrist region and the finger root region in the upper limb region, and the hip region, the knee region, the ankle region and the toe region in the lower limb region.

[0080] For the four sub-regions, the key point detection algorithm (such as the MediaPipe human pose model) is used to locate the subdivided regions with action reference significance.

[0081] The head region focuses on positioning the nose tip region (centered on the nose tip vertex, 5x5 pixel range, as the head position reference point), the head top center region (the midpoint of the top end of the head contour, assisting in judging the head tilt direction); the torso region positions the neck region (the middle position between the bottom end of the chin line and the top end of the shoulder, reflecting the connection posture of the torso and the head), the left shoulder region (the most protruding point on the left shoulder), and the right shoulder region (the most protruding point on the right shoulder, the starting point of the upper limb action); the upper limb region positions the elbow region (the connection between the upper arm and the forearm, reflecting the bending degree of the upper limb), the wrist region (the connection between the forearm and the hand, reflecting the hand position), and the finger root region (the connection between the palm and the fingers, assisting in judging the hand gripping action); the lower limb region positions the hip region (the midpoint of the top end of the hip, the starting point of the lower limb action), the knee region (the connection between the thigh and the shank, reflecting the bending degree of the lower limb), the ankle region (the connection between the shank and the foot), and the toe region (the front end of the foot, assisting in judging whether the leg is close to the pedal); the center pixel coordinates of each subdivided region are recorded, and the positioning error is ≤ 2 pixels.

[0082] Step S505: determining key point data based on the nose tip region, the head top center region, the neck region, the left shoulder region, the right shoulder region, the elbow region, the wrist region, the finger root region, the hip region, the knee region, the ankle region, and the toe region.

[0083] The 17 subdivided feature regions (nose tip, head top center, neck, both shoulders, both elbows, both wrists, both finger roots, hip, both knees, both ankles, and both toes) positioned in step S504 are taken as core key points for data integration.

[0084] In the coordinate conversion process, the pixel coordinates of each key point are combined with the camera intrinsic parameters (preset focal length and pixel size) and the cockpit coordinate system (with the cockpit ground as the origin, the X-axis along the front-rear direction of the vehicle, the Y-axis along the left-right direction, and the Z-axis along the up-down direction) to be converted into actual three-dimensional coordinates (unit: mm); in the data labeling process, a type label can be added to each key point (such as "nose tip key point (X1, Y1, Z1)" and "right knee key point (X2, Y2, Z2)"), forming structured key point data; in the effectiveness verification process, abnormal key points with coordinates exceeding the actual range of the cockpit (such as "toe coordinates in the instrument panel area" due to image blur) are removed, ensuring that the key point data can truly reflect the position of the personnel's limbs and providing accurate position reference for subsequent action data judgment (step S403).

[0085] Optionally, based on the age data and the gear data collected by the gear state sensor, a risk judgment strategy of the personnel and a corresponding early warning control strategy are determined, as shown in FIG. 8, including: Figure 6 ​Step S601: Determine the age value of the person based on the age data, and determine the gear state corresponding to the target vehicle based on the gear data collected by the gear state sensor.

[0086] The age value determination process is based on the age data determined in step S502 (child / adult), and further outputs the specific age value; if it is a child, it is refined into "low age child (≤6 years old)" and "school age child (7-12 years old)" in combination with facial features and weight data (from the second pressure sensor); if it is an adult, directly label the age value range (>12 years old), and preset the age threshold as 12 years old (industry general child safety definition standard, to ensure the focus of risk objects); In the gear state determination process, by real-time reading of the gear data collected by the gear state sensor, the specific gear state of the target vehicle is determined; including P (parking still), R (reverse ready to move / drive), N (neutral no power), D (forward ready to move / drive), and recording the gear duration (such as P has been parked for 5 minutes, R has been switched for 10 seconds), to avoid strategy lag caused by static gear data.

[0087] Step S602: When the age value is not greater than the preset age threshold, determine the risk judgment strategy and early warning control strategy corresponding to the gear state according to the behavior data corresponding to the person; when the age value is greater than the age threshold, determine the risk judgment strategy and early warning control strategy according to the gear state.

[0088] According to the relationship between the age value and the age threshold (12 years old), the strategy is differentiated, and the core is to distinguish between "child risk scene" and "adult normal scene".

[0089] Scenario 1: Age value < 12 years old (child scene, need to combine behavior data); develop strategies based on "whether child behavior affects vehicle safety state", and associate behavior data with gear state to subdivide risks: If the gear is P (still), the behavior data is "close to the gear lever but not touch", the risk judgment strategy is: low risk; the early warning control strategy is: slight buzzing (500Hz, 3 seconds / time) in the cockpit, and the instrument panel displays a blue "child close to the central control" prompt; the behavior data is "touch and try to move the gear lever", the risk judgment strategy is: medium risk; the early warning control strategy is: buzzing + gear electronic locking (prevent gear shifting), and simultaneously push the "child misoperation of gear" notification to the owner's APP; If the gear is R / D (ready to drive / drive), the behavior data is "step on the accelerator / brake pedal (depth >5mm)", the risk judgment strategy is: high risk; the early warning control strategy is: immediately cut off the vehicle power, automatically switch to P and activate the hand brake, and the vehicle voice broadcast "detect child misoperation, emergency brake".

[0090] Scenario 2: Age value ≥ 12 years old (adult scenario, based only on gear state); default adult operation conforms to regular driving logic, strategy focuses on "whether there is a regular safety hazard in gear state", without the need to associate child behavior data: If the gear is in N (neutral) and the vehicle is on a slope (judged by the body inclination sensor), the risk judgment strategy is: low risk; the early warning control strategy is: the instrument panel displays a red text prompt "slope neutral, suggest P gear and pull the handbrake to prevent the car from sliding"; If the gear is in R (reverse) and the vehicle has been reversing for more than 30 seconds (without brake action), the risk judgment strategy is: low risk; the early warning control strategy is: the reverse radar enhances the prompt sound (the frequency increases as the distance to the obstacle shortens); If the gear is in P / D (normal parking / running) and there is no abnormal operation, the risk judgment strategy is: no risk; the early warning control strategy is: no early warning is triggered, only the regular driving state monitoring is maintained.

[0091] Optionally, the risk level corresponding to the behavior data is determined by the risk judgment strategy, as shown in Figure 7 , including: Step S701: determining the action data, first pressure data, second pressure data, acceleration data and pedal depth data corresponding to the personnel based on the behavior data.

[0092] Based on the previously determined personnel behavior data (such as "touching the gear lever", "approaching the pedal", etc.), five types of core data supporting risk judgment are reverse disassembled.

[0093] The action data corresponds to the limb movement trajectory in the behavior data (such as "hand moving towards the gear lever" and "leg stretching towards the pedal"); The first pressure data corresponds to the force information of "touching the gear lever" in the behavior data (such as the pressure value of pressing the gear lever and the contact duration); The second pressure data corresponds to the pressure distribution change of "contacting the seat" in the behavior data (such as the pressure offset from sitting to half-standing); The acceleration data corresponds to the vibration feedback of "touching the instrument table" in the behavior data (such as the acceleration value generated by tapping and pressing); Pedal depth data: corresponding to the operation amplitude of "contacting the pedal" in the behavior data (such as the depth of the accelerator / brake pedal being stepped on and the depth change rate), ensuring that each type of data can be directly associated with the risk dimension.

[0094] Step S702: determining the limb position data of the personnel by using the key point data corresponding to the action data.

[0095] In combination with the key point data (such as wrist, elbow, toe coordinates) contained in the action data, the limb position data is calculated and output, and the core focuses on the relative position of the limb and the key components of the vehicle.

[0096] The hand position is calculated by the wrist and finger root key point coordinates, the straight line distance between the hand and the gear lever (such as distance ≤10cm for "close distance approach", >10cm for "long distance approach"), and the vertical height of the hand and the steering wheel (such as height <30cm for "below the steering wheel plane", with the risk of accidental touch); The leg position is calculated by the knee and toe key point coordinates, the horizontal distance between the foot and the pedal (such as distance ≤5cm for "about to contact the pedal"), and the angle between the knee and the seat (such as angle <90° for "leg bending force, possible pedal stepping"), and the limb position data directly reflects whether there is a space condition for operating the vehicle.

[0097] Step S703: Determine the gear holding data corresponding to the person by using the first pressure data.

[0098] Feature extraction is performed on the first pressure data (collected by the gear handle pressure sensor) to form the gear holding data, focusing on quantifying the holding force and holding state: Holding force: distinguish between light touch (pressure value 1-5N, such as children's fingertips touching briefly), half holding (pressure value 6-15N, such as children's palms wrapping around the gear handle), and full holding (pressure value >15N, such as adult normal operation or children's forceful grip); Holding state: record the pressure duration (such as continuous >2 seconds for continuous holding, <2 seconds for brief touch), pressure distribution (such as pressure concentrated on the top of the gear handle for trying to lift or press the gear handle, and uniform distribution for simple holding), and the gear holding data directly reflects the operation intention strength of the gear.

[0099] Step S704: Determine the seat climbing data corresponding to the person by using the second pressure data.

[0100] Analyze the dynamic changes of the second pressure data (collected by the seat pressure sensor) to determine the seat climbing data, focusing on capturing abnormal changes in sitting posture.

[0101] Pressure distribution change: If the seat cushion pressure changes from "uniform distribution" (normal sitting posture) to "front side concentration + backrest pressure sudden increase" (such as children supporting the seat backrest with both hands and moving their hips forward), it is determined that the child is "climbing the front of the seat and approaching the central control"; Pressure total value change: If the total pressure value changes from "stable value" (such as 20kg child corresponding pressure) to "fluctuating value" (such as pressure instantaneously increasing by 5kg and then falling back), it is determined that the child is "shaking the body and trying to get up"; Seat area change: if the pressure spreads from "single seat area" (such as rear seats) to "cross seat area" (such as rear pressure - front seat edge pressure), it is determined that "climbing from rear to front cockpit", and the seat climbing data directly reflects "whether to break through the safety area and approach the operation core area".

[0102] Step S705: Determine the instrument force data corresponding to the person using acceleration data.

[0103] Filter the acceleration data (collected by the instrument table acceleration sensor) (remove normal engine vibration interference, threshold set to 0.1g), output instrument force data.

[0104] Force intensity: distinguish "light touch" (acceleration value 0.1-0.3g, such as a child's fingertips touching the center control key), "pats" (acceleration value 0.4-0.8g, such as a child's palm patting the instrument table), "impact" (acceleration value > 0.8g, such as a child's body hitting the instrument table); Force position: through multi-sensor data association (such as instrument table left side acceleration changes to "touch left side key", middle changes to "touch center control screen / gear area"), instrument force data mainly assists in determining "whether to mis-touch instrument table control key, indirectly affecting vehicle state".

[0105] Step S706: Determine the driving control data corresponding to the target vehicle using pedal depth data.

[0106] Perform dynamic analysis on pedal depth data (collected by pedal depth sensor) to generate driving control data, focusing on operation amplitude and operation risk.

[0107] Pedal depth: distinguish no operation (depth 0mm), mis-touch (depth 1-5mm, such as a child's foot accidentally touching the pedal), effective operation (depth > 5mm, such as an adult normally stepping or a child stepping hard); Operation risk: combined with gear state supplement determination (such as accelerator pedal depth > 5mm in R gear is high risk, which may cause reverse acceleration, brake pedal depth > 5mm in P gear is low risk, no moving hidden danger), depth change rate (such as depth from 0mm to 10mm in 1 second is fast stepping, higher risk), driving control data directly correlates whether the vehicle driving state can be changed.

[0108] Step S707: Obtain the risk level corresponding to the limb position data, gear holding data, seat climbing data, instrument force data and driving control data according to the risk determination strategy.

[0109] Call the risk determination strategy determined in the foregoing (divide "child scene" and "adult scene", associate with gear state), match the above five types of data with the risk standards in the strategy one by one, and output the corresponding risk level.

[0110] Child + P-gear scene: if the limb position data is "hand 5 cm away from the gear lever" and the gear holding data is "light touch (3N, 1 second)", the risk level is low risk; if the limb position data is "hand 0 cm away from the gear lever" and the gear holding data is "half holding (10N, 3 seconds)", the risk level is medium risk.

[0111] Child + R / D gear scene: if the driving control data is "pedal depth 8mm, fast pedal", the risk level is high risk; if only the instrument force data is "light touch (0.2g)", the risk level is low risk.

[0112] Adult scene: if the seat climbing data is "no abnormality" and the driving control data is "normal pedal depth", the risk level is no risk; if the driving control data is "N-gear slope, pedal depth 0mm", the risk level is low risk.

[0113] Each type of data needs to meet multiple data cross verification to avoid single data misjudgment risk level.

[0114] Optionally, the step S104 of controlling the target vehicle to perform the pre-warning behavior corresponding to the pre-warning control strategy according to the risk level corresponding to the pre-warning control strategy, as shown in the following table, includes: Figure 8 Step S801: Obtain a first risk level corresponding to the limb position data, and control the target vehicle to perform one or more pre-warning behaviors of sound and light warning, central locking, window lifting and double flash opening based on a first pre-warning control strategy corresponding to the first risk level.

[0115] First, according to the limb position data (such as the distance between the hand and the gear lever, the distance between the leg and the pedal), the first risk level (low / medium / high) is divided, and then the first pre-warning control strategy is matched to perform the corresponding behavior: Low risk (such as hand 5-10 cm away from gear lever, not touching): only execute sound and light warning, the cabin buzzer intermittently sounds at a frequency of 500Hz (3 seconds / time), and the instrument panel turns on a blue warning light (displaying that a child is close to the operation area); Medium risk (such as hand 0-5 cm away from gear lever, about to touch): execute sound and light warning + central locking, on the basis of low risk, lock the front door central control (prevent the child from opening the door by mistake), and at the same time, the window is automatically lifted to the closed state (avoid the child's body stretching out); High risk (such as hand close to gear lever, leg close to pedal): execute sound and light warning + central locking + double flash opening, on the basis of medium risk, activate the vehicle double flash (alert surrounding vehicles), and continuously buzz (frequency increased to 800Hz) until the risk is removed.

[0116] ​Step S802: Obtain the second risk level corresponding to the gear holding data, and control the target vehicle to perform one or more of the following pre-warning behaviors based on the second pre-warning control strategy corresponding to the second risk level: sound and light warning, central lock, speed reduction and parking, double flash opening and starting parking.

[0117] According to the gear holding data (holding force, duration), the second risk level (low / medium / high) is divided, and the second pre-warning control strategy is matched.

[0118] Low risk (such as touching the gear lever, pressure 1-5N, duration <2 seconds): execute sound and light warning + central lock, 600Hz intermittent sound of buzzer, yellow light flashing of instrument panel, and lock the gear electronic mechanism (prevent gear shifting); Medium risk (such as half holding the gear lever, pressure 6-15N, duration 2-5 seconds): execute sound and light warning + central lock + double flash opening + start parking, on the basis of low risk, open double flash light, automatically activate electronic hand brake (prevent vehicle from moving unexpectedly); High risk (such as full holding the gear lever, pressure >15N, duration >5 seconds, or trying to dial): execute sound and light warning + central lock + speed reduction and parking + double flash opening + start parking, if the vehicle is in driving state (R / D gear), slow down to 0 with an acceleration of 0.5m / s 2 , and then start parking; if stationary (P / N gear), only intensify sound and light warning (continuous buzzer + red light always on).

[0119] Step S803: Obtain the third risk level corresponding to the seat climbing data, and control the target vehicle to perform one or more of the following pre-warning behaviors based on the third pre-warning control strategy corresponding to the third risk level: sound and light warning and central lock.

[0120] According to the seat climbing data (pressure distribution, cross-region situation), the third risk level (low / medium / high) is divided, and the third pre-warning control strategy is matched.

[0121] Low risk (such as slight shaking of sitting posture, pressure fluctuation <5kg): only execute sound and light warning, rear buzzer 400Hz low frequency sound (avoid shocking children), rear roof light soft green light; High risk (such as climbing the front of the seat, pressure transfer across the seat area): execute sound and light warning + central lock, on the basis of low risk, lock the front door (prevent children from climbing into the driver's cabin and mistakenly opening the door), and at the same time, the front seat is automatically adjusted backward by 5cm (to enlarge the distance between children and gear lever).

[0122] Step S804: Obtain the fourth risk level corresponding to the instrument force data, and control the target vehicle to perform sound and light warning pre-warning behavior based on the fourth pre-warning control strategy corresponding to the fourth risk level.

[0123] The fourth risk level (low / medium / high) is determined based on the instrument force data (acceleration value, force location), and a fourth warning control strategy is matched (only audible and visual warnings, because the instrument force has little impact on vehicle driving).

[0124] Low risk (light touch, acceleration 0.1-0.3g, such as touching a non-control button): The dashboard emits a 300Hz low-frequency beep (1 second / time), and the central control screen displays a green "Do not touch the instrument panel" warning; Medium to high risk (slapping / impact, acceleration > 0.3g, such as touching buttons near the gear shift): The dashboard emits a 700Hz high-frequency beep (lasts for 2 seconds and stops for 1 second), and the central control screen flashes a red light to enhance the warning and prevent further touch.

[0125] Step S805: Obtain the fifth risk level corresponding to the driving control data, and control the target vehicle to perform one or more of the above warning behaviors based on the fifth warning control strategy corresponding to the fifth risk level, such as audible and visual warning, central locking, speed reduction and parking, hazard lights activation, starting parking and calling for rescue.

[0126] The fifth risk level (low / medium / high) is determined based on driving control data (pedal depth, rate of change), and a fifth warning control strategy (with the most comprehensive warning behavior, including rescue triggering) is matched.

[0127] Low risk (accidental pedal touch, depth 1-5mm, speed <2mm / s): Execute audible and visual warning + hazard lights, the vehicle horn sounds a short blast (to alert outside the vehicle), the hazard lights are activated, and the instrument panel displays a yellow "accidental pedal touch" message; Medium risk (effective operation, depth 5-15mm, speed 2-5mm / s, such as pressing the accelerator in P gear): Execute audible and visual warning + central locking + hazard lights + start parking, lock all central locking systems, start electronic parking brake, and the buzzer sounds continuously (800Hz). High risk (depth > 15mm, speed > 5mm / s, such as pressing the accelerator in R / D gear): Execute audible and visual warnings + central locking + deceleration to stop + hazard lights + engage parking brake + call for roadside assistance. The vehicle is traveling at 1m / s. 2 After accelerating smoothly to a standstill, the vehicle will automatically call the owner via the in-vehicle system (sending location and risk type). If there is no response from the owner within 10 seconds, the system will further contact emergency contacts (such as preset family phone numbers) to ensure timely intervention in high-risk scenarios.

[0128] like Figure 9 The flowchart of another vehicle warning control method shown includes the following steps: Step 1: Activate the monitoring system after the vehicle is powered on; Step 2: Detect the characteristics (height / facial features) of the child in the vehicle; Height can be measured using a binocular camera (baseline distance 60mm) and calculated through parallax. Where f is the focal length, B is the baseline distance, d is the parallax, and H is the height value; the detection range is 0.8-1.5m, corresponding to the height of children aged 3-13 years.

[0129] Facial feature recognition can be performed using a convolutional neural network (CNN) architecture, which consists of an input layer (112*112) - ResNet18 - feature vector (512D) - classifier.

[0130] The key identification dimensions are shown in Table 1:

[0131] Table 1 Step 3: Analyze data to determine if the child has triggered dangerous actions; Specifically, the analysis process can be implemented through Table 2:

[0132] Table 2 The tiered response mechanism is as follows: Warning stage (level 1): Feature matching degree 60% to 80% - trigger audio-visual prompt (frequency 2Hz); Intervention stage (level 2): ​​When a limb extends out of the car window, the window automatically rises; when the pedal is accidentally pressed, the power output is limited to 20%.

[0133] The actual hierarchical response strategy used can be found in Table 3:

[0134] Table 3 Step 4: Send control commands via CAN bus (with higher priority than driver operation).

[0135] The core algorithms involved in the above process are as follows: Multi-feature fusion recognition is adopted ;in: This represents a risk feature function that maps the input feature x to a risk score value, reflecting the contribution of a specific feature to the overall risk (e.g., when x = braking frequency). Output braking risk coefficient); For real-time risk assessment of autonomous driving systems (such as collision warning and path planning), the threshold conditions are: 0 to 30 maintain the current state, 30 to 60 issue a warning, and 60 to 100 require active intervention (such as automatic braking). Specific thresholds need to be determined through real-vehicle calibration, and may be adjusted within ±5% for different vehicle models. =0.4 (for height <140cm); =0.3 (hand contact with driving controls); =0.3 (abnormal vehicle acceleration).

[0136] The relevant detection model used takes a single frame of in-vehicle image It as input and outputs detection bounding boxes bi, class probabilities pi, and keypoint coordinates ki. It satisfies the following relationship: ; in, :Detect network parameters; : Coordinates and dimensions of the target bounding box center; : Target category confidence (children, adults, objects, etc.); Coordinates of key points on the human body (head, shoulder, elbow, hand, hip, etc.).

[0137] The joint loss function for detection and keypoint regression is: ; Wherein, the bounding box loss is ; indicates the degree of overlap between the predicted bounding box and the ground truth bounding box; the classification loss is ;in To predict class probabilities (recommended) , ); Category balance coefficient; The keypoint regression loss is: Overall weighting: .

[0138] When extracting key motion features, the Euclidean distance from the hand to the steering wheel is: Hand speed (inter-frame shift): Proximity Index (ReachIndex): ;in: For the Sigmoid function; The action is close to the threshold; Controlling sensitivity.

[0139] In multimodal feature fusion algorithms, the input feature vector is defined as The parameters have the following meanings:

[0140] Bayesian weighted fusion confidence calculation ;in: Confidence levels from different modalities; Weighted according to signal-to-noise ratio; The ratio of the mean to the variance of the modal signals is used to obtain the fusion confidence score. .

[0141] To reduce false alarms, an exponential moving average (EMA) is used: ;in: Smoothing factor, recommended value 0.3; : Smoothing confidence level at time t.

[0142] The algorithm for identifying erroneous operations uses , ;in, *: Network weight matrix; *: Temporal convolution operation; : Non-linear activation function (ReLU); Time window length. Action category set: .

[0143] In the safety decision-making process, a child misoperation risk score is defined: ;in: Weighting coefficients, satisfying =1; Normalized upper limit of hand speed; : Upper limit of radar amplitude normalization. When Furthermore, the vehicle is currently running (ignition = ON), and the system determines that there is a high-risk erroneous operation.

[0144] During gradual intervention, the vehicle state constraints are: ; The actions at each level are as follows:

[0145] As can be seen from the above vehicle warning and control method, this method makes full use of various sensors in the vehicle to accurately analyze children's behavior, and combines them with corresponding warning and control strategies to control the target vehicle to issue warnings, thereby achieving accurate warnings about children's vehicle operation behavior and significantly reducing the probability of such accidents.

[0146] Corresponding to the above embodiments of the vehicle warning control method, this invention also provides a vehicle warning control system, such as... Figure 10 As shown, the system includes: The initialization module 1010 is used to acquire the preset driver status monitoring sensor, gear status sensor and sensing sensor in the cockpit area of ​​the target vehicle, and control the driver status monitoring sensor, gear status sensor and sensing sensor to be in working state after the vehicle is powered on. The perception module 1020 is used to collect early warning feature data in the cockpit area in real time using the driver status monitoring sensor and the sensing sensor, and to determine the age data and behavior data of the people in the cockpit area based on the early warning feature data. The decision module 1030 is used to determine the risk assessment strategy and corresponding early warning control strategy for personnel based on age data and gear position data collected by the gear position status sensor, and to determine the risk level corresponding to the behavioral data using the risk assessment strategy. The execution module 1040 is used to control the target vehicle to perform the warning behavior corresponding to the warning control strategy according to the warning control strategy corresponding to the risk level.

[0147] As can be seen from the above-mentioned vehicle warning and control system, the system makes full use of various sensors in the vehicle to accurately analyze children's behavior, and combines them with corresponding warning and control strategies to control the target vehicle to issue warnings, thereby achieving accurate warnings about children's vehicle operation behavior and significantly reducing the probability of such accidents.

[0148] The vehicle warning control system provided in this embodiment of the invention has the same implementation principle and technical effects as the aforementioned vehicle warning control method embodiment. For the sake of brevity, any parts not mentioned in the system embodiment can be referred to the corresponding content in the aforementioned vehicle warning control method embodiment.

[0149] This embodiment also provides an electronic device, the structural schematic diagram of which is shown below. Figure 11 As shown, the device includes a processor 101 and a memory 102; wherein, the memory 102 is used to store one or more computer instructions, which are executed by the processor to implement the steps of the above-described vehicle warning control method.

[0150] Figure 11 The electronic device shown also includes a bus 103 and a communication interface 104, with the processor 101, communication interface 104 and memory 102 connected via the bus 103.

[0151] The memory 102 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. The bus 103 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0152] The communication interface 104 is used to connect to at least one user terminal and other network units through a network interface, and to send encapsulated IPv4 packets or IPv4 packets to the user terminal through the network interface.

[0153] Processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 101 or by instructions in software form. The processor 101 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 102. The processor 101 reads the information in memory 102 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

Claims

1. A vehicle early warning control method, characterized in that, The method includes: The system acquires preset driver status monitoring sensors, gear status sensors, and sensing sensors in the cockpit area of ​​the target vehicle, and controls the driver status monitoring sensors, gear status sensors, and sensing sensors to be in working state after the vehicle is powered on. The driver status monitoring sensor and the sensing sensor are used to collect early warning feature data in the cockpit area in real time, and the age data and behavior data of the people in the cockpit area are determined based on the early warning feature data. Based on the age data and the gear position data collected by the gear position sensor, a risk assessment strategy and its corresponding early warning control strategy for the person are determined, and the risk level corresponding to the behavioral data is determined using the risk assessment strategy. According to the warning control strategy corresponding to the risk level, the target vehicle is controlled to perform the warning behavior corresponding to the warning control strategy.

2. The vehicle early warning control method according to claim 1, characterized in that, Acquire preset driver status monitoring sensors, gear status sensors, and sensing sensors in the cockpit area of ​​the target vehicle, including: The driver status monitoring sensor is acquired based on the color camera, infrared camera, depth camera, and millimeter-wave radar sensor deployed in the dashboard and roof console of the target vehicle. The gear position sensor is obtained based on the position sensor deployed in the gear position of the target vehicle; The sensing sensors are acquired based on a first pressure sensor deployed in the gear shift lever, a second pressure sensor deployed in the seat, an acceleration sensor deployed in the dashboard, and a depth sensor deployed in the pedals of the target vehicle.

3. The vehicle early warning control method according to claim 2, characterized in that, The driver status monitoring sensor and the sensing sensor are used to collect early warning feature data in the cockpit area in real time, including: The color camera and the infrared camera are used to acquire digital images of the cockpit area in real time, and the personnel contained in the cockpit area are identified through the digital images, and the image data corresponding to the personnel is acquired. The depth camera and the millimeter-wave radar are used to acquire real-time distance data corresponding to the personnel in the cockpit area; The first pressure sensor is used to acquire the first pressure data corresponding to the person gripping the gear lever in real time. The second pressure sensor is used to acquire the second pressure data corresponding to when the person touches the seat in real time. The acceleration sensor is used to acquire real-time acceleration data when the person touches the dashboard. The depth sensor is used to acquire, in real time, the pedal depth data corresponding to when the person controls the accelerator pedal and brake pedal of the vehicle; The warning feature data is determined based on the image data, the distance data, the first pressure data, the second pressure data, the acceleration data, and the pedal depth data.

4. The vehicle early warning control method according to claim 3, characterized in that, Based on the aforementioned warning feature data, the age and behavioral data of personnel in the cockpit area are determined, including: Based on the distance data, the personnel included within a preset range in the cockpit area are obtained; The age and key point data of the person are determined using the image data; The action data of the personnel are determined based on the key point data; The action data, the first pressure data, the second pressure data, the acceleration data, and the pedal depth data are used to determine the behavioral data corresponding to the person.

5. The vehicle early warning control method according to claim 4, characterized in that, The step of determining the age data and key point data of the person using the image data includes: The human body region corresponding to the person is obtained based on the image data, and the face region of the person is determined based on the human body region; Calculate the interpupillary distance, face width-to-height ratio, and chin curvature of the person in the face region, and determine the age data corresponding to the person based on the calculation results of the interpupillary distance, face width-to-height ratio, and chin curvature; Identify and acquire the head region, torso region, upper limb region, and lower limb region corresponding to the person in the human body region; Identify and acquire the nose tip region or the center region of the top of the head region corresponding to the person, the neck region, left shoulder region and right shoulder region corresponding to the person in the torso region, the elbow region, wrist region and finger root region corresponding to the person in the upper limb region, and the hip region, knee region, ankle region and toe region corresponding to the person in the lower limb region; The key point data are determined based on the nose tip region, the center of the top of the head region, the neck region, the left shoulder region, the right shoulder region, the elbow region, the wrist region, the base of the fingers region, the hip region, the knee region, the ankle region, and the toe region.

6. The vehicle early warning control method according to claim 1, characterized in that, Based on the age data and the gear position data collected by the gear position sensor, a risk assessment strategy and its corresponding early warning and control strategy for the personnel are determined, including: The age of the person is determined based on the age data, and the gear position of the target vehicle is determined based on the gear position data collected by the gear position sensor. When the age value is not greater than a preset age threshold, the risk assessment strategy and the early warning control strategy corresponding to the gear position are determined based on the behavioral data of the person. When the age value is greater than the age threshold, the risk assessment strategy and the early warning control strategy are determined based on the gear position.

7. The vehicle early warning control method according to claim 4, characterized in that, Determining the risk level corresponding to the behavioral data using the risk assessment strategy includes: Based on the behavioral data, the action data, the first pressure data, the second pressure data, the acceleration data, and the pedal depth data corresponding to the person are determined; The limb position data of the person is determined by using the key point data corresponding to the motion data; The first pressure data is used to determine the file holding data corresponding to the personnel. The second pressure data is used to determine the seat climbing data corresponding to the person. The acceleration data is used to determine the instrument force data corresponding to the person. The driving control data corresponding to the target vehicle is determined using the pedal depth data. The risk level corresponding to the limb position data, gear grip data, seat climbing data, instrument force data, and driving control data is obtained according to the risk assessment strategy.

8. The vehicle early warning control method according to claim 7, characterized in that, According to the warning control strategy corresponding to the risk level, the target vehicle is controlled to perform the warning behavior corresponding to the warning control strategy, including: The system acquires the first risk level corresponding to the limb position data, and controls the target vehicle to perform one or more of the above-mentioned warning behaviors, such as audible and visual warnings, central locking, window raising, and hazard lights activation, based on the first warning control strategy corresponding to the first risk level. The system obtains the second risk level corresponding to the file holding data, and controls the target vehicle to perform one or more of the above-mentioned warning behaviors based on the second warning control strategy corresponding to the second risk level, including audible and visual warnings, central locking, speed reduction and parking, hazard lights activation, and starting parking. Obtain the third risk level corresponding to the seat climbing data, and control the target vehicle to perform one or more of the above-mentioned warning behaviors, such as audible and visual warnings and central locking, based on the third warning control strategy corresponding to the third risk level; The fourth risk level corresponding to the instrument force data is obtained, and the target vehicle is controlled to perform audible and visual warning actions based on the fourth early warning control strategy corresponding to the fourth risk level. The fifth risk level corresponding to the driving control data is obtained, and the target vehicle is controlled to perform one or more of the above-mentioned warning behaviors based on the fifth warning control strategy corresponding to the fifth risk level, such as audible and visual warning, central locking, speed reduction and parking, hazard lights activation, starting parking and calling for rescue.

9. A vehicle early warning control system, characterized in that, The system includes: An initialization module is used to acquire preset driver status monitoring sensors, gear status sensors and sensing sensors in the cockpit area of ​​the target vehicle, and to control the driver status monitoring sensors, gear status sensors and sensing sensors to be in working state after the vehicle is powered on. The perception module is used to collect early warning feature data in the cockpit area in real time using the driver status monitoring sensor and the sensing sensor, and to determine the age data and behavior data of the people in the cockpit area based on the early warning feature data. The decision-making module is used to determine the risk assessment strategy and its corresponding early warning control strategy for the person based on the age data and the gear position data collected by the gear position sensor, and to determine the risk level corresponding to the behavior data using the risk assessment strategy. The execution module is used to control the target vehicle to perform the warning behavior corresponding to the warning control strategy according to the risk level.

10. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the steps of the vehicle warning control method according to any one of claims 1 to 8.

Images