Door opening risk early warning method and device, electronic equipment and storage medium

Abstract

The invention provides a door opening risk early warning method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting a first image of a preset external region and a second image of a preset internal region of a vehicle in real time if the vehicle enters a parking state; identifying a moving object in the first image, and acquiring a state parameter of the moving object according to the first image; a driver and passengers in the second image are recognized, and state characteristic parameters of the driver and passengers are obtained according to the second image; and according to the state parameters of the moving object and the state characteristic parameters of the driver and passengers, the vehicle is controlled to send preset reminding information and / or execute preset protection actions. According to the invention, the assessment capability of the personnel risk cognition state is enhanced, the door opening collision risk caused by personnel negligence or insufficient environmental perception is effectively reduced, and the reliability of the safety protection system is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle safety technology, and in particular to a door opening risk warning method, device, electronic device and storage medium. Background Art

[0002] With the development of the economy and technology, cars have become an essential means of transportation in people's lives. During daily commutes, people often choose to park on the roadside for various reasons. However, after parking, drivers and passengers may neglect to check for vehicles or pedestrians behind them and rashly open the door, which can easily lead to "door-opening" traffic accidents. This occurs when a vehicle or pedestrian behind them hits a suddenly opened door, causing an accident. This not only causes unnecessary physical injury and psychological trauma to the parties involved, but can also negatively impact road traffic order. Although current vehicles are equipped with some safety devices, they still have many shortcomings in preventing accidents in this specific scenario. Summary of the Invention

[0003] In view of the above, it is necessary to propose a door opening risk warning method, device, electronic device and storage medium to solve the technical problem of how to prevent vehicle accidents in the specific scenario of "door opening kill".

[0004] In a first aspect, the present application provides a door opening risk warning method, which is applied to a vehicle, and the method includes: if the vehicle enters a parking state, collecting a first image of a preset external area and a second image of a preset internal area of the vehicle in real time; identifying a moving object in the first image, and obtaining state parameters of the moving object based on the first image; identifying a driver and passenger in the second image, and obtaining state characteristic parameters of the driver and passenger based on the second image; and controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger.

[0005] In the door opening risk warning method in the embodiment of the present application, first, if the vehicle enters the parking state, a first image of a preset external area and a second image of a preset internal area of the vehicle are collected in real time; a moving object in the first image is identified, and the state parameters of the moving object are obtained based on the first image; the driver and passenger in the second image are further identified, and the state characteristic parameters of the driver and passenger are obtained based on the second image; and further, based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action. Based on this, by comprehensively judging the state parameters of the moving object and the state characteristic parameters of the driver and passenger in multiple dimensions, not only the risk state of the moving object is identified, but also the risk perception ability of the driver and passenger is paid more attention, thereby enhancing the ability to assess the risk cognition state of the personnel, making the warning trigger mechanism more in line with the complexity of the actual driving scene, effectively reducing the risk of door opening collision caused by negligence of the personnel or insufficient environmental perception, realizing the spatiotemporal matching of risk warning, and improving the reliability of the safety protection system.

[0006] In some embodiments of the present application, the state characteristic parameters include the driver and passenger's line of sight and gesture movements, and obtaining the driver and passenger's state characteristic parameters based on the second image includes: obtaining the driver and passenger's first key feature and second key feature based on the second image, wherein the first key feature includes at least hand posture, and the second key feature includes at least eye posture, head posture, and body posture; determining the driver and passenger's gesture movements based on the first key feature, and determining the driver and passenger's line of sight based on the second key feature.

[0007] In some embodiments of the present application, the state parameters include the interval distance between the moving object and the vehicle, the driving speed and actual size of the moving object, and the controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger includes: determining the size information of a preset first area and the size information of a preset second area based on the actual size of the moving object, wherein the first area is an area within a warning range, and the second area is an area within an alarm range; judging whether the moving object is located in the first area or the second area based on the interval distance to obtain a first judgment result, and judging whether the driving speed is greater than a preset speed threshold to obtain a second judgment result; judging whether the line of sight is located in a preset safe area or a preset dangerous area in the vehicle to obtain a third judgment result, wherein the safe area and the dangerous area are formed by the coverage range of the driver and passenger's line of sight in the vehicle; controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result.

[0008] In some embodiments of the present application, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result, including: if the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the car door, the vehicle is controlled to issue the reminder message for a first preset number of times.

[0009] In some embodiments of the present application, controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also includes: if the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, controlling the vehicle to issue the reminder message a second preset number of times, wherein the second preset number is greater than the first preset number.

[0010] In some embodiments of the present application, controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also includes: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue the reminder message for the second preset number of times; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the safe area, and the gesture action is to open the door, controlling the vehicle to issue the reminder message for the second preset number of times; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, controlling the vehicle to perform the protective action, wherein the protective action includes locking the door.

[0011] In some embodiments of the present application, controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also includes: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue the reminder message for the first preset number of times; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in the safe area, and the gesture action is to open the door, controlling the vehicle to issue the reminder message for the first preset number of times; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, controlling the vehicle to issue the reminder message for the second preset number of times.

[0012] In the second aspect, the present application also provides a door opening risk warning device, which is applied to a vehicle, and the device includes: an acquisition module, which is used to collect a first image of a preset external area and a second image of a preset internal area of the vehicle in real time if the vehicle enters a parking state; a first acquisition module, which is used to identify a moving object in the first image and obtain the state parameters of the moving object based on the first image; a second acquisition module, which is used to identify the driver and passenger in the second image and obtain the state characteristic parameters of the driver and passenger based on the second image; a warning module, which is used to control the vehicle to issue a preset reminder message and / or perform a preset protective action based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger.

[0013] In a third aspect, the present application also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the door opening risk warning method described in the above embodiment.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the door opening risk warning method described in the above embodiment is implemented.

[0015] It can be understood that the door opening risk warning device of the second aspect, the electronic device of the third aspect, and the computer-readable storage medium of the fourth aspect provided above all correspond to the door opening risk warning method of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding door opening risk warning method provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a door opening risk warning method provided in one embodiment of the present application.

[0017] Figure 2 This is a schematic diagram of an application scenario of a door opening risk warning method provided in an embodiment of the present application.

[0018] Figure 3 This is a schematic diagram of an application scenario of a door opening risk warning method provided in another embodiment of the present application.

[0019] Figure 4 This is a schematic diagram of the functional modules of a door opening risk warning device provided in one embodiment of the present application.

[0020] Figure 5 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application.

[0021] Component Symbol Description Electronic device 10 Memory 11 Processor 12 Door opening risk warning device 100 Acquisition module 110 First acquisition module 120 Second acquisition module 130 Early warning module 140 The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0022] In order to make the technical problems, technical solutions and beneficial effects solved by this application more clearly understood, this application is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0023] In order to more clearly understand the embodiments of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other in the absence of conflict.

[0024] See also Figure 1 , which is a flow chart of a door opening risk warning method provided in one embodiment of the present application.

[0025] The door opening risk warning method of the embodiment of the present application can be applied to one or more electronic devices 10. The electronic device 10 is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc.

[0026] The electronic device 10 may be an onboard device of a vehicle, such as a body control module (BCM), a vehicle control unit (VCU), etc.

[0027] Specifically, the door opening risk warning method includes the following steps. According to different needs, the order of some steps in the flowchart can be changed, and some steps can be omitted.

[0028] S10: If the vehicle enters a parking state, a first image of a preset external area and a second image of a preset internal area of the vehicle are collected in real time.

[0029] The preset external area refers to the monitoring area within a specific range behind and to the sides of the vehicle, and the preset internal area refers to the monitoring area within a specific range inside the vehicle. Furthermore, the coverage of the preset external and preset internal areas can be set by adjusting the relevant camera's viewing angle parameters.

[0030] S20: Identify a moving object in the first image, and obtain state parameters of the moving object according to the first image.

[0031] The moving objects include but are not limited to vehicles, pedestrians, etc. The state parameters include but are not limited to the distance between the moving object and the vehicle, the speed and actual size of the moving object, etc., which are not limited in this application.

[0032] Specifically, a target detection algorithm can be used to identify moving objects in the first image. Common target detection algorithms include those based on traditional computer vision and those based on deep learning. Furthermore, the vehicle's speed sensor can be used to collect the moving object's speed. Furthermore, the Euclidean distance formula can be used to calculate the distance between the moving object's coordinate position in the real world and the vehicle's own coordinate position in the real world. Furthermore, during the target detection process, after detecting the moving object, a bounding box of the moving object in the first image can be obtained. Using known camera parameters (e.g., focal length, pixel size, etc.) and the actual distance of the moving object from the camera, i.e., the distance between the moving object and the vehicle, the size of the bounding box can be converted to the actual size of the moving object based on the geometric relationship of perspective projection.

[0033] S30: Identify the driver and passenger in the second image, and obtain state characteristic parameters of the driver and passenger based on the second image.

[0034] Specifically, the state characteristic parameters include, but are not limited to, the driver's or passenger's gaze point and hand gestures. The specific steps for obtaining the driver's or passenger's state characteristic parameters based on the second image include: obtaining, based on a preset first key feature and a second key feature of the driver's or passenger, based on the second image, wherein the first key feature includes at least hand posture, and the second key feature includes at least eye posture, head posture, and body posture; determining the driver's or passenger's hand gestures based on the first key feature, and determining the driver's or passenger's gaze point based on the second key feature.

[0035] Among them, the sight point refers to the specific location of the driver's gaze in the interior space of the vehicle. The hand posture in the first key feature refers to the hand joint angle and finger extension status information captured by the camera. Specifically, a convolutional neural network can be used to extract features of the hand area in the second image to achieve recognition of action intentions such as opening the door and unfastening the seat belt. The second key feature refers to the eye movement trajectory, head deflection angle and torso orientation information collected by the camera. Specifically, it can be achieved by three-dimensional spatial modeling of the eye posture, head posture and body posture based on a three-dimensional modeling algorithm of multi-view geometry to determine whether the driver observes moving objects outside the vehicle before getting off the vehicle.

[0036] The above-mentioned embodiment, by fusing eye, head, and body posture data and combining it with hand motion recognition, can more accurately determine whether the driver or passenger intends to observe moving objects outside the vehicle and whether they have the intention to open the door. Through multi-source feature fusion, it can accurately distinguish between the driver or passenger's actual observation area and unconscious posture changes, reducing the risk of door opening collision caused by observation blind spots.

[0037] S40: According to the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action.

[0038] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action according to the state parameters of the moving object and the state characteristic parameters of the driver and passenger include: determining the size information of the preset first area and the size information of the preset second area according to the actual size of the moving object, wherein the first area is the area within the warning range, that is, Figure 2 The second area is the area within the warning range, that is, Figure 2 The warning area shown; judging whether the moving object is located in the first area or the second area according to the interval distance, obtaining a first judgment result, and judging whether the driving speed is greater than a preset speed threshold, obtaining a second judgment result; judging whether the line of sight is located in a preset safe area or a preset dangerous area in the vehicle, obtaining a third judgment result, wherein the safe area and the dangerous area are formed by the coverage range of the driver's line of sight in the vehicle; controlling the vehicle to issue a preset reminder message and / or perform a preset protective action according to the gesture action, the first judgment result, the second judgment result, and / or the third judgment result.

[0039] Among them, combined Figure 2As shown, it is assumed that the moving object is a vehicle coming from behind and the vehicle itself is the target vehicle; the longitudinal distance between the vehicle coming from behind and the target vehicle is set to L0, and the longitudinal distance between the vehicle coming from behind and the target vehicle is divided into two parts: L0=L1+L2. If the vehicle enters the L1 range, it is the warning distance; if the vehicle enters the L2 range, it is the alarm distance. The lateral distance between the vehicle coming from behind and the target vehicle is set to H0, and the lateral distance between the vehicle coming from behind and the target vehicle is divided into two parts. If the vehicle enters the H1 range, it is the warning distance; if the vehicle enters the H2 range, it is the alarm distance. Among them, the actual size of the vehicle coming from behind is used to dynamically adjust the coverage of the warning area and the alarm area to accommodate moving objects of different sizes. The actual size of the vehicle coming from behind is set differently, and the size settings of L0, L1, L2, H0, H1, and H2 are also different. For example, a larger warning distance and alarm distance are set for large vehicles coming from behind. The specific settings are made according to actual needs, and this application does not impose any restrictions on this.

[0040] Among them, combined Figure 3 As shown, the safe area and the dangerous area refer to the spatial areas inside the vehicle where the driver and passenger's line of sight falls. Assume that the driver and passenger opens the door from the left side of the vehicle to get off, and the driver and passenger's line of sight falls in area d (i.e. the left window area of the vehicle), indicating that the driver and passenger has the intention to observe the moving objects on the left side outside the vehicle, then the line of sight falls in the safe area. If the driver and passenger's line of sight falls in area a (i.e. the left door handle area of the vehicle), area b (i.e. the left footrest and carpet area of the vehicle), or area c (i.e. the area directly in front of the vehicle), indicating that the driver and passenger has no intention to observe the moving objects on the left side outside the vehicle, then the line of sight falls in the dangerous area. On the contrary, if the passenger opens the door on the right side to get out of the vehicle, the passenger's line of sight is in area D (i.e. the right window area of the vehicle), indicating that the driver and passenger have the intention to observe moving objects on the right side outside the vehicle, and the line of sight is in the safe area. If the driver and passenger's line of sight is in area A (i.e. the right door handle area of the vehicle), area B (i.e. the right footrest and carpet area of the vehicle), and area C (i.e. the area directly in front of the vehicle), indicating that the driver and passenger do not have the intention to observe moving objects on the right side outside the vehicle, the line of sight is in the dangerous area.

[0041] In the above embodiment, the boundary range of the warning area and the alarm area is dynamically optimized according to the actual size of the moving object, and a multi-dimensional risk assessment is performed in combination with the interval distance, driving speed and the line of sight of the driver and passengers. The warning level is accurately matched before the driver and passengers perform the door opening action, which not only reduces excessive prompts for low-risk scenarios, but also can promptly activate the protection mechanism for high-risk scenarios, effectively reducing the risk of collision caused by suddenly opening the car door.

[0042] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on a gesture action, a first judgment result, a second judgment result, and / or a third judgment result include: if the first judgment result is that the vehicle is located in a first area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in a dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue a first preset number of reminder messages.

[0043] In the above embodiment, when a moving object is detected to be in a first area, the driving speed does not exceed a speed threshold, the driver's line of sight is in the danger zone, and there is an intention to open the door, indicating that the risk of a "door-opening kill" accident is low, a first preset number of reminder messages (e.g., 1) are triggered. For example, the reminder message can be a voice prompt or a flashing instrument panel warning light. Based on this, when the risk is low, the driver and passengers are reminded to pay attention and observe in a timely manner, while avoiding distractions caused by excessive reminders, thereby ensuring safety while optimizing the user experience.

[0044] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue a second preset number of reminder messages, wherein the second preset number is greater than the first preset number.

[0045] In the above embodiment, when a moving object is detected in the first area, the speed exceeds the speed threshold, the driver's line of sight is in the danger zone, and there is an intention to open the door, indicating a slightly higher risk of a "door-opening collision" accident, a second preset number of reminder messages (for example, two) are triggered. Based on this, increasing the number of reminder messages in slightly higher-risk situations can more effectively attract the driver's attention, compared to a single reminder in low-speed scenarios, and effectively reduce the risk of door collisions caused by driver negligence.

[0046] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue a second preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is in a safe area, and the gesture action is to open the door, controlling the vehicle to issue a second preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is in a dangerous area, and the gesture action is to open the door, controlling the vehicle to perform a protective action, wherein the protective action includes locking the door.

[0047] In the above-described embodiment, when a moving object is detected in the second area and the driving speed exceeds the speed threshold, even if there is no intention to open the door, the risk of the vehicle being involved in a "door-opening kill" accident is slightly higher, and a second preset number of reminder messages (for example, 2) are triggered. Based on this, increasing the number of reminder messages in situations where the risk is slightly higher can more effectively attract the attention of drivers and passengers, strengthen the warning effect, and effectively reduce the risk of door collisions caused by driver negligence, compared to a single reminder in low-speed scenarios. When a moving object is detected in the second area, the driving speed exceeds the speed threshold, the line of sight is in the danger zone, and there is an intention to open the door, it indicates that the risk of the vehicle being involved in a "door-opening kill" accident is extremely high, and dangerous behavior is blocked. Based on this, while ensuring the effectiveness of the reminder warning, physical blocking measures can be directly implemented for extremely dangerous scenarios, significantly reducing the probability of collision accidents caused by accidentally opening the door.

[0048] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue a first preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is in a safe area, and the gesture action is to open the door, controlling the vehicle to issue a first preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is in a dangerous area, and the gesture action is to open the door, controlling the vehicle to issue a second preset number of reminder messages.

[0049] In the above embodiment, when a moving object is detected in the second area and its speed does not exceed the speed threshold, even if there is no intention to open the door; or when a moving object is detected in the second area and its speed does not exceed the speed threshold, but there is an intention to open the door, but the line of sight is within a safe area, there may still be a risk of a "door-opening kill" accident. A first preset number of reminder messages (e.g., one) are triggered. This ensures timely reminders to drivers and passengers when the risk is low, while avoiding distractions caused by excessive reminders, thereby ensuring safety while optimizing the user experience. When a moving object is detected in the second area and its speed does not exceed the speed threshold, and there is an intention to open the door, but the line of sight is within a dangerous area, indicating a slightly higher risk of a "door-opening kill" accident, a second preset number of reminder messages (e.g., two) are triggered. Increasing the number of reminder messages in situations of slightly higher risk more effectively draws the attention of drivers and passengers, compared to a single reminder in low-speed scenarios, effectively reducing the risk of door collisions caused by driver negligence.

[0050] In the above embodiment, by introducing a collaborative analysis mechanism of gaze tracking and behavioral actions, it is possible to effectively distinguish between the active safety behavior and potentially dangerous operations of the driver and passengers, realizing a dual evaluation system based on the degree of environmental danger and the behavioral characteristics of the driver and passengers. It can automatically match the optimal reminder strategy in different risk level scenarios, ensuring necessary safety warnings while avoiding excessive interference with normal operations, avoiding excessive alarms or missed reports, and effectively improving the adaptability of warning scenarios and user experience.

[0051] In the door opening risk warning method in the embodiment of the present application, first, if the vehicle enters the parking state, a first image of a preset external area and a second image of a preset internal area of the vehicle are collected in real time; a moving object in the first image is identified, and the state parameters of the moving object are obtained based on the first image; the driver and passenger in the second image are further identified, and the state characteristic parameters of the driver and passenger are obtained based on the second image; and further, based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action. Based on this, by comprehensively judging the state parameters of the moving object and the state characteristic parameters of the driver and passenger in multiple dimensions, not only the risk state of the moving object is identified, but also the risk perception ability of the driver and passenger is paid more attention, thereby enhancing the ability to assess the risk cognition state of the personnel, making the warning trigger mechanism more in line with the complexity of the actual driving scene, effectively reducing the risk of door opening collision caused by negligence of the personnel or insufficient environmental perception, realizing the spatiotemporal matching of risk warning, and improving the reliability of the safety protection system.

[0052] See also Figure 4 , is a schematic diagram of the functional modules of the door opening risk warning device 100 provided in one embodiment of the present application.

[0053] In this embodiment, based on the above Figure 1 The present application also provides a door opening risk warning device 100 that is similar in concept to the door opening risk warning method in the illustrated embodiment, and can be used to implement the aforementioned door opening risk warning method. For ease of explanation, the schematic diagram of the embodiment of the door opening risk warning device 100 only shows the parts relevant to the embodiment of the present application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the door opening risk warning device 100, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0054] Specifically, the door opening risk warning device 100 provided in the embodiment of the present application includes a collection module 110 , a first acquisition module 120 , a second acquisition module 130 , and a warning module 140 .

[0055] The acquisition module 110 is used to acquire a first image of a preset external area and a second image of a preset internal area of the vehicle in real time when the vehicle enters a parking state.

[0056] The preset external area refers to the monitoring area within a specific range behind and to the sides of the vehicle, and the preset internal area refers to the monitoring area within a specific range inside the vehicle. Furthermore, the coverage of the preset external and preset internal areas can be set by adjusting the relevant camera's viewing angle parameters.

[0057] The first acquisition module 120 is used to identify a moving object in a first image and acquire state parameters of the moving object according to the first image.

[0058] The moving objects include but are not limited to vehicles, pedestrians, etc. The state parameters include but are not limited to the distance between the moving object and the vehicle, the speed and actual size of the moving object, etc., which are not limited in this application.

[0059] Specifically, a target detection algorithm can be used to identify moving objects in the first image. Common target detection algorithms include those based on traditional computer vision and those based on deep learning. Furthermore, the vehicle's speed sensor can be used to collect the moving object's speed. Furthermore, the Euclidean distance formula can be used to calculate the distance between the moving object's coordinate position in the real world and the vehicle's own coordinate position in the real world. Furthermore, during the target detection process, after detecting the moving object, a bounding box of the moving object in the first image can be obtained. Using known camera parameters (e.g., focal length, pixel size, etc.) and the actual distance of the moving object from the camera, i.e., the distance between the moving object and the vehicle, the size of the bounding box can be converted to the actual size of the moving object based on the geometric relationship of perspective projection.

[0060] The second acquisition module 130 is used to identify the driver and passenger in the second image and acquire state characteristic parameters of the driver and passenger based on the second image.

[0061] Specifically, the state characteristic parameters include, but are not limited to, the driver's or passenger's gaze point and hand gestures. The specific steps for obtaining the driver's or passenger's state characteristic parameters based on the second image include: obtaining, based on a preset first key feature and a second key feature of the driver's or passenger, based on the second image, wherein the first key feature includes at least hand posture, and the second key feature includes at least eye posture, head posture, and body posture; determining the driver's or passenger's hand gestures based on the first key feature, and determining the driver's or passenger's gaze point based on the second key feature.

[0062] Among them, the sight point refers to the specific location of the driver's gaze in the interior space of the vehicle. The hand posture in the first key feature refers to the hand joint angle and finger extension status information captured by the camera. Specifically, a convolutional neural network can be used to extract features of the hand area in the second image to achieve recognition of action intentions such as opening the door and unfastening the seat belt. The second key feature refers to the eye movement trajectory, head deflection angle and torso orientation information collected by the camera. Specifically, it can be achieved by three-dimensional spatial modeling of the eye posture, head posture and body posture based on a three-dimensional modeling algorithm of multi-view geometry to determine whether the driver observes moving objects outside the vehicle before getting off the vehicle.

[0063] The above-mentioned embodiment, by fusing eye, head, and body posture data and combining it with hand motion recognition, can more accurately determine whether the driver or passenger intends to observe moving objects outside the vehicle and whether they have the intention to open the door. Through multi-source feature fusion, it can accurately distinguish between the driver or passenger's actual observation area and unconscious posture changes, reducing the risk of door opening collision caused by observation blind spots.

[0064] The warning module 140 is used to control the vehicle to issue a preset warning message and / or perform a preset protective action according to the state parameters of the moving object and the state characteristic parameters of the driver and passenger.

[0065] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action according to the state parameters of the moving object and the state characteristic parameters of the driver and passenger include: determining the size information of the preset first area and the size information of the preset second area according to the actual size of the moving object, wherein the first area is the area within the warning range, that is, Figure 2 The second area is the area within the warning range, that is, Figure 2The warning area shown; judging whether the moving object is located in the first area or the second area according to the interval distance, obtaining a first judgment result, and judging whether the driving speed is greater than a preset speed threshold, obtaining a second judgment result; judging whether the line of sight is located in a preset safe area or a preset dangerous area in the vehicle, obtaining a third judgment result, wherein the safe area and the dangerous area are formed by the coverage range of the driver's line of sight in the vehicle; controlling the vehicle to issue a preset reminder message and / or perform a preset protective action according to the gesture action, the first judgment result, the second judgment result, and / or the third judgment result.

[0066] Among them, combined Figure 2 As shown, it is assumed that the moving object is a vehicle coming from behind and the vehicle itself is the target vehicle; the longitudinal distance between the vehicle coming from behind and the target vehicle is set to L0, and the longitudinal distance between the vehicle coming from behind and the target vehicle is divided into two parts: L0=L1+L2. If the vehicle enters the L1 range, it is the warning distance; if the vehicle enters the L2 range, it is the alarm distance. The lateral distance between the vehicle coming from behind and the target vehicle is set to H0, and the lateral distance between the vehicle coming from behind and the target vehicle is divided into two parts. If the vehicle enters the H1 range, it is the warning distance; if the vehicle enters the H2 range, it is the alarm distance. Among them, the actual size of the vehicle coming from behind is used to dynamically adjust the coverage of the warning area and the alarm area to accommodate moving objects of different sizes. The actual size of the vehicle coming from behind is set differently, and the size settings of L0, L1, L2, H0, H1, and H2 are also different. For example, a larger warning distance and alarm distance are set for large vehicles coming from behind. The specific settings are made according to actual needs, and this application does not impose any restrictions on this.

[0067] Among them, combined Figure 3As shown, the safe area and the dangerous area refer to the spatial areas inside the vehicle where the driver and passenger's line of sight falls. Assume that the driver and passenger opens the door from the left side of the vehicle to get off, and the driver and passenger's line of sight falls in area d (i.e. the left window area of the vehicle), indicating that the driver and passenger has the intention to observe the moving objects on the left side outside the vehicle, then the line of sight falls in the safe area. If the driver and passenger's line of sight falls in area a (i.e. the left door handle area of the vehicle), area b (i.e. the left footrest and carpet area of the vehicle), or area c (i.e. the area directly in front of the vehicle), indicating that the driver and passenger has no intention to observe the moving objects on the left side outside the vehicle, then the line of sight falls in the dangerous area. On the contrary, if the passenger opens the door on the right side to get out of the vehicle, the passenger's line of sight is in area D (i.e. the right window area of the vehicle), indicating that the driver and passenger have the intention to observe moving objects on the right side outside the vehicle, and the line of sight is in the safe area. If the driver and passenger's line of sight is in area A (i.e. the right door handle area of the vehicle), area B (i.e. the right footrest and carpet area of the vehicle), and area C (i.e. the area directly in front of the vehicle), indicating that the driver and passenger do not have the intention to observe moving objects on the right side outside the vehicle, the line of sight is in the dangerous area.

[0068] In the above embodiment, the boundary range of the warning area and the alarm area is dynamically optimized according to the actual size of the moving object, and a multi-dimensional risk assessment is performed in combination with the interval distance, driving speed and the line of sight of the driver and passengers. The warning level is accurately matched before the driver and passengers perform the door opening action, which not only reduces excessive prompts for low-risk scenarios, but also can promptly activate the protection mechanism for high-risk scenarios, effectively reducing the risk of collision caused by suddenly opening the car door.

[0069] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on a gesture action, a first judgment result, a second judgment result, and / or a third judgment result include: if the first judgment result is that the vehicle is located in a first area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in a dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue a first preset number of reminder messages.

[0070] In the above embodiment, when a moving object is detected to be in a first area, the driving speed does not exceed a speed threshold, the driver's line of sight is in the danger zone, and there is an intention to open the door, indicating that the risk of a "door-opening kill" accident is low, a first preset number of reminder messages (e.g., 1) are triggered. For example, the reminder message can be a voice prompt or a flashing instrument panel warning light. Based on this, when the risk is low, the driver and passengers are reminded to pay attention and observe in a timely manner, while avoiding distractions caused by excessive reminders, thereby ensuring safety while optimizing the user experience.

[0071] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue a second preset number of reminder messages, wherein the second preset number is greater than the first preset number.

[0072] In the above embodiment, when a moving object is detected in the first area, the speed exceeds the speed threshold, the driver's line of sight is in the danger zone, and there is an intention to open the door, indicating a slightly higher risk of a "door-opening collision" accident, a second preset number of reminder messages (for example, two) are triggered. Based on this, increasing the number of reminder messages in slightly higher-risk situations can more effectively attract the driver's attention, compared to a single reminder in low-speed scenarios, and effectively reduce the risk of door collisions caused by driver negligence. In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue a second preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is in a safe area, and the gesture action is to open the door, controlling the vehicle to issue a second preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is in a dangerous area, and the gesture action is to open the door, controlling the vehicle to perform a protective action, wherein the protective action includes locking the door.

[0073] In the above-described embodiment, when a moving object is detected in the second area and the driving speed exceeds the speed threshold, even if there is no intention to open the door, the risk of the vehicle being involved in a "door-opening kill" accident is slightly higher, and a second preset number of reminder messages (for example, 2) are triggered. Based on this, increasing the number of reminder messages in situations where the risk is slightly higher can more effectively attract the attention of drivers and passengers, strengthen the warning effect, and effectively reduce the risk of door collisions caused by driver negligence, compared to a single reminder in low-speed scenarios. When a moving object is detected in the second area, the driving speed exceeds the speed threshold, the line of sight is in the danger zone, and there is an intention to open the door, it indicates that the risk of the vehicle being involved in a "door-opening kill" accident is extremely high, and dangerous behavior is blocked. Based on this, while ensuring the effectiveness of the reminder warning, physical blocking measures can be directly implemented for extremely dangerous scenarios, significantly reducing the probability of collision accidents caused by accidentally opening the door.

[0074] In some embodiments of the present application, the specific steps of controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result also include: if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, and the gesture action is to unfasten the seat belt, controlling the vehicle to issue a first preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is in a safe area, and the gesture action is to open the door, controlling the vehicle to issue a first preset number of reminder messages; if the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is in a dangerous area, and the gesture action is to open the door, controlling the vehicle to issue a second preset number of reminder messages.

[0075] In the above embodiment, when a moving object is detected in the second area and its speed does not exceed the speed threshold, even if there is no intention to open the door; or when a moving object is detected in the second area and its speed does not exceed the speed threshold, but there is an intention to open the door, but the line of sight is within a safe area, there may still be a risk of a "door-opening kill" accident. A first preset number of reminder messages (e.g., one) are triggered. This ensures timely reminders to drivers and passengers when the risk is low, while avoiding distractions caused by excessive reminders, thereby ensuring safety while optimizing the user experience. When a moving object is detected in the second area and its speed does not exceed the speed threshold, and there is an intention to open the door, but the line of sight is within a dangerous area, indicating a slightly higher risk of a "door-opening kill" accident, a second preset number of reminder messages (e.g., two) are triggered. Increasing the number of reminder messages in situations of slightly higher risk more effectively draws the attention of drivers and passengers, compared to a single reminder in low-speed scenarios, effectively reducing the risk of door collisions caused by driver negligence. In the above embodiment, by introducing a collaborative analysis mechanism of gaze tracking and behavioral actions, it is possible to effectively distinguish between the active safety behavior and potentially dangerous operations of the driver and passengers, realizing a dual evaluation system based on the degree of environmental danger and the behavioral characteristics of the driver and passengers. It can automatically match the optimal reminder strategy in different risk level scenarios, ensuring necessary safety warnings while avoiding excessive interference with normal operations, avoiding excessive alarms or missed reports, and effectively improving the adaptability of warning scenarios and user experience.

[0076] In the door opening risk warning device 100 of the present application, first, if the vehicle enters the parking state, a first image of a preset external area and a second image of a preset internal area of the vehicle are collected in real time; a moving object in the first image is identified, and the state parameters of the moving object are obtained based on the first image; the driver and passenger in the second image are further identified, and the state characteristic parameters of the driver and passenger are obtained based on the second image; and further, based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action. Based on this, by comprehensively judging the state parameters of the moving object and the state characteristic parameters of the driver and passenger in multiple dimensions, not only the risk state of the moving object is identified, but also the risk perception ability of the driver and passenger is paid more attention, thereby enhancing the ability to assess the risk cognition state of the personnel, making the warning trigger mechanism more in line with the complexity of the actual driving scene, effectively reducing the risk of door opening collision caused by negligence of the personnel or insufficient environmental perception, realizing the spatiotemporal matching of risk warnings, and improving the reliability of the safety protection system.

[0077] See also Figure 5 , is a schematic diagram of the hardware structure of an electronic device 10 provided in one embodiment of the present application.

[0078] The electronic device 10 provided in the embodiment of the present application includes but is not limited to a memory 11, a processor 12, and a computer program stored in the memory 11 and executable on the processor 12, such as a door opening risk warning program. When the computer program is executed by the processor 12, the door opening risk warning method as described in the above embodiment is implemented.

[0079] Figure 5 Only the electronic device 10 having the memory 11 and the processor 12 is shown. It can be understood by those skilled in the art that Figure 5 The structure shown does not limit the electronic device 10 , and the electronic device 10 may include fewer or more components than shown, or combine certain components, or arrange the components differently.

[0080] In some embodiments of the present application, the electronic device 10 can be communicatively connected to devices such as a desktop computer, a notebook, a PDA, a cloud server, etc.

[0081] In some embodiments of the present application, the electronic device 10 can perform human-computer interaction with the user through a keyboard, mouse, remote control, touchpad or voice control device.

[0082] In some embodiments of the present application, the electronic device 10 may further include a network device and / or a client device. The network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, and a cloud server based on cloud computing consisting of a large number of hosts or network servers.

[0083] In some embodiments of the present application, the network where the electronic device 10 is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (VPN), etc.

[0084] In some embodiments of the present application, the memory 11 stores multiple computer-readable instructions to implement a door opening risk warning method, and the processor 12 can execute multiple instructions to achieve: if the vehicle enters a parking state, a first image of a preset external area and a second image of a preset internal area of the vehicle are collected in real time; a moving object in the first image is identified, and the state parameters of the moving object are obtained based on the first image; the driver and passenger in the second image are identified, and the state characteristic parameters of the driver and passenger are obtained based on the second image; according to the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action.

[0085] Specifically, the specific implementation method of the processor 12 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0086] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 10 and does not constitute a limitation on the electronic device 10. The electronic device 10 may have a bus structure or a star structure. The electronic device 10 may also include more or less other hardware or software than shown in the figure, or a different arrangement of components. For example, the electronic device 10 may also include input and output devices, network access devices, etc.

[0087] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The diagram is represented by only one arrow, but this does not mean that there is only one bus or one type of bus. The bus is configured to implement connection and communication between the memory 11 and at least one processor 12, etc.

[0088] It should be noted that the electronic device 10 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the scope of protection of the present application and incorporated herein by reference.

[0089] In some embodiments of the present application, the processor 12 may be comprised of an integrated circuit, such as a single packaged integrated circuit or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 12 is the control core (Control Unit) of the electronic device 10, connecting the various components of the electronic device 10 using various interfaces and circuits. It executes or runs programs or modules stored in the memory 11 (e.g., executing a door opening risk warning program) and accessing data stored in the memory 11 to execute various functions of the electronic device 10 and process data.

[0090] The processor 12 executes the operating system of the electronic device 10 and various installed applications. The processor 12 executes the applications to implement the steps in each of the above-mentioned embodiments of the door opening risk warning method, for example Figure 1 Steps shown.

[0091] Exemplarily, the computer program may be divided into one or more modules / units, one or more of which are stored in the memory 11 and executed by the processor 12 to complete the present application. One or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program in the electronic device 10. For example, the computer program may be divided into an acquisition module 110, a first acquisition module 120, a second acquisition module 130, and an early warning module 140.

[0092] The integrated unit implemented as a software functional module can be stored in a computer-readable storage medium. The software functional module stored in a storage medium includes instructions for causing a computer device (which can be a personal computer, computer device, or network device, etc.) or a processor to execute a portion of a door opening risk warning method according to various embodiments of the present application.

[0093] If the modules / units integrated in the electronic device 10 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete them through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments.

[0094] The computer program includes computer program code, which may be in source code form, object code form, executable file, or some intermediate form. Computer-readable media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory, and other memory devices.

[0095] An embodiment of the present application further provides a computer-readable storage medium (not shown), in which computer-readable instructions are stored. The computer-readable instructions are executed by a processor in the electronic device 10 to implement a door opening risk warning method according to any of the above embodiments.

[0096] Specifically, the computer-readable storage medium can be non-volatile or volatile. Computer-readable storage media include flash memory, mobile hard disks, multimedia cards, card-type memories (such as SD memories, DX memories, etc.), magnetic memories, magnetic disks, optical disks, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 10, such as a mobile hard disk of the electronic device 10. In other embodiments, the memory 11 can also be an external storage device of the electronic device 10, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, etc. equipped on the electronic device 10. The memory 11 can not only be used to store application software installed on the electronic device 10 and various types of data, such as the code of a door opening risk warning program, but can also be used to temporarily store data that has been output or is about to be output.

[0097] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0098] In the embodiments of the present application, it should be noted that, unless otherwise clearly specified or limited, the meaning of "multiple" refers to two or more.

[0099] In the embodiments of this application, it should be noted that, unless otherwise expressly specified or limited, words such as "for example" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" in the embodiments of this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Rather, the use of words such as "for example" is intended to present related concepts in a concrete manner.

[0100] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections, electrical connections, or mutual communication; they can refer to direct connections or indirect connections through an intermediate medium; they can refer to internal communication between two components or the interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on specific circumstances.

[0101] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more features.

[0102] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, "and / or" is merely a description of the association relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally indicates that the related objects are in an "or" relationship.

[0103] Unless otherwise specified, all steps of this application may be performed sequentially or randomly. For example, "a method includes steps A and B" means that the method may include steps A and B performed sequentially, or steps B and A performed sequentially. For example, "a method may also include step C" means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or steps A, C, and B, or steps C, A, and B, etc.

[0104] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the module division is only a logical function division, and other division methods may be used in actual implementation.

[0106] The functional modules in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0107] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not intended to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A door opening risk warning method, characterized in that: Applied to a vehicle, the method comprises: If the vehicle enters a parking state, capturing a first image of a preset external area and a second image of a preset internal area of the vehicle in real time; identifying a moving object in the first image, and acquiring state parameters of the moving object based on the first image; Identifying a driver or passenger in the second image, and obtaining state characteristic parameters of the driver or passenger based on the second image; According to the state parameters of the moving object and the state characteristic parameters of the driver and passenger, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action.

2. The door opening risk warning method according to claim 1, characterized in that: The state characteristic parameters include the sight point and gesture of the driver and passenger, and obtaining the state characteristic parameters of the driver and passenger according to the second image includes: Acquire a first key feature and a second key feature of the driver and passenger based on the second image, wherein the first key feature includes at least a hand posture, and the second key feature includes at least an eye posture, a head posture, and a body posture; The gesture action of the driver and passenger is determined based on the first key feature, and the sight point of the driver and passenger is determined based on the second key feature.

3. The door opening risk warning method according to claim 2, characterized in that: The state parameters include the distance between the moving object and the vehicle, the speed of the moving object, and the actual size of the moving object. Controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the state parameters of the moving object and the state characteristic parameters of the driver and passenger includes: Determining size information of a preset first area and size information of a preset second area according to the actual size of the moving object, wherein the first area is an area within the warning range, and the second area is an area within the alarm range; Determining whether the moving object is located in the first area or the second area based on the interval distance to obtain a first determination result, and determining whether the traveling speed is greater than a preset speed threshold to obtain a second determination result; determining whether the sight point is located in a preset safe area or a preset dangerous area in the vehicle, to obtain a third determination result, wherein the safe area and the dangerous area are formed by dividing the coverage range of the driver's sight line in the vehicle; According to the gesture action, the first judgment result, the second judgment result, and / or the third judgment result, the vehicle is controlled to issue a preset reminder message and / or perform a preset protective action.

4. The door opening risk warning method according to claim 3, characterized in that: The controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result includes: If the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue the reminder message a first preset number of times.

5. The door opening risk warning method according to claim 4, characterized in that: The controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result further includes: If the first judgment result is that the vehicle is located in the first area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue the reminder message a second preset number of times, wherein the second preset number of times is greater than the first preset number of times.

6. The door opening risk warning method according to claim 5, characterized in that: The controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result further includes: If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, and the gesture action is unfastening the seat belt, the vehicle is controlled to issue the reminder message for the second preset number of times; If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the sight point is located in the safe area, and the gesture action is opening the door, the vehicle is controlled to issue the reminder message the second preset number of times; If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is greater than the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the car door, the vehicle is controlled to perform the protective action, wherein the protective action includes locking the car door.

7. The door opening risk warning method according to claim 5, characterized in that: The controlling the vehicle to issue a preset reminder message and / or perform a preset protective action based on the gesture action, the first judgment result, the second judgment result, and / or the third judgment result further includes: If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, and the gesture action is unfastening the seat belt, the vehicle is controlled to issue the reminder message the first preset number of times; If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the sight point is located in the safe area, and the gesture action is opening the door, the vehicle is controlled to issue the reminder message the first preset number of times; If the first judgment result is that the vehicle is located in the second area, the second judgment result is that the driving speed is less than or equal to the speed threshold, the third judgment result is that the line of sight is located in the dangerous area, and the gesture action is to open the door, the vehicle is controlled to issue the reminder message for the second preset number of times.

8. A door opening risk warning device, characterized in that: Applied to a vehicle, the device comprises: an acquisition module, configured to acquire, in real time, a first image of a preset external area and a second image of a preset internal area of the vehicle when the vehicle enters a parking state; a first acquisition module, configured to identify a moving object in the first image and acquire a state parameter of the moving object according to the first image; a second acquisition module, configured to identify a driver or passenger in the second image and acquire state characteristic parameters of the driver or passenger based on the second image; The early warning module is used to control the vehicle to issue a preset warning message and / or perform a preset protective action according to the state parameters of the moving object and the state characteristic parameters of the driver and passenger.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the door opening risk warning method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the door opening risk warning method according to any one of claims 1 to 7 is implemented.

Images