Blind area prompting method and device, equipment and storage medium

Abstract

The invention discloses a blind area prompting method and device, equipment and a storage medium, and belongs to the field of intelligent driving. The safety problem when a driver or passengers load and unload articles in the trunk can be solved. The method comprises the steps that under the condition that a driver opens a trunk, the action intention of the driver is determined based on the action made by the driver and the weight of an object in the trunk; according to the action intention of the driver, the staying time of the driver in the trunk is predicted, and the staying time of the driver is obtained; when the distance between the surrounding vehicle and the vehicle is smaller than a preset distance and the difference value between the residence time of the driver and the residence time is smaller than a preset time difference, corresponding reminding measures are executed according to the distance between the surrounding vehicle and the vehicle and the speed of the surrounding vehicle.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent driving, and particularly relates to a blind area prompting method and device, equipment and a storage medium. BACKGROUND

[0002] The driver or passenger may stop by the roadside at any time during driving, load or unload objects in the trunk, and in this process, when a vehicle comes from the rear or side of the vehicle, the driver or passenger cannot effectively observe the oncoming vehicles in all directions due to the attention on loading or unloading objects, which may endanger the safety of the personnel. In the prior art, the blind area prompting is generally performed when the driver or passenger opens the door.

[0003] In the prior art, the blind area prompting does not cover the scenario of loading or unloading objects in the trunk, and cannot effectively ensure the safety of the personnel when loading or unloading objects in the trunk. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a blind area prompting method, device, equipment and storage medium, which can solve the safety problem of the driver or passenger when loading or unloading objects in the trunk.

[0005] In order to solve the above technical problems, the present application is implemented as follows: In a first aspect, the embodiments of the present application provide a blind area prompting method, which comprises: In the case that the driver opens the trunk, determining the action intention of the driver based on the action made by the driver and the weight of the object in the trunk; According to the action intention of the driver, predicting the time for the driver to stay at the trunk to obtain the stay time of the driver; In the case that the distance between the surrounding vehicle and the vehicle is less than a preset distance, and the difference between the stay time of the driver and the stay time is less than a preset time difference, performing a corresponding prompting measure according to the distance between the surrounding vehicle and the vehicle and the speed of the surrounding vehicle.

[0006] Optionally, before determining the action intention of the driver based on the action made by the driver and the weight of the object in the trunk, the method further comprises: obtaining an image of the driver through a camera; constructing a joint node model of the driver according to the image; determining the action made by the driver according to the joint node model.

[0007] Optionally, the method further comprises: monitoring the weight and the change of the center of gravity of the object in the trunk through a gravity sensor installed on the object loading plane of the trunk.

[0008] Optionally, determining the driver's action based on the keypoint model includes: The angle of the driver's waist-knee line is determined by the location of the skeletal joints, thereby determining the driver's body bending movements; The driver's hand movements are determined by the position of the hand joints.

[0009] Optionally, determining the driver's intention based on the driver's actions and the weight of the objects in the trunk includes: If the driver's bending angle is greater than a preset degree, the time the driver's hand joint is below the knee joint is greater than a preset time, and the weight of the object in the trunk increases, the driver's intention is determined to be a loading intention. If the driver's bending angle is greater than a preset degree, the time the driver's hand joint is below the knee joint is less than a preset time, and the weight of the object in the trunk decreases, then the driver's intention is determined to be the intention to retrieve the object. If the driver's bending angle is less than a preset degree and the distance between the driver's hand joint and the waist joint is less than a preset distance, the driver's intention is determined to be a stopping intention.

[0010] Optionally, predicting the time the driver will stay in the trunk based on the driver's intention to stay, and obtaining the driver's dwell time, includes: Determine the historical dwell time corresponding to the action intention, wherein the historical dwell time is the dwell time of the driver each time the action corresponding to the action intention is performed; The average of the historical stay times is taken as the dwell time.

[0011] Optionally, the step of implementing corresponding reminder measures based on the distance between the vehicle and the surrounding vehicles and the speed of the surrounding vehicles includes: If the distance between the vehicle and the surrounding vehicles is greater than a first preset distance and the speed of the surrounding vehicles is less than a first preset speed, a first-level warning is issued, which includes illuminating a warning light. If the distance between the surrounding vehicles and the vehicle is less than a first preset distance, and the speed of the surrounding vehicles is greater than a second preset speed, a secondary alert is executed. The secondary alert includes illuminating an alert light and providing a voice prompt. If the distance between the surrounding vehicles and the vehicle is less than a second preset distance, and the speed of the surrounding vehicles is greater than a third preset speed, a level two alert will be issued.

[0012] Secondly, embodiments of this application provide a blind spot warning device, the device comprising: The action intent recognition module is used to determine the driver's action intent based on the actions performed by the driver and the weight of the objects in the trunk when the driver opens the trunk. The dwell time determination module is used to predict the time the driver will stay in the trunk based on the driver's intention to move, and to obtain the driver's dwell time. The blind spot warning module is used to implement corresponding warning measures based on the distance between the surrounding vehicles and the vehicle and the speed of the surrounding vehicles when the distance between the surrounding vehicles and the vehicle is less than a preset distance, and the difference between the driver's dwell time and the dwell time is less than a preset time difference.

[0013] Optionally, the device further includes: The image acquisition module is used to acquire images of the driver via a camera; A joint point model building module is used to construct a joint point model of the driver based on the image. The action recognition module is used to determine the actions performed by the driver based on the joint model.

[0014] Optionally, the device further includes: The object weight sensing module is used to monitor the weight of the object in the trunk and the change of the center of gravity through a gravity sensor installed on the loading plane of the trunk.

[0015] Optionally, the action recognition module includes: The first motion recognition submodule is used to determine the angle of the driver's waist-knee line by the position of the skeletal joints, so as to determine the driver's body bending motion; The second motion recognition submodule is used to determine the driver's hand motion by the position of the hand joints.

[0016] Optionally, the action intent recognition module includes: The first intention recognition submodule is used to determine the driver's action intention as a loading intention when the driver's bending angle is greater than a preset degree, the time when the driver's hand joint is lower than the knee joint is greater than a preset time, and the weight of the object in the trunk increases. The second intention recognition submodule is used to determine that the driver's action intention is to retrieve an object when the driver's bending angle is greater than a preset degree, the time when the driver's hand joint is lower than the knee joint is less than a preset time, and the weight of the object in the trunk decreases. The third intention recognition submodule is used to determine that the driver's intention is to stay when the driver's bending angle is less than a preset degree and the distance between the driver's hand joint and the waist joint is less than a preset distance.

[0017] Optionally, the dwell time determination module includes: The dwell time prediction submodule is used to determine the historical dwell time corresponding to the action intention. The historical dwell time is the dwell time of the driver each time the action corresponding to the action intention is performed, which is recorded in advance. The dwell time determination submodule is used to take the average of the historical dwell times as the dwell time.

[0018] Optionally, the blind spot warning module includes: The first reminder submodule is used to execute a first-level reminder when the distance between the surrounding vehicles and the vehicle is greater than a first preset distance and the speed of the surrounding vehicles is less than a first preset speed. The first-level reminder includes illuminating a reminder light. The second reminder submodule is used to perform a secondary reminder when the distance between the vehicle and the surrounding vehicles is less than a first preset distance and the speed of the surrounding vehicles is greater than a second preset speed. The secondary reminder includes illuminating a reminder light and providing a voice prompt. The third reminder submodule is used to execute a secondary reminder when the distance between the surrounding vehicles and the vehicle is less than a second preset distance and the speed of the surrounding vehicles is greater than a third preset speed.

[0019] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0020] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0021] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0022] In the blind spot warning method provided in this application, when the driver opens the trunk, the driver's intention is determined based on the driver's actions and the weight of the objects in the trunk; based on the driver's intention, the time the driver stays in the trunk is predicted to obtain the driver's dwell time; if the distance between the surrounding vehicles and the vehicle is less than a preset distance, and the difference between the driver's dwell time and the dwell time is less than a preset time difference, corresponding reminder measures are executed based on the distance between the surrounding vehicles and the vehicle and the speed of the surrounding vehicles.

[0023] In this method, the driver's intention is determined by the driver's actions and the weight of the objects in the trunk. The driver's dwell time in the trunk is then calculated based on the driver's intention. If the driver's dwell time reaches a critical value when vehicles approach from the vicinity, the driver is alerted based on the distance and speed of the surrounding vehicles. This prevents the driver from being scratched or hit by surrounding vehicles when getting up and walking away after loading or unloading items, thus ensuring the driver's safety when loading and unloading items in the trunk. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of a blind spot warning scenario proposed in an embodiment of this application; Figure 2 This is a schematic diagram of a blind spot warning control module according to an embodiment of this application; Figure 3 This is a flowchart of a blind spot warning method proposed in an embodiment of this application; Figure 4 This is a monitoring logic diagram proposed in one embodiment of this application; Figure 5 This is a schematic diagram of a reminder process proposed in an embodiment of this application; Figure 6 This is a schematic diagram of a blind spot warning device according to an embodiment of this application; Figure 7 This is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0027] refer to Figure 1 , Figure 1 This is a schematic diagram of a blind spot warning scenario proposed in an embodiment of this application, such as... Figure 1 As shown in the picture, the driver and passengers are loading and unloading items in the trunk. At this time, a vehicle is approaching from the surrounding area. The vehicle sends a voice reminder through the in-vehicle speaker, asking the vehicle to proceed only after confirming that it is safe to do so. At the same time, the taillights light up to indicate the presence of the vehicle.

[0028] refer to Figure 2 , Figure 2 This is a schematic diagram of a blind spot warning control module according to an embodiment of this application, as shown below. Figure 2 As shown, the VIU (Vehicle Infotainment Unit) collects the tailgate's opening and closing status and reports it to the MDC (Mobile Data Center), then sends voice prompts to the LDM (Light Driver Module). The MDC, based on intelligent driving hardware such as cameras and radar, detects whether vehicles and pedestrians in the blind spot are at an unsafe distance when the vehicle is in Park and the tailgate is open. If they are, it sends a warning request to the VIU and CDC. The CDC (Central Control Unit) provides voice prompts upon receiving the warning request. The LDM illuminates the corresponding indicator light upon receiving a warning light request.

[0029] refer to Figure 3 , Figure 3 This is a flowchart of a blind spot warning method proposed in an embodiment of this application, as follows: Figure 3 As shown, this method is applied to the vehicle's infotainment system, and specifically includes the following steps: S11: When the driver opens the trunk, the driver's intention is determined based on the driver's action and the weight of the object in the trunk.

[0030] In this embodiment, the driver's action intent is identified based on the actions performed by the driver. Action intent includes loading intent, retrieving intent, and stopping intent, etc.

[0031] In this embodiment, the vehicle-mounted sensing system is used to collect the motion characteristics of people in the trunk. The vehicle-mounted sensing system includes a rear bumper radar, a camera on the top of the tailgate, an ultrasonic radar sensor array at the trunk sill, and a pressure-sensitive floor at the cargo surface of the trunk.

[0032] In this embodiment, when the driver opens the trunk, the camera in the vehicle perception system captures images of the driver. Based on the captured images, 18 2D joints (such as shoulders, elbows, wrists, knees, etc.) of the driver's body are detected. These joints are represented by coordinates (x, y) on the image plane. The depth information corresponding to the joints is estimated from a single image, thereby expanding the 2D coordinates into 3D coordinates (x, y, z). The 3D joints are connected to form a skeletal system (such as using cylinders or spheres to represent bones). The skeleton is bound to the driver model (such as a virtual human). The model's movements are driven in real time through the joint data. Radar assists in detecting the driver's position and other information. Gravity sensors on the pressure-sensitive floor detect the weight of objects in the trunk and the movement of the center of gravity. By modeling the driver, the actions performed by the driver are determined. Combined with the increase or decrease in the weight of objects in the trunk, it is identified whether the driver is loading or unloading items, or simply standing still. At the same time, combined with the movement of the center of gravity of objects in the trunk, it is identified whether the objects are moving.

[0033] For example, if a driver needs to retrieve items from the trunk while driving, and pulls over to the side of the road to open the trunk, the vehicle uses radar to detect the driver approaching the trunk and simultaneously uses a camera to capture an image of the driver in the trunk, thus judging the driver's actions and identifying the driver's intentions.

[0034] S12: Based on the driver's intention, predict the time the driver will stay in the trunk, and obtain the driver's dwell time.

[0035] In this embodiment, the driver's dwell time is the time the driver spends in the trunk.

[0036] In this embodiment, after recognizing the driver's intention, the driver's dwell time in the trunk is predicted based on the time the driver has spent when making the same intention in the past.

[0037] For example, if the driver's intention is to load items, the driver's current time spent in the trunk can be predicted based on the average time the driver has spent in the trunk when performing loading actions in the past.

[0038] S13: If the distance between the surrounding vehicles and the vehicle is less than a preset distance, and the difference between the driver's dwell time and the dwell time is less than a preset time difference, execute the corresponding reminder measures based on the distance between the surrounding vehicles and the vehicle and the speed of the surrounding vehicles.

[0039] In this embodiment, the preset distance is the safe distance that vehicles should maintain. When the distance between vehicles is less than the preset distance, people outside the vehicle are at risk of scraping against surrounding vehicles.

[0040] In this embodiment, surrounding vehicles refer to vehicles passing in front of or behind this vehicle in the same lane or within a certain distance when this vehicle is parked on the roadside.

[0041] In this embodiment, when the distance between the driver and surrounding vehicles is less than a preset distance, it indicates that the surrounding vehicles are relatively close. If the difference between the driver's dwell time and the dwell time is less than a preset time difference, it means that the driver's dwell time is about to reach a critical value, and the driver is very likely to get up and return to the vehicle. At this point, the probability of the driver coming into contact with surrounding vehicles is high. Based on the distance between the driver and surrounding vehicles and the speed of the surrounding vehicles, corresponding reminder measures are implemented.

[0042] For example, if the predicted driver's dwell time is 120 seconds, and the driver has been stationary for 100 seconds, the vehicle radar detects oncoming vehicles at relatively high speeds and then initiates blind spot warning.

[0043] In this embodiment, when the driver opens the trunk to load or unload items, the system identifies the driver's intention and predicts the driver's dwell time. Then, it combines the distance and speed of surrounding vehicles to provide the driver with corresponding blind spot warnings, ensuring the driver's safety when loading or unloading items from the trunk.

[0044] In another embodiment of this application, before determining the driver's intention based on the driver's actions and the weight of the objects in the trunk, the method further includes: S21: Obtain an image of the driver using a camera.

[0045] In this embodiment, the driver's image is captured by a camera on the tailgate when the driver opens the trunk.

[0046] S22: Based on the image, construct the driver's joint model.

[0047] In this embodiment, the joint model is based on multi-view geometry principles, reconstructing the joint coordinates of the human body in 3D space using 2D key point information collected by multiple synchronous cameras. In this embodiment, after acquiring the driver's image, a joint model of the driver is constructed based on the driver's image. The driver's image is then input into a 3D model building program, which is a pre-designed program specifically for generating models from images.

[0048] For example, OpenPose 3D is used to build a model containing 18 joints of the driver in real time.

[0049] S23: Determine the action performed by the driver based on the joint model.

[0050] In this embodiment, after obtaining the joint point model, the driver's actions are determined by real-time analysis of the joint point model based on the positions of important joint points such as the driver's waist, knees, and hands, as well as the time each joint point spends at each position.

[0051] For example, the angle of bending / leaning forward can be obtained by calculating the waist-knee line angle through 3D skeletal joints, and the hand joints can be obtained by determining whether they are continuously lower than the knee height through the Z-axis position of the model, thereby determining whether something is being picked up or organized.

[0052] In this embodiment, by establishing a key point model of the driver to identify the driver's actions, the driver's actions can be analyzed accurately.

[0053] In another embodiment of this application, the method further includes: S31: The weight of the objects in the trunk and the change of the center of gravity are monitored by a gravity sensor installed on the loading plane of the trunk.

[0054] In this embodiment, when the driver opens the trunk, the vehicle-mounted sensing system uses gravity sensors installed on the cargo plane of the trunk to monitor the weight of the objects in the trunk and the changes in the center of gravity in real time. This determines whether the weight of the objects in the trunk increases, decreases, or simply changes in the center of gravity without a change in weight. The system then analyzes the changes in the weight of the objects in the trunk.

[0055] For example, an 8×8 grid (64 contacts) of distributed piezoelectric thin-film sensors is deployed on the trunk floor to monitor changes in pressure distribution. Combined with time series analysis and spatial motion modeling, the continuous movement of the object's weight and center of gravity is determined.

[0056] gross weight ,in Indicates the total weight. This represents the i-th contact point.

[0057] Calculation of centroid coordinates: (1), (2) in, The x-coordinate of the centroid The vertical coordinate is the centroid. This represents the x-coordinate of the i-th contact point. This represents the ordinate of the i-th contact point.

[0058] In this embodiment, the weight changes of objects in the trunk are obtained by gravity sensors on the cargo plane of the trunk, and then the driver's actions are effectively analyzed by combining the key point model.

[0059] In another embodiment of this application, determining the driver's action based on the keypoint model includes: S41: Determine the angle of the driver's waist-knee line by the position of the skeletal joints, so as to determine the driver's body bending motion.

[0060] In this embodiment, when analyzing the joint model, the angle of the driver's waist-knee line is determined by the position of the skeletal joints, and the driver's bending / forward angle is determined based on the angle of the driver's waist-knee line, thereby determining the driver's body bending action.

[0061] For example, obtain the coordinates of the lumbar joint: W(wx, wy, wz), obtain the coordinates of the knee joint: K(kx, ky, kz), and construct the lumbar-knee vector. .

[0062] When projecting along the anterior-posterior direction of the human body (sagittal projection), the X-axis component is discarded, while the Y-axis and Z-axis are retained to obtain the sagittal projection vector. Calculate the angle between the projection vector and the perpendicular vector.

[0063] Wherein, vertical reference vector (0,1,0). The vector is the projection of the waist-knee vector onto the sagittal plane.

[0064] S42: Determine the driver's hand movements by the position of the hand joints.

[0065] In this embodiment, the position coordinates of the hand joints are obtained, and the driver's hand movements are determined based on the changing trend of the hand joint coordinates. If the hand coordinates move continuously in the horizontal direction, it may be that the driver is organizing items; if they move vertically, it may be that the driver is loading or unloading items.

[0066] In this embodiment, a depth map is also obtained through a ToF depth camera (below the rear windshield), and an RGB image is provided by a binocular vision system (top of the tailgate).

[0067] For example, 2D joints are first obtained from the image, and then the 2D joints are mapped to 3D coordinates to extract depth values ​​and transform the coordinate system.

[0068] In this embodiment, for each frame of the image, the Y coordinate of the key point of the current frame is extracted: ,

[0069] in, The Y-coordinate of the key hand point The Y-coordinate of the knee key point.

[0070] The height difference between the key hand points and the knee is: (6) Among them, This represents the average percentage change in the height difference between the hands and knees, where n is the sample size. Let be the hand height of the i-th sample.

[0071] In this embodiment, the driver's actions are calculated based on the position of each joint point to ensure accurate identification of the driver's actions.

[0072] In another embodiment of this application, determining the driver's intention based on the driver's actions and the weight of objects in the trunk includes: S51: If the driver's bending angle is greater than a preset degree, the time the driver's hand joint is lower than the knee joint is greater than a preset time, and the weight of the object in the trunk increases, the driver's intention is determined to be a loading intention.

[0073] In this embodiment, the driver's different intentions are determined based on multiple combinations of actions. If the driver's bending angle is greater than a preset degree and the time the driver's hand joint is below the knee joint is greater than a preset time, it indicates that the driver is bending over to load or unload something. If an increase in the weight of the object in the trunk is detected at this time, the driver's intention is determined to be loading.

[0074] For example, the bending angle / torso leaning forward is 30~60° (depending on the trunk depth, height and object placement of different models), the key points of the hands are continuously below the knee height (the duration is greater than the threshold), and the center of gravity of the objects in the trunk is continuously shifting (if the overall weight remains unchanged, it indicates that the intention is to organize the items; if the weight continuously increases or decreases, it indicates that the items are being loaded or unloaded).

[0075] S52: If the driver's bending angle is greater than a preset degree, the time the driver's hand joint is below the knee joint is less than a preset time, and the weight of the object in the trunk decreases, the driver's intention is determined to be the intention to retrieve the object.

[0076] In this embodiment, if the driver's bending angle is greater than a preset angle and the time when the driver's hand joint is lower than the knee joint is less than a preset time, it indicates that the driver may be taking something out of the trunk. If the weight of the object in the trunk is detected to have decreased at this time, it is determined that the driver's intention is to take something out.

[0077] For example, the action combination for retrieving an object is: bending over at an angle / torso leaning forward 30~60°, the key point of the hand briefly lowering below knee height / single bend / short bend, and a single change in the weight of the object in the trunk. S53: If the driver's bending angle is less than a preset degree and the distance between the driver's hand joint and the waist joint is less than a preset distance, the driver's intention is determined to be a stopping intention.

[0078] In this embodiment, if the driver's bending angle is less than a preset degree and the distance between the driver's hand joint and the waist joint is less than a preset distance, it indicates that the driver may be in a standing state. At this time, the driver's intention is determined to be a stopping intention.

[0079] For example, the combination of pause intention actions is: body upright (torso tilt angle <15°), feet stationary, key hand points at waist height (Z-axis position ±10cm), gaze not focused on the trunk (eye tracking shows looking at a phone / distant object), and recognition of whether the user is facing the trunk or the opposite direction.

[0080] In this embodiment, recognizing the driver's intentions through combinations of driver actions helps predict the time the driver will spend in the trunk.

[0081] In another embodiment of this application, the step of predicting the time the driver will stay in the trunk based on the driver's intention to stay, and obtaining the driver's dwell time, includes: S61: Determine the historical dwell time corresponding to the action intention, wherein the historical dwell time is the dwell time of the driver each time the action corresponding to the action intention is performed, which is recorded in advance.

[0082] In this embodiment, the historical dwell time is the pre-recorded dwell time of the driver under each action corresponding to the driver's intention to perform an action.

[0083] In this embodiment, after determining the driver's intention, the system determines the historical dwell time corresponding to the driver's intention. The system records the time the driver spends at the trunk each time and categorizes the time according to the intention.

[0084] For example, if the driver's intention is to stay, then the system retrieves the historical dwell time corresponding to the driver's intention to stay, as recorded by the system.

[0085] S62: The average of the historical stay times is taken as the stay time.

[0086] In this embodiment, after determining the historical dwell time corresponding to the driver's intention, the average of the historical dwell time is used as the dwell time.

[0087] In this embodiment, the median of historical stay times can also be used as the stay time.

[0088] In this embodiment, the driver's current dwell time is predicted based on the driver's historical dwell time, which helps to effectively remind the driver.

[0089] In another embodiment of this application, the step of executing corresponding reminder measures based on the distance between the surrounding vehicles and the vehicle itself, and the speed of the surrounding vehicles, includes: S71: When the distance between the surrounding vehicles and the vehicle is greater than a first preset distance and the speed of the surrounding vehicles is less than a first preset speed, a first-level reminder is executed, the first-level reminder including illuminating the reminder light.

[0090] In this embodiment, when the distance between the surrounding vehicles and the vehicle is greater than a first preset distance, and the speed of the surrounding vehicles is less than a first preset speed, the surrounding vehicles are farther away and slower, posing less threat to the driver. Therefore, a first-level warning is issued, and the warning measure is to turn on the warning light.

[0091] refer to Figure 4 , Figure 4 This is a monitoring logic diagram proposed in one embodiment of this application, such as... Figure 4 As shown, when the vehicle is stationary, the distance between the vehicle on the side and the vehicle itself is x. The warning logic is that the distance of the vehicle on the side is X≥1m and V≤20km / h. The secondary warning logic is that the distance of the vehicle on the side is (X<1m and V≥3km / h) or (X≤5m and V≥60km / h).

[0092] For example, the first preset distance is 1m and the first preset speed is 20km / h.

[0093] S72: If the distance between the surrounding vehicles and the vehicle is less than a first preset distance and the speed of the surrounding vehicles is greater than a second preset speed, a secondary reminder is executed. The secondary reminder includes illuminating a reminder light and providing a voice prompt.

[0094] In this embodiment, when the distance between the surrounding vehicles and the vehicle is less than a first preset distance, and the speed of the surrounding vehicles is greater than a second preset speed, the vehicles are close together and the surrounding vehicles are moving at high speeds, posing a greater threat to the driver. Therefore, a secondary warning is issued, and the measures for the secondary warning include illuminating a warning light.

[0095] For example, the second preset speed is 3 km / h.

[0096] S73: If the distance between the surrounding vehicles and the vehicle is less than the second preset distance, and the speed of the surrounding vehicles is greater than the third preset speed, execute a level two alert.

[0097] In this embodiment, the third preset speed is greater than the second preset speed.

[0098] In this embodiment, when the distance between the surrounding vehicles and the vehicle is less than the second preset distance and the speed of the surrounding vehicles is greater than the third preset speed, the speed of the surrounding vehicles is relatively fast, and a secondary warning measure is executed.

[0099] For example, the third preset speed is 60 km / h.

[0100] In this embodiment, if the user is facing away from the trunk and is not engaging in any distracting activities such as using a mobile phone, the monitoring and alert for vehicles approaching from behind will not be triggered.

[0101] In this embodiment, graded alerts are provided based on the distance of surrounding vehicles to the vehicle and the speed of the vehicles. This effectively alerts the driver to blind spots while conserving vehicle battery power, ensuring the driver's safety when loading and unloading items from the trunk.

[0102] refer to Figure 5 , Figure 5 This is a schematic diagram of a reminder process proposed in an embodiment of this application, such as... Figure 5 As shown, when the user opens the trunk in Park (P) gear, the trunk loading blind spot monitoring function is activated. The MDC (Mount Control Center) monitors the surrounding environment in real time. If a vehicle approaches from the side beyond a safe distance and the conditions for a secondary alert are met, the system checks if the conditions for a secondary alert are also met. If so, the MDC sends a secondary alert to the VIU (Vehicle Utility Unit) / CDC (Dual Access Control Unit). The VIU controls the LDM (Lane Deployment Management Unit) to illuminate the warning light, and the CDC provides a voice prompt. If not, the MDC sends a primary alert to the VIU, which in turn controls the LDC to illuminate the warning light. After the surrounding vehicles have moved away, the MDC alert request stops, and the function is deactivated.

[0103] In the above embodiments of this application, when the driver opens the trunk, the vehicle-mounted perception system extracts the driver's features, models the driver, analyzes the driver's actions, identifies the driver's intentions, and determines the driver's dwell time in the trunk. At the same time, it detects the distance and speed of surrounding vehicles. When the driver's dwell time behind the vehicle is close to the predicted dwell time, the system provides graded reminders to the driver based on the distance and speed of surrounding vehicles, effectively ensuring the safety of the driver or passengers when loading and unloading items from the trunk.

[0104] refer to Figure 6 , Figure 6 This is a schematic diagram of a blind spot warning device 600 according to an embodiment of this application, as shown below. Figure 6 As shown, the device includes: The action intent recognition module 601 is used to determine the driver's action intent based on the action performed by the driver and the weight of the object in the trunk when the driver opens the trunk. The dwell time determination module 602 is used to predict the time the driver will stay in the trunk based on the driver's intention to move, and to obtain the driver's dwell time. The blind spot warning module 603 is used to perform corresponding warning measures based on the distance between the surrounding vehicles and the vehicle and the speed of the surrounding vehicles when the distance between the surrounding vehicles and the vehicle is less than a preset distance and the difference between the driver's dwell time and the dwell time is less than a preset time difference.

[0105] Optionally, the device further includes: The image acquisition module is used to acquire images of the driver via a camera; A joint point model building module is used to construct a joint point model of the driver based on the image. The action recognition module is used to determine the actions performed by the driver based on the joint model.

[0106] Optionally, the device further includes: The object weight sensing module is used to monitor the weight of the object in the trunk and the change of the center of gravity through a gravity sensor installed on the loading plane of the trunk.

[0107] Optionally, the action recognition module includes: The first motion recognition submodule is used to determine the angle of the driver's waist-knee line by the position of the skeletal joints, so as to determine the driver's body bending motion; The second motion recognition submodule is used to determine the driver's hand motion by the position of the hand joints.

[0108] Optionally, the action intent recognition module includes: The first intention recognition submodule is used to determine the driver's action intention as a loading intention when the driver's bending angle is greater than a preset degree, the time when the driver's hand joint is lower than the knee joint is greater than a preset time, and the weight of the object in the trunk increases. The second intention recognition submodule is used to determine that the driver's action intention is to retrieve an object when the driver's bending angle is greater than a preset degree, the time when the driver's hand joint is lower than the knee joint is less than a preset time, and the weight of the object in the trunk decreases. The third intention recognition submodule is used to determine that the driver's intention is to stay when the driver's bending angle is less than a preset degree and the distance between the driver's hand joint and the waist joint is less than a preset distance.

[0109] Optionally, the dwell time determination module includes: The dwell time prediction submodule is used to determine the historical dwell time corresponding to the action intention. The historical dwell time is the dwell time of the driver each time the action corresponding to the action intention is performed, which is recorded in advance. The dwell time determination submodule is used to take the average of the historical dwell times as the dwell time.

[0110] Optionally, the blind spot warning module includes: The first reminder submodule is used to execute a first-level reminder when the distance between the surrounding vehicles and the vehicle is greater than a first preset distance and the speed of the surrounding vehicles is less than a first preset speed. The first-level reminder includes illuminating a reminder light. The second reminder submodule is used to perform a secondary reminder when the distance between the vehicle and the surrounding vehicles is less than a first preset distance and the speed of the surrounding vehicles is greater than a second preset speed. The secondary reminder includes illuminating a reminder light and providing a voice prompt. The third reminder submodule is used to execute a secondary reminder when the distance between the surrounding vehicles and the vehicle is less than a second preset distance and the speed of the surrounding vehicles is greater than a third preset speed.

[0111] The blind spot warning device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0112] The blind spot warning device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0113] The blind spot warning device provided in this application embodiment can achieve... Figures 1 to 5 The various processes implemented by the blind spot warning device in any of the method embodiments are not described here to avoid repetition.

[0114] Optionally, this application embodiment also provides an electronic device, including a processor 110, a memory 109, and a program or instructions stored in the memory 109 and executable on the processor 110. When the program or instructions are executed by the processor 110, they implement the various processes of the above-described blind spot warning method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0115] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0116] Figure 7 This is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of this application. The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0117] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. 5 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here. This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described blind spot warning method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0118] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0119] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described blind spot warning method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0120] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0121] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0122] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0123] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A blind area prompting method characterized by comprising: The method comprises: In the case that the driver opens the trunk, determining the action intention of the driver based on the action made by the driver and the weight of the objects in the trunk; According to the action intention of the driver, predicting the time for the driver to stay at the trunk, obtaining the stay time of the driver; In the case that the distance between the surrounding vehicle and the vehicle is less than a preset distance, and the difference between the stay time of the driver and the stay time is less than a preset time difference, performing corresponding reminding measures according to the distance between the surrounding vehicle and the vehicle and the speed of the surrounding vehicle.

2. The blind area prompting method according to claim 1, characterized by Before determining the action intention of the driver based on the action made by the driver and the weight of the objects in the trunk, the method further comprises: obtaining the image of the driver through the camera; constructing the joint model of the driver according to the image; determining the action made by the driver according to the joint model.

3. The blind area prompting method according to claim 1, characterized by The method further comprises: monitoring the weight and the change of the center of gravity of the objects in the trunk through the gravity sensor installed on the object-carrying plane of the trunk.

4. The blind area prompting method according to claim 2, characterized by The determination of the action made by the driver according to the joint model comprises: determining the waist-knee connection angle of the driver through the position of the skeletal joint to determine the body bending action of the driver; determining the hand action of the driver through the position of the hand joint.

5. The blind area prompting method of claim 1, wherein The determination of the action intention of the driver based on the action made by the driver and the weight of the objects in the trunk comprises: in the case that the bending angle of the driver is greater than a preset degree, the hand joint of the driver is lower than the knee joint for more than a preset time, and the weight of the objects in the trunk increases, determining that the action intention of the driver is a loading intention; in the case that the bending angle of the driver is greater than a preset degree, the hand joint of the driver is lower than the knee joint for less than a preset time, and the weight of the objects in the trunk decreases, determining that the action intention of the driver is a taking intention; in the case that the bending angle of the driver is less than a preset degree, and the hand joint of the driver is located at the waist joint for less than a preset distance, determining that the action intention of the driver is a stay intention.

6. The blind area prompting method of claim 1, wherein, The prediction of the time for the driver to stay at the trunk according to the action intention of the driver, obtaining the stay time of the driver, comprises: determining the historical stay time corresponding to the action intention, which is the stay time of the driver recorded in advance when making the action corresponding to the action intention each time; taking the average value of the historical stay time as the stay time.

7. The blind area prompting method of claim 1, wherein The execution of the corresponding reminding measures according to the distance between the surrounding vehicle and the vehicle and the speed of the surrounding vehicle comprises: in the case that the distance between the surrounding vehicle and the vehicle is greater than a first preset distance, and the speed of the surrounding vehicle is less than a first preset speed, performing a first-level reminding, which comprises lighting the reminding lamp; In a case where the distance between the surrounding vehicle and the ego vehicle is less than a first preset distance and the speed of the surrounding vehicle is greater than a second preset speed, a second level of prompting is performed, the second level of prompting including lighting a prompting lamp and voice prompting; In a case where the distance between the surrounding vehicle and the ego vehicle is less than a second preset distance and the speed of the surrounding vehicle is greater than a third preset speed, a second level of prompting is performed.

8. A blind spot prompting device characterized by comprising: The device comprises: An action intention recognition module configured to determine an action intention of a driver based on an action made by the driver and a weight of an object in a trunk in a case where the driver opens the trunk; A stay time determination module configured to predict a time for the driver to stay at the trunk based on the action intention of the driver, and obtain a stay time of the driver; A blind area prompting module configured to perform corresponding prompting measures based on a distance between a surrounding vehicle and the ego vehicle and a speed of the surrounding vehicle in a case where the distance between the surrounding vehicle and the ego vehicle is less than a preset distance and a difference between a stay time of the driver and the stay time of the driver is less than a preset time difference.

9. An electronic device, comprising: A processor, a memory, and a program or instructions stored on the memory and executable on the processor, the program or instructions being executed by the processor to implement steps in any of the methods of claims 1-7.

10. A readable storage medium, characterized by, A readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement steps in any of the methods of claims 1-7.

Images