Driving risk prediction method, device and system, vehicle and medium

Abstract

The embodiment of the invention discloses a driving risk prediction method, device and system, a vehicle and a medium. A dynamic target of the vehicle in the driving process is determined; determining a first position of the dynamic target in the vehicle; determining that the dynamic target corresponds to a first static target in the vehicle based on the first position and the structure of the vehicle; and predicting the driving risk of the dynamic target based on the interaction action between the dynamic target and the first static target. According to the embodiment of the invention, based on the interaction action between the dynamic targets such as the driver and the passenger in the vehicle and the first static target, whether the dynamic targets such as the driver and the passenger have the safety risk can be judged, so that corresponding treatment measures can be taken in time to avoid the risk when the safety risk is found; therefore, the safety guarantee of drivers and passengers is improved.

Description

Technical Field

[0001] The present disclosure relates to safe driving technology, three-dimensional reconstruction technology and computer-assisted driving technology, and in particular to a driving risk prediction method, device, system, vehicle and medium. Background Art

[0002] During vehicle driving, some dangerous actions or improper operations of the driver and passengers may cause safety risks. For example, the driver's hands are off the steering wheel, and the accelerator is mistaken for the brake. These actions may cause safety risks. The current cockpit monitoring system has weak monitoring and recognition capabilities for these actions or operations, and cannot effectively identify dangerous actions or improper operations of the driver and passengers, which poses certain safety risks. Summary of the invention

[0003] The present disclosure provides a driving risk prediction method, device, system, vehicle and medium to effectively identify dangerous actions of dynamic targets in a vehicle.

[0004] A first aspect of the present disclosure provides a method for predicting driving risk during vehicle driving, comprising:

[0005] Determine the dynamic goals of the vehicle during driving;

[0006] determining a first position of the dynamic object in the vehicle;

[0007] determining, based on the first position and the configuration of the vehicle, that the dynamic object corresponds to a first static object in the vehicle;

[0008] Based on the interactive action between the dynamic object and the first static object, the driving risk of the dynamic object is predicted.

[0009] A second aspect of the present disclosure provides a driving risk prediction device during vehicle driving, comprising:

[0010] A first determination module is used to determine the dynamic target of the vehicle during driving;

[0011] A second determination module, used to determine a first position of the dynamic target in the vehicle;

[0012] a third determination module, configured to determine, based on the first position and the structure of the vehicle, that the dynamic object corresponds to a first static object in the vehicle;

[0013] The prediction module is used to predict the driving risk of the dynamic target based on the interaction between the dynamic target and the first static target.

[0014] A third aspect of the present disclosure provides a driving risk prediction system during vehicle driving, comprising:

[0015] A first sensor deployed on the vehicle, for collecting data on a dynamic target in the vehicle;

[0016] The driving risk prediction device during vehicle driving described in the second aspect above is used to determine the dynamic target of the vehicle during driving based on the data collected by the first sensor for the dynamic target; determine the first position of the dynamic target in the vehicle; based on the first position and the structure of the vehicle, determine that the dynamic target corresponds to the first static target in the vehicle; and predict the driving risk of the dynamic target based on the interaction between the dynamic target and the first static target.

[0017] A fourth aspect of the present disclosure provides a vehicle, the vehicle comprising: a processor, and a memory communicatively connected to the processor, and further comprising the driving risk prediction device during vehicle driving as described in the second aspect above;

[0018] A memory, configured to store instructions executable by the processor;

[0019] A processor is used to read executable instructions from a memory and execute the instructions to control a driving risk prediction device during vehicle driving to implement any driving risk prediction method during vehicle driving as described in the first aspect above.

[0020] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any one of the methods of the first aspect.

[0021] The sixth aspect of the present disclosure proposes a computer program product, which, when an instruction processor in the computer program product executes, executes the driving risk prediction method during vehicle driving proposed in the embodiment of the first aspect of the present disclosure.

[0022] Based on the embodiments of the present disclosure, dynamic targets are first detected during vehicle driving. For example, dynamic targets may include people or objects that may move in positions such as drivers and passengers. The dynamic targets during vehicle driving and their first positions in the vehicle are determined. Then, based on the first position and the structure of the vehicle, it is determined that the dynamic target corresponds to a first static target in the vehicle. The interaction between the dynamic target and the first static target is detected, and it is identified whether the interaction is a dangerous action. On this basis, it can be determined whether dynamic targets such as drivers and passengers pose safety risks, so that when safety risks are found, corresponding disposal measures can be taken in time to avoid risks, thereby improving the safety of drivers and passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1It is a flowchart of a method for predicting driving risks during vehicle driving provided by an exemplary embodiment of the present disclosure.

[0024] Figure 2 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0025] Figure 3 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0026] Figure 4 It is a schematic diagram of a three-dimensional reconstruction process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0027] Figure 5 It is a schematic diagram of a three-dimensional reconstruction process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0028] Figure 6 It is a schematic diagram of a three-dimensional reconstruction process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0029] Figure 7 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0030] Figure 8 It is a schematic diagram of a three-dimensional reconstruction process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0031] Fig. 9 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0032] Fig.10 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0033] Fig.11 It is a risk warning flow chart of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0034] Fig.12 It is a flowchart of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0035] Fig.13 It is a schematic diagram of the system architecture of a method for predicting driving risks during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0036] Fig.14 It is a schematic diagram of a vehicle accident cause analysis process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0037] Fig.15 A schematic diagram of the structure of a driving risk prediction device during vehicle driving provided by an exemplary embodiment of the present disclosure.

[0038] Fig.16 A schematic structural diagram of a driving risk prediction device during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0039] Fig.17 A schematic structural diagram of a driving risk prediction device during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0040] Fig.18 A schematic structural diagram of a driving risk prediction device during vehicle driving provided by another exemplary embodiment of the present disclosure.

[0041] Fig.19 is a structural diagram of a vehicle provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0043] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0044] Application Overview

[0045] The current cockpit safety monitoring systems include DMS (Driver Monitor System), OMS (Occupancy Monitoring System), RMS (Rear Monitoring System), etc. These systems have weak monitoring and identification capabilities for dangerous actions or improper operations, and cannot accurately reflect the real safety risks of drivers and passengers. For example, in the DMS system, taking the behavior of leaving the steering wheel with both hands as an example, the monitoring party only relies on the pressure sensor set on the steering wheel to determine whether the driver's hands are off the steering wheel. The driver can easily bypass this check by hanging heavy objects on the steering wheel.

[0046] Exemplary Methods

[0047] Figure 1 FIG. 1 is a flow chart of a method for predicting driving risk during vehicle driving provided by an exemplary embodiment of the present disclosure. In addition to vehicles, this embodiment can be applied to mobile devices such as flying equipment (e.g., airplanes, aircraft), ships, etc. Figure 1 As shown, this embodiment includes the following steps:

[0048] Step S101, determining the dynamic target of the vehicle during driving.

[0049] In order to ensure the driving safety of the vehicle during driving, the target objects in the vehicle cabin can be monitored. The target objects can include static targets and dynamic targets. Static targets include the hardware system of the vehicle itself, such as seats, steering wheels, accelerator pedals, brake pedals, etc. in the cabin. Dynamic targets can include people or objects that may move in addition to static targets, such as drivers, passengers, and various personal belongings. During the driving process of the vehicle, the video information in the vehicle cabin can be obtained in real time through the synchronous camera system in the cabin, and the moving targets in the video that move can be identified and located through the moving target detection algorithm, and the detected moving targets can be determined as dynamic targets of the vehicle during driving. For example, the driver and passengers may make various movements of their hands, feet or limbs during the driving process of the vehicle. By performing moving target detection on the video information in the vehicle cabin, the driver and passengers can be determined as dynamic targets of the vehicle during driving.

[0050] Step S102, determining a first position of the dynamic target in the vehicle.

[0051] Through the synchronous camera system in the cockpit, after pre-calibration, the three-dimensional position coordinates of dynamic targets and static targets can be obtained, and the first position of the dynamic target in the vehicle can be determined based on the three-dimensional position coordinates of the dynamic target.

[0052] Step S103: determining, based on the first position and the structure of the vehicle, that the dynamic target corresponds to a first static target in the vehicle.

[0053] The structure of the vehicle may include layout information of various components of the hardware system of the vehicle, and may specifically include the appearance, position and connection relationship of various static objects in the vehicle. In step S103, based on the first position and the structure of the vehicle, a static object that is in contact with or close to the dynamic object may be searched, and the static object may be determined as the dynamic object corresponding to the first static object in the vehicle.

[0054] In one embodiment, the dynamic target in the driving risk prediction system as the execution subject of the driving risk prediction method disclosed herein may be a passenger in the cabin. By comparing the coordinates of the first position of the passenger determined in step S102 with the position coordinates of each static target in the cabin, it can be seen that the passenger is in direct contact with the seat numbered "05A", and the seat numbered "05A" is determined as the first static target corresponding to the dynamic target.

[0055] In another embodiment, the dynamic target in the driving risk prediction system may be a driver. By comparing the coordinates of the first position of the driver determined in step S102 with the position coordinates of each static target in the vehicle cabin, it can be seen that the driver is in direct contact with the steering wheel and the main driver's seat, and the driver's feet are relatively close to the accelerator pedal and the brake pedal, and the steering wheel, the main driver's seat, the accelerator pedal and the brake pedal are determined as the first static target corresponding to the dynamic target.

[0056] Step S104: predicting the driving risk of the dynamic target based on the interaction between the dynamic target and the first static target.

[0057] Among them, the interactive actions between the dynamic target and the first static target may include dangerous actions that are not allowed. For example, when the first static target is a seat, its corresponding interactive action may include the dynamic target leaving the seat and standing up; when the first static target is a steering wheel, its corresponding interactive action may include the action of the driver of the dynamic target taking his hands off the steering wheel. In addition, the interactive actions between the dynamic target and the first static target may also include possible misoperation behaviors. For example, when the first static target is an accelerator pedal and a brake pedal, their corresponding interactive actions include the actions of stepping on the accelerator and the brake, respectively.

[0058] Specifically, the interaction action between the dynamic target and the first static target can be determined based on the interaction information between the two, thereby predicting the driving risk of the dynamic target. The interaction information between the dynamic target and the first static target can include the positional relationship and interaction relationship between the two, such as whether the dynamic target is in direct contact with the first static target, the distance between the dynamic target and the first static target, and the interaction force between the dynamic target and the first static target. The interaction force between the dynamic target and the first static target can be detected by a force sensor.

[0059] In one embodiment, based on interaction information such as whether the driver's hands are in direct contact with the steering wheel, the distance between the driver's hands and the steering wheel, and the interaction force between the driver's hands and the steering wheel, it can be determined whether the driver has an interaction action of taking his hands off the steering wheel, and then predict whether the above-mentioned action of the driver may cause safety risks.

[0060] In another embodiment, based on interaction information such as whether the passenger's buttocks are in direct contact with the seat, the distance between the passenger's buttocks and the seat, and the interaction force between the passenger's buttocks and the seat, it can be determined whether the passenger has an interactive action of standing up from the seat, and then it can be predicted whether the above-mentioned action of the passenger may cause safety risks.

[0061] In summary, based on the embodiments of the present disclosure, dynamic targets are first detected during vehicle driving. For example, dynamic targets may include people or objects that may move, such as drivers and passengers. The dynamic targets during vehicle driving and their first positions in the vehicle are determined. Then, based on the first position and the structure of the vehicle, it is determined that the dynamic target corresponds to the first static target in the vehicle, and the interaction between the dynamic target and the first static target is detected, and it is identified whether the interaction is a dangerous action. On this basis, it can be determined whether there are safety risks for dynamic targets such as drivers and passengers, so that when safety risks are found, corresponding disposal measures can be taken in time to avoid risks, thereby improving the safety of drivers and passengers.

[0062] like Figure 2 As shown in the above Figure 1 Based on the illustrated embodiment, step S102 may include:

[0063] Step S201 , performing three-dimensional reconstruction based on data collected by the first sensor for the dynamic target, to obtain a three-dimensional reconstruction result corresponding to the dynamic target.

[0064] Among them, the first sensor may include an image sensor. Multiple image sensors arranged at different positions in the vehicle cabin can collect image data in real time. After pre-calibration, accurate 3D (three-dimensional) modeling results of dynamic targets can be obtained based on the collected image data. When system resources can meet the requirements, real-time three-dimensional reconstruction of dynamic targets can be performed. In this step, three-dimensional reconstruction can be performed based on the data collected by at least two first sensors for the dynamic target to obtain a three-dimensional reconstruction result corresponding to the dynamic target.

[0065] Step S202: Generate a digital twin system corresponding to the vehicle based on the modeling information of the vehicle and the three-dimensional reconstruction results corresponding to the dynamic target. The digital twin system uses physical models and data collected by sensors to simulate and simulate physical entities, processes or systems in an information platform, thereby reflecting the entire life cycle of the corresponding physical equipment.

[0066] First, a three-dimensional reconstruction result of a static target is constructed based on the modeling information of the vehicle. The static target may include the 3D modeling information of the vehicle itself, which may be provided by the automobile manufacturer or obtained through laser scanning, etc., wherein static targets of the same model may reuse the same 3D modeling information. For example, laser sensors may be set corresponding to multiple preset positions in the cockpit, and the 3D modeling information of the static target may be obtained by using the laser sensor to perform three-dimensional reconstruction of the static target. After the position of the static target is fixed, the static target may be reconstructed in three dimensions at least once. The three-dimensional position coordinates of the static target may be obtained based on the three-dimensional reconstruction result.

[0067] The static targets and dynamic targets together constitute the digital twin system corresponding to the vehicle, and the digital twin system corresponding to the vehicle can be generated based on the three-dimensional reconstruction results of the static targets and dynamic targets. In the disclosed embodiment, the digital twin system is a virtual representation of the vehicle to reflect the behavior, characteristics and performance of the dynamic targets and static targets in the vehicle.

[0068] Step S203: determine the first position of the dynamic target in the digital twin system based on the three-dimensional reconstruction result.

[0069] According to the three-dimensional reconstruction results of the dynamic target in the digital twin system of the vehicle, the three-dimensional position coordinates of the dynamic target can be obtained. The three-dimensional position coordinates can represent the first position of the dynamic target in the digital twin system, that is, the first position of the dynamic target in the vehicle.

[0070] In the related art, the target objects in the cockpit are monitored and identified based on 2D images. This method makes it difficult to obtain accurate depth information and high-precision results. Depth information is crucial for understanding and representing three-dimensional scenes. Depth information can be used to perceive the three-dimensional structure and spatial relationship of objects. In the disclosed embodiment, the digital twin system of the vehicle is constructed based on the three-dimensional reconstruction results containing depth information. The digital twin system based on the vehicle can accurately obtain the position coordinates of dynamic and static targets in the vehicle, so that the actions of dynamic targets can be accurately identified.

[0071] In one embodiment, Figure 3 As shown in the above Figure 2 Based on the illustrated embodiment, the above step S201 may include the following steps:

[0072] Step S1001, inputting at least one first image captured by a first sensor for a dynamic target into a neural network model, and outputting modeling parameters of the dynamic target through the neural network model;

[0073] Step S1002, inputting modeling parameters into the three-dimensional reconstruction model, and outputting the three-dimensional reconstruction result of the dynamic target through the three-dimensional reconstruction model.

[0074] The modeling parameters include shape parameters, posture parameters and camera parameters. The shape parameters and posture parameters are used to represent the shape characteristics and posture characteristics of the dynamic target. The camera parameters can include the image's intrinsic parameters (such as focal length, principal point coordinates) and extrinsic parameters (such as camera position and orientation).

[0075] In step S1001, based on at least one first image of a dynamic target acquired by a first sensor, the image is processed by a neural network model to output modeling parameters of the dynamic target. In step S1002, based on the modeling parameters, the three-dimensional reconstruction result of the dynamic target is output through a three-dimensional reconstruction model.

[0076] The dynamic targets in the driving risk prediction system may generally include the human body, and the three-dimensional reconstruction model in the above step S1002 may include a human body model. The human body model is a digital simulation model of the human body based on human body parameters, which can be used to accurately describe the morphological and mechanical characteristics of the human body. In this step, human body models such as SMPL (Skinned Multi-Person Linear Model), STAR (Sparse Trained Articulated Human Body Regressor), SMPL-X (SMPL eXpressive), and MANO (Model for Articulated Hands) can be used to reconstruct the human body in three dimensions.

[0077] In one example, a single-camera based 3D human body reconstruction online inference method can be used to perform human body reconstruction. Figure 4 , input a single human image into the deep neural network model, and use the deep neural network model to predict the human body's morphological parameters, posture parameters, and camera parameters. Then input the human body's morphological parameters, posture parameters, and camera parameters into the human body model, and use the human body model to complete the human body surface level prediction. The distance from the static target to the human body surface can be predicted, and the three-dimensional reconstruction result of the human body can be obtained. Based on the high-precision prediction results output by the above human body model, the actions of dynamic targets can be accurately judged in subsequent steps, thereby improving the safety of drivers and passengers.

[0078] In another example, a multi-camera based 3D human body reconstruction online inference method can be used to perform human body reconstruction. Figure 5 , using multiple images collected by multiple cameras as input information for the model, which is essentially similar to Figure 4 The monocular input method shown in the figure is different from the Figure 5The process adds a unified alignment step for multi-view representation, so that the features of multiple viewpoints can be unified in the same space. In addition, multi-view input is not limited to RGB images, and images that combine RGB and depth images can be used. Compared with the monocular method, the multi-view method can give a more stable and accurate reconstruction result based on multiple images collected by multiple cameras.

[0079] Figure 5 The multi-view fusion method used in the process adds a fusion step of unified multi-view representation in the entire algorithm process. This method is called the mid-fusion method. The multiple steps in the mid-fusion method are an overall set of processes. In addition to the mid-fusion method, the pre-fusion or post-fusion method can also be used. The pre-fusion method is to perform pre-processing first, perform a perspective fusion separately, unify the images of the three perspectives together, and then directly input the fused image into the network. Post-fusion is to obtain a processing result for the image of each perspective, and then fuse multiple processing results.

[0080] See also Figure 6 In another example, the human body model can also be combined with NERF (Neural Radiance Field) technology to perform three-dimensional reconstruction of the human body. Among them, the radiation field can describe the color and brightness information of light in three-dimensional space. The human body model reconstructs the three-dimensional human body contour and defines dense three-dimensional points on the human body surface. The neural radiation field uses the dense three-dimensional points on the human body surface defined by the human body model to map the three-dimensional points to the radiation field to obtain the information of the clothes on the human body surface. In other words, the neural radiation field is used to add the information of human clothes to the surface of the human body contour to obtain the surface reconstruction result of clothing, so as to achieve more refined behavior discrimination and improve the accuracy of the three-dimensional reconstruction results.

[0081] In one embodiment, Figure 7 As shown in the above Figure 2 Based on the illustrated embodiment, the above step S201 may include the following steps:

[0082] Step S1004, triangulating the two-dimensional key points to obtain the three-dimensional key points of the dynamic target;

[0083] Step S1005 , fitting the three-dimensional key points using the three-dimensional reconstruction model to obtain a three-dimensional reconstruction result of the dynamic target.

[0084] In scenarios where real-time reasoning is not required, such as static reconstruction of accident scenes, multiple cameras can be used for offline fitting to perform three-dimensional reconstruction of dynamic targets to achieve a more fitting and higher-precision effect.

[0085] The dynamic target in the driving risk prediction system may generally include a human body. The first sensor may include multiple cameras disposed at different positions in the vehicle cabin. For the same dynamic target, the multiple cameras may obtain multiple images of the dynamic target from multiple different positions. Figure 7 and Figure 8 In step S1003, human posture estimation is performed on images obtained by cameras set at multiple different orientations to obtain 2D key points at the same position under multiple viewing angles. The triangulation processing in step S1004 is to process the two-dimensional key points using triangulation technology. Based on the above multiple 2D key points, the 2D key points are converted into 3D key points using triangulation technology to obtain the 3D coordinates of the position. In an example, based on the 2D key points of a certain position in the left shoulder image of the human body in the front, left, and right directions, the absolute 3D key point coordinates of the position can be obtained.

[0086] In step S1005, the fitting process uses the coordinates of the 3D key points as the true value and target of the training to train the human body model to make it fit the 3D key points better. The human body model defines a series of dense points on the surface of the human body and also defines some driving key points. For example, a key point on the left shoulder can drive multiple dense points within the left shoulder to move. If the key point on the left shoulder moves, it will drive a series of dense points nearby to move as well. During the fitting process, the key points in the human body model are matched with the 3D key points obtained in step S1004.

[0087] and Figures 4 to 6 Compared with the online process, Figure 8 In the offline process shown, the human body model itself has modeling parameters. During the model training process, the body parameters, posture parameters and camera parameters are changed so that the key points obtained by the human body model are closer and closer to the key points obtained by triangulation. During the model training process, the modeling parameters are modified to make the two more matched. During fitting, the parameters of the human body model are modified according to the information of the 3D key points until the appropriate parameter values ​​are found, and then the surface is reconstructed based on these parameters to obtain the human body surface reconstruction result.

[0088] The difference between the online process and the offline process is that in the online process, the body parameters, posture parameters, and camera parameters are obtained by the neural network, and the modeling parameters do not need to be changed or trained when they are used; in the offline process, the modeling parameters are obtained by training, and the modeling parameters are changed by fitting methods to make them match as much as possible. The training process is a cyclic process that requires continuous fitting and iteration. This process is relatively time-consuming and is suitable for offline scenarios that do not require real-time reasoning. In contrast, based on the appropriate parameter values ​​obtained by iteration, the offline fitting results are more accurate and the reconstructed image effect is more fitting.

[0089] In one embodiment, Fig. 9 As shown in the above Figure 1 Based on the illustrated embodiment, the above step S104 may include the following steps:

[0090] Step S1041, obtaining a positional relationship between the dynamic target and the first static target.

[0091] Specifically, first, according to the three-dimensional reconstruction result of the dynamic target in the digital twin system of the vehicle, the first position of the dynamic target in the digital twin system can be determined. Then, the position of the first static target corresponding to the dynamic target is obtained in the digital twin system of the vehicle. Finally, the first position of the dynamic target is compared with the position of the first static target to obtain the positional relationship between the dynamic target and the first static target. For example, when the dynamic target is a passenger and the first static target corresponding to the dynamic target is determined to be a window, the positional relationship between the dynamic target and the first static target is obtained by comparing the position of the passenger's hand with the window. Specifically, it can be determined whether all or part of the passenger's hand is outside the window.

[0092] Step S1042: determining an interactive action between the dynamic object and the first static object based on the position relationship.

[0093] In the above example, based on the positional relationship of all or part of the passenger's hand outside the window, the interaction action between the dynamic target and the first static target can be determined, and specifically, the interaction action can be determined as the passenger's hand extending outside the window.

[0094] Step S1043: Based on the interactive action, predict the driving risk corresponding to the first static target appearing in the dynamic target.

[0095] In the above example, based on the interactive action of the passenger extending his hand out of the car window, it can be predicted that the above action of the passenger may cause safety risks.

[0096] In another example, the interaction between the dynamic target and the first static target can be determined based on whether the positional relationship between the dynamic target and the first static target meets the preset condition. The preset condition may include that the distance between the dynamic target and the first static target is not within a reasonable preset range. If the condition is met, it can be determined that the dynamic target has an interaction with the first static target. The corresponding preset conditions can be set according to different first static targets. For example, in a vehicle driving risk prediction system, when the first static target is a car window, the preset conditions are set to set the position of all or part of the key points on the dynamic target to be outside the car window. In step S102, it is determined that the coordinates of the first position of the passenger are compared with the position coordinates of each static target in the cabin, and it can be known that the position of the passenger is relatively close to the car window, and the car window is determined as the first static target corresponding to the dynamic target in step S103. In step S104, if the position of all or part of the key points of the passenger's hand is monitored to be outside the car window, it is determined that the passenger's hand is extended out of the window, that is, it can be determined that the passenger has an interaction with the first static target, and on this basis, the driving risk that may be caused is predicted.

[0097] In this embodiment, based on the positional relationship between the dynamic target and the first static target, relevant preset conditions of the corresponding positional relationship are reasonably set according to different first static targets. The interaction action between the dynamic target and the first static target can be more accurately identified based on whether the positional relationship meets the preset conditions. On this basis, it can be judged whether there are safety risks for dynamic targets such as drivers and passengers, thereby improving the safety of drivers and passengers.

[0098] like Fig.10 As shown in the above Fig. 9 Based on the illustrated embodiment, the above step S1042 may include the following steps:

[0099] Step S1032, in response to the position relationship satisfying a preset condition, determining that the dynamic target has an interactive action corresponding to the first static target; or

[0100] Step S1033, in response to the position relationship not satisfying a preset condition, obtaining second sensor information on the first static target;

[0101] Step S1034: determining, based on the second sensor information, whether the dynamic target has an interactive action corresponding to the first static target.

[0102] Fig.11 FIG. 1 is a schematic diagram of a risk warning process of a driving risk prediction method during vehicle driving provided by another exemplary embodiment of the present disclosure. Fig.10 and Fig.11, a multi-eye synchronous camera collects images in the cockpit in real time, first identifies the human body from the images collected by the camera, and then reconstructs the various parts of the human body based on the human body image through a high-precision multi-eye 3D human body reconstruction algorithm, and further obtains the first position of the 3D human body in the cockpit 3D system. Among them, the 3D human body reconstruction algorithm includes but is not limited to the SMPL series method, NERF, 3D Gaussian Splatting, 3D human body posture estimation, etc. According to the 3D structural information of the cockpit, the steering wheel close to the position of the human hand can be determined as the first static target. The preset condition of the position relationship corresponding to the steering wheel can be set as the distance between the human hand and the steering wheel is greater than a preset threshold D. Further obtain the interaction information between the human hand and the steering wheel, for example, the position relationship between the human hand and the steering wheel in the 3D space can be determined, and then preliminarily determine whether there is an interactive action in which the human hand is separated from the steering wheel.

[0103] See also Fig.10 and Fig.11 , if the distance between the human hand and the steering wheel is greater than or equal to a certain threshold value D, the above preset conditions are met, and it is judged that the human hand is obviously off the steering wheel, and a dangerous behavior warning is immediately initiated; if the distance between the human hand and the steering wheel is less than the threshold value D, the above preset conditions are not met, and it is judged to be a suspected detachment state. The next step is to read the steering wheel pressure sensor information, that is, to obtain the second sensor information on the first static target. According to the matching of the sensor pressure position and the suspected contact position, the result of whether the human hand is off the steering wheel is corrected. If the sensor pressure position matches the suspected contact position successfully, and the pressure sensor detects a pressure value greater than or equal to the preset threshold value F, it is judged that there is no interactive action of the human hand leaving the steering wheel. If the sensor pressure position does not match the suspected contact position successfully, or the pressure sensor detects a pressure value less than the preset threshold value F, it is judged that there is an interactive action of the human hand leaving the steering wheel. If it is judged that there is an interactive action of the human hand leaving the steering wheel, a warning is also initiated.

[0104] In this embodiment, a second sensor is used to collect interaction information such as pressure between the dynamic target and the first static target, and the positional relationship between the dynamic target and the first static target is combined with the second sensor information on the first static target. For example, the 3D position of the human body is combined with the sensor information on the first static target. A comprehensive judgment is made based on the above information to determine whether the dynamic target exhibits the interactive action corresponding to the first static target, which can further ensure the accuracy of behavior judgment.

[0105] In one embodiment, in the above Figure 1 On the basis of the embodiment shown, after step S104, the method further includes:

[0106] In response to determining that the dynamic target has taken an impermissible action, an alarm is issued for the impermissible action.

[0107] Among them, the unacceptable actions may include passengers sticking their heads or hands out of the window, the driver's hands off the steering wheel, and other dangerous actions that are not allowed to occur. Fig.11 When a dynamic target makes an unauthorized action, an alarm will be issued for the unauthorized action, prompting relevant personnel to take appropriate measures in time to avoid risks, thereby improving the safety of drivers and passengers.

[0108] In one embodiment, if Fig.12 As shown in the above Figure 1 On the basis of the embodiment shown, after step S104, the method further includes:

[0109] Step S105, in response to the interaction between the dynamic target and the first static target, first relevant information of the interaction is stored for analyzing whether the interaction is an erroneous operation, the first relevant information including: external environment information of the vehicle, motion information of the vehicle, structure of the vehicle, and three-dimensional reconstruction result corresponding to the dynamic target.

[0110] Among them, misoperation behavior and disallowed actions are two different interactive actions. For example, sticking your head out of the car window is a disallowed action. The allowed action of stepping on the accelerator can be divided into two cases. In the first case, stepping on the accelerator may be a correct operation; in the second case, mistaking the accelerator for the brake when it should be the brake is a misoperation behavior.

[0111] The external environment information of the vehicle may include the road condition information of the vehicle, such as whether there is an obstacle target in front of the vehicle, the distance between the obstacle target and the vehicle, etc. The motion information of the vehicle may include the speed, position, acceleration, etc. of the vehicle. The motion information and external environment information may be obtained through multiple sensors arranged inside and outside the vehicle.

[0112] If the dynamic target interacts with the first static target, for example, the driver steps on the accelerator, the first relevant information of the interaction can be stored. The first relevant information of the action of stepping on the accelerator may include external environmental information such as the vehicle's speed, position, acceleration, external environmental information such as road conditions, the vehicle's structure, and the driver's three-dimensional reconstruction results. In the subsequent scenario where the cause of the accident needs to be analyzed, based on the first relevant information stored above, it can be analyzed whether the action of stepping on the accelerator is an erroneous operation.

[0113] In this embodiment, based on the pre-stored first relevant information of the interactive action, if an accident occurs, the accident process can be restored by reading the stored data, and then it can be determined whether the interactive action is an erroneous operation. Based on the data storage, the cause of the accident can be accurately identified, thereby improving the accuracy and efficiency of accident handling.

[0114] like Fig.13 As shown, in one example, the system architecture of the vehicle's driving risk prediction system may include a cockpit digital twin system, a central computing system, a storage system, and a cockpit replay system. The central computing system acquires 3D information of static and dynamic targets in the cockpit by collecting information from the first sensor, and constructs a cockpit digital twin system. According to the 3D position of the dynamic target in the cockpit system and the interaction information with the first static target, it is determined whether there are any unallowed actions with safety risks during driving, and whether there are any driver errors. The various components in the system architecture are described in detail as follows:

[0115] 1) Cockpit digital twin system: Static targets and dynamic targets together constitute the cockpit digital twin system. Through the cockpit digital twin system, 3D interaction information between dynamic targets such as passengers or drivers and the cockpit can be obtained, and then it can be determined whether there are dangerous behaviors that cause safety risks.

[0116] 2) Central computing system: This system can be the AI ​​(artificial intelligence) computing center of the vehicle. The system reconstructs the cabin system by calling the sensors of the entire vehicle to build a digital twin system of the cabin. Among them, the sensors of the entire vehicle include DMS / OMS cameras, lidar, RGBD (red, green, blue depth) cameras, ToF (Time of Flight, time of flight technology), pressure sensors, etc.

[0117] 3) Storage system: This system receives vehicle driving information such as vehicle speed, sensor information, and information from the reconstructed cockpit digital twin system of the central computing system, and saves it separately. The data in the system can be read by users to analyze the cause of the accident after the accident occurs, and its function is similar to the "black box" on the aircraft. The storage system can be set in the safest position of the vehicle body and protected by high temperature and anti-collision materials. The internal data is set to be readable but cannot be manually modified by the user to avoid malicious tampering or accidental loss.

[0118] 4) Cockpit replay system: This system is a software system that is also applicable to vehicles such as airplanes. It does not need to be directly installed on the moving vehicle. For example, the system can be set up on a remote server, similar to a ground monitoring center. The cockpit replay system can directly call and read the data in the storage system, and reconstruct all the information in the cockpit at high frequency to reconstruct the driving process or the process of the accident. After an accident occurs, the cause of the accident can be analyzed through this system.

[0119] In one embodiment, in the above Fig.12 Based on the embodiment shown, the above method further includes:

[0120] In response to receiving an acquisition request from a user, acquiring second relevant information corresponding to an acquisition condition in the acquisition request from the first relevant information;

[0121] According to the second relevant information, it is determined whether there is any erroneous operation behavior.

[0122] In the above step S105, the first relevant information of the interactive action is pre-stored. After the accident occurs, the user can request to obtain the first relevant information of the interactive action generated during the accident period. In response to receiving the user's acquisition request, the driving risk prediction system obtains the second relevant information corresponding to the acquisition condition in the acquisition request from the first relevant information, that is, obtains the first relevant information generated during the accident period, and then determines whether there is an erroneous operation behavior of the dynamic target based on the information.

[0123] For example, if the second relevant information shows that there is an obstacle target in front of the current vehicle and the distance between the obstacle target and the vehicle is very close, and the current vehicle is moving very fast, then the correct operation in this case should be to step on the brake. If it is detected that the driver steps on the accelerator at this time, it can be determined that the driver's stepping on the accelerator is an erroneous operation.

[0124] In summary, based on the first relevant information of the interactive action pre-stored in the storage system, if an accident occurs, the driving risk prediction system can restore the accident process by reading the stored data, and determine whether the interactive action is an erroneous operation by analyzing the stored data. This method can improve the accuracy and efficiency of accident handling.

[0125] In one embodiment, the above process of restoring accidents and determining behaviors can be performed in a driving risk prediction system installed in a vehicle; in another embodiment, the above process of restoring accidents and determining behaviors can also be performed in a cockpit replay system installed on a remote server.

[0126] Fig.14 This is a schematic diagram of a vehicle accident cause analysis process of a driving risk prediction method provided by another exemplary embodiment of the present disclosure. In an example of a traffic accident, the current vehicle collides with an obstacle vehicle in front, and it is necessary to determine whether the driver has stepped on the brakes to determine the cause of the accident. Fig.14The dashed box on the left shows the real vehicle monitoring process, which can be used to monitor the behavior of dynamic targets in real time while the vehicle is driving. First, the human body data is collected by synchronous cameras, and high-precision 3D human body reconstruction results are obtained based on the human body data. Then, based on the human body reconstruction results, the 3D structure information of the cockpit, the pressure sensor information of the brake pedal, the external obstacle information and vehicle motion information obtained by the assisted driving system, it is judged whether the user has stepped on the brakes. Among them, the vehicle radar or camera can be used to obtain the external obstacle information; the sensor can be used to detect the vehicle speed and other vehicle motion information.

[0127] Specifically, see Fig.14 First, the positional relationship between the driver's foot and the brake pedal can be determined based on the human body reconstruction result and the brake position of the brake pedal in the cockpit 3D structure information. If the positional relationship meets the preset conditions, for example, the distance between the driver's foot and the brake pedal is less than or equal to a certain threshold, it can be determined that the driver has stepped on the brake. If the positional relationship does not meet the preset conditions, it can only be concluded that the driver's foot is suspected to be in contact with the brake position. Furthermore, the pressure sensor information of the brake position can be obtained, and combined with the pressure sensor information to determine whether the driver has stepped on the brake. When it is detected that the brake position is under greater pressure, it is determined that the driver has stepped on the brake, otherwise it is determined that the driver has not stepped on the brake.

[0128] After the interactive action of the dynamic target is detected in the real vehicle monitoring process, the first relevant information of the interactive action is saved in the storage system. The storage system can be set in a memory outside the ground. The first relevant information includes: external obstacle information, vehicle motion information, 3D human body reconstruction results, cockpit 3D structure and recorded braking behavior, etc.

[0129] Fig.14 The dashed box on the right side of the figure shows the process of the cockpit replay system. After an accident occurs, the cockpit replay system reads the historical data of the vehicle before the accident from the storage system and reconstructs the accident process. In one way, the reconstruction process can adopt the same process as the real vehicle monitoring process, that is, similar to Figures 4 to 6 The online inference method shown in Figure 2; in another approach, the reconstruction process can also be done using Figure 8 In comparison, the offline fitting method can obtain a reconstruction result with higher accuracy.

[0130] In the above traffic accident example, the current vehicle collides with the obstacle vehicle in front, and it is necessary to determine whether the driver has braked, and on this basis determine the cause of the accident. For example, the cockpit replay system reconstructs the accident process and determines that the driver has braked based on the reconstruction results, but from the obtained vehicle motion information, it is determined that the vehicle has not slowed down. Based on this, it can be determined that the accident may be caused by a brake device failure, and the driver's braking behavior is a correct operation.

[0131] In summary, with the help of the above-mentioned reconstruction method in the cockpit replay system, the historical data before the vehicle accident is read from the storage system, and the accident process is reconstructed. Based on the reconstruction results, it is possible to accurately determine whether there was any user misoperation at the time of the accident, thereby avoiding accident disputes and improving the fairness, accuracy and efficiency of accident handling.

[0132] Exemplary Devices

[0133] like Fig.15 As shown, the present disclosure also provides a corresponding embodiment of a driving risk prediction device during vehicle driving. In the embodiment of the driving risk prediction device during vehicle driving, the device includes:

[0134] A first determination module 100 is used to determine a dynamic target of the vehicle during driving;

[0135] A second determination module 200, configured to determine a first position of the dynamic target in the vehicle;

[0136] A third determination module 300, for determining, based on the first position and the structure of the vehicle, that the dynamic object corresponds to a first static object in the vehicle;

[0137] The prediction module 400 is used to predict the driving risk of the dynamic object based on the interaction between the dynamic object and the first static object.

[0138] like Fig.16 As shown, in one implementation, the second determining module 200 may include:

[0139] The reconstruction submodule 210 is used to perform three-dimensional reconstruction based on the data collected by the first sensor for the dynamic target, and obtain a three-dimensional reconstruction result corresponding to the dynamic target;

[0140] A generating submodule 220 is used to generate a digital twin system corresponding to the vehicle based on the modeling information of the vehicle and the three-dimensional reconstruction result corresponding to the dynamic target;

[0141] The determination submodule 230 is used to determine the first position of the dynamic target in the digital twin system based on the three-dimensional reconstruction result.

[0142] In one embodiment,

[0143] The reconstruction submodule 210 is used to:

[0144] Inputting at least one first image captured by the first sensor for the dynamic target into a neural network model, and outputting modeling parameters of the dynamic target through the neural network model;

[0145] The modeling parameters are input into the three-dimensional reconstruction model, and the three-dimensional reconstruction result of the dynamic target is output through the three-dimensional reconstruction model.

[0146] In one embodiment,

[0147] The reconstruction submodule 210 is used to:

[0148] Extracting two-dimensional key points from a first image captured by a first sensor for the dynamic target;

[0149] Triangulate the two-dimensional key points to obtain the three-dimensional key points of the dynamic target;

[0150] The 3D reconstruction model is used to fit the 3D key points to obtain the 3D reconstruction result of the dynamic target.

[0151] In one embodiment,

[0152] The prediction module 400 is used to:

[0153] Acquire a positional relationship between the dynamic target and the first static target;

[0154] Based on the position relationship, determining an interactive action between the dynamic target and the first static target;

[0155] Based on the interactive action, the driving risk corresponding to the first static target is predicted to appear in the dynamic target.

[0156] like Fig.17 As shown, in one embodiment, the above device may further include an alarm module 600, which is used to:

[0157] After predicting the driving risk of the dynamic target, in response to determining that the dynamic target exhibits an impermissible behavior, an alarm is issued for the impermissible behavior.

[0158] like Fig.18 As shown, in one embodiment, the above device may further include a storage module 700, which is used to:

[0159] After predicting the driving risk of the dynamic target, in response to the interaction between the dynamic target and the first static target, first relevant information of the interaction is stored for analyzing whether the interaction is an erroneous operation. The first relevant information includes: external environment information of the vehicle, motion information of the vehicle, structure of the vehicle, and three-dimensional reconstruction results corresponding to the dynamic target.

[0160] like Fig.18 As shown, in one embodiment, the above device may further include an analysis module 800, which is used to:

[0161] In response to receiving an acquisition request from a user, acquiring second relevant information corresponding to an acquisition condition in the acquisition request from the first relevant information;

[0162] According to the second relevant information, it is determined whether there is any erroneous operation behavior.

[0163] The beneficial technical effects corresponding to the exemplary embodiment of the present device can be found in the corresponding beneficial technical effects of the above exemplary method section, which will not be repeated here.

[0164] Exemplary Systems

[0165] The present disclosure also provides a corresponding embodiment of a driving risk prediction system during vehicle driving. In the embodiment of the driving risk prediction system during vehicle driving, the system includes:

[0166] A first sensor deployed on the vehicle, for collecting data on a dynamic target in the vehicle;

[0167] The above-mentioned driving risk prediction device during vehicle driving is used to determine the dynamic target of the vehicle during driving based on the data collected by the first sensor for the dynamic target; determine the first position of the dynamic target in the vehicle; based on the first position and the structure of the vehicle, determine that the dynamic target corresponds to the first static target in the vehicle; based on the interaction between the dynamic target and the first static target, predict the driving risk of the dynamic target.

[0168] In one embodiment, the vehicle includes a plurality of first static targets and at least one second sensor deployed on each of the first static targets, and the at least one second sensor is used to obtain second sensor information corresponding to the first static target.

[0169] In one embodiment, the driving risk prediction device during vehicle driving is further used to: perform three-dimensional reconstruction based on data collected by the first sensor for the dynamic target to obtain a three-dimensional reconstruction result corresponding to the dynamic target; generate a digital twin system corresponding to the vehicle based on the modeling information of the vehicle and the three-dimensional reconstruction result corresponding to the dynamic target;

[0170] The above-mentioned driving risk prediction system during vehicle driving may also include: a storage unit for storing data of the digital twin system.

[0171] In one embodiment, the driving risk prediction system during vehicle driving may further include:

[0172] At least one third sensor deployed on the vehicle, used to collect external environment information of the vehicle;

[0173] at least one fourth sensor deployed on the vehicle, for collecting movement information of the vehicle;

[0174] The driving risk prediction device during vehicle driving is also used to respond to the interaction between the dynamic target and the first static target, and store first relevant information of the interaction action in a storage unit for analyzing whether the interaction action is an erroneous operation behavior. The first relevant information includes: external environment information of the vehicle, motion information of the vehicle, structure of the vehicle, and three-dimensional reconstruction results corresponding to the dynamic target.

[0175] In one embodiment, the driving risk prediction device during vehicle driving is further used to:

[0176] In response to receiving an acquisition request from a user, second related information corresponding to an acquisition condition in the acquisition request is acquired from a storage unit; and whether there is an erroneous operation behavior is determined based on the second related information.

[0177] The exemplary embodiments of the driving risk prediction method during vehicle driving, the driving risk prediction device during vehicle driving, and the driving risk prediction system during vehicle driving of the present disclosure correspond to each other, and can be mutually referenced and quoted in the corresponding implementation methods, and no further description is given. The central computing system of the driving risk prediction system during vehicle driving provided by the exemplary embodiment of the present disclosure may include the driving risk prediction device during vehicle driving provided by the exemplary embodiment of the present disclosure; or, the above-mentioned central computing system is configured as the driving risk prediction device during vehicle driving provided by the exemplary embodiment of the present disclosure, which is used to implement the driving risk prediction method during vehicle driving provided by the exemplary embodiment of the present disclosure.

[0178] The beneficial technical effects corresponding to the exemplary embodiments of the driving risk prediction method during vehicle driving, the driving risk prediction device during vehicle driving, and the driving risk prediction system during vehicle driving disclosed in the present invention may also be referenced to each other and will not be repeated here.

[0179] Example Vehicles

[0180] The present disclosure also provides a corresponding embodiment of a vehicle.

[0181] In an embodiment of the vehicle, the vehicle includes a processor, and a memory in communication with the processor, and also includes the above-mentioned driving risk prediction device during vehicle driving;

[0182] The memory is used to store processor executable instructions;

[0183] The processor is used to read executable instructions from the memory and execute the instructions to control the driving risk prediction device during vehicle driving to implement the above-mentioned driving risk prediction method during vehicle driving.

[0184] Fig.19 A structural diagram of a vehicle provided for an embodiment of the present disclosure includes at least one processor 11 and a memory 12.

[0185] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the vehicle 10 to perform desired functions.

[0186] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run one or more computer program instructions to implement the driving risk prediction method and / or other desired functions of the various embodiments of the present disclosure during vehicle driving.

[0187] In one example, the vehicle 10 may further include an input device 13 and an output device 14 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0188] The input device 13 may also include, for example, a keyboard, a mouse, etc.

[0189] The output device 14 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0190] Of course, to simplify, Fig.19 Only some of the components related to the present disclosure in the vehicle 10 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, the vehicle 10 may further include any other appropriate components according to specific application scenarios.

[0191] Exemplary computer program products and computer-readable storage media

[0192] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the driving risk prediction method during vehicle driving of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0193] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0194] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the driving risk prediction method during vehicle driving of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0195] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0196] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0197] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for predicting driving risk during vehicle driving, comprising: Determine the dynamic goals of the vehicle during driving; determining a first position of the dynamic object in the vehicle; determining, based on the first position and the structure of the vehicle, that the dynamic object corresponds to a first static object in the vehicle; Based on the interaction between the dynamic object and the first static object, a driving risk of the dynamic object is predicted.

2. The method according to claim 1, wherein determining the first position of the dynamic target in the vehicle comprises: Performing three-dimensional reconstruction based on the data collected by the first sensor for the dynamic target to obtain a three-dimensional reconstruction result corresponding to the dynamic target; Based on the modeling information of the vehicle and the three-dimensional reconstruction result corresponding to the dynamic target, generating a digital twin system corresponding to the vehicle; The first position of the dynamic target in the digital twin system is determined based on the three-dimensional reconstruction result.

3. The method according to claim 2, wherein: The performing three-dimensional reconstruction based on the data collected by the first sensor for the dynamic target to obtain a three-dimensional reconstruction result corresponding to the dynamic target includes: Inputting at least one first image collected by the first sensor for the dynamic target into a neural network model, and outputting modeling parameters of the dynamic target through the neural network model; The modeling parameters are input into a three-dimensional reconstruction model, and a three-dimensional reconstruction result of the dynamic target is output via the three-dimensional reconstruction model.

4. The method according to claim 2, wherein: The performing three-dimensional reconstruction based on the data collected by the first sensor for the dynamic target to obtain a three-dimensional reconstruction result corresponding to the dynamic target includes: Extracting two-dimensional key points from a first image captured by the first sensor for the dynamic target; Triangulating the two-dimensional key points to obtain three-dimensional key points of the dynamic target; The three-dimensional key points are fitted using a three-dimensional reconstruction model to obtain a three-dimensional reconstruction result of the dynamic target.

5. The method according to any one of claims 1 to 4, wherein the step of predicting the driving risk of the dynamic object based on the interaction between the dynamic object and the first static object comprises: Acquire a positional relationship between the dynamic target and the first static target; Based on the position relationship, determining an interaction action between the dynamic object and the first static object; Based on the interactive action, predict that the dynamic target has a driving risk corresponding to the first static target.

6. The method according to any one of claims 1 to 5, wherein: After predicting the driving risk of the dynamic target, the method further includes: In response to determining that the dynamic target has taken an impermissible action, an alarm is issued for the impermissible action.

7. The method according to any one of claims 1 to 6, wherein: After predicting the driving risk of the dynamic target, the method further includes: In response to an interaction between a dynamic target and the first static target, first relevant information of the interaction is stored for analyzing whether the interaction is an erroneous operation. The first relevant information includes: external environment information of the vehicle, motion information of the vehicle, structure of the vehicle, and a three-dimensional reconstruction result corresponding to the dynamic target.

8. The method according to claim 7, further comprising: In response to receiving an acquisition request from a user, acquiring second relevant information corresponding to an acquisition condition in the acquisition request from the first relevant information; Determine whether there is any erroneous operation behavior based on the second relevant information.

9. A driving risk prediction device during vehicle driving, comprising: A first determination module is used to determine the dynamic target of the vehicle during driving; A second determination module, configured to determine a first position of the dynamic target in the vehicle; a third determination module, configured to determine, based on the first position and the structure of the vehicle, that the dynamic target corresponds to a first static target in the vehicle; A prediction module is used to predict the driving risk of the dynamic object based on the interaction between the dynamic object and the first static object.

10. A driving risk prediction system during vehicle driving, comprising: A first sensor deployed on the vehicle, for collecting data on a dynamic target in the vehicle; The driving risk prediction device during vehicle driving as claimed in claim 9, which is used to determine the dynamic target of the vehicle during driving based on the data collected by the first sensor for the dynamic target; determining a first position of the dynamic object in the vehicle; and determining, based on the first position and a structure of the vehicle, that the dynamic object corresponds to a first static object in the vehicle; Based on the interaction between the dynamic object and the first static object, a driving risk of the dynamic object is predicted.

11. A vehicle, comprising: A processor, and a memory in communication with the processor, further comprising the driving risk prediction device during vehicle driving according to claim 9; The memory is used to store instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to control the driving risk prediction device during vehicle driving to implement the driving risk prediction method during vehicle driving as described in any one of claims 1-8.

12. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the driving risk prediction method during vehicle driving as described in any one of claims 1 to 8.

Images