Driver simulation method and device in collision scene, storage medium and terminal

Abstract

The invention discloses a driver simulation method and device in a collision scene, a storage medium and a terminal, and the method comprises the steps: obtaining a to-be-detected image which is a posture image when a driver operates a simulation driver; creating a posture recognition model, wherein the posture recognition model comprises a 2D posture recognition unit, a 3D posture recognition unit and a fine adjustment unit; the 2D posture recognition unit is used for recognizing a 2D posture of a driver, the 3D posture recognition unit is used for converting the 2D posture into a 3D posture, and the fine adjustment unit is used for adjusting parameters of a posture recognition model; recognizing the to-be-detected image based on a posture recognition model to obtain the posture of the driver, and judging whether the driver makes an emergency posture in a collision scene or not; the dummy is controlled to make corresponding actions according to the postures; and if the driver makes the emergency posture in the collision scene, acquiring collision damage information of the dummy. According to the method, the accuracy of human injury evaluation in the vehicle collision test is greatly improved, and help is provided for improvement and optimization of vehicle safety.

Description

Technical Field

[0001] This invention relates to the field of vehicle collision testing technology, and specifically to a method, device, storage medium, and terminal for simulating a driver in a collision scenario. Background Technology

[0002] In the field of vehicle safety, crash testing is an important part of assessing vehicle safety. However, in existing crash tests, the dummies generally only passively absorb the impact force of the collision and fail to realize the driver's active emergency response in real collision scenarios.

[0003] In real-world accidents, drivers instinctively react to collision threats, including emergency braking, emergency steering, and emergency avoidance. These behaviors are influenced by various factors, including the type and severity of the emergency, the driver's experience, psychological qualities and reaction ability, vehicle performance and configuration, as well as road conditions and environmental factors. Driver posture is a crucial component of driver emergency behavior, reflecting the driver's behavioral patterns in emergency situations and influencing the final injury outcome.

[0004] In existing technologies, the actions of dummy figures are usually matched based on a preset posture database or the posture response is predicted through a pre-trained model. However, the actual actions of the driver's posture are ignored, which leads to a deviation between the driver's posture and the driver's instinctive reaction. Ultimately, the collected data on human injury does not match the actual situation, affecting vehicle safety.

[0005] Therefore, how to effectively address the deficiencies and shortcomings of existing technologies has become an urgent technical challenge. Summary of the Invention

[0006] The purpose of this invention is to address the above-mentioned problems by providing a method, apparatus, storage medium, and terminal for driver simulation in collision scenarios.

[0007] The technical solution of this invention is as follows: a driver simulation method in a collision scenario, comprising the following steps: acquiring a target image, wherein the target image is a posture image of a driver operating a driving simulator; creating a posture recognition model, wherein the posture recognition model includes a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit; the 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model; obtaining the driver's posture based on the target image identified by the posture recognition model and determining whether the driver has made an emergency posture in a collision scenario; controlling a dummy to make corresponding actions according to the posture; if the driver makes an emergency posture in a collision scenario, obtaining the collision damage information of the dummy.

[0008] As an improvement of this embodiment of the invention, the "acquiring the image to be detected" specifically includes: continuously acquiring the driver's posture image when the driver operates the driving simulator to obtain the image to be detected.

[0009] As an improvement to this embodiment of the invention, the step of "identifying the driver's posture based on the posture recognition model and determining whether the driver has made an emergency posture in a collision scenario" specifically includes: the 2D posture recognition unit in the posture recognition model processes the image to be detected, identifies the driver's 2D human key points, and constructs a 2D skeleton; based on the 2D skeleton, the 3D posture recognition unit processes the image to convert the 2D posture into a 3D posture, obtains the driver's human key points in 3D space, and constructs a 3D skeleton; the 2D skeleton and the 3D skeleton are quantitatively analyzed to obtain quantitative data reflecting the driver's posture; and the driver is determined to have made an emergency posture in a collision scenario based on the quantitative data.

[0010] As an improvement of this embodiment of the invention, the quantification data includes a number of preset quantification indicators and a number of values ​​corresponding to the preset quantification indicators in a continuous time period.

[0011] As an improvement to this embodiment of the invention, the step of "determining whether the driver has made an emergency posture in a collision scenario based on the quantitative data" specifically includes: comparing the changes in the values ​​corresponding to preset quantitative indicators in the quantitative data over a continuous time period; if the change in at least one indicator is greater than a preset threshold and continues for a preset duration, then the driver has made an emergency posture in a collision scenario.

[0012] As an improvement of this invention, the quantitative index is an index that reflects the spatial position and angular relationship of various parts of the body.

[0013] As an improvement to an embodiment of the present invention, the "controlling the dummy to perform corresponding actions according to the posture" specifically includes: the dummy controller receives the action signal corresponding to the posture, converts the action signal into driving signals for each joint of the dummy, and controls each joint of the dummy to perform actions corresponding to the driver's posture based on the driving signals.

[0014] To achieve one of the aforementioned objectives, one embodiment of the present invention provides a driver simulation device for collision scenarios, comprising the following modules: a data acquisition module for acquiring an image to be detected, wherein the image to be detected is a posture image of a driver operating a driving simulator; a model creation module for creating a posture recognition model, wherein the posture recognition model includes a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit; the 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model; a recognition module for recognizing the driver's posture based on the image to be detected using the posture recognition model and determining whether the driver has adopted an emergency posture in a collision scenario; and a control module for controlling the dummy to perform corresponding actions based on the posture; if the driver adopts an emergency posture in a collision scenario, the collision damage information of the dummy is acquired.

[0015] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a storage medium storing program instructions, which, when executed, implement the driver simulation method in a collision scenario as described in any of the preceding claims.

[0016] To achieve one of the above-mentioned objectives, one embodiment of the present invention provides a terminal, including a processor and a memory, wherein the memory stores program instructions, and the processor executes the program instructions to implement the driver imitation method in a collision scenario as described in any of the preceding claims.

[0017] The driver simulation method, device, storage medium, and terminal provided in this invention have the following advantages: by capturing and reproducing the driver's real emergency actions in a collision scenario in real time, it overcomes the limitations of passive response of dummies or indirect deduction of actions based on preset data in the prior art, and significantly improves the authenticity and accuracy of human injury assessment in collision tests; it provides a more reliable test basis for vehicle safety design optimization and effectively makes up for the deviation between existing tests and actual collision scenarios. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the driver imitation method in a collision scenario as described in this invention. Figure 2 This is a schematic diagram of the posture recognition model described in this invention; Figure 3 This is a posture comparison diagram of the driver imitation method under the collision scenario described in this invention; Figure 4 This is a schematic diagram of the driver simulation device in the collision scenario described in this invention; Figure 5 This is a schematic diagram of the structure of the terminal described in this invention. Detailed Implementation

[0019] The present invention will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of the present invention.

[0020] If the present invention involves orientation (e.g., up, down, left, right, front, back, outside, inside, etc.) when described, then the orientations involved need to be defined.

[0021] The scope of the embodiments described herein includes the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitations, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0022] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientation or positional relationships are based on the orientation or positional relationships shown in the accompanying drawings and are used only for the convenience of describing this document and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements, or direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0023] Embodiment 1 of the present invention provides a method for driver simulation in a collision scenario, such as... Figure 1 As shown, it includes the following steps: Step 101: Obtain the image to be detected, which is the posture image of the driver operating the driving simulator; In practice, the camera is fixed inside the cockpit of the driving simulator, ensuring that its field of view fully covers the driver's full posture and operating actions. When the driver performs simulated driving operations on the simulator, the camera continuously captures images of the driver's posture at a preset frame rate to obtain the image to be detected. The captured posture images include information such as the driver's limb positions, joint angles, and operating gestures. Collision scenarios include frontal collisions, side collisions, or rear-end collisions. It can be understood that by capturing the driver's real emergency actions in collision scenarios in real time, the accuracy of the dummy's response posture is greatly improved.

[0024] Step 102: Create a posture recognition model, which includes a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit; the 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model; Here, the structure of the pose recognition model is as follows: Figure 2 As shown, preferably, the 2D pose recognition unit in this invention is ViTPose. The ViTPose network structure includes a backbone network and a decoder. The backbone network is a pure visual Transformer, which can be a basic visual Transformer, a large visual Transformer, an ultra-large visual Transformer, or a large visual autoencoder. The backbone network segments the input image into multiple image patches and converts each image patch into a vector representation. Multiple Transformer blocks process the image patch vectors. Each Transformer block contains a multi-head self-attention layer and a feedforward network. The multi-head self-attention layer captures global contextual information in the image. Then, a lightweight decoder processes the features extracted by the backbone network. The decoder contains two deconvolutional layers and a prediction layer. The deconvolutional layers restore the low-resolution feature map to the original image resolution through upsampling operations. The prediction layer outputs a keypoint heatmap. Each heatmap corresponds to a keypoint, and the pixel value on the heatmap represents the probability that the pixel belongs to the keypoint, thereby obtaining the 2D human keypoints of the driver and constructing a 2D skeleton. Understandably, ViTPose has a simple structure, is easy to understand and implement, and has better parallelism, which improves inference speed. It can also be easily extended to multiple pose estimation tasks by adding multiple decoders.

[0025] The 3D pose recognition unit described in this invention is MotionAGFormer. It divides the number of channels by using two parallel Transformer and graph convolutional network transformer streams, alternately extracting spatial and temporal features. The graph convolutional network transformer utilizes the local relationships between adjacent joints, outputting a new representation complementary to the Transformer, and adaptively fusing these two representations. The feature extraction module of the MotionAGFormer backbone network is divided into a spatial transformer and a temporal transformer. Each input marker represents a single joint of the human body, and the spatial transformer models the dependencies between various human key points. Each input marker represents a frame of a pose sequence, and the temporal transformer models the dependencies of the 2D skeleton at different times. The spatial graph convolutional network transformer represents the human body topology using an adjacency matrix, while the temporal graph convolutional network transformer uses the K most temporally relevant frames as adjacent nodes in the graph, thereby obtaining the driver's human key points in 3D space and constructing a 3D skeleton. It is understood that MotionAGFormer can better capture dependencies and improve recognition accuracy.

[0026] The fine-tuning unit described in this invention is LoRA, which is a class of efficient model fine-tuning algorithms. It can achieve similar performance to training with all parameters by adjusting a small number of model parameters, thus obtaining a bias that is more consistent with the driver's emergency posture.

[0027] Step 103: Based on the posture recognition model, identify the driver's posture in the image to be detected and determine whether the driver has made an emergency posture in the collision scenario; In practice, the 2D posture recognition unit in the posture recognition model processes the image to be detected, identifies the driver's 2D human key points, and constructs a 2D skeleton. Based on the 2D skeleton, the 3D posture recognition unit processes the image to convert the 2D posture into a 3D posture, obtains the driver's human key points in 3D space, and constructs a 3D skeleton. The 2D skeleton and the 3D skeleton are quantitatively analyzed to obtain quantitative data reflecting the driver's posture. Based on the quantitative data, it is determined whether the driver should adopt an emergency posture in a collision scenario.

[0028] Here, the quantitative data includes several preset quantitative indicators and several values ​​corresponding to these preset quantitative indicators over a continuous time period. The quantitative indicators reflect the spatial position and angular relationship of various body parts, and may include trunk angle, head tilt angle, upper arm tilt angle, forearm tilt angle, elbow joint angle, and arm-trunk angle. The trunk angle is the absolute angle of the driver's spine, reflecting the degree of trunk tilt, including upper trunk angle and lower trunk angle. The head tilt angle is the absolute angle of the driver's head, reflecting the degree of head tilt. The upper arm tilt angle is the angle between the driver's shoulder to elbow and a line perpendicular to the ground, reflecting the posture of the upper arm. The forearm tilt angle is the angle between the driver's elbow to wrist and a line perpendicular to the ground, reflecting the posture of the forearm. The elbow joint angle is the angle between the driver's upper arm and forearm, reflecting the degree of elbow flexion. The arm-trunk angle is the angle between the driver's upper arm and trunk, reflecting the relative position of the upper limb and trunk. The postures corresponding to the preset quantitative indicators include leaning forward, sitting upright, and leaning back, and their corresponding values ​​are shown in Table 1.

[0029] Table 1. Values ​​corresponding to preset quantitative indicators

[0030] In this embodiment, the step of "determining whether the driver has made an emergency posture in a collision scenario based on the quantitative data" specifically includes: comparing the changes in the values ​​corresponding to the preset quantitative indicators in the quantitative data over a continuous time period; if the change in at least one indicator is greater than a preset threshold and continues for a preset duration, then the driver has made an emergency posture in a collision scenario.

[0031] In practice, such as Figure 3 As shown, two frames from the video of the driver in an emergency situation were extracted for analysis. Initially, the driver stood upright and looked straight ahead. When the driver noticed the sudden situation to the right front, the head and body tilted slightly to the right, and both the head tilt angle and neck tilt angle increased. During the turn, the tilt angles of the left forearm and upper arm increased significantly, while the angles of the upper arm and body decreased, and the angle of the right hand remained basically unchanged. This indicates that the driver adopted an emergency posture in a collision scenario. The quantitative analysis values ​​are shown in Table 2.

[0032] Table 2 Figure 3 Medium-quantitative analysis numerical

[0033] Step 104: Control the dummy to make corresponding movements according to the posture; if the driver makes an emergency posture in the collision scenario, obtain the collision damage information of the dummy.

[0034] In practice, the dummy controller receives the motion signals corresponding to the posture and converts them into drive signals for each joint of the dummy. Based on these drive signals, it controls the dummy's joints to perform actions corresponding to the driver's posture. Specifically, the dummy controller parses and converts the received motion signals, breaking them down into specific drive commands for each joint based on the joint distribution of the dummy's mechanical structure. If the driver makes a sudden steering wheel turn, the controller calculates the rotation angles and timing of the corresponding shoulder and elbow joints of the dummy's arms, converting them into pulse or torque signals for the motors. Then, the drive mechanism inside the dummy activates according to the drive signals, controlling each joint to move at preset angles, speeds, and forces, thereby ensuring that the dummy's overall posture matches the driver's movements.

[0035] If the driver adopts an emergency posture during a collision, the dummy's built-in sensors simultaneously collect data on forces, deformations, and accelerations at various points during the collision, as the dummy performs the corresponding action. This data is then processed to generate collision damage information, providing a basis for subsequent vehicle safety assessments. It is understandable that through signal conversion and drive control by the controller, synchronization between the dummy's movements and the driver's posture is achieved, while ensuring real-time acquisition of collision damage information.

[0036] Embodiment 2 of the present invention provides a driver simulation device for collision scenarios, such as Figure 4 As shown, it includes the following modules: Data acquisition module 201 is used to acquire the image to be detected, which is the posture image of the driver operating the driving simulator; The model creation module 202 is used to create a posture recognition model, which includes a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit. The 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model. The recognition module 203 is used to recognize the driver's posture in the image to be detected based on the posture recognition model and to determine whether the driver has made an emergency posture in a collision scenario. The control module 204 is used to control the dummy to make corresponding actions according to the posture; if the driver makes an emergency posture in a collision scenario, the collision damage information of the dummy is obtained.

[0037] Embodiment 3 of the present invention provides a storage medium storing program instructions, which, when executed, implement the driver imitation method in a collision scenario as described in any of the preceding embodiments.

[0038] Embodiment 4 of the present invention provides a terminal, such as Figure 5As shown, it includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the driver simulation method in the collision scenario as described in any of the preceding claims.

[0039] This invention can be an apparatus, method, and / or computer program product. A computer program product may include a readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0040] Storage media can be tangible devices that hold and store instructions for use by instruction execution devices. Storage media can include, but are not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0041] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0042] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for driver simulation in a collision scenario, characterized in that, Includes the following steps: Acquire an image to be detected, which is a posture image of the driver operating the driving simulator; A posture recognition model is created, comprising a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit; the 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model. Based on the posture recognition model, the driver's posture is obtained from the image to be detected, and it is determined whether the driver has made an emergency posture in the collision scenario. The dummy is controlled to perform corresponding actions based on the stated posture; if the driver adopts an emergency posture in a collision scenario, the collision damage information of the dummy is obtained.

2. The driver simulation method in a collision scenario according to claim 1, characterized in that, The "acquiring the image to be detected" specifically includes: continuously acquiring the driver's posture image when the driver operates the driving simulator to obtain the image to be detected.

3. The driver simulation method in a collision scenario according to claim 1, characterized in that, The phrase "identifying the driver's posture based on the posture recognition model and determining whether the driver has adopted an emergency posture in a collision scenario" specifically includes: the 2D posture recognition unit in the posture recognition model processes the image to be detected, identifies the driver's 2D human key points, and constructs a 2D skeleton; based on the 2D skeleton, the 3D posture recognition unit processes the image to convert the 2D posture into a 3D posture, obtains the driver's human key points in 3D space, and constructs a 3D skeleton; the 2D skeleton and the 3D skeleton are quantitatively analyzed to obtain quantitative data reflecting the driver's posture; and the quantitative data is used to determine whether the driver has adopted an emergency posture in a collision scenario.

4. The driver simulation method in a collision scenario according to claim 3, characterized in that, The quantitative data includes several preset quantitative indicators and several values ​​corresponding to the preset quantitative indicators in a continuous time period.

5. The driver simulation method in a collision scenario according to claim 4, characterized in that, The step of "determining whether the driver has made an emergency posture in a collision scenario based on the quantitative data" specifically includes: comparing the changes in the values ​​corresponding to preset quantitative indicators in the quantitative data over a continuous time period; if the change in at least one indicator is greater than a preset threshold and continues for a preset duration, then the driver has made an emergency posture in a collision scenario.

6. The driver simulation method in a collision scenario according to claim 3, characterized in that, The quantitative indicators are those that reflect the spatial position and angular relationship of various parts of the body.

7. The driver simulation method in a collision scenario according to claim 1, characterized in that, The "controlling the dummy to perform corresponding actions according to the posture" specifically includes: the dummy controller receives the action signal corresponding to the posture, converts the action signal into drive signals for each joint of the dummy, and controls each joint of the dummy to perform actions corresponding to the driver's posture based on the drive signals.

8. A driver simulation device for collision scenarios, characterized in that, Includes the following modules: The data acquisition module is used to acquire the image to be detected, which is the posture image of the driver operating the driving simulator; The model creation module is used to create a posture recognition model, which includes a 2D posture recognition unit, a 3D posture recognition unit, and a fine-tuning unit. The 2D posture recognition unit is used to recognize the driver's 2D posture, the 3D posture recognition unit is used to convert the 2D posture to a 3D posture, and the fine-tuning unit is used to adjust the parameters of the posture recognition model. The recognition module is used to identify the driver's posture in the image to be detected based on the posture recognition model and to determine whether the driver has made an emergency posture in a collision scenario. The control module is used to control the dummy to make corresponding actions according to the posture; if the driver makes an emergency posture in a collision scenario, the collision damage information of the dummy is obtained.

9. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they implement the driver simulation method in a collision scenario as described in any one of claims 1 to 7.

10. A terminal, characterized in that, It includes a processor and a memory, the memory storing program instructions, and the processor executing the program instructions to implement the driver simulation method in a collision scenario as described in any one of claims 1 to 7.

Images