Method for recognizing head excursion of car crash dummy based on machine vision

Abstract

The application relates to the technical field of automobile passive safety testing and evaluation, in particular to a head out-of-bound recognition method for a car crash dummy based on machine vision. The method comprises the following steps: constructing a three-dimensional visual measurement reference field to obtain multiple head images; training a visual perception model; recognizing a head bounding box and head key point two-dimensional coordinates in each head image; performing weighted absolute orientation based on the head key point three-dimensional coordinates to obtain a head three-dimensional pose; calculating the geometric distance between each judgment point in a judgment point set and the physical safety boundary; and calculating an out-of-bound probability at each time and in a time window. In a vehicle crash test, the application realizes real-time and high-precision trajectory tracking and automatic out-of-bound judgment of the head space movement of a dummy based on multi-view machine vision, three-dimensional reconstruction and deep learning technology.

Description

Technical Field

[0001] This application relates to the field of automotive passive safety testing and evaluation technology, and more specifically, to a machine vision-based method for recognizing the head crossing boundaries of a car collision dummy. Background Technology

[0002] In automotive crash safety performance testing, the dynamic response of a dummy's head during the instantaneous impact, particularly its maximum displacement relative to fixed structures within the vehicle, is a key quantitative indicator for evaluating the effectiveness of occupant restraint systems such as seat belts and airbags. The current mainstream testing method involves attaching high-contrast markers to the dummy's head, using a high-speed camera system deployed inside the vehicle to record two-dimensional motion images of these markers, and then manually or semi-automatically analyzing the images after the test using specialized software to determine whether the head exceeds preset physical boundaries. This method has inherent limitations: 1) Dimensional gaps and accuracy limitations: Based on single-viewpoint or non-precisely calibrated two-dimensional image analysis, it is difficult to restore the true three-dimensional spatial movement of the head. The measurement accuracy is greatly affected by the viewpoint and lens distortion, and is easily affected by the subjective judgment of the analyst.

[0003] 2) Poor anti-interference capability: During the collision, the deployment of airbags and the splashing of debris can easily cause the marker points to be obscured or lost, resulting in the interruption of key data and seriously affecting the integrity and reliability of the analysis.

[0004] 3) Insufficient real-time performance: The entire process relies heavily on post-processing, making it impossible to provide real-time warnings and feedback during the test, which reduces testing efficiency and makes it difficult to support rapid iterative development.

[0005] 4) Complex and costly system: The multi-sensor (such as high-speed camera + inertial sensor) fusion scheme adopted to improve accuracy has problems such as complex system integration, high time synchronization requirements, cumbersome calibration and high cost.

[0006] Therefore, there is an urgent need for a technical solution that can achieve non-contact, fully automatic, high-precision, and real-time three-dimensional head displacement measurement and boundary judgment in complex and harsh collision environments. Summary of the Invention

[0007] The purpose of this application is to provide a machine vision-based method for recognizing the boundary crossing of a dummy head in a car collision test. This method is used in vehicle collision tests to perform real-time, high-precision trajectory tracking and automatic boundary crossing determination of the dummy head's spatial motion based on multi-view machine vision, 3D reconstruction, and deep learning technologies.

[0008] To achieve the above objectives, this application provides a machine vision-based method for recognizing the head crossing boundaries of a car collision dummy, comprising: A three-dimensional visual measurement reference field is constructed, multiple cameras are set up within the three-dimensional visual measurement reference field, and the intrinsic and extrinsic parameters are calibrated to transform the physical safety boundary of the collision test to the world coordinate system. Multiple head images are obtained by simultaneously capturing images of the dummy's head during the car collision using the multiple cameras. A two-stage training framework combining general-collision domain progressive self-supervised pre-training and few-sample supervised fine-tuning was adopted to train the visual perception model. The visual perception model is used to identify the head bounding box and the two-dimensional coordinates of the head key points in each head image. The three-dimensional coordinates of the head key points in the world coordinate system are obtained by performing three-dimensional reconstruction based on the two-dimensional coordinates of the head key points; the three-dimensional pose of the head is obtained by performing weighted absolute orientation based on the three-dimensional coordinates of the head key points. Select a set of decision points from multiple head key points, and calculate the geometric distance from each decision point in the decision point set to the physical security boundary in the world coordinate system; Based on the geometric distance and the multi-view fusion comprehensive visibility confidence of the decision point at the current moment, calculate the out-of-bounds probability at each moment and within the time window; The training employs a two-stage framework combining general-collision domain progressive self-supervised pre-training with few-sample supervised fine-tuning to train a visual perception model, including: The first network is trained using a general head image dataset, enabling the first network to learn general head representations. The first network is trained again using unlabeled, real vehicle crash test or high-speed dynamic test video frames, so that the first network learns the head representation of the collision domain; during this training process, cross-frame consistency loss is constructed based on the features of adjacent frame pairs. The trained first network is used as the feature extraction backbone network and connected to a lightweight single-stage detector to form the second network. The second network is trained using real collision test keyframes labeled with the bounding boxes of the dummy head, so that the second network learns the bounding boxes of the dummy head. The trained first network is used as the feature extraction backbone network and connected to the key point localization network to form the third network. The third network is trained using real collision test keyframes labeled with key points of the dummy head, so that the third network learns the two-dimensional key points of the dummy head and the visibility confidence of each key point.

[0009] Compared with the prior art, this application has the following beneficial effects: It can accurately measure head pose in three-dimensional space; it can stably track key points even under conditions of occlusion, blurring, and drastic changes in lighting; it can calculate displacement in near real-time and output boundary judgment results during the test; it only relies on a conventional high-speed camera, without the need to attach markers or additional sensors, thus lowering the barrier to entry. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a machine vision-based method for recognizing the head crossing boundaries of a car collision dummy, as provided in an embodiment of this application. Detailed Implementation

[0012] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0013] Figure 1 This is a flowchart illustrating a machine vision-based method for recognizing head boundaries of a car collision dummy. This application is applicable to situations where multiple cameras are used to capture real-time images of the head of a collision dummy, and a series of algorithms are used to identify whether the head has crossed the boundary.

[0014] See Figure 1 The method provided in this embodiment includes: S110. Construct a three-dimensional visual measurement reference field, set up multiple cameras in the three-dimensional visual measurement reference field, and complete the calibration of internal and external parameters to transform the physical safety boundary of the collision test into the world coordinate system.

[0015] The purpose of this step is to construct a geometrically accurate, coordinate-system-unified, and metrically reliable three-dimensional visual measurement reference field. This step is completed in a static environment before the experiment, and its accuracy is the foundation for the reliability of subsequent dynamic motion analysis.

[0016] Inside the test vehicle or crash trolley, multiple cameras are installed, such as N (N≥2) high-speed cameras, adhering to the principles of non-coplanarity and full field of view coverage. The non-coplanarity principle means that the optical centers of each camera should, as far as possible, avoid being on the same plane to form a geometrically constrained multi-view observation network, thereby significantly reducing the uncertainty in the depth direction of 3D reconstruction. The full field of view coverage principle means that the joint field of view of all cameras must completely cover the entire motion envelope space of the dummy's head from its static initial position to the expected maximum dynamic displacement, with a margin of at least 20% to accommodate accidental deviations in the motion trajectory.

[0017] All cameras must have a global shutter (to eliminate the rolling shutter effect caused by high-speed motion), a high frame rate (≥1000fps to freeze transient motion), and high dynamic range (HDR) to cope with extreme lighting changes such as airbag deployment. All cameras are physically secured to the vehicle frame or a robust rig structure using rigid mounting brackets to ensure no relative displacement between cameras or between the cameras and the object being measured under impact.

[0018] A central synchronization controller sends a unified TTL hardware trigger pulse to all cameras to control the start and stop of all camera captures, achieving microsecond-level start-stop synchronization. The cameras' built-in clocks perform continuous network synchronization, assigning a globally consistent timestamp to each frame.

[0019] The calibration board was moved at various distances, azimuth angles, and tilt angles (e.g., a checkerboard with a side length of 30mm ± 0.005mm), and the pixel coordinates of the corner points of the calibration board in images captured by multiple cameras were extracted. Based on the pinhole camera model and the Brown-Conrady distortion model, the intrinsic parameter matrix was initially solved by minimizing the reprojection error. K i (Including focal length, principal point, and tilt factor) and distortion coefficient vector.

[0020] The core function of global bundle adjustment is to deduce the most consistent 3D structure (object coordinates), camera spatial pose (i.e., extrinsic matrix), and internal imaging parameters (intrinsic matrix, distortion coefficients) from noisy 2D image observations. By minimizing the "reprojection error" on all observed images, information is transferred from the two-dimensional image to the three-dimensional space, achieving a globally optimal solution. The global bundle adjustment algorithm is existing technology and will not be elaborated upon here.

[0021] After the vehicle / trolley is in place and before the dummy is placed, a high-precision coordinate measuring machine or laser tracker is used to directly measure and define the physical safety boundary for head crossing. For example, the three-dimensional coordinates of three (or more) non-collinear points on the steering wheel rim are measured, and their plane equations are determined by least-squares fitting to obtain a plane datum; the three-dimensional coordinates of two discrete points on the edge of the B-pillar interior panel are measured, and a linear datum is obtained by fitting.

[0022] The physical safety boundaries are transformed to the world coordinate system. Specifically, to ensure consistency between the physical measurement coordinate system and the visual coordinate system, coordinate system registration is required. After visual calibration, the same measuring instrument is used to measure a set of common feature points (such as a reference sphere attached to the vehicle body) that are clearly visible and fixed in position in the visual image, obtaining their coordinates in the measuring machine coordinate system. Common points in the visual coordinate system coordinates below Given that, by solving the absolute orientation problem, the coordinate system from the measuring machine coordinate system to the world coordinate system (i.e., the visual coordinate system) is obtained. Optimal rigid body transformation (rotation matrix R) align Translation vector t align Using the obtained rigid body transformations, the linear datum and planar datum are transformed to the world coordinate system. This achieves a complete and traceable mapping from physical entities to digital models. Physical security boundaries are represented in mathematical form in the world coordinate system, such as plane equations and line equations.

[0023] S120: Multiple cameras simultaneously capture images of the dummy's head during a car collision, obtaining multiple head images.

[0024] Although the hardware is synchronized, slight drift in the camera's internal clock, frame buffer differences, or data transmission delays may cause sub-frame level misalignment in the frame index of the decoded multi-channel video streams. Therefore, an interpolation alignment algorithm is used to align the timestamps of head images captured by multiple cameras. The interpolation alignment algorithm is as follows: for each global time point t... k For each camera i: find the global time point t in the timestamp sequence. k Two adjacent frames are used to generate a synchronization frame by weighting the two frames. The weights can be customized or based on t. k It is determined by the time difference between the preceding and following frames.

[0025] For each global time t k This results in a set of strictly synchronized multi-path images, ensuring the effectiveness of multi-view geometric constraints.

[0026] Geometric distortion correction and dynamic lighting robustness enhancement are performed on each head image. The geometric distortion correction model is the Brown-Conrady distortion model, which can be found in existing technologies. To address instantaneous highlights (overexposure) and complex shadows (underexposure) caused by airbag deployment and glass breakage reflections, the following dynamic lighting robustness enhancement processing is performed: First, the image is converted from the RGB color space to the YUV space, separating the Y component representing luminance. For the luminance component Y, contrast-limited adaptive histogram equalization is applied; specifically, the image is divided into multiple non-overlapping local regions (tiles). A luminance histogram is then calculated for each local region. According to the preset cutting limit To crop the histogram, i.e.: ; This is the cropped brightness histogram. This is the brightness histogram before cropping.

[0027] The cropped histogram is then equalized to obtain the transformation function T. m,n By eliminating boundary artifacts between regions through bilinear interpolation, a globally enhanced luminance component Y is obtained. clahe (u,v). Please refer to existing technologies for details; further explanation is unnecessary.

[0028] By combining time-domain information, the instantaneously highlighted region Ω is identified. highlight and persistent shadow area Ω shadow For the instantaneously bright area Ω highlight Recovery using Retinex theory: ; in, Represents the spatial coordinates of a pixel. The luminance component of the input image. For Gaussian kernel function, This represents the illuminance component estimate obtained after applying Gaussian filtering to the luminance image. Y recover (x) represents the reflection component of the logarithmic domain output. By subtracting from the logarithmic domain to remove the influence of illumination, the detail of the highlight area is restored.

[0029] For Ω shadow Perform gamma correction: ; Where x represents the spatial coordinates of a pixel, Y(x) is the luminance component of the input image, and γ is the gamma correction coefficient used for non-linear luminance enhancement of dark areas. Y corrected(x) represents the luminance component output after gamma correction.

[0030] The processed region is then compared with the globally enhanced luminance component Y. clahe The fusion yields the final enhanced luminance component Y. enhanced Y enhanced The image is then merged with the original U and V chromaticity components and converted back to the RGB color space to obtain the final pre-processed image.

[0031] Through this step, the original video stream is transformed into a set of high-quality multi-view image sequences that are time-synchronized, geometrically distortion-free, have minimized motion blur, and suppress illumination interference.

[0032] S130 employs a two-stage training framework that combines general-collision domain progressive self-supervised pre-training with small-sample supervised fine-tuning to train a visual perception model.

[0033] Step 1: Train the first network using a general head image dataset so that the first network learns general head representations.

[0034] This dataset includes real human face images in various poses and lighting conditions, as well as static images of heads from various commercial mannequins (Hybrid III, THOR, WorldSID, etc.), covering different poses, lighting conditions, and backgrounds. A SwAV (Swapping Assignments between Views) contrastive learning method is employed. The backbone network Φ uses a deep residual network (such as ResNet-50). For each image in the general head image dataset, two independent random data augmentation methods (including cropping, color jittering, Gaussian blur, etc.) are applied to generate two augmented views. SwAV is used for online clustering and swapping prediction. It maps the features of the two augmented views to a set of learnable prototype vectors to generate encodings. The loss function encourages predicting the encoding assignment of one augmented view from features of the other, and vice versa. After pre-training, the first network already possesses good general head representation capabilities.

[0035] Step 2: Continue training the first network using unlabeled, real vehicle collision test or high-speed dynamic test video frames, so that the first network learns the head representation of the collision domain; during this training process, construct cross-frame consistency loss based on the features of adjacent frame pairs.

[0036] Specifically, a large number of unlabeled, real-world vehicle crash test or high-speed dynamic test videos were collected, and frame sequences were extracted from them to construct a crash domain dataset. These images contain complex interior backgrounds, airbag deployments, debris splashes, severe motion blur, and abrupt changes in lighting.

[0037] The first network, pre-trained in the first step, is then trained again on the collision domain dataset. For a continuous video frame sequence in the collision domain dataset... Selecting adjacent frame pairs Because the inter-frame interval is extremely short (e.g., Δt = 1 ms), the head's movement in the image is continuous and minute. The first network processes these two frames separately and outputs their features. and Design the following cross-frame consistency loss. : ; Where θ is the angle between feature vectors. Adjacent frame pairs The features are input into the first network Φ to obtain the features. and By minimizing The goal is to have the first network Φ extract the most similar features possible for the same head target at adjacent time points (despite potential appearance changes and interference). This process effectively "pulls" the network's feature space from a general domain to a collision-specific domain, enabling it to learn to ignore transient, non-rigid interference (such as blurring or occlusions) unique to collision videos and focus on the stable semantic features of the head target itself. This ultimately yields a domain-adaptive, strong feature extractor Φ. adapted .

[0038] The overall goal of the first and second steps is to inject powerful, collision-oriented prior knowledge into the first network without manual annotation.

[0039] Step 3: Use the trained first network as the feature extraction backbone network and connect it to a lightweight single-stage detector to form the second network; train the second network using real collision test keyframes labeled with the bounding box of the dummy head, so that the second network learns the bounding box of the dummy head.

[0040] Specifically, on a limited number (e.g., thousands of frames) of real-world collision test keyframes, experts annotate the precise bounding boxes of the dummy's head. A lightweight, single-stage detector architecture (such as YOLOX's detector head) is used to measure the Φ... adapted As the backbone network for feature extraction, the shallow weights of the feature extraction backbone network are frozen, and only the deep layers and the detection head are trained. Data augmentation optimized for collision scenarios is used, including simulated airbag occlusion, motion blur rendering, and random brightness perturbation. The loss function adopts a combination of classification loss (varifocal loss) and regression loss (e.g., GIoU Loss + L1 Loss), and the construction of the loss function is described in existing techniques and will not be repeated here. This stage outputs the bounding box of the dummy head, providing a precise region of interest for subsequent keypoint localization.

[0041] Step 4: Use the trained first network as the feature extraction backbone network and connect it to the key point localization network to form the third network; train the third network using real collision test keyframes labeled with key points of the dummy head, so that the third network learns the two-dimensional key points of the dummy head and the visibility confidence of each key point.

[0042] The construction of the training samples includes defining a set of M anatomically significant key points within the bounding box of the dummy head, such as the top of the head, between the eyebrows, the root of the nose, the tip of the nose, the midpoint of the chin, the outer canthi of the left and right eyes, the tragus of the left and right ears, and the midpoint of the back of the head. In the same batch of images with the bounding box of the dummy head annotated, these key points are labeled with pixel-level coordinates, and each point is labeled with a visibility label (fully visible = 1, partially occluded = 0.5, invisible = 0).

[0043] A high-resolution network (HRNet) is used as the keypoint localization network. HRNet can consistently maintain high-resolution feature maps, making it particularly suitable for tasks requiring precise localization. Φ adapted Integrated as the front end of HRNet, it undergoes end-to-end training using the aforementioned training samples. The following combined loss function is used to address the localization challenge under severe occlusion: ; in, To train the loss function of the third network, For heatmap regression loss, It is a geometric constraint loss. It is a weighted balance; the third network predicts a Gaussian heatmap for the j-th head keypoint. , This is the true Gaussian heatmap of the j-th head keypoint, where M is the number of head keypoints, and smoothed L1 loss is used. The heatmap regression loss is calculated; S is a predefined set of point pairs with stable distance relationships. and The 3D coordinates of head keypoints obtained from multi-view triangulation during the training phase, or directly corresponding to the real 3D coordinates in the labeled dataset. The prior distance between the m-th and n-th head keypoints. The three-dimensional coordinates of the m-th head keypoint 3D coordinates of the nth head keypoint The distance.

[0044] The purpose of steps three and four is to utilize limited, precisely labeled collision image data to analyze Φ. adapted Monitor and fine-tune it to enable it to complete specific tasks (detection and positioning).

[0045] S140. Using a visual perception model, identify the head bounding box and the two-dimensional coordinates of the head key points in each head image.

[0046] The head image is input into a third network, which ultimately outputs the predicted 2D pixel-level coordinates of each head keypoint, along with a visibility confidence score reflecting the reliability of those coordinates (generated by the peak intensity of the predicted heatmap or a parallel occlusion prediction head). In cases of severe occlusion... This effectively constrains predicted points to lie within a reasonable spatial configuration, while visibility confidence is used to weight or eliminate low-quality observations in subsequent 3D reconstruction. Through this step, a robust and accurate visual perception model is obtained, capable of outputting, in real time, head bounding boxes unaffected by transient disturbances and a set of head keypoint coordinates with reliability assessment from a single frame of head image.

[0047] S150. Perform 3D reconstruction based on the 2D coordinates of the head key points to obtain the 3D coordinates of the head key points in the world coordinate system; perform weighted absolute orientation based on the 3D coordinates of the head key points to obtain the 3D pose of the head.

[0048] The purpose of this step is to stably and accurately reconstruct the three-dimensional motion trajectory and pose (i.e., six-DOF pose) of the dummy head in a unified world coordinate system from a multi-view two-dimensional keypoint sequence with confidence assessment using geometric and optimization algorithms. This achieves a complete mathematical description of the rigid body motion of the head, providing direct and reliable physical quantity input for the final boundary violation determination.

[0049] For each head keypoint, the 3D coordinates in the world coordinate system are solved using the pixel coordinates and confidence levels observed by multiple cameras, employing a weighted least squares method to give higher-confidence observations greater weight. Even with only two cameras active, the 3D coordinate error remains less than 5mm. The specific steps of the weighted least squares method are detailed in existing technologies and will not be repeated here.

[0050] By solving the weighted absolute orientation problem, the 3D coordinates of head keypoints are aligned with a predefined standard 3D rigid body model of the head, thereby calculating the 3D coordinates and pose of the head in the world coordinate system. Weighted singular value decomposition (SVD) is used to solve the rigid body transformation of the head, a well-known technique in the field, and will not be elaborated here. Through weighted absolute orientation, the set of 3D keypoints of the head observed in the current frame is aligned with a predefined standard 3D rigid body model of the head, thereby calculating the rigid body motion (rotation and translation) of the head model relative to the world coordinate system.

[0051] Then, the head position sequence calculated from the continuous time series is smoothed and filtered to obtain the displacement curve, and the head attitude (represented by Euler angles) sequence calculated from the continuous time series is smoothed and filtered to obtain the attitude curve. For example, a Savitzky-Golay filter is used for filtering. The displacement curve is numerically differentiated to obtain the velocity trajectory and acceleration trajectory. Specifically, the velocity trajectory is obtained by performing a first differentiation on the displacement curve, and the acceleration trajectory is obtained by performing a second differentiation.

[0052] Finally, this step outputs complete and smooth time history data of the head's six degrees of freedom motion, including: head three-dimensional displacement, three-dimensional velocity, three-dimensional acceleration, three-dimensional attitude angle, and three-dimensional coordinates of head key points.

[0053] It should be noted that the end-to-end latency of preprocessing, deep learning inference, and post-processing on the embedded GPU module meets near real-time requirements. A pipelined buffering mechanism is employed, with the processing of the current frame and the acquisition of the next frame occurring in parallel. The output results exhibit a fixed but acceptable latency relative to real time, and the inter-frame decision interval is consistent with the acquisition frame rate. Vibration table testing has demonstrated that the embedded GPU module operates continuously without failure under high-intensity random vibration and impact, proving its suitability for collision testing environments. A high-configuration industrial PC solution with even lower latency is also available, allowing users to choose according to their testing environment.

[0054] S160. Select a set of decision points from multiple head key points, and calculate the geometric distance from each decision point in the decision point set to the physical safety boundary in the world coordinate system.

[0055] The key points most prone to collision intrusion are selected from multiple head key points to form a decision point set, including the chin midpoint, forehead midpoint, vertex, and nasal tip. The use of multiple points provides spatial redundancy.

[0056] If the physical security boundary is a plane, then in the world coordinate system, the distance from the determination point to the plane is calculated according to the formula for the distance from a point to a plane. A positive sign indicates that the point is on the positive side of the plane (the safe side), and a negative sign indicates intrusion. If the physical security boundary is a straight line, then in the world coordinate system, the distance from the determination point to the straight line is calculated according to the formula for the distance from a point to a straight line. A positive sign indicates that the point is on the positive side of the plane (the safe side), and a negative sign indicates intrusion.

[0057] S170. Calculate the out-of-bounds probability for each time step and within the time window based on the geometric distance and the multi-view fusion comprehensive visibility confidence of the decision point at the current time.

[0058] Among them, "multi-view fusion comprehensive visibility confidence level" The visibility confidence of the decision point output in step S140 under each independent viewpoint is weighted and fused according to the spatial geometric weights, unoccluded ratio, and image sharpness of each viewpoint, resulting in a single normalized scalar with a value range of [0,1]. For example, the spatial geometric weight is determined based on the angle between the camera optical axis and the normal to the head surface. The spatial geometric weight is highest when the viewpoint is directly facing the forehead (angle close to 0°); the spatial geometric weight drops sharply when the side or pitch angle is too large. The proportion of the visible decision point to the total area of ​​all decision points is calculated according to the semantic segmentation method, i.e., the unoccluded ratio; the image sharpness can be calculated using existing methods such as Tenengrad gradient variance. The spatial geometric weights, unoccluded ratio, and image sharpness under each viewpoint are normalized to obtain three normalized values; the three normalized values ​​are added together to obtain the weight under each viewpoint. The weights of each viewpoint are weighted and summed with the visibility confidence scores for that viewpoint, then normalized to the [0,1] interval to obtain the multi-view fusion comprehensive visibility confidence score for that decision point. The closer the value is to 1, the more reliable the comprehensive observation of the decision point from multiple perspectives; conversely, if... If the value approaches 0, it indicates that the point is severely occluded in three-dimensional space or has poor observation quality, and the contribution weight of the point to the overall result should be reduced in the calculation of the out-of-bounds probability.

[0059] Set a dynamic security threshold This threshold includes not only the static distance specified by regulations. δ 0 (as specified in FMVSS208, 80mm), and also incorporates the uncertainty of the pose estimation at the current moment. (Can be obtained from covariance estimates or empirical values). An adaptive safety buffer is formed.

[0060] At each time t, define the weighted out-of-bounds probability based on multiple decision points: ; ; in, It is the probability of going out of bounds at time t. It is a very small positive number, used to prevent the denominator from being zero. This is the static distance. k For a safety factor, for example, let's take 2. Let be the uncertainty of the pose estimation at time t. Let be the dynamic safety threshold at time t. It is time t. ml The geometric distance from each decision point to the physical safety boundary, where L is the total number of decision points. It is the first mlThe multi-view fusion comprehensive visibility confidence of each decision point at time t is used as a weight; It is an indicator function. It takes the value 1 when the condition in parentheses is met, and takes the value 0 otherwise. This means that only intrusions observed with high confidence are accepted. The more intrusions there are and the higher the confidence level, the greater the overall probability of going out of bounds.

[0061] To avoid momentary false alarms caused by single-frame noise or misdetection, a sliding time window of length W (e.g., W=7 frames) is introduced. The probability of exceeding the limit within the time window is calculated as follows: ; Where W is the length of the time window, for The average probability of exceeding the limit within the time window from time t. For time variables, For time The probability of exceeding the boundary at that time.

[0062] Only when An "out-of-bounds confirmation" event is only determined when the threshold value η (e.g., η=0.6) is exceeded for five consecutive times. Simultaneously, the following is recorded: 1) First boundary crossing moment: meets the requirements The start time.

[0063] 2) Maximum intrusion: During the entire event .

[0064] 3) The most dangerous point: the moment when the maximum intrusion occurs.

[0065] The method provided in this embodiment will be described in detail below through several specific implementation methods.

[0066] Example 1: Determination of the driver's head crossing the steering wheel boundary in a frontal collision.

[0067] The test vehicle was a certain type of SUV, subjected to a 100% overlap rigid barrier collision according to C-NCAP regulations at a speed of 50 km / h. Before the test, three high-speed cameras were installed inside the vehicle, with a lens focal length of 12mm, a frame rate of 1000fps, and a resolution of 1280×800. Camera 1 was fixed above the A-pillar, with its optical axis pointing towards the driver's head area; Camera 2 was fixed in the center of the roof, shooting vertically downwards; Camera 3 was fixed above the dashboard, aimed at the steering wheel. A synchronous controller was used for triggering, with measured trigger jitter less than 2μs. Calibration used a 30mm×30mm ceramic checkerboard (corner accuracy ±0.005mm), acquiring 25 sets of synchronized images. After bundle adjustment, the average reprojection error was 0.38 pixels. A laser tracker measured three points on the steering wheel rim, and after coordinate transformation, the steering wheel plane equation was obtained: 0.707X + 0.707Z - 1100 = 0 (X to the right, Z forward). The calibration file was stored on the server.

[0068] During the experiment, three video streams were transmitted to the industrial control computer in near real-time. The preprocessing module sequentially performed interpolation synchronization, distortion correction, optical flow deblurring, CLAHE, and Retinex specular suppression. The visual perception model was pre-trained on 2 million general head images and fine-tuned using approximately 2000 frames of real collision images (including various occlusion scenarios). The visual perception model outputs a head bounding box and 11 head key points, with the nose tip coordinates at t=80ms being (612,208,835) mm with a confidence level of 0.92. After 3D reconstruction, the chin midpoint coordinates were (608,185,820) mm, and its signed distance to the steering wheel plane was calculated to be approximately -91 mm (intrusion).

[0069] Dynamic threshold calculation: at t=80ms, Value is 4mm, dynamic threshold =80 + 2 × 4 = 88 mm.

[0070] Weighted out-of-bounds probability calculation: Among the four decision points, the chin and forehead have confidence scores of 0.89 and 0.85 respectively, and both are intrusive. The other two points are not intrusive, therefore P0. viol =(0.89+0.85) / (0.89+0.85+0.92+0.88)=0.51. Window length W=7, threshold η=0.6, the judgment condition is not met. Continuing to t=85ms, the chin point intrusion deepens to -105mm, the confidence scores of all four points are higher than 0.8, the probability of exceeding the boundary rises to 0.73, and exceeds 0.6 for 5 consecutive frames. The system judges "confirmed boundary violation", records the first boundary violation time at 85ms, and the maximum intrusion amount is 105mm.

[0071] Example 2: Determination of the dummy's head crossing the B-pillar in a side collision.

[0072] The test vehicle was a sedan, and a side pole impact test was conducted according to the Euro NCAP procedure at a speed of 32 km / h. Before the test, two high-speed cameras (1000fps, 1280×800 resolution) were installed inside the vehicle: Camera 1 was fixed to the roof above the B-pillar, with its optical axis pointing diagonally downwards towards the dummy's head; Camera 2 was fixed to the opposite side seat back, extending to the side of the dummy via a rigid bracket. Synchronous trigger jitter was less than 3 μs. Calibration used the same ceramic checkerboard pattern as in Example 1, with an average reprojection error of 0.42 pixels.

[0073] Digitalization of safety boundaries: Using a laser tracker, the two endpoints of the leading edge of the B-pillar interior panel were measured. After coordinate transformation, the equation of the straight line was obtained: P=(320,150,680)+λ(0,1,0.2) (unit: mm, X to the right, Y to the back, Z to the up). Among the 11 key points of the dummy's head, the three points most likely to be close to the B-pillar—the left tragus, the left edge of the top of the head, and the outer edge of the left eye socket—were selected as the judgment points.

[0074] Experimental Procedure: 30ms after the collision, the head begins to move towards the B-pillar. At t=55ms, the three-dimensional coordinates of the left tragus are (298, 172, 695) mm. The calculated straight-line distance is approximately 13mm (geometric distance, negative for signed distances, -13mm). Side-impact regulations do not specify a static distance; therefore, δ0=10mm (empirical safety margin), σ=3.5mm, and δ=17mm are used, indicating the intrusion did not exceed the threshold. By t=68ms, the left tragus intrusion reached -22mm, with a three-point average confidence level of 0.91. The weighted probability of exceeding the limit is P0. viol =0.78, sliding window mean 0.69, five consecutive frames exceeding 0.6, the system determines "boundary violation confirmed", the first boundary violation time is recorded at 68ms, the maximum intrusion is 31mm. This embodiment verifies the effectiveness of the system under side collision, only 2 cameras, and straight boundary conditions.

[0075] Example 3: Determination of driver's head crossing the steering wheel boundary in an offset collision (2 cameras).

[0076] The test vehicle was a sedan of a certain type, and a crash test was conducted according to the IIHS 25% small offset crash procedure at a speed of 64 km / h. Before the test, only two high-speed cameras (1000fps, 1280×800 resolution) were installed inside the vehicle: Camera 1 was fixed above the A-pillar, and Camera 2 was fixed vertically downward from the center of the roof. Synchronous trigger jitter was less than 3μs. Calibration used the same checkerboard pattern as in Example 1, with an average reprojection error of 0.45 pixels. The steering wheel plane equation was obtained by measuring with a laser tracker: 0.65X + 0.65Z - 420 = 0.

[0077] This embodiment verifies that the system can still operate stably under conditions of only two cameras and high lateral acceleration due to offset collision. During the test, the head motion trajectory was successfully reconstructed. At t=72ms, the intrusion at the midpoint of the chin reached -98mm, the dynamic threshold δ≈88mm (σ=4.2mm), the weighted out-of-bounds probability was 0.69, and it exceeded 0.6 for 5 consecutive frames, confirming an out-of-bounds violation. This embodiment, together with Embodiments 1 and 2, supports the protection range of "2 to 4 cameras".

[0078] The embodiments of this application have the following technical effects: 1) Significantly improved accuracy: With multiple cameras working effectively, the 3D positioning accuracy reaches the millimeter level, the reprojection error is controlled at the sub-pixel level, and the overall accuracy is significantly better than traditional methods.

[0079] 2) Extremely robust: Even in complex situations where key points are 30% obscured, the tracking success rate remains extremely high, far exceeding the capabilities typically demonstrated by existing methods.

[0080] 3) Significant synergistic effect: Ablation experiments show that when temporal consistency loss and geometric constraint loss are used together, the proportions of positioning error and extreme error are significantly reduced, resulting in a significant synergistic enhancement effect rather than a simple superposition.

[0081] 4) Easy and convenient deployment: No need to paste any markers or install inertial sensors. All functions can be completed using only a high-speed camera, which greatly reduces the threshold for use and the complexity of preparation.

[0082] 5) Near real-time feedback capability: The end-to-end processing latency is extremely low. Combined with the pipeline buffer mechanism, it can fully meet the real-time requirements of engineering scenarios such as collision tests.

[0083] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0084] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A machine vision-based method for recognizing the head crossing boundaries of a car collision dummy, characterized in that, include: A three-dimensional visual measurement reference field is constructed, multiple cameras are set up within the three-dimensional visual measurement reference field, and the intrinsic and extrinsic parameters are calibrated to transform the physical safety boundary of the collision test to the world coordinate system. Multiple head images are obtained by simultaneously capturing images of the dummy's head during the car collision using the multiple cameras. A two-stage training framework combining general-collision domain progressive self-supervised pre-training and few-sample supervised fine-tuning was adopted to train the visual perception model. The visual perception model is used to identify the head bounding box and the two-dimensional coordinates of the head key points in each head image. The three-dimensional coordinates of the head key points in the world coordinate system are obtained by performing three-dimensional reconstruction based on the two-dimensional coordinates of the head key points; the three-dimensional pose of the head is obtained by performing weighted absolute orientation based on the three-dimensional coordinates of the head key points. Select a set of decision points from multiple head key points, and calculate the geometric distance from each decision point in the decision point set to the physical security boundary in the world coordinate system; Based on the geometric distance and the multi-view fusion comprehensive visibility confidence of the decision point at the current moment, calculate the out-of-bounds probability at each moment and within the time window; The training employs a two-stage framework combining general-collision domain progressive self-supervised pre-training with few-sample supervised fine-tuning to train a visual perception model, including: The first network is trained using a general head image dataset, enabling the first network to learn general head representations. The first network is trained again using unlabeled, real vehicle crash test or high-speed dynamic test video frames, so that the first network learns the head representation of the collision domain; during this training process, cross-frame consistency loss is constructed based on the features of adjacent frame pairs. The trained first network is used as the feature extraction backbone network and connected to a lightweight single-stage detector to form the second network. The second network is trained using real collision test keyframes labeled with the bounding boxes of the dummy head, so that the second network learns the bounding boxes of the dummy head. The trained first network is used as the feature extraction backbone network and connected to the key point localization network to form the third network. The third network is trained using real collision test keyframes labeled with key points of the dummy head, so that the third network learns the two-dimensional key points of the dummy head and the visibility confidence of each key point.

2. The method according to claim 1, characterized in that, Constructing a three-dimensional vision measurement reference field includes: Inside the test vehicle or crash trolley, multiple cameras are set up according to the principles of non-coplanarity and full field of view coverage; a central synchronous controller sends a unified TTL hardware trigger pulse to all cameras to control the start and stop of all camera acquisition. The calibration board is moved at various distances, azimuth angles, and tilt angles to extract the pixel coordinates of the corner points of the calibration board in images captured by multiple cameras; based on the pinhole camera model, Brown-Conrady distortion model, and global bundle adjustment algorithm, the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficient vector of the camera are solved. Transform the physical security boundary into the world coordinate system.

3. The method according to claim 2, characterized in that, The physical security boundary includes: a planar reference and a linear reference; The physical security boundary is expressed in mathematical form in the world coordinate system.

4. The method according to claim 3, characterized in that, After obtaining multiple head images by simultaneously capturing images of the dummy's head during a car collision using the multiple cameras, the process further includes: An interpolation alignment algorithm is used to align the timestamps of head images captured by multiple cameras; Geometric distortion correction and dynamic lighting robustness enhancement are performed on each head image.

5. The method according to claim 4, characterized in that, The third network is trained using the following loss function: ； in, To train the loss function of the third network, For heatmap regression loss, It is a geometric constraint loss. It is a weighted balance; the third network predicts a Gaussian heatmap for the j-th head keypoint. , This is the true Gaussian heatmap of the j-th head keypoint, where M is the number of head keypoints, and smoothed L1 loss is used. The heatmap regression loss is calculated; S is a predefined set of point pairs with stable distance relationships. and The 3D coordinates of head keypoints obtained from multi-view triangulation during the training phase, or directly corresponding to the real 3D coordinates in the labeled dataset. The prior distance between the m-th and n-th head keypoints. The three-dimensional coordinates of the m-th head key point 3D coordinates of the nth head keypoint The distance.

6. The method according to claim 5, characterized in that, Weighted absolute orientation is performed based on the three-dimensional coordinates of the head key points to obtain the three-dimensional pose of the head, including: By solving the weighted absolute orientation problem, the three-dimensional coordinates of the head key points are aligned with the predefined standard three-dimensional rigid body model of the head, thereby calculating the three-dimensional coordinates and attitude of the head in the world coordinate system.

7. The method according to claim 6, characterized in that, After calculating the head's 3D coordinates and pose in the world coordinate system, the following steps are also included: The displacement curve is obtained by smoothing and filtering the head position sequence calculated from the continuous time series, and the attitude curve is obtained by smoothing and filtering the head attitude sequence calculated from the continuous time series. The displacement curve is numerically differentiated to obtain the velocity trajectory and acceleration trajectory.

8. The method according to claim 7, characterized in that, Based on the geometric distance and the multi-view fusion comprehensive visibility confidence of the decision point at the current moment, the out-of-bounds probability at each moment is calculated, including: ； ； in, It is the probability of going out of bounds at time t. It is a very small positive number, used to prevent the denominator from being zero. This is the static distance. For safety reasons, Let be the uncertainty of the pose estimation at time t. Let be the dynamic safety threshold at time t. It is time t. ml The geometric distance from each decision point to the physical safety boundary, where L is the total number of decision points. It is the first ml The confidence level of the multi-view fusion comprehensive visibility of each decision point at time t; It is an indicator function that takes the value 1 when the condition inside the parentheses is met, and takes the value 0 otherwise.

9. The method according to claim 8, characterized in that, Calculate the probability of exceeding the boundary within the time window, including: ； Where W is the length of the time window, for The average probability of exceeding the limit within the time window from time t. For time variables, For time The probability of exceeding the boundary at that time.

Images