Video image analysis method and system based on AI large model

Abstract

The invention relates to the field of video image analysis, in particular to a video image analysis method and system based on an AI large model, and the method comprises the steps: calling the driving data of a driving subject to extract the usual driving characteristics of the driving subject within a preset driving distance of a target observation point; in combination with the usual driving characteristics and the response time of changing the driving state, evaluating the driving state characterization value of the driving subject, and determining the driving state grade of the driving subject; evaluating and analyzing the driving condition of the driving subject based on the driving state grade; and determining the concentration deviation degree of the driving main body and the gap distance between the hand and the steering wheel to judge whether a prompt signal is sent out or not. According to the invention, based on the video image analysis processing of the AI large model, the data processing efficiency is improved, and the accuracy of the evaluation of the driving state of the driver is improved.

Description

Technical Field

[0001] This invention relates to the field of video image analysis, and in particular to a video image analysis method and system based on a large AI model. Background Technology

[0002] With the continuous growth of car ownership and the increasing complexity of road traffic scenarios, driving safety has become a core issue of social concern. A driver's driving condition directly affects road safety, and poor driving habits such as fatigue and distracted driving are significant contributing factors to traffic accidents.

[0003] Against the backdrop of the rapid development of intelligent transportation technology, driving status monitoring technology based on video image analysis has been widely researched and applied. This type of technology collects video image data from the driver's cab using onboard cameras, and combines image processing, pattern recognition, and other technologies to achieve real-time monitoring of driver behavior and status.

[0004] With the evolution of AI technology, large-scale models, with their powerful feature extraction, multi-source data fusion, and adaptability to complex scenarios, provide a new technological path for driving status monitoring. This enables in-depth mining and precise analysis of multi-dimensional data, improving the accuracy and real-time performance of driving status assessment, while simultaneously driving the development of driving safety monitoring systems towards intelligence and personalization.

[0005] Chinese Patent Application Publication No. CN106651910A discloses an intelligent image analysis method and alarm system for abnormal driver behavior. The method includes: a camera device acquiring images from the driver's cab; an image analysis device performing intelligent analysis and calculation on the acquired images; comparing the acquired images with a locomotive driver state model to retrieve video images of irregular actions and event point images; an analysis host issuing audible and visual alarms for drivers in irregular driving states; and generating a video file database from the video images of irregular actions, event point images, and event logs. This invention analyzes and compares real-time video images of driver operations captured by the camera device, retrieving video images of irregular actions and event point images, thereby achieving effective analysis and alarm for irregular driver behavior. It analyzes, stores, and replays irregular driver behavior to prevent such behavior and reduce the probability of locomotive accidents.

[0006] However, the following problems still exist in the existing technology. Most analyses focus on a single behavior or a single data dimension, lacking the ability to adapt to drivers' individual driving habits. Furthermore, the risk coverage scenarios are relatively limited, reducing the accuracy of driving status assessment. Summary of the Invention

[0007] To address this, the present invention provides a video image analysis method and system based on an AI large model, which overcomes the problems of existing technologies that mostly focus on the analysis of a single behavior or a single data dimension, lack the ability to adapt to the driver's personalized driving habits, and have limited risk coverage scenarios, thus reducing the accuracy of driving status assessment.

[0008] To achieve the above objectives, this invention provides a video image parsing method based on a large AI model, comprising: The driving data of the driver is called to extract the driver's usual driving characteristics within a predetermined driving distance from the target observation point. The usual driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. By combining the usual driving characteristics and the reaction time to changes in driving status, the driving status characterization value of the driving subject is evaluated, and the driving status level of the driving subject is determined. The driving status of the driver is evaluated and analyzed based on the driving status level, including identifying the abnormal time of the driver within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driver based on the driving waiting characteristics and corresponding head rotation amplitude of the driver within a predetermined time period from the abnormal time, so as to determine whether the driver meets the waiting status benchmark. Determine the driver's level of focus deviation and the distance between their hands and the steering wheel to decide whether to issue a warning signal; The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

[0009] Furthermore, the process of evaluating the driving state characterization values ​​of the driving entity includes: The sum of the ratio of the distance to the vehicle in front to the distance threshold and the ratio of the distance to the stop line when in the lead to the distance threshold is used as the first driving state feature. The ratio of the reaction time threshold to the reaction time for changing driving states is used as the second driving state feature. The first driving state feature and the second driving state feature are weighted and summed to obtain the driving state representation value.

[0010] Further, determining the driving status level of the driving entity includes: If the driving status representation value of the driver is less than the driving status representation threshold, then the driving status level of the driver is determined to be a dangerous level. If the driving status representation value of the driving subject is greater than or equal to the driving status representation threshold, then the driving status level of the driving subject is determined as a safety level.

[0011] Furthermore, the driving status of the driver is evaluated and analyzed based on the driving status level, including: If the driving status of the driver is classified as dangerous, then the driving situation of the driver will be assessed and analyzed.

[0012] Furthermore, the process of identifying unusual moments in the driving entity's journey within a predetermined distance from the target observation point includes: If it is detected that the driving entity does not meet the driving state baseline conditions, the corresponding moment will be identified as the abnormal moment. The driving state reference conditions include maintaining a distance from the vehicle in front that is less than a maintaining distance reference threshold, or being in the lead and the distance relative to the stop line that is less than a distance reference threshold.

[0013] Further, the process of calculating the waiting attention representation parameters of the driving subject includes: The sum of the ratio of the maximum gaze deviation angle to the maximum gaze deviation angle threshold and the ratio of the duration threshold to the duration of looking straight ahead is used as the first waiting focus feature; The ratio of head rotation amplitude to head rotation amplitude threshold is used as the second waiting focus feature; The first waiting focus feature and the second waiting focus feature are weighted and summed to obtain the waiting focus characterization parameter.

[0014] Further, determining whether the driving entity meets the waiting state benchmark includes: If the waiting focus representation parameter of the driver is less than the waiting focus representation parameter threshold of the driver, then the driver is determined to meet the waiting state benchmark.

[0015] Further, determining whether to issue a prompt signal includes: If the driver's attention deviation exceeds the attention deviation threshold, and the gap between the hands and the steering wheel exceeds the gap distance threshold, a warning signal will be issued.

[0016] Furthermore, the process of determining the driver's attention deviation includes: Identify the duration of the driver's downward gaze and the extent of eye opening; The sum of the ratio of the duration of downward gaze to the threshold duration of downward gaze and the ratio of the threshold for eye opening amplitude to the eye opening amplitude is used as the focus deviation.

[0017] Furthermore, a system applying video image analysis methods based on AI large models is also provided, including, The inertial analysis module is used to call the driving data of the driver to extract the inertial driving characteristics of the driver within a predetermined driving distance from the target observation point. The inertial driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. A driving analysis module, which is connected to the inertial analysis module, is used to combine the inertial driving characteristics and the reaction time of changing driving state to evaluate the driving state characterization value of the driving subject and determine the driving state level of the driving subject. The status assessment module, which is connected to the driving analysis module, is used to assess and analyze the driving situation of the driving subject based on the driving status level. This includes identifying abnormal times for the driving subject within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driving subject based on the driving waiting characteristics and corresponding head rotation amplitude of the driving subject within a predetermined time range from the abnormal time to determine whether the driving subject meets the waiting status benchmark. A focus alert module, which is connected to the state assessment module, is used to determine the driver's focus deviation and the gap distance between the hands and the steering wheel, so as to determine whether to issue a warning signal. The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

[0018] Compared with existing technologies, this invention includes a habit analysis module, which retrieves the driver's driving data to extract the driver's habitual driving characteristics within a predetermined driving distance from the target observation point; a driving analysis module, which combines habitual driving characteristics with reaction time to change driving states to evaluate the driver's driving state characterization value and determine the driver's driving state level; a state assessment module, which evaluates and analyzes the driver's driving situation based on the driving state level; and a focus prompting module, which determines the driver's focus deviation and the gap between the driver's hands and the steering wheel to determine whether to issue a prompt signal. This invention utilizes AI large-scale model video image analysis processing to improve data processing efficiency, thereby enhancing the accuracy of driver state assessment.

[0019] In particular, this invention considers the individualized habitual behaviors of drivers, reducing the misjudgment rate of compliant driving or overlooking hidden risks. It establishes a baseline for individual driving habits, including quantifying the driver's dynamic safety redundancy by maintaining a safe distance from the vehicle in front. This distance directly reflects the risk buffer space when following another vehicle; the closer the distance is to the driver's habitual baseline, the stronger the driver's awareness of collision risk avoidance during dynamic driving and the more stable their operation. The distance relative to the stop line when in the lead quantifies the driver's static waiting safety compliance. This distance reflects the driver's operational compliance in stationary waiting scenarios such as red lights; by comparing it with their habitual baseline, the consistency of their safe operation in static scenarios can be accurately judged. Furthermore, this invention incorporates reaction time to changes in driving status, quantifying the driver's emergency response capability and level of focus. Moreover, reaction time is directly related to the driver's controllability. Therefore, this invention combines driving operation stability, safety redundancy control, and static waiting standardization dimensions to evaluate the driving entity's driving status characterization value, quantifying the driving entity's overall driving safety level and characterizing the driving entity's operational stability, providing data support for subsequently determining the driving entity's driving status level. This makes the driving status assessment more closely aligned with the driving entity's actual operating mode, improving assessment accuracy and the targeted nature of risk identification.

[0020] In particular, this invention pinpoints moments when a driver's real-time behavior deviates from their usual baseline, quickly identifying abnormal states and improving the detection rate of hidden risks. It focuses on the waiting process in potential risk scenarios such as excessively close following distances or proximity to the stop line, assessing the driver's focus and operational compliance. The degree of head rotation quantifies the reasonableness of the driver's visual focus range. In reality, moderate head rotation, such as observing the rearview mirror or side road conditions, is normal driving operation; excessive rotation indicates that the driver's attention has deviated from the core driving area ahead, potentially indicating distraction. The maximum angle of visual deviation quantifies the degree of deviation of the driver's visual focus, directly reflecting the severity of attentional distraction and serving as a key indicator for judging a state of distraction. Furthermore, the duration of looking straight ahead quantifies the stability of the driver's focused attention. Thus, it achieves full-scenario safety coverage, calculates the driver's waiting focus characteristics, quantifies the driver's level of focus in waiting scenarios, and provides data support for subsequent determination of whether the driver meets the waiting state benchmark. This invention ensures that the assessment of the driver's waiting status is accurate and closely matches the actual driving scenario, thereby improving the accuracy of risk identification and the targeted nature of intervention.

[0021] In particular, this invention accurately targets high-risk scenarios, enabling on-demand triggering of safety warning signals while balancing driving safety and user experience. It distinguishes between normal driving behavior and high-risk behavior by using a dual criterion: the driver's level of focus deviation and the distance between their hands and the steering wheel. Excessive driver focus indicates an inability to respond promptly to emergencies. Conversely, hands leaving the steering wheel prevents quick execution of maneuvers; the combined effect of these two factors exponentially increases the risk of an accident. Warnings are only issued when the high-risk combination of insufficient focus and hands leaving the steering wheel occurs, avoiding frequent invalid warnings due to a single condition and reducing interference with the driver's normal operations. This invention precisely intervenes in high-risk scenarios, improving the accuracy of risk identification and reducing invalid warnings. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the steps of the video image parsing method based on an AI large model, as described in an embodiment of the invention. Figure 2 A logic diagram for determining the driving status level of the driving subject in an embodiment of the invention; Figure 3 A logic decision diagram for identifying non-standard moments in an embodiment of the invention; Figure 4 This is a logic diagram for determining whether the driving subject meets the waiting state criteria in an embodiment of the invention. Detailed Implementation

[0023] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0024] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0025] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0026] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0027] Please see Figure 1 The diagram illustrates the steps of the AI-based large-scale model-based video image parsing method according to an embodiment of the present invention. The AI-based large-scale model-based video image parsing method according to an embodiment of the present invention includes: Step S1: Call the driving data of the driver to extract the driver's usual driving characteristics within a predetermined driving distance from the target observation point. The usual driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. Step S2: Combine the usual driving characteristics and the reaction time of changing driving state to evaluate the driving state characterization value of the driving subject and determine the driving state level of the driving subject. Step S3: Evaluate and analyze the driving status of the driving subject based on the driving status level, including identifying the abnormal time of the driving subject within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driving subject based on the driving waiting characteristics and corresponding head rotation amplitude of the driving subject within a predetermined time range from the abnormal time, so as to determine whether the driving subject meets the waiting status benchmark. Step S4: Determine the driver's attention deviation and the gap between the hands and the steering wheel to determine whether to issue a warning signal; The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

[0028] Specifically, the driving data includes habitual driving characteristics, reaction time to changes in driving status, driving waiting characteristics, and corresponding head rotation amplitude, focus deviation, and hand-to-steering wheel clearance, etc. The change in driving state refers to the state corresponding to the driving behavior of the driver responding to the trigger signals in the driving scenario, such as changes in the state of the vehicle in front or changes in traffic lights, and actively taking action to step on the brake or accelerator, thereby realizing the "transition of driving state from one stable mode to another stable mode".

[0029] Specifically, the purpose of setting up target observation points is to serve the assessment of driving status along traffic light routes, providing a reliable observational basis for accurately capturing the critical "waiting-starting" state transition scenario. Based on this, in implementation, target observation points can be set 50-100 meters before the stop line at traffic light intersections. This location can fully cover the entire process of vehicle queuing, traffic light switching response, and acceleration, ensuring effective collection of the driver's habitual driving characteristics in this scenario, such as the usual distance from the stop line and reaction time to changes in driving status, such as the reaction time to start from a green light. These details will not be elaborated further.

[0030] Understandably, the core observation scenario in implementation focuses on the complete process of a vehicle accelerating from a queue at a traffic light intersection. Therefore, the predetermined driving distance needs to ensure the integrity of data collection. From the perspective of scenario adaptability, 30-50 meters before the stop line is the core area for vehicle queuing, which can comprehensively cover the driving status of vehicles at different queuing positions. At the same time, this distance range can fully support the transition of a vehicle from a standstill to low-speed acceleration, accurately capturing the reaction time of changes in driving status, such as the delay from a standstill to starting after the green light turns on, avoiding the loss of key data due to too short a distance and the introduction of irrelevant driving interference due to too long a distance. Based on this, in this embodiment, the predetermined driving distance is selected within the range [30m, 50m], and 30m is preferred in implementation. This preferred value can not only meet the data collection needs of the core scenario, but also focus on the key observation range, reduce redundant data, and improve the efficiency of subsequent feature extraction and driving status assessment.

[0031] Specifically, the purpose of setting the predetermined time range is to capture the driving state continuously exhibited by the driver after the occurrence of an unusual moment, ensuring that the collected relevant feature data can accurately and comprehensively represent the driver's level of focus and driving operation, and avoiding feature fragmentation and judgment distortion caused by insufficient observation time. Based on this, in this embodiment, the predetermined time range is selected within the interval [5s, 8s], with 8s being the preferred option in practice, which will not be elaborated further.

[0032] Specifically, the process of evaluating the driving state representation values ​​of the driving subject includes: The sum of the ratio of the distance to the vehicle in front to the distance threshold and the ratio of the distance to the stop line when in the lead to the distance threshold is used as the first driving state feature. The ratio of the reaction time threshold to the reaction time for changing driving states is used as the second driving state feature. The first driving state feature and the second driving state feature are weighted and summed to obtain the driving state representation value.

[0033] Specifically, in real-world situations, reaction time to changes in driving status, such as braking response when the vehicle in front brakes suddenly or starting response when traffic lights change, directly determines whether an accident occurs or its severity. Furthermore, reaction time to changes in driving status is a dynamic response characteristic, directly reflecting the driver's level of focus, physical condition (fatigue / distraction), and emergency handling ability. It is a core human factor in driving safety and better reflects the true level of risk. Deviations in maintaining distance from the vehicle in front and distance from the stop line can be corrected through minor adjustments, such as appropriately increasing the following distance, thus providing a buffer against risk. Therefore, the second driving status characteristic calculated based on reaction time to changes in driving status is given a higher weighting coefficient, set to 0.6. Correspondingly, the weighting coefficient for the first driving status characteristic calculated based on habitual driving characteristics, namely maintaining distance from the vehicle in front and distance from the stop line when in the lead, is set to 0.4.

[0034] In this embodiment, the purpose of setting the maintaining distance threshold, distance threshold, and reaction time threshold is to characterize situations where the driving subject's operational stability is low. By acquiring several historical driving data of the driving subject, and calling historical data on maintaining distance from the vehicle in front within a predetermined driving distance from the target observation point, historical data on distance relative to the stop line when in the lead, and historical data on reaction time when changing driving state, the average maintaining distance, average distance, and average reaction time are calculated and used as the benchmark values ​​under normal conditions. Based on the purpose of setting the above three thresholds, the maintaining distance threshold is determined as the product of the average maintaining distance and a first offset coefficient, the distance threshold is determined as the product of the average distance and a second offset coefficient, and the reaction time threshold is determined as the product of the average reaction time and a time deviation coefficient. The first offset coefficient is selected within the interval [0.9, 0.95], preferably 0.9 in practice; the second offset coefficient is selected within the interval [0.85, 0.95], preferably 0.85 in practice; and the time deviation coefficient is selected within the interval [1.05, 1.1], preferably 1.05 in practice.

[0035] Specifically, this invention considers the individualized habitual behaviors of drivers to reduce the misjudgment rate of compliant driving or overlooking hidden risks. It establishes a baseline for individual driving habits, including quantifying the driver's dynamic safety redundancy by maintaining a safe distance from the vehicle in front. This distance directly reflects the risk buffer space when following another vehicle; the closer the distance is to the driver's habitual baseline, the stronger the driver's awareness of collision risk avoidance during dynamic driving and the more stable their operation. The distance relative to the stop line when in the lead quantifies the driver's level of safety compliance during static waiting. This distance reflects the driver's operational compliance in stationary waiting scenarios such as red lights; too close a distance may lead to insufficient reaction space when starting. By comparing with their own habitual baseline, the consistency of their safe operation in static scenarios can be accurately judged. Furthermore, this invention incorporates reaction time to changes in driving status, quantifying the driver's emergency response capability and level of focus. A shorter reaction time indicates a driver's faster perception and execution of signals indicating changes in driving conditions, such as a vehicle slowing down or traffic lights changing. This indirectly reflects the driver's focus and physical condition while driving, such as the absence of fatigue or distraction. Conversely, a long reaction time suggests insufficient emergency response capabilities and potential safety risks. Furthermore, reaction time is directly related to the driver's controllability; a faster reaction indicates greater controllability, while a delayed reaction increases the risk of loss of control. Therefore, this invention combines driving operation stability, safety redundancy control, and static waiting standardization dimensions to evaluate the driver's driving status characterization value, quantifying the driver's overall driving safety level and characterizing the driver's operational stability, providing data support for subsequently determining the driver's driving status level. This makes the driving status assessment more closely aligned with the driver's actual operating patterns, improving assessment accuracy and the targeted nature of risk identification.

[0036] Specifically, please refer to Figure 2 As shown, this is a logic diagram for determining the driving status level of the driver in an embodiment of the present invention. Determining the driving status level of the driver includes: If the driving status representation value of the driver is less than the driving status representation threshold, then the driving status level of the driver is determined to be a dangerous level. If the driving status representation value of the driving subject is greater than or equal to the driving status representation threshold, then the driving status level of the driving subject is determined as a safety level.

[0037] The driving state characterization threshold is predetermined. The driving state characterization value calculated under the following conditions is equal to the keeping distance from the vehicle in front, equal to the keeping distance threshold, equal to the distance threshold when in the lead position relative to the stop line, and equal to the reaction time threshold when changing driving state.

[0038] Specifically, the driving condition of the driver is evaluated and analyzed based on the driving status level, including: If the driving status of the driver is classified as dangerous, then the driving situation of the driver will be assessed and analyzed.

[0039] Specifically, please refer to Figure 3 As shown, it is a logic decision diagram for identifying unusual moments in an embodiment of the present invention. The process of identifying unusual moments of the driving subject within a predetermined driving distance from the target observation point includes: If it is detected that the driving entity does not meet the driving state baseline conditions, the corresponding moment will be identified as the abnormal moment. The driving state reference conditions include maintaining a distance from the vehicle in front that is less than a maintaining distance reference threshold, or being in the lead and the distance relative to the stop line that is less than a distance reference threshold.

[0040] In this embodiment, the purpose of setting a maintaining distance reference threshold and a distance reference threshold is to accurately identify abnormal deviations in the driver's current driving state compared to the normal state. Therefore, the maintaining distance reference threshold is determined based on the maintaining distance threshold, which is 0.95 times the maintaining distance threshold, and the distance reference threshold is determined based on the distance threshold, which is 0.9 times the distance threshold. This adapts to the safety requirements of following and waiting scenarios, ensuring the targeted and accurate identification of state differences.

[0041] Specifically, the process of calculating the waiting attention representation parameters of the driving subject includes: The sum of the ratio of the maximum gaze deviation angle to the maximum gaze deviation angle threshold and the ratio of the duration threshold to the duration of looking straight ahead is used as the first waiting focus feature; The ratio of head rotation amplitude to head rotation amplitude threshold is used as the second waiting focus feature; The first waiting focus feature and the second waiting focus feature are weighted and summed to obtain the waiting focus characterization parameter.

[0042] Specifically, the core of safety in waiting scenarios is being ready to respond to changes in the situation ahead, such as traffic light changes or the start of the vehicle in front. The key to this ability lies in whether the driver's gaze is focused on the driving-related area. The maximum angle of gaze deviation and the duration of forward gaze can quantify the degree of gaze deviation and the stability of gaze focus, respectively. Integrating these two features directly reflects whether the driver has locked their gaze on the road ahead, traffic lights, and other key areas, making it a core dimension for determining the driver's level of focus while waiting. However, head rotation is easily affected by driving posture and seat position; for example, a tilted driver posture may lead to misjudgment of rotation range, resulting in slightly weaker stability. Therefore, the first waiting focus feature, calculated based on the driving waiting characteristics—maximum angle of gaze deviation and duration of forward gaze—is given a higher weighting coefficient, set to 0.6. Correspondingly, the weighting coefficient for the second waiting focus feature, calculated based on the driver's head rotation range, is set to 0.4.

[0043] In this embodiment, the purpose of setting the maximum gaze deviation angle threshold, duration threshold, and head rotation amplitude threshold is to characterize the low level of driver focus during waiting scenarios. By acquiring several historical driving data points of the driver, historical data on the maximum gaze deviation angle, duration of forward gaze, and corresponding head rotation amplitude are retrieved from a predetermined time period starting from an unusual moment. The average maximum gaze deviation angle, average duration, and average head rotation amplitude are calculated and used as baseline values ​​under normal conditions. Based on the purpose of setting the above three thresholds, the... The maximum line-of-sight deviation angle threshold is determined as the product of the average maximum line-of-sight deviation angle and the angle deviation coefficient. The duration threshold is determined as the product of the average duration and the duration deviation coefficient. The head rotation amplitude threshold is determined as the product of the average head rotation amplitude and the rotation deviation coefficient. The angle deviation coefficient is selected within the interval [1.1, 1.2], preferably 1.1 in practice. The duration deviation coefficient is selected within the interval [0.85, 0.95], preferably 0.85 in practice. The rotation deviation coefficient is selected within the interval [1.2, 1.3], preferably 1.2 in practice.

[0044] Specifically, this invention identifies moments when a driver's behavior deviates from their usual baseline, quickly recognizing abnormal states and improving the detection rate of hidden risks. It focuses on the waiting process in potential risk scenarios such as being too close to other vehicles or too close to the stop line, assessing the driver's focus and operational compliance. The degree of head movement quantifies the reasonableness of the driver's visual focus. In reality, moderate head movement, such as observing the rearview mirror or side road conditions, is normal driving operation. Excessive movement indicates that the driver's attention has deviated from the core driving area ahead, potentially indicating distraction, such as looking at unrelated objects or excessive conversation with passengers. The maximum angle of visual deviation quantifies the degree of deviation in the driver's visual focus. A smaller angle indicates a more focused view of the road ahead or driving-related areas, such as the dashboard; a larger angle means the gaze has strayed from the core area of ​​focus, such as looking down at a phone or turning to look out the window, directly reflecting the severity of distraction and serving as a key indicator of inattentiveness. Furthermore, the duration of looking straight ahead quantifies the stability of the driver's attention. The longer the duration of the waiting period, the more stable the driver's focus is, indicating a more consistent visual focus on the road ahead. Conversely, a shorter duration suggests that the driver's attention is easily distracted, making it difficult to maintain focus on driving-related scenarios for an extended period, posing a potential safety risk, such as a rear-end collision due to failure to pay attention to the road ahead when starting. This invention achieves full-scenario safety coverage by calculating the driver's attention span parameters during waiting, quantifying the level of focus in waiting scenarios, and providing data support for subsequent determination of whether the driver meets the waiting state benchmark. This invention ensures accurate assessment of the driver's waiting state and closely reflects actual driving scenarios, improving the accuracy of risk identification and the targeted nature of interventions.

[0045] Specifically, please refer to Figure 4 As shown, this is a logic diagram for determining whether a driving entity meets the waiting state criteria according to an embodiment of the present invention. Determining whether the driving entity meets the waiting state criteria includes: If the waiting attention representation parameter of the driver is less than the waiting attention representation parameter threshold of the driver, then the driver is determined to meet the waiting state benchmark. If the waiting focus characterization parameter of the driver is greater than or equal to the waiting focus characterization parameter threshold of the driver, then the driver is determined to not meet the waiting state benchmark.

[0046] The threshold for the waiting focus characterization parameter is predetermined. It is determined by calculating the waiting focus characterization parameter when the maximum line of sight deviation angle is equal to the maximum line of sight deviation angle threshold, the duration threshold is equal to the duration of looking forward, and the head rotation amplitude is equal to the head rotation amplitude threshold.

[0047] Specifically, determining whether to issue a prompt signal includes: If the driver's attention deviation exceeds the attention deviation threshold, and the gap between the hands and the steering wheel exceeds the gap distance threshold, a warning signal will be issued.

[0048] In this embodiment, the purpose of setting the attention deviation threshold and the gap distance threshold is to characterize the situation where the driver's timely response to operations that change the driving state is low. By acquiring some historical driving data of the driver, calling the historical data of the driver's attention deviation and the historical data of the gap distance between the hand and the steering wheel, the average attention deviation and the average gap distance are calculated and used as the benchmark values ​​under normal conditions. Based on the purpose of setting the above two thresholds, the attention deviation threshold is determined as the product of the average attention deviation and the attention deviation coefficient, and the gap distance threshold is determined as the product of the average gap distance and the gap deviation coefficient. The attention deviation coefficient is selected in the interval [0.9, 0.95], preferably 0.9 in the implementation, and the gap deviation coefficient is selected in the interval [1.05, 1.1], preferably 1.05 in the implementation.

[0049] Specifically, this invention precisely targets high-risk scenarios, enabling on-demand triggering of safety warning signals while balancing driving safety and user experience. It distinguishes between normal driving behavior and high-risk behavior through dual criteria: the driver's level of focus and the distance between their hands and the steering wheel. Excessive driver focus means an inability to respond promptly to unexpected situations, such as changes in traffic lights or the start of a vehicle ahead. Conversely, hands leaving the steering wheel prevents quick execution of maneuvers; the combined effect of these two factors exponentially increases the risk of an accident. For example, if the driver's focus is off but their hands remain on the steering wheel (e.g., slight fatigue), or if their hands briefly leave the wheel but their focus is normal (e.g., adjusting the air conditioning), no warning is triggered. Warnings are only issued when a high-risk combination occurs: insufficient focus (e.g., distraction / fatigue) and hands leaving the steering wheel (e.g., loss of control). This avoids frequent invalid warnings due to a single condition and reduces interference with normal driver operation. This invention precisely intervenes in high-risk scenarios, improving the accuracy of risk identification and reducing invalid warnings.

[0050] Specifically, the process of determining the driver's attention deviation includes: Identify the duration of the driver's downward gaze and the extent of eye opening; The sum of the ratio of the duration of downward gaze to the threshold duration of downward gaze and the ratio of the threshold for eye opening amplitude to the eye opening amplitude is used as the focus deviation.

[0051] In this embodiment, the purpose of setting the threshold for the duration of downward gaze shift and the threshold for the amplitude of eye opening is to characterize situations where the driver's gaze deviates significantly from its focus. By acquiring several historical driving data of the driver, the historical data of the duration of downward gaze shift and the historical data of the amplitude of eye opening are called, and the average duration of downward gaze shift and the average amplitude of eye opening are calculated. These are then used as the baseline values ​​under normal circumstances. Based on the purpose of setting the above two thresholds, the threshold for the duration of downward gaze shift is determined to be the product of the average duration of downward gaze shift and the downward gaze shift deviation coefficient, and the threshold for the amplitude of eye opening is determined to be the product of the average amplitude of eye opening and the amplitude deviation coefficient. The downward gaze shift deviation coefficient is selected within the interval [1.05, 1.1], preferably 1.05 in practice, and the amplitude deviation coefficient is selected within the interval [1.1, 1.15], preferably 1.1 in practice.

[0052] Specifically, it also provides systems that apply video image analysis methods based on AI large models, including, The inertial analysis module is used to call the driving data of the driver to extract the inertial driving characteristics of the driver within a predetermined driving distance from the target observation point. The inertial driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. A driving analysis module, which is connected to the inertial analysis module, is used to combine the inertial driving characteristics and the reaction time of changing driving state to evaluate the driving state characterization value of the driving subject and determine the driving state level of the driving subject. The status assessment module, which is connected to the driving analysis module, is used to assess and analyze the driving situation of the driving subject based on the driving status level. This includes identifying abnormal times for the driving subject within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driving subject based on the driving waiting characteristics and corresponding head rotation amplitude of the driving subject within a predetermined time range from the abnormal time to determine whether the driving subject meets the waiting status benchmark. A focus alert module, which is connected to the state assessment module, is used to determine the driver's focus deviation and the gap distance between the hands and the steering wheel, so as to determine whether to issue a warning signal. The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

[0053] Specifically, there are no specific restrictions on the method of collecting and acquiring driving data. The forward millimeter-wave radar installed on the vehicle can be used to monitor the environment in front of the vehicle and determine the real-time distance to the vehicle in front, the distance relative to the stop line, etc. The vehicle can visually identify road conditions ahead and the timing of traffic light switching through the onboard camera located at the front of the vehicle. Combined with the onboard CAN bus, it can read the vehicle's own status data and identify the status of the brake pedal, such as pressing / releasing, the status of the accelerator pedal, and the vehicle's stationary / moving status. This allows it to determine the reaction time for changes in driving status, such as the time difference between "green light turning on - accelerator being pressed". In some possible implementations, the driver's facial and limb movements are captured by the vehicle's DMS camera, thereby determining characteristic data such as the driver's gaze direction, eye opening and closing, head rotation range, and the relative position of the hands and steering wheel, which will not be elaborated further.

[0054] Specifically, there are no restrictions on the specific structure of the routine analysis module, driving analysis module, status assessment module, and attention prompting module. Each module or its units can be composed of logic components or combinations of logic components. Logic components include field-programmable processors, computers, or microprocessors in computers.

[0055] Specifically, this invention leverages the multi-source data parallel processing capabilities of a large AI model, simultaneously integrating driving data and video image data, such as head, hand, and gaze data, to achieve real-time parallel computation of "feature extraction - parameter calculation - state determination." Furthermore, relying on the deep learning capabilities of the large AI model, it improves the accuracy of recognizing relevant subtle features through training with massive amounts of driving data. Consequently, it adapts to complex road conditions, covers more driving scenarios, meets real-time requirements, and enhances the recognition accuracy of relevant features.

[0056] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A video image parsing method based on a large AI model, characterized in that, include: The driving data of the driver is called to extract the driver's usual driving characteristics within a predetermined driving distance from the target observation point. The usual driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. By combining the usual driving characteristics and the reaction time to changes in driving status, the driving status characterization value of the driving subject is evaluated, and the driving status level of the driving subject is determined. The driving status of the driver is evaluated and analyzed based on the driving status level, including identifying the abnormal time of the driver within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driver based on the driving waiting characteristics and corresponding head rotation amplitude of the driver within a predetermined time period from the abnormal time, so as to determine whether the driver meets the waiting status benchmark. Determine the driver's level of focus deviation and the distance between their hands and the steering wheel to decide whether to issue a warning signal; The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

2. The video image parsing method based on an AI large model according to claim 1, characterized in that, The process of evaluating the driving state characterization values ​​of the driving subject includes: The sum of the ratio of the distance to the vehicle in front to the distance threshold and the ratio of the distance to the stop line when in the lead to the distance threshold is used as the first driving state feature. The ratio of the reaction time threshold to the reaction time for changing driving states is used as the second driving state feature. The first driving state feature and the second driving state feature are weighted and summed to obtain the driving state representation value.

3. The video image parsing method based on an AI large model according to claim 2, characterized in that, Determining the driving status level of the driving entity includes: If the driving status representation value of the driver is less than the driving status representation threshold, then the driving status level of the driver is determined to be a dangerous level. If the driving status representation value of the driving subject is greater than or equal to the driving status representation threshold, then the driving status level of the driving subject is determined as a safety level.

4. The video image parsing method based on an AI large model according to claim 3, characterized in that, The driving status of the driver is evaluated and analyzed based on the driving status level, including: If the driving status of the driver is classified as dangerous, then the driving situation of the driver will be assessed and analyzed.

5. The video image parsing method based on an AI large model according to claim 1, characterized in that, The process of identifying unusual moments when the driving entity is within a predetermined driving distance from the target observation point includes: If it is detected that the driving entity does not meet the driving state baseline conditions, the corresponding moment will be identified as the abnormal moment. The driving state reference conditions include maintaining a distance from the vehicle in front that is less than a maintaining distance reference threshold, or being in the lead and the distance relative to the stop line that is less than a distance reference threshold.

6. The video image parsing method based on an AI large model according to claim 1, characterized in that, The process of calculating the waiting attention characterization parameters of the driving subject includes: The sum of the ratio of the maximum gaze deviation angle to the maximum gaze deviation angle threshold and the ratio of the duration threshold to the duration of looking straight ahead is used as the first waiting focus feature; The ratio of head rotation amplitude to head rotation amplitude threshold is used as the second waiting focus feature; The first waiting focus feature and the second waiting focus feature are weighted and summed to obtain the waiting focus characterization parameter.

7. The video image parsing method based on an AI large model according to claim 6, characterized in that, Determining whether the driving entity meets the waiting state criteria includes: If the waiting focus representation parameter of the driver is less than the waiting focus representation parameter threshold of the driver, then the driver is determined to meet the waiting state benchmark.

8. The video image parsing method based on an AI large model according to claim 1, characterized in that, Determining whether to issue a prompt signal includes: If the driver's attention deviation exceeds the attention deviation threshold, and the gap between the hands and the steering wheel exceeds the gap distance threshold, a warning signal will be issued.

9. The video image parsing method based on an AI large model according to claim 1, characterized in that, The process of determining the driver's attention deviation includes: Identify the duration of the driver's downward gaze and the extent of eye opening; The sum of the ratio of the duration of downward gaze to the threshold duration of downward gaze and the ratio of the threshold for eye opening amplitude to the eye opening amplitude is used as the focus deviation.

10. A system for applying the video image parsing method based on an AI large model as described in any one of claims 1-9, characterized in that, include, The inertial analysis module is used to call the driving data of the driver to extract the inertial driving characteristics of the driver within a predetermined driving distance from the target observation point. The inertial driving characteristics include the distance maintained with the vehicle in front and the distance relative to the stop line when in the lead position. A driving analysis module, which is connected to the inertial analysis module, is used to combine the inertial driving characteristics and the reaction time of changing driving state to evaluate the driving state characterization value of the driving subject and determine the driving state level of the driving subject. The status assessment module, which is connected to the driving analysis module, is used to assess and analyze the driving situation of the driving subject based on the driving status level. This includes identifying abnormal times for the driving subject within a predetermined driving distance from the target observation point, and calculating the waiting focus characterization parameters of the driving subject based on the driving waiting characteristics and corresponding head rotation amplitude of the driving subject within a predetermined time range from the abnormal time to determine whether the driving subject meets the waiting status benchmark. A focus alert module, which is connected to the state assessment module, is used to determine the driver's focus deviation and the gap distance between the hands and the steering wheel, so as to determine whether to issue a warning signal. The driving waiting characteristics include the maximum angle of visual deviation and the duration of looking straight ahead.

Images