Vehicle violation handling method and device, electronic equipment, vehicle and medium

Abstract

This invention relates to a method, apparatus, electronic device, vehicle, and medium for handling vehicle violations. The method includes real-time monitoring of the driver's state changes in a vehicle, including at least one of emotional state changes and physiological parameter changes; when the state change information meets a preset threshold at a target time, acquiring the vehicle's onboard video stream within a preset time window prior to the target time; performing local analysis on the onboard video stream to identify whether a target vehicle exists in the surrounding environment and whether the target vehicle is engaging in illegal driving behavior within the preset time window; if a target vehicle exists, constructing violation description information for the target vehicle, including at least one of the illegal behavior, the time of occurrence of the illegal behavior, and target vehicle identification information. This method can capture the violation description information of a vehicle in real time when a violation is detected, enabling rapid evidence collection of the violation.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle technology, specifically to a method, device, electronic equipment, vehicle, and medium for handling vehicle violations. Background Technology

[0002] In the field of intelligent vehicles, existing technologies typically use dashcams to record video data during vehicle operation. When a driver discovers a traffic violation or accident, they can use a mobile device such as a smartphone to download the video data from the dashcam and then replay it to pinpoint the moment the traffic violation or accident occurred, thereby capturing the entire video stream of the traffic violation or accident.

[0003] However, the current method still requires drivers to review the footage and search for evidence afterward, resulting in low response efficiency and difficulty in meeting the needs of actual driving scenarios in a timely manner. Summary of the Invention

[0004] One objective of this invention is to provide a method for handling vehicle violations, thereby solving the problem that in the prior art, vehicle violations require manual evidence collection by the user afterward, resulting in slow response speed and low evidence collection efficiency; a second objective is to provide a device for handling vehicle violations; a third objective is to provide an electronic device; a fourth objective is to provide a vehicle; and a fifth objective is to provide a computer-readable storage medium.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for handling vehicle violations includes:

[0007] Real-time monitoring of the driver's state changes in the vehicle, including at least one of emotional state changes and physiological parameter changes;

[0008] When the state change information meets the preset threshold judgment condition at the target time, the vehicle's in-vehicle video stream within the preset time window before the target time is obtained.

[0009] Perform local analysis on the in-vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle, and whether the target vehicle is engaging in illegal driving behavior within the preset time window;

[0010] If the target vehicle exists, then violation description information of the target vehicle is constructed, and the violation description information includes at least one of the following: violation behavior, time of occurrence of violation behavior, and target vehicle identification information.

[0011] Furthermore, before acquiring the vehicle's in-vehicle video stream within a preset time window prior to the target time when the state change information meets a preset threshold determination condition at the target time, the method further includes:

[0012] Based on the state change information, it is identified whether the driver's heart rate enters a preset heart rate threshold range within a preset time and whether the driver's emotions become abnormal, the abnormal emotions including at least one of anxiety, anger, fear and worry.

[0013] If the driver's heart rate enters the preset heart rate threshold range within the preset time and the driver's emotion changes to the abnormal emotion, then the state change information is determined to meet the preset threshold judgment condition.

[0014] The moment when the driver's heart rate enters the preset heart rate threshold range is selected as the target moment.

[0015] Furthermore, the step of performing local analysis on the in-vehicle video stream to identify whether a target vehicle exists in the environment surrounding the vehicle includes:

[0016] Extract at least one frame of the target image from the in-vehicle video stream;

[0017] Based on the at least one frame of the target image, other vehicles in the environment surrounding the vehicle whose distance from the vehicle is within a preset distance threshold range are identified.

[0018] Prioritize each other vehicle within the preset distance threshold range and determine the first priority of each other vehicle.

[0019] Based on the first priority, it is determined in turn whether each other vehicle is the target vehicle.

[0020] Furthermore, the step of prioritizing each other vehicle within the preset distance threshold range and determining the first priority of each other vehicle includes:

[0021] Get the timestamp of each frame of the target image;

[0022] Based on the time interval between the timestamp of each target image frame and the target time, and the distance between each other vehicle in each target image frame and the vehicle itself, the first priority of each other vehicle is determined.

[0023] Furthermore, after constructing the violation description information of the target vehicle, the method further includes:

[0024] Obtain the violation level corresponding to the violation of each target vehicle;

[0025] Based on the aforementioned violation level, a second priority is determined for each target vehicle;

[0026] Based on the second priority of each target vehicle, the violation description information of each target vehicle is fed back to the driver in a differentiated manner.

[0027] Furthermore, it also includes:

[0028] Obtain the feedback confirmation signal issued by the driver based on the feedback description information of the violation;

[0029] Based on the feedback confirmation signal issued by the driver, the final target vehicle is determined from among the target vehicles;

[0030] The violation description information of the final target vehicle is reported to the remote terminal connected to the vehicle.

[0031] Furthermore, it also includes:

[0032] Detect whether the vehicle is currently in a scenario where violations occur frequently;

[0033] If the vehicle is currently in a high-frequency violation scenario, then at least one monitoring target is determined in the high-frequency violation scenario according to the priority ranking rule, which includes at least one of the relative acceleration ranking rule and the relative distance ranking rule.

[0034] Predict the impending violation of the monitored target;

[0035] Based on the target violation, the vehicle's driving data, and the driver's status, determine the collision risk between the vehicle and the monitored target;

[0036] Based on the collision risk, a pre-processing strategy for violations is generated.

[0037] Furthermore, the step of generating a pre-processing strategy for violations based on the collision risk includes:

[0038] If the collision risk is greater than the first preset risk threshold and less than the second preset risk threshold, a warning and / or an avoidance operation plan will be issued to the driver.

[0039] If the collision risk is greater than or equal to the second preset risk threshold, the vehicle is controlled to perform an avoidance operation on the monitored target.

[0040] Furthermore, identifying at least one monitoring target in the scenario where the violation occurs frequently includes:

[0041] Detect whether there are frequently violating vehicles in the scenarios where the violations occur frequently;

[0042] If the high-frequency violation vehicles exist, then at least the high-frequency violation vehicles will be marked as the monitoring targets.

[0043] A vehicle violation processing device, comprising:

[0044] The monitoring module is used to monitor the driver's state changes in the vehicle in real time, including at least one of emotional state changes and physiological parameter changes.

[0045] The acquisition module is used to acquire the vehicle's in-vehicle video stream within a preset time window before the target time when the state change information meets the preset threshold judgment condition at the target time.

[0046] The analysis module is used to perform local analysis on the in-vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle, and the target vehicle is engaging in illegal driving behavior within the preset time window;

[0047] A construction module is used to construct violation description information for the target vehicle if the target vehicle exists. The violation description information includes at least one of the following: violation behavior, time of occurrence of violation behavior, and target vehicle identification information.

[0048] An electronic device includes: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the processor to perform the method described above.

[0049] A vehicle includes a vehicle body and an electronic device as described above disposed in the vehicle body, the electronic device being used to perform the method described above.

[0050] A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described method.

[0051] The beneficial effects of this invention are:

[0052] (1) The driver’s heart rate, emotional changes and vehicle speed fluctuations are detected in real time through the emotion monitoring module. When an abnormal emotional state is triggered, the local large model on the vehicle is dynamically started to identify the illegal vehicle. This minimizes resource consumption while accurately responding to the user’s emotional needs and improving the efficiency of violation identification and the effect of emotional soothing.

[0053] (2) By constructing a prediction model based on historical violation data, potential violations can be proactively analyzed when drivers approach high-risk areas. The warning strategy can be dynamically adjusted in conjunction with the user's emotional state to achieve closed-loop control from passive detection to proactive intervention, thereby reducing the risk of traffic accidents. Attached Figure Description

[0054] Figure 1 A flowchart illustrating the vehicle violation handling method provided in this application embodiment;

[0055] Figure 2 A flowchart of the target time determination method provided in the embodiments of this application;

[0056] Figure 3 This is a schematic diagram of the target vehicle determination process provided in an embodiment of this application;

[0057] Figure 4 A flowchart of a method for preprocessing violations provided in this application embodiment;

[0058] Figure 5 This application provides a flowchart of a violation detection process triggered based on driver status change information in an embodiment of the present application.

[0059] Figure 6 This is a structural diagram of a violation detection system triggered by abnormal emotions provided in an embodiment of this application;

[0060] Figure 7 A flowchart of high-frequency scene prediction provided in the embodiments of this application;

[0061] Figure 8 This is a structural diagram of the emotion abnormality prediction system provided in the embodiments of this application;

[0062] Figure 9 A schematic diagram of the vehicle violation processing device provided in this application;

[0063] Figure 10 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0064] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0065] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0066] Intelligent vehicles can be equipped with a Driver Monitoring System (DMS) or an In-cabin Monitoring System (IMS) to perceive the driver's status while the vehicle is in motion. Additionally, intelligent vehicles can be equipped with dashcams for recognizing the behavior of surrounding vehicles and collecting evidence of violations during driving.

[0067] In intelligent vehicles, traditional solutions for recognizing surrounding vehicle behavior and collecting evidence of violations mainly rely on dashcam video streams to detect violations such as crossing lane lines, changing lanes, and running red lights, and then provide evidence retrieval and reporting support afterward. However, in practical use, during the violation processing phase, drivers typically need to manually review the video, pinpoint the time point, and locate the target vehicle. This interactive process is cumbersome and can easily lead to missing valuable evidence.

[0068] Based on the above-mentioned technical problems, this application provides a method for handling vehicle violations. During vehicle operation, changes in the driver's state can be used as the basis for initiating video analysis. When the driver's state changes meet predetermined judgment conditions, the in-vehicle video stream from a period of time prior to that moment is retrieved for local identification. This allows the video analysis scope to converge within a preset time window, thereby reducing the consumption of local vehicle-side computing resources. After automatically identifying the target vehicle with the violation, the violation description information of the target vehicle can be constructed to quickly provide evidence of the violation to the driver, thus avoiding the tedious process of the driver manually locating and collecting evidence, and improving the response efficiency to violations.

[0069] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0070] Figure 1 This is a flowchart illustrating the vehicle violation processing method provided in an embodiment of this application. Figure 1 As shown, the method for handling vehicle violations can be achieved through the following steps:

[0071] Step 110: Monitor the driver's status changes in the vehicle in real time.

[0072] The status change information includes at least one of emotional state changes and physiological parameter changes.

[0073] Step 120: When the state change information meets the preset threshold judgment condition at the target time, acquire the vehicle's in-vehicle video stream within the preset time window before the target time.

[0074] Step 130: Perform local analysis on the vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle and whether the target vehicle is engaging in illegal driving behavior within a preset time window.

[0075] Step 140: If a target vehicle exists, construct the violation description information for the target vehicle.

[0076] The violation description information includes at least one of the following: the violation behavior, the time of the violation, and the identification information of the target vehicle.

[0077] The vehicle violation processing method provided in this application embodiment can complete the analysis on the vehicle side when the driver sees a target vehicle with a violation, and generate violation description information in the first time. Because the timing is accurate and the target is clear, it can maximize the use of the local computing resources on the vehicle side and reduce the dependence on cloud traffic resources.

[0078] In step 110 above, the driver's state change information is used to characterize whether the driver is experiencing any abnormal changes that could trigger subsequent video retrieval and analysis processes. Specifically, emotional state changes reflect whether the driver's emotional state changes during vehicle operation, and physiological parameter changes reflect whether parameters characterizing the driver's immediate physical and mental state change.

[0079] In this application, the executing entity can be an in-vehicle processing unit installed in the vehicle, which is communicatively connected to the in-vehicle monitoring device, the in-vehicle camera acquisition unit, and the interactive display unit.

[0080] In the specific implementation of step 110 above, the in-vehicle monitoring device can acquire changes in the driver's emotional state and / or physiological parameters. The vehicle processing unit generates state change information corresponding to the current time period based on the acquired information, which can be used for subsequent steps to make judgments and calls.

[0081] In addition, to ensure that the monitoring results can be directly used to trigger subsequent processing, the vehicle-mounted processing unit can establish a time correlation for the status change information.

[0082] Step 110 continuously generates time-correlated information on driver state changes inside the vehicle, so that subsequent video analysis no longer runs continuously on the entire driving video, but instead uses the abnormal driver state as the triggering basis, thereby establishing a linkage between driver perception and surrounding violation identification in the overall method.

[0083] In one possible embodiment, the state change information may consist only of changes in emotional state or only of changes in physiological parameters, as long as it can reflect a definite change in the driver's state relative to the baseline state during the current period.

[0084] In step 120 above, the target time is the time point when the state change information reaches the trigger condition. The preset threshold judgment condition is used to determine whether the current state change has reached the level that requires the initiation of violation behavior processing. The preset time window is used to limit the range of video data that needs to be backtracked and obtained. The vehicle video stream is used to provide image input for subsequent identification of the target vehicle and its illegal driving behavior.

[0085] In this embodiment, the vehicle-mounted processing unit continuously reads the state change information generated in step 110 above, and determines the target time based on whether the state change information reaches a preset threshold. Subsequently, the vehicle-mounted processing unit backtracks a preset time window based on the target time to obtain the in-vehicle video stream within the corresponding time interval.

[0086] After acquiring the in-vehicle video stream, the vehicle's processing unit writes the video segments to a local analysis cache for use in the next step.

[0087] In this embodiment, by extracting the vehicle video stream within a preset time window before the target time only when the state change reaches a preset threshold judgment condition, the analysis object is converged to the video segment adjacent to the time of the driver's state change, reducing the computing power and storage occupation required for continuous full analysis, and making subsequent identification more focused on road events that may cause driver state fluctuations.

[0088] In step 130 above, local analysis means that the violation identification operation is completed on the vehicle local processing platform. The target vehicle means the vehicle that has been identified as having committed a violation of driving behavior within a preset time window and needs to output further descriptive information. The violation of driving behavior means the specific behavior that violates traffic rules or road traffic regulations.

[0089] In this embodiment, the vehicle-mounted processing unit performs local analysis and processing on the video clips obtained in step 120 above to identify vehicles in the vehicle's surrounding environment and whether they exhibit any illegal driving behavior within a preset time window.

[0090] In the specific implementation of step 130 above, the vehicle processing unit can identify vehicle objects in the video based on the vehicle video stream, and determine whether there is a target vehicle committing illegal driving behavior based on the vehicle's behavior in the video.

[0091] If a target vehicle is detected, the corresponding target vehicle identification result and illegal driving behavior identification result will be output; if no target vehicle is detected, a non-detection result will be output.

[0092] This embodiment extracts in-vehicle video within a preset time window, concentrates the vehicle's analysis resources on this video stream, and completes the identification of surrounding vehicles and their illegal driving behaviors on the vehicle side. It can quickly determine whether there are illegal vehicles associated with changes in driver status without relying on the cloud.

[0093] In step 140 above, the violation description information is used to structure the recognition results, the violation behavior is used to indicate the specific violation action performed by the target vehicle, the violation occurrence time is used to record the corresponding time of the action in the video, and the target vehicle identification information is used to distinguish specific vehicle objects.

[0094] In this embodiment, the vehicle-mounted processing unit can write at least one of the following into the corresponding record: the violation behavior of the target vehicle, the time of occurrence of the violation behavior, and the identification information of the target vehicle, so as to form result information that can be stored, retrieved, or displayed.

[0095] In addition, in cases where there are multiple target vehicles, the vehicle-mounted processing unit can construct and distinguish the corresponding violation description information for each vehicle.

[0096] This application embodiment uses driver status change information as the basis for initiating vehicle video acquisition and violation identification, and restricts video analysis to a preset time window before the target time. It can perform local analysis around the key time period corresponding to abnormal driver status, and then structure and organize the identified violations, the time of occurrence of the violations, and the target vehicle identification information, thereby realizing the linkage between driver status monitoring and surrounding violation handling. This reduces the computing power, bandwidth, and storage consumption caused by continuous all-time analysis, and makes the violation clue location and evidence generation process more direct, thereby improving the response efficiency to the violation behavior of the target vehicle.

[0097] Furthermore, in order to more accurately pinpoint the time of the violation and thus ensure the capture of more accurate vehicle video streams, in some embodiments, Figure 2 A flowchart of the target time determination method provided in the embodiments of this application is shown below. Figure 2 As shown, it includes the following steps:

[0098] Step 210: Based on the status change information, identify whether the driver's heart rate enters the preset heart rate threshold range within a preset time and whether the driver's emotions become abnormal.

[0099] Abnormal emotions include at least one of anxiety, anger, fear, and worry.

[0100] Step 220: If the driver's heart rate enters the preset heart rate threshold range within a preset time and the driver's emotion changes to an abnormal emotion, then the state change information is determined to meet the preset threshold judgment condition.

[0101] Step 230: Select the moment when the driver's heart rate enters the preset heart rate threshold range as the target moment.

[0102] In this embodiment, the preset duration can be a relatively short period of time (e.g., 3 seconds). This is to account for the fact that in practice, if a driver sees a target vehicle engaging in illegal behavior while driving, the driver may experience abnormal emotional changes or a rapid increase in heart rate within a short period of time.

[0103] Among them, the driver's emotional changes can be obtained by analyzing the changes in the driver's facial expressions through IMS or DMS, while the driver's heart rate changes can be obtained by continuously collecting the time-series signal of the pixel brightness of the driver's facial region of interest through IMS or DMS and filtering to calculate the heart rate.

[0104] Alternatively, the vehicle's infotainment system can communicate with the driver's smart bracelet to obtain information about the driver's heart rate changes.

[0105] In this embodiment, the vehicle processing unit continuously receives the driver's heart rate data and compares the current heart rate with a preset heart rate threshold range. At the same time, it combines facial expressions, voice features, or posture changes to identify whether the driver's emotions have changed from stable to abnormal.

[0106] When the driver's heart rate is detected to enter the threshold range for the first time within a preset time period, and the emotion recognition result simultaneously indicates that the driver is in an abnormal emotion, the vehicle's processing unit will determine that the corresponding state change information meets the preset threshold judgment condition, and record the moment when the heart rate enters the threshold range as the target moment. This target moment can be used as the reference time point for subsequent playback of the in-vehicle video stream, and video clipping can be completed in conjunction with the preset time window before the target moment.

[0107] The preset heart rate threshold range represents the driver's heart rate range when facing a target vehicle engaging in illegal behavior, and can be obtained through calibration.

[0108] The target time determination method provided in this embodiment uses a linkage triggering relationship between heart rate changes and emotional changes. The target time corresponds to the key node where the driver's physiological abnormality occurs, so that the subsequent vehicle video stream can be analyzed around a more relevant time period. This can limit the video analysis scope to a preset time window before the target time, making the acquisition and analysis of vehicle video streams more targeted and reducing the amount of irrelevant video processing.

[0109] Furthermore, in practice, there may be multiple vehicles surrounding the target vehicle. In such cases, although the driver may experience changes in emotional state and physiological parameters, it is still necessary to identify each specific vehicle as the target vehicle. For example, Figure 3 This is a schematic diagram of the target vehicle determination process provided in the embodiments of this application, such as... Figure 3 As shown, it includes the following steps:

[0110] Step 310: Extract at least one frame of the target image from the vehicle video stream;

[0111] Step 320: Based on at least one frame of the target image, identify other vehicles in the surrounding environment of the vehicle whose distance from the vehicle is within a preset distance threshold range;

[0112] Step 330: Prioritize each other vehicle within the preset distance threshold range and determine the first priority of each other vehicle;

[0113] Step 340: Based on the first priority, determine whether each other vehicle is the target vehicle in turn.

[0114] In this embodiment, local analysis refers to the process performed independently by the vehicle's local processing unit, without the need for cloud-based analysis resources.

[0115] After locating the vehicle video stream within a preset time window from the continuous video stream acquired by the vehicle-mounted camera acquisition unit, if the total duration of the vehicle video stream is 10 seconds, a frame can be extracted from it every 0.5 seconds as the target image.

[0116] The image processing module in the vehicle can perform lane region segmentation, vehicle contour detection, and target box extraction on the target image. It can also combine monocular visual ranging, binocular disparity estimation, or perspective conversion based on calibration parameters to calculate the distance between each vehicle and the vehicle, thereby filtering out other vehicles within a preset distance threshold range.

[0117] For each other vehicle selected, a ranking value can be generated based on the distance from the vehicle's center point to its own reference point, the time offset from the target time, and the image clarity. Based on this, a first priority is determined, and then the matching of vehicle type features, license plate features, or motion trajectory features is performed in descending order of the first priority to determine whether the vehicle belongs to the target vehicle.

[0118] In some implementations, the timestamp of each frame of the target image can be obtained; based on the interval between the timestamp of each frame of the target image and the target time, and the distance between each other vehicle and the vehicle in each frame of the target image, the first priority of each other vehicle can be determined.

[0119] In this embodiment, the target time is assumed to be 10 seconds before the current time. If both target image A (timestamp is 9.5 seconds before the current time) and target image B (timestamp is 9 seconds before the current time) exist simultaneously, since the timestamp of target image A is closer to the target time, the priority of other vehicles in target image A can be set to be higher than the priority of other vehicles in target image B.

[0120] In addition, there may be multiple other vehicles in target vehicle A. In this case, the priority of each other vehicle in target vehicle A can be further determined based on the distance between each other vehicle and the target vehicle.

[0121] In a specific implementation of this embodiment, the vehicle-mounted camera acquisition unit in the vehicle can continuously output the target image sequence within a preset time window, and write the timestamp corresponding to the acquisition time for each frame of the target image.

[0122] The image processing module in the vehicle reads the timestamps of adjacent frames based on the target time, calculates the interval between each frame and the target time, and performs vehicle detection and lane scene analysis on each frame to obtain the pixel distance of each other vehicle relative to the vehicle. This distance is then converted into the actual distance by combining the camera calibration parameters, perspective mapping relationship, or depth estimation results.

[0123] Subsequently, the image processing module uses the interval duration and spacing corresponding to the timestamp as the basis for determining the first priority, and sorts other vehicles within the preset distance threshold range. Among them, other vehicles with shorter interval duration and smaller spacing have a higher first priority.

[0124] In this embodiment, the first priority result of each of the other vehicles can be written into the vehicle candidate set for subsequent target vehicle determination.

[0125] This implementation combines time and space factors for priority determination, enabling the vehicle system to process vehicles that are closer to the target time and closer to the vehicle itself first. This makes the selection of candidate vehicles more closely related to the event at the target time, reduces the amount of computation involving irrelevant vehicles, and improves the accuracy and efficiency of target vehicle positioning.

[0126] Returning to step 340 in the above embodiment, after determining the first priority of each other vehicle, the other vehicles can be judged in descending order of priority to determine whether they are the target vehicle. For example, if there is a first vehicle and a second vehicle in the target image A, and the first vehicle has a higher first priority, then it can be determined first whether the first vehicle is the target vehicle.

[0127] In this embodiment, when determining whether other vehicles are the target vehicles, the main determination is whether they have committed specific acts that violate traffic rules or road traffic regulations.

[0128] The target vehicle determination method provided in this application first captures the target image, then limits the distance threshold range and judges them one by one according to priority. This allows the local analysis to focus on vehicle objects more closely related to the vehicle itself, reducing the participation of irrelevant candidate vehicles and lowering the computational burden on the vehicle-mounted terminal. Since the recognition order is related to vehicle distance and time, the system can more quickly identify vehicles more likely to correspond to violations, improving the targeting and processing efficiency of target vehicle recognition.

[0129] In addition, in practice, there may be multiple target vehicles, and different target vehicles may have committed different violations. In some embodiments, the violation level corresponding to the violation of each target vehicle can be obtained; then, based on the violation level, the second priority of each target vehicle is determined; and according to the second priority of each target vehicle, the violation description information of each target vehicle is fed back to the driver in a differentiated manner.

[0130] In this embodiment, after constructing the violation description information of the target vehicle, the local violation assessment module in the vehicle can be called to determine the level of the violation behavior corresponding to each target vehicle. The local violation assessment module can pre-set a violation level mapping table to map behaviors such as crossing the line, sudden lane change, driving against the flow of traffic, and running a red light to different violation levels. It can also further correct the same violation category by combining the duration of the violation, the relative distance to the vehicle, and the impact of the target vehicle on the current driving risk.

[0131] Subsequently, the vehicle-mounted processing unit generates a second priority sequence based on the grade result and writes the second priority sequence into the feedback queue. The interactive display unit in the vehicle outputs the violation description information of each target vehicle to the driver's display screen, voice broadcast unit, or vehicle-mounted message center according to the second priority sequence, so as to realize the hierarchical prompts for violation information of different target vehicles.

[0132] In this embodiment, the differentiated feedback of the violation description information can be completed by the display control module in the vehicle. The display control module determines the information presentation position, presentation duration and presentation style according to the second priority.

[0133] For example, for target vehicles with higher second priority, they are displayed using a highlighted color, a fixed window, and an immediate prompt; for target vehicles with lower second priority, they are displayed using a normal color, a collapsed display, or a delayed presentation.

[0134] In addition, when multiple target vehicles meet the feedback conditions at the same time, the corresponding violation description information can be pushed in sequence according to the second priority, and a traceable record can be retained after a round of feedback is completed so that the driver can view it later.

[0135] This embodiment can directly map the severity of violations to the feedback priority, so that drivers receive vehicle information for more serious violations first, and form a hierarchical presentation effect in multi-target scenarios, thereby aligning violation prompts with the level of driving risk and improving the relevance and processing efficiency of violation description information.

[0136] In some embodiments, after the differentiated feedback of violation description information is completed and sent to the driver, the driver can determine whether it is necessary to send the violation description information of one or more target vehicles to the violation processing platform. This can save the driver's time and eliminate the need for the driver to manually review the video for evidence afterwards. The specific steps include the following:

[0137] Step 11: Obtain the feedback confirmation signal issued by the driver based on the feedback description of the violation;

[0138] Step 12: Based on the feedback confirmation signal issued by the driver, determine the final target vehicle among all target vehicles;

[0139] Step 13: Report the violation description information of the final target vehicle to the remote terminal connected to the vehicle.

[0140] In this embodiment, after the vehicle's interactive display unit presents the violation description information of each target vehicle to the driver in a differentiated manner, it receives the feedback confirmation signal issued by the driver through touch, button or voice, and transmits the feedback confirmation signal to the vehicle's local processing unit.

[0141] The vehicle's local processing unit analyzes the feedback confirmation signal to confirm the target vehicle number, image marker location, or corresponding violation record identifier selected by the driver, and thereby locks the final target vehicle among multiple target vehicles.

[0142] In this scenario, the driver can select multiple target vehicles as the final target vehicle. For example, in a real-world scenario, vehicle A illegally changes lanes, causing the driver to brake suddenly, and at the same time, vehicle B rear-ends the driver. Since both vehicles bear some responsibility in the traffic accident, the driver can choose both vehicle A and vehicle B as the final target vehicles.

[0143] Furthermore, the vehicle's local processing unit sends the violation description information corresponding to the final target vehicle to a remote terminal connected to the vehicle via the vehicle communication module. The vehicle communication module can establish a connection with the remote terminal using cellular communication or short-range communication links.

[0144] Among them, remote terminals can refer to relevant platforms for handling violations, such as traffic accident handling platforms.

[0145] This embodiment determines the final target vehicle through the driver's feedback confirmation signal, which ensures that the reported object is consistent with the driver's subjective confirmation. This further narrows down the multiple candidate vehicles that have been reported to a single target object, and then uploads its violation description information to the remote terminal. This makes the reported information more targeted and facilitates the remote terminal to centrally retain and process the violation event. It also avoids the driver having to manually collect evidence, reducing the complexity of evidence collection.

[0146] Furthermore, the violation description information reported by the driver after confirmation can be recorded, including the corresponding time, vehicle, and location. This allows for advance warnings or preventative measures to be taken when other drivers encounter areas with high violation rates. Specifically, in some embodiments, Figure 4 A flowchart of the violation preprocessing method provided in the embodiments of this application is shown below. Figure 4 As shown, it includes the following steps:

[0147] Step 410: If the vehicle is currently in a high-frequency violation scenario, then determine at least one monitoring target in the high-frequency violation scenario according to the priority ranking rules.

[0148] The priority sorting rules include at least one of the relative acceleration sorting rules and the relative distance sorting rules.

[0149] Step 420: Predict the target violation that is about to occur.

[0150] Step 430: Based on the target violation, the vehicle's driving data, and the driver's status, determine the collision risk between the vehicle and the monitored target.

[0151] Step 440: Generate a pre-processing strategy for violations based on collision risk.

[0152] In this embodiment, the vehicle's in-vehicle processing unit can continuously receive data from the positioning module, the vehicle posture module, and the camera acquisition module, and compare the vehicle's current position with the pre-stored high-frequency violation areas. When the current position falls into the corresponding area and meets the preset time or density conditions, it is determined that the vehicle has entered a high-frequency violation scenario.

[0153] In this process, after confirming that the vehicle has entered a scenario where the violation occurs frequently, the vehicle-mounted processing unit extracts candidate targets based on the vehicle detection results in the video frame, and generates a ranking result based on the relative acceleration of the candidate targets and the distance between the candidate targets and the vehicle.

[0154] The relative acceleration can be calculated from the target displacement changes in consecutive frames combined with the timestamp, and the spacing can be converted from the target box scale, calibration parameters, or depth estimation results.

[0155] After sorting the candidate targets according to the priority sorting rules, at least one candidate target whose sorting value meets the preset conditions is selected as the monitoring target, and a continuous tracking mark is established for it.

[0156] Subsequently, the vehicle-mounted processing unit inputs the trajectory changes, lane location, speed changes, and historical violation tags of the monitored target into the evaluation model. The evaluation model can be a machine learning-based classification model. The evaluation model outputs the violation that the monitored target is about to commit and the probability value of each violation. The violation with the highest probability value is selected as the target violation.

[0157] In step 420 above, the monitoring target can specifically refer to motor vehicles and non-motor vehicles (such as pedestrians or animals).

[0158] In step 430 above, the vehicle's driving data may include vehicle speed, acceleration, driving direction, and distance between the vehicle and the monitored target. Driver status can refer to the driver's emotions; for example, if the driver is tense or anxious, the vehicle speed may increase, increasing the risk of collision if the monitored target is about to illegally change lanes.

[0159] This allows for the generation of corresponding processing strategies based on different collision risk levels. For example, in some implementations, if the collision risk is greater than a first preset risk threshold but less than a second preset risk threshold, a warning and / or an evasive maneuver plan is issued to the driver; if the collision risk is greater than or equal to the second preset risk threshold, the vehicle is controlled to perform an evasive maneuver towards the monitored target.

[0160] In this embodiment, the first preset risk threshold and the second preset risk threshold are used to measure the probability of a collision between the monitored target and the vehicle. The higher the risk threshold, the more likely a collision is to occur between the vehicle and the monitored target. For example, the first preset risk threshold can be 90%, and the second preset risk threshold can be 99.9%.

[0161] In this specific implementation, if the likelihood of a collision is judged based on the driver's emotional state, and exceeds the first preset risk threshold of 90% (i.e., if the vehicle continues driving with the current driving data, the driver's emotional state makes it difficult to make a timely and correct judgment), the driver will be promptly alerted to the impending violation and corresponding actions (e.g., turning the steering wheel 30 degrees to the left and accelerating to avoid a vehicle violating the right-hand rule without changing lanes). If the risk of a collision is high, and exceeds the first preset risk threshold of 99.9%, the vehicle's intelligent driving system will be notified to execute the correct pre-planned avoidance operation (e.g., if a flock of sheep is approaching and the driver's heart rate is detected to rise rapidly to 160 while the vehicle speed remains unchanged, the intelligent driving system will be notified to perform a deceleration action).

[0162] Through the above embodiments and implementation methods, the vehicle can promptly lock onto key monitoring targets after entering high-frequency violation scenarios and formulate corresponding handling measures in advance based on collision risks, thereby narrowing the monitoring scope to high-risk targets and ensuring that subsequent analysis and handling match the risk level of the scenario, thus improving the pertinence of violation prediction and the timeliness of handling.

[0163] In addition, in some embodiments, it is possible to detect whether there are high-frequency violation vehicles in scenarios where violations occur frequently, and if high-frequency violation vehicles are found, then at least the high-frequency violation vehicles are marked as monitoring targets.

[0164] In this embodiment, the vehicle controller can combine image frames collected by the vehicle-mounted camera, roadside perception data, and historical violation cache data to count the number of violations by each vehicle in the preset area, and generate a frequency score based on the number of violations per unit time, the duration of the violation, and the repetition of the violation type.

[0165] When a vehicle's frequency score reaches a preset threshold, the system identifies it as a high-frequency violation vehicle. When multiple vehicles meet the criteria, the system can select one or more vehicles as monitoring targets based on their scores from highest to lowest, and write the corresponding vehicle identifier into the local monitoring list or the vehicle's storage unit.

[0166] Vehicle identification can be used to assign monitoring identifiers to vehicles, update vehicle status fields, bind monitoring priority parameters, or trigger continuous tracking tasks. After identification, the vehicle's trajectory, speed, and relative position are continuously recorded in subsequent analysis.

[0167] Additionally, before the vehicle enters the location and / or time period of a high-frequency violation, it can alert the driver to the presence of such violations ahead and to proceed with caution. When a license plate number associated with a high-frequency violation is detected, the driver is alerted that the vehicle has recently committed M violations and should proceed with caution.

[0168] This embodiment first identifies high-frequency violating vehicles and then directly converts them into monitoring targets. This establishes a correspondence between the determination of monitoring targets and high-risk vehicles in the scene, and the monitoring range can be narrowed to target vehicles that are more likely to engage in violations. This provides a stable input for subsequent probability prediction and violation preprocessing strategy generation, and reduces invalid analysis of irrelevant vehicles.

[0169] Figure 5 The flowchart for violation detection triggered by driver status change information provided in this application embodiment is as follows: Figure 5 As shown, it includes the following steps:

[0170] Step 510: When the driver's heart rate is detected to rise rapidly and their emotions become anxious, angry, fearful, or worried, notify the reporting module of the time when the heart rate begins to rise and start collecting information on whether there have been any recent vehicle violations; or when the driver initiates "I want to report a vehicle for violation" using voice, notify the reporting module and start collecting information on whether there have been any recent vehicle violations.

[0171] Step 520: Extract the video stream of the dashcam for about 10 seconds, and use the small model on the vehicle to extract the clearest picture at 0.5 seconds per frame. Send the picture folder and picture time to the large model.

[0172] Step 530: Transfer the image folder to the large model, call the large model to identify the vehicle's illegal operation, generate the corresponding license plate number and the corresponding illegal operation, record the violation time, and edit the video of the corresponding time period from the dashcam according to the violation time. Based on the description language generated by the large model, a pop-up window is recommended to be displayed to the driver.

[0173] After receiving the images, the large model notifies the reporting module to analyze them, collecting information on vehicles within 10 meters of the vehicle. These vehicles are then prioritized based on their distance from the vehicle and the trigger signal time. Vehicles closest to the vehicle with the closest trigger time (the target time mentioned above) have the highest priority. A violation is assessed from highest to lowest priority, categorized as dangerous (red), illegal (orange), or unintentional (blue) based on the degree of danger of the driving behavior. If no violating vehicle is detected, the information is sent to the reporting module. If the system is activated by emotion recognition, no action is taken; if activated by voice, a voice reply indicates no violating vehicle was detected. If only one vehicle is detected, the violation event, license plate number, violation behavior, and image are directly returned to the reporting module, along with the violation time. The image processing module then transmits the video of that time period to the reporting module.

[0174] The reporting module receives the video stream and displays the license plate, the location of the violation, and a thumbnail preview of the violation video through the recommendation module. If multiple vehicles are identified, they are sorted by priority, and the license plate numbers are displayed. The corresponding colors are displayed according to the degree of danger. The reporting module displays multiple license plate number modules for drivers to select. When a driver clicks on the corresponding license plate number, the corresponding violation information is displayed.

[0175] Step 540: After the driver selects to report and collect evidence with one click, if it is a single vehicle violation, the violation description information of that vehicle can be directly reported to the remote terminal. If multiple vehicles violate the rules, the driver can selectively report the violation description information of one or more vehicles.

[0176] The remote terminal can be a platform for handling violations, such as a traffic management platform.

[0177] Step 550: Monitor the driver's mood and heart rate changes again, and comfort the driver to alleviate abnormal emotions.

[0178] Furthermore, with improved model performance, this solution can also perform reverse model analysis based on successfully uploaded and approved violation scenarios by remote terminals, conduct scenario simulations, and provide contingency plans for corresponding scenarios. For example, when a vehicle on the right crosses a solid line to change lanes, the response could be to swerve to the left and slow down or brake.

[0179] Figure 6 This is a structural diagram of a violation detection system triggered by abnormal emotions provided in an embodiment of this application, as shown below. Figure 6 As shown, it includes a basic vehicle control system 61, an AI box 62, an intelligent driving module 63, a vehicle infotainment system 64, a driver monitoring system / in-vehicle occupant monitoring system 65, a four-way surround view / forward view camera 66, an in-vehicle audio system 67, an in-vehicle remote communication terminal 68 (Telematics Box, TBOX) and a cloud platform 69.

[0180] The driver monitoring system / occupant monitoring system connects to the vehicle's infotainment system via Universal Serial Bus (USB) or Controller Area Network (CAN), transmitting video data streams to the infotainment system for analysis using the Inter-Integrated Circuit (I2C) protocol. Four surround-view / forward-view cameras are connected to the infotainment system via bypass hardwired connections and are primarily responsible for recording video. The display screen mainly displays recommendation pop-ups and information on vehicles violating traffic rules. The in-vehicle audio system broadcasts text and voice messages to the driver. The TBOX is mainly used for networking between the infotainment system and the cloud server, enabling interaction between the driver and the traffic management platform. The AI ​​box is mainly responsible for running the large-scale vehicle model. The basic vehicle control system is mainly responsible for providing driving data. Additionally, a monitor on the steering wheel can accurately obtain the driver's heart rate, blood pressure, and blood oxygen saturation.

[0181] The voice processing module invokes the in-vehicle intelligent function interface through the driver's voice information, triggering the violation detection process. The emotional health module mainly uses IMS / DMS cameras to identify the driver's emotions, heart rate, and blood pressure, triggering the violation detection process and following up on the user's subsequent emotional urgency. The recommendation module mainly displays the detection results identified by the reporting module and provides an interactive pop-up window for the driver. The image processing module contains a small model responsible for generating a photo of the license plate number clearly visible within 10 seconds of the video stream at the trigger detection time and providing it to the large model. Based on the time point returned by the large model, it crops the video stream and sends it to the reporting module. The reporting module is responsible for displaying the specific video of the violating vehicle at a specific time, the violation behavior, the license plate number, the violation event, and the location of the violation. The algorithm model is responsible for receiving images transmitted from the image processing module, identifying the violating vehicle information based on the images, generating corresponding descriptions and time points, and returning them to the reporting module. The intelligent driving module mainly provides raw driving video stream data and actively controls the vehicle's driving avoidance operations.

[0182] For example, Figure 7 A flowchart of high-frequency scene prediction provided in the embodiments of this application is shown below. Figure 7 As shown, it includes the following steps:

[0183] Step 710: The large model identifies scenarios where violations occur frequently or identifies vehicles that violate frequent rules.

[0184] Step 720: After receiving the prediction start signal, the large model requests to drive the original video stream and analyzes the video stream.

[0185] Step 730: Prioritize the candidate targets, evaluate them from small to large relative acceleration and from far to near distance, identify high priority targets first, evaluate up to ten targets in parallel, and determine the monitoring targets.

[0186] Step 740: Based on the vehicle's driving data, determine the impact of potential violations by the monitored target on the vehicle's driving behavior.

[0187] Step 750: When the risk exceeds 90%, a pop-up warning will be displayed; if it exceeds 99%, the vehicle will be controlled to perform an avoidance maneuver against the monitored target.

[0188] in, Figure 8 This is a structural diagram of the emotion abnormality prediction system provided in the embodiments of this application, as shown below. Figure 8 As shown, the emotion abnormality prediction system includes an emotion module 81, a large model 82, a recommendation module 83, and a driving module 84. This system can be used to perform the above steps 710 to 750.

[0189] Figure 9 This is a structural diagram of the vehicle violation processing device provided in this application, such as... Figure 9 As shown, the vehicle violation processing device 90 provided in this embodiment includes:

[0190] The monitoring module 901 is used to monitor the driver's status changes in the vehicle in real time.

[0191] Among them, the information on changes in state includes at least one of changes in emotional state and changes in physiological parameters;

[0192] The acquisition module 902 is used to acquire the vehicle's in-vehicle video stream within a preset time window before the target time when the state change information meets the preset threshold judgment condition at the target time.

[0193] Analysis module 903 is used to perform local analysis on the vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle and whether the target vehicle has illegal driving behavior within a preset time window.

[0194] Module 904 is used to construct violation description information for the target vehicle if the target vehicle exists.

[0195] The violation description information includes at least one of the following: the violation behavior, the time of the violation, and the identification information of the target vehicle.

[0196] Optionally, the vehicle violation processing device also includes a time determination module for:

[0197] Based on the status change information, identify whether the driver's heart rate enters the preset heart rate threshold range within a preset time and whether the driver's emotions become abnormal. Abnormal emotions include at least one of anxiety, anger, fear and worry.

[0198] If the driver's heart rate enters the preset heart rate threshold range within a preset time and the driver's emotions become abnormal, then the state change information is determined to meet the preset threshold judgment condition.

[0199] The moment when the driver's heart rate enters the preset heart rate threshold range is selected as the target moment.

[0200] Optionally, the analysis module can be specifically used for:

[0201] Extract at least one frame of the target image from the in-vehicle video stream;

[0202] Based on at least one frame of target image, identify other vehicles in the surrounding environment that are within a preset distance threshold range from the vehicle.

[0203] Prioritize each other vehicle within a preset distance threshold range and determine the first priority of each other vehicle.

[0204] Based on the first priority, determine in turn whether each other vehicle is the target vehicle.

[0205] Optionally, the analysis module can be used to: obtain the timestamp of each frame of the target image; and determine the first priority of each other vehicle based on the interval between the timestamp of each frame of the target image and the target time, as well as the distance between each other vehicle and the vehicle in each frame of the target image.

[0206] Optionally, the vehicle violation processing device further includes a feedback module, used to: obtain the violation level corresponding to the violation of each target vehicle; determine the second priority of each target vehicle based on the violation level; and provide differentiated feedback of the violation description information of each target vehicle to the driver according to the second priority of each target vehicle.

[0207] Optionally, the vehicle violation processing device also includes a reporting module, used to obtain a feedback confirmation signal issued by the driver based on the feedback violation description information; determine the final target vehicle among the target vehicles according to the feedback confirmation signal issued by the driver; and report the violation description information of the final target vehicle to a remote terminal connected to the vehicle.

[0208] Optionally, the vehicle violation processing device also includes a strategy generation module for:

[0209] Detect whether the vehicle is currently in a scenario where violations frequently occur;

[0210] If the vehicle is currently in a scenario where violations occur frequently, then at least one monitoring target is identified in the scenario where violations occur frequently, according to the priority ranking rules. The priority ranking rules include at least one of the relative acceleration ranking rules and the relative distance ranking rules.

[0211] Predict and monitor impending violations by the target entity;

[0212] Based on the target's violation, the vehicle's driving data, and the driver's status, determine the collision risk between the vehicle and the monitored target;

[0213] Based on collision risk, generate a pre-processing strategy for violations.

[0214] Optionally, the policy generation module is also used for:

[0215] If the collision risk is greater than the first preset risk threshold but less than the second preset risk threshold, a warning and / or an evasive action plan will be issued to the driver.

[0216] If the collision risk is greater than or equal to the second preset risk threshold, the vehicle will be controlled to perform an avoidance maneuver against the monitored target.

[0217] Optionally, the policy generation module is also used to: detect whether there are high-frequency violation vehicles in scenarios where violations occur frequently; if there are high-frequency violation vehicles, then at least mark the high-frequency violation vehicles as monitoring targets.

[0218] Figure 10 A schematic diagram of the structure of the electronic device provided in this application. Figure 10 As shown, the electronic device 1000 provided in this embodiment includes at least one processor 1001 and a memory 1002. Optionally, the electronic device 1000 further includes a communication component 1003. The processor 1001, the memory 1002, and the communication component 1003 are connected via a bus.

[0219] In a specific implementation, at least one processor 1001 executes computer execution instructions stored in memory 1002, causing at least one processor 1001 to perform the above-described method.

[0220] The specific implementation process of processor 1001 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0221] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0222] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0223] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0224] This application also provides a vehicle, which includes a vehicle body and an electronic device disposed in the vehicle body. The electronic device may be the cockpit domain controller mentioned above, used to perform the steps of the above method.

[0225] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0226] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0227] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0228] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0229] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0230] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0231] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0232] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0233] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0234] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

[0235] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A method for handling vehicle violations, characterized in that, include: Real-time monitoring of the driver's state changes in the vehicle, including at least one of emotional state changes and physiological parameter changes; When the state change information meets the preset threshold judgment condition at the target time, the vehicle's in-vehicle video stream within the preset time window before the target time is obtained. Perform local analysis on the in-vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle, and whether the target vehicle is engaging in illegal driving behavior within the preset time window; If the target vehicle exists, then violation description information of the target vehicle is constructed, and the violation description information includes at least one of the following: violation behavior, time of occurrence of violation behavior, and target vehicle identification information.

2. The method according to claim 1, characterized in that, When the state change information meets a preset threshold judgment condition at a target time, before acquiring the vehicle's in-vehicle video stream within a preset time window before the target time, the method further includes: Based on the state change information, it is identified whether the driver's heart rate enters a preset heart rate threshold range within a preset time and whether the driver's emotions become abnormal, the abnormal emotions including at least one of anxiety, anger, fear and worry. If the driver's heart rate enters the preset heart rate threshold range within the preset time and the driver's emotion changes to the abnormal emotion, then the state change information is determined to meet the preset threshold judgment condition. The moment when the driver's heart rate enters the preset heart rate threshold range is selected as the target moment.

3. The method according to claim 1, characterized in that, The step of performing local analysis on the in-vehicle video stream to identify whether a target vehicle exists in the environment surrounding the vehicle includes: Extract at least one frame of the target image from the in-vehicle video stream; Based on the at least one frame of the target image, other vehicles in the environment surrounding the vehicle whose distance from the vehicle is within a preset distance threshold range are identified. Prioritize each other vehicle within the preset distance threshold range and determine the first priority of each other vehicle. Based on the first priority, it is determined in turn whether each other vehicle is the target vehicle.

4. The method according to claim 3, characterized in that, The step of prioritizing each other vehicle within the preset distance threshold range and determining the first priority of each other vehicle includes: Obtain the timestamp of each frame of the target image; Based on the time interval between the timestamp of each target image frame and the target time, and the distance between each other vehicle in each target image frame and the vehicle itself, the first priority of each other vehicle is determined.

5. The method according to any one of claims 1-4, characterized in that, After constructing the violation description information of the target vehicle, the method further includes: Obtain the violation level corresponding to the violation of each target vehicle; Based on the aforementioned violation level, a second priority is determined for each target vehicle; Based on the second priority of each target vehicle, the violation description information of each target vehicle is fed back to the driver in a differentiated manner.

6. The method according to claim 5, characterized in that, Also includes: Obtain the feedback confirmation signal issued by the driver based on the feedback description information of the violation; Based on the feedback confirmation signal issued by the driver, the final target vehicle is determined from among the target vehicles; The violation description information of the final target vehicle is reported to the remote terminal connected to the vehicle.

7. The method according to claim 1, characterized in that, Also includes: Detect whether the vehicle is currently in a scenario where violations occur frequently; If the vehicle is currently in a high-frequency violation scenario, then at least one monitoring target is determined in the high-frequency violation scenario according to the priority ranking rule, and the priority ranking rule includes at least one of the relative acceleration ranking rule and the relative distance ranking rule; Predict the impending violation of the monitored target; Based on the target violation, the vehicle's driving data, and the driver's status, determine the collision risk between the vehicle and the monitored target; Based on the collision risk, a pre-processing strategy for violations is generated.

8. The method according to claim 7, characterized in that, The step of generating a pre-processing strategy for violations based on the collision risk includes: If the collision risk is greater than the first preset risk threshold and less than the second preset risk threshold, a warning and / or an avoidance operation plan will be issued to the driver. If the collision risk is greater than or equal to the second preset risk threshold, the vehicle is controlled to perform an avoidance operation on the monitored target.

9. The method according to claim 7, characterized in that, The step of identifying at least one monitoring target in the scenario where the violation occurs frequently includes: Detect whether there are frequently violating vehicles in the scenarios where the violations occur frequently; If the high-frequency violation vehicles exist, then at least the high-frequency violation vehicles will be marked as the monitoring targets.

10. A device for processing vehicle violations, characterized in that, include: The monitoring module is used to monitor the driver's state changes in the vehicle in real time, including at least one of emotional state changes and physiological parameter changes. The acquisition module is used to acquire the vehicle's in-vehicle video stream within a preset time window before the target time when the state change information meets the preset threshold judgment condition at the target time. The analysis module is used to perform local analysis on the in-vehicle video stream to identify whether there is a target vehicle in the environment around the vehicle, and the target vehicle is engaging in illegal driving behavior within the preset time window; A construction module is used to construct violation description information for the target vehicle if the target vehicle exists. The violation description information includes at least one of the following: violation behavior, time of occurrence of violation behavior, and target vehicle identification information.

11. An electronic device, characterized in that, include: Memory and processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-9.

12. A vehicle, characterized in that, The device includes a vehicle body and an electronic device disposed therein as described in claim 11, the electronic device being used to perform the method as described in any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-9.

Images