Traffic behavior visual detection method and system and readable storage medium

Abstract

The invention discloses a traffic behavior visual detection method and system and a readable storage medium. The method comprises the following steps: acquiring a monitoring image; inputting the monitoring image into a trained target detection model for feature extraction and feature screening so as to detect and obtain the category and the position of a violation target; generating a corresponding violation track based on the position of the violation target, and visualizing the violation track; and performing visual display and early warning based on the category of the violation target, and configuring a corresponding early warning scheme based on the identified category. According to the invention, the violation behavior can be detected based on the monitoring video, and early warning is performed based on the violation behavior.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to a traffic behavior visual detection method, system and readable storage medium. Background Art

[0002] In recent years, with the rapid development of the economy, the number of vehicles in cities has increased rapidly, and traffic problems have become an important issue in urban development. In order to meet people's travel needs, urban roads are expanded and upgraded, and new roads are built. The construction time is long and the manpower and material resources are huge. In addition, the upgrade of traffic control means can make full use of the road capacity under the existing road network and hardware conditions, with low construction cost and significant effect.

[0003] In the prior art, the vehicle congestion problem in the traffic flow is usually considered, and reasonable diversion is carried out to solve the traffic problem and resolve the traffic conflict to the greatest extent. However, the violation problem inside the vehicle is ignored, which poses certain safety hazards.

[0004] For example, the "traffic detection method and traffic detection device" disclosed in the Chinese patent literature has a publication number of CN104751644B and an application date of April 14, 2015. The traffic detection method of the invention includes: the traffic detection device divides each lane in the area to be processed into at least one unit, detects the vehicle passing information of each unit according to a preset period, calculates the unit congestion parameters of each unit within the sampling time according to the vehicle passing information and the preset period, calculates the connection incremental congestion parameters of the connection according to the unit congestion parameters and the number of lanes, and detects the traffic status of the area to be processed according to the connection incremental congestion parameters. However, there is a problem that it cannot detect illegal behaviors inside vehicles. Summary of the invention

[0005] The purpose of the present invention is to provide a traffic behavior visual detection method, system and readable storage medium, which can detect illegal behavior based on monitoring video and issue early warning based on the illegal behavior.

[0006] A first aspect of the present invention provides a method for visually detecting traffic behavior, comprising the following steps: Acquire monitoring images; Input the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; Generate a corresponding violation trajectory based on the location of the violation target, and visualize the violation trajectory; Visual display and warning are performed based on the categories of violation targets, wherein corresponding warning plans are configured based on the identified categories.

[0007] In this solution, the acquisition of monitoring images specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

[0008] In this solution, the monitoring image is input into a trained target detection model for feature extraction and feature screening to detect the category and location of the illegal target, specifically including: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

[0009] In this solution, the visual display and early warning based on the category of violation targets specifically include: Matching scheme levels based on the category and behavior of the violation target, wherein the scheme levels include at least a primary scheme, a secondary scheme, a tertiary scheme, and a quaternary scheme; Warnings are differentiated based on the plan level. The first-level plan warning method includes simultaneously enabling optical warnings, sound warnings, pop-up warnings and SMS warnings; the second-level plan warning method includes simultaneously enabling sound warnings, pop-up warnings and SMS warnings; the third-level plan warning method includes simultaneously enabling pop-up warnings and SMS warnings; the fourth-level plan warning method includes only enabling pop-up warnings.

[0010] In this solution, the method further includes training the target detection model, specifically including: Acquire a training data set, wherein the training data set includes violation categories and violation locations; Classifying the training data set into a training set, a validation set and a test set based on a preset ratio; The initial model is loaded, trained based on the training set, and each training result is verified using the validation set, and finally the model that meets the preset conditions is tested with the test set to be used as the target detection model.

[0011] In this solution, when the initial neural network model is trained using the training set, the method further includes preprocessing the images in the training data set, wherein the preprocessing method includes at least image translation, image scaling, and image distortion.

[0012] A second aspect of the present invention further provides a traffic behavior visual detection system, comprising a memory and a processor, wherein the memory comprises a traffic behavior visual detection method program, and when the traffic behavior visual detection method program is executed by the processor, the following steps are implemented: Acquire monitoring images; Input the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; Generate a corresponding violation trajectory based on the location of the violation target, and visualize the violation trajectory; Visual display and warning are performed based on the categories of violation targets, wherein corresponding warning plans are configured based on the identified categories.

[0013] In this solution, the acquisition of monitoring images specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

[0014] In this solution, the monitoring image is input into a trained target detection model for feature extraction and feature screening to detect the category and location of the illegal target, specifically including: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

[0015] In this solution, the visual display and early warning based on the category of violation targets specifically include: Matching scheme levels based on the category and behavior of the violation target, wherein the scheme levels include at least a primary scheme, a secondary scheme, a tertiary scheme, and a quaternary scheme; Warnings are differentiated based on the plan level. The first-level plan warning method includes simultaneously enabling optical warnings, sound warnings, pop-up warnings and SMS warnings; the second-level plan warning method includes simultaneously enabling sound warnings, pop-up warnings and SMS warnings; the third-level plan warning method includes simultaneously enabling pop-up warnings and SMS warnings; the fourth-level plan warning method includes only enabling pop-up warnings.

[0016] In this solution, the method further includes training the target detection model, specifically including: Acquire a training data set, wherein the training data set includes violation categories and violation locations; Classifying the training data set into a training set, a validation set and a test set based on a preset ratio; The initial model is loaded, trained based on the training set, and each training result is verified using the validation set, and finally the model that meets the preset conditions is tested with the test set to be used as the target detection model.

[0017] In this solution, when the initial neural network model is trained using the training set, the method further includes preprocessing the images in the training data set, wherein the preprocessing method includes at least image translation, image scaling, and image distortion.

[0018] The third aspect of the present invention provides a computer-readable storage medium, which includes a machine-readable traffic behavior visual detection method program. When the traffic behavior visual detection method program is executed by a processor, the steps of a traffic behavior visual detection method as described in any one of the above items are implemented.

[0019] The present invention discloses a traffic behavior visual detection method, system and readable storage medium, which can detect illegal behaviors based on monitoring videos and issue early warnings based on the illegal behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flow chart of a traffic behavior visual detection method of the present invention is shown; Figure 2 A block diagram of a traffic behavior visual detection system of the present invention is shown. DETAILED DESCRIPTION

[0021] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0022] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0023] Figure 1 A flow chart of a traffic behavior visual detection method of the present application is shown.

[0024] like Figure 1 As shown, the present application discloses a traffic behavior visual detection method, comprising the following steps: S102, acquiring a monitoring image; S104, inputting the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; S106, generating a corresponding violation trajectory based on the position of the violation target, and visualizing the violation trajectory; S108, performing visual display and warning based on the category of the violation target, wherein a corresponding warning plan is configured based on the identified category.

[0025] It should be noted that, in this embodiment, a monitoring image is first obtained, wherein the obtained monitoring image includes violations, and the violations include at least changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands off the steering wheel, speeding, etc., and then the monitoring image is input into a trained target detection model for feature extraction and feature screening to detect the category and position of the violation target accordingly. Accordingly, a corresponding violation trajectory is generated based on the position of the violation target, and the violation trajectory is visualized, as well as a visual display and warning based on the category of the violation target, wherein a corresponding warning scheme is configured based on the identified category, specifically, the category includes at least overloading, not wearing a helmet, not wearing a seat belt, etc. Corresponding warning schemes are configured for different categories, which will be described in subsequent instructions, so that targeted warnings can be made according to different schemes to reduce driving safety hazards.

[0026] According to an embodiment of the present invention, acquiring a monitoring image specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

[0027] It should be noted that, in this embodiment, the monitoring image is divided based on the frame number of the monitoring stream video, and feature extraction and feature screening are performed on the monitoring image to improve feature recognition, detect the moving target in the monitoring image, and if the moving target has any illegal behavior, it will be regarded as a illegal target, and the image with the corresponding frame number will be used as the monitoring image, wherein the illegal behavior recognition can be judged by the image target recognition model, and the use of the recognition model to match and identify the behavior in the image is not described in detail in this embodiment. For those skilled in the art, it is an achievable existing technology. This embodiment is only used as an application, and the purpose is to distinguish different moving targets in the monitoring stream video to determine whether there is an illegal behavior and obtain the required illegal target.

[0028] According to an embodiment of the present invention, the step of inputting the monitoring image into a trained target detection model for feature extraction and feature screening to detect the category and position of the illegal target specifically includes: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

[0029] It should be noted that, in this embodiment, the K-means algorithm is used as the clustering algorithm, and the K-means algorithm is used to cluster the sizes of the illegal targets in the monitoring image to obtain candidate frame parameters, and then based on the candidate frame parameter set, the monitoring image is subjected to feature extraction and feature screening using the trained target detection model to obtain the model output, thereby obtaining the category and position of the illegal target based on the model output, wherein the target detection model in this embodiment is a trained model, and the specific training steps will be described in the subsequent explanation, and in this embodiment, it is directly applied to obtain the corresponding model output, that is, the type and position of the illegal target are obtained.

[0030] According to an embodiment of the present invention, the visual display and early warning based on the category of the violation target specifically include: Matching scheme levels based on the category and behavior of the violation target, wherein the scheme levels include at least a primary scheme, a secondary scheme, a tertiary scheme, and a quaternary scheme; Warnings are differentiated based on the plan level. The first-level plan warning method includes simultaneously enabling optical warnings, sound warnings, pop-up warnings and SMS warnings; the second-level plan warning method includes simultaneously enabling sound warnings, pop-up warnings and SMS warnings; the third-level plan warning method includes simultaneously enabling pop-up warnings and SMS warnings; the fourth-level plan warning method includes only enabling pop-up warnings.

[0031] It should be noted that in this embodiment, since the application includes dual detection of violation behaviors and violation categories, when issuing early warnings for violation targets, scheme level matching and early warnings can be performed based on violation behaviors and violation categories. Specifically, the first-level scheme is when the category is two or more of overloading, not wearing a helmet, and not wearing a seat belt; the second-level scheme is when the category is one of overloading, not wearing a helmet, and not wearing a seat belt; the third-level scheme is when the category is not any of overloading, not wearing a helmet, and not wearing a seat belt, and the violation is two or more of crossing the solid line to change lanes, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, and speeding; the fourth-level scheme is when the category is not any of overloading, not wearing a helmet, and not wearing a seat belt, and the violation is one of crossing the solid line to change lanes, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, and speeding.

[0032] Furthermore, warnings are carried out based on warning schemes, specifically including: a first-level scheme, in which optical warnings, sound warnings, pop-up warnings and SMS warnings are enabled at the same time; a second-level scheme, in which sound warnings, pop-up warnings and SMS warnings are enabled at the same time; a third-level scheme, in which pop-up warnings and SMS warnings are enabled at the same time; a fourth-level scheme, in which only pop-up warnings are enabled; among which, the optical warning is to enable flashing lights for light flashing warnings, the sound warning is to enable loudspeakers for voice broadcasts, the pop-up warning is to analyze the computer pop-up categories and violations for text warnings, and the SMS warning is to send categories and violations to the mobile phones of relevant personnel via SMS for text warnings.

[0033] According to an embodiment of the present invention, the method further includes training the target detection model, specifically including: Acquire a training data set, wherein the training data set includes violation categories and violation locations; Classifying the training data set into a training set, a validation set and a test set based on a preset ratio; The initial model is loaded, trained based on the training set, and each training result is verified using the validation set, and finally the model that meets the preset conditions is tested with the test set to be used as the target detection model.

[0034] It should be noted that, in this embodiment, the acquired training data set can be a historically collected surveillance stream video or an image in a third-party database, and the training data set has corresponding annotation information, which specifically annotates information such as violation behavior, location, and size. The training data set is then classified based on a preset ratio to obtain a training set, a verification set, and a test set, wherein the preset ratio can be "7:2:1", and the initial model is loaded, the initial model is trained based on the training set, and the verification set is used to verify each training result, and finally the test set is used to test to obtain a model that meets the preset conditions as the target detection model.

[0035] According to an embodiment of the present invention, when the initial neural network model is trained using the training set, the method further includes preprocessing the images in the training data set, wherein the preprocessing method includes at least image translation, image scaling, and image distortion.

[0036] It should be noted that in this embodiment, for the images in the training data set, since they have been labeled in advance, preprocessing is required when applying them so that all images in the training data set meet the requirements of unified standardization. Therefore, at least image translation, image scaling and image distortion are required.

[0037] It is worth mentioning that the corresponding violation trajectory is generated based on the location of the violation target, including: Tracking the same traffic violation target that appears in the monitoring image in the monitoring video stream; Generate the corresponding violation track based on the position and size information at each frame.

[0038] It should be noted that, in this embodiment, when generating a judgment on whether a moving target is a traffic violation target, if so, the monitoring image is saved as a traffic violation image, and the traffic violation target is tracked, and the position and size information of the traffic violation target in the monitoring image is calculated, wherein the traffic violation behaviors of the traffic violation target include changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, speeding, etc.; the judgment condition for speeding is that the vehicle speed exceeds a certain threshold; the judgment condition for running a red light is that the car crosses the zebra crossing and does not stop immediately when the red light is detected; the judgment condition for not giving way to pedestrians is when the distance between the motor vehicle and the pedestrian is less than A certain threshold is reached, and pedestrians have a tendency to cross the road, and the car has no obvious deceleration or braking behavior; the judgment condition for changing lanes across the solid line is that the car changes lanes across the solid line in the solid line section; the judgment condition for the driver's hands leaving the steering wheel is that the driver's hands do not touch the car steering wheel within a certain time range. When a violation occurs, the violation behavior, occurrence time, subject snapshot photos, license plate number and other information are saved, and it is determined whether the position is between the initial position and the end position of the monitoring position. If so, go to the next frame of the monitoring image until the tracking is ended, thereby generating the corresponding violation trajectory based on the monitoring image of the violation target.

[0039] Figure 2 A block diagram of a traffic behavior visual detection system of the present invention is shown.

[0040] like Figure 2 As shown, the present invention discloses a traffic behavior visual detection system, including a memory and a processor, wherein the memory includes a traffic behavior visual detection method program, and when the traffic behavior visual detection method program is executed by the processor, the following steps are implemented: Acquire monitoring images; Input the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; Generate a corresponding violation trajectory based on the location of the violation target, and visualize the violation trajectory; Visual display and warning are performed based on the categories of violation targets, wherein corresponding warning plans are configured based on the identified categories.

[0041] It should be noted that, in this embodiment, a monitoring image is first obtained, wherein the obtained monitoring image includes violations, and the violations include at least changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands off the steering wheel, speeding, etc., and then the monitoring image is input into a trained target detection model for feature extraction and feature screening to detect the category and position of the violation target accordingly. Accordingly, a corresponding violation trajectory is generated based on the position of the violation target, and the violation trajectory is visualized, as well as a visual display and warning based on the category of the violation target, wherein a corresponding warning scheme is configured based on the identified category, specifically, the category includes at least overloading, not wearing a helmet, not wearing a seat belt, etc. Corresponding warning schemes are configured for different categories, which will be described in subsequent instructions, so that targeted warnings can be made according to different schemes to reduce driving safety hazards.

[0042] According to an embodiment of the present invention, acquiring a monitoring image specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

[0043] It should be noted that, in this embodiment, the monitoring image is divided based on the frame number of the monitoring stream video, and feature extraction and feature screening are performed on the monitoring image to improve feature recognition, detect the moving target in the monitoring image, and if the moving target has any illegal behavior, it will be regarded as a illegal target, and the image with the corresponding frame number will be used as the monitoring image, wherein the illegal behavior recognition can be judged by the image target recognition model, and the use of the recognition model to match and identify the behavior in the image is not described in detail in this embodiment. For those skilled in the art, it is an achievable existing technology. This embodiment is only used as an application, and the purpose is to distinguish different moving targets in the monitoring stream video to determine whether there is an illegal behavior and obtain the required illegal target.

[0044] According to an embodiment of the present invention, the step of inputting the monitoring image into a trained target detection model for feature extraction and feature screening to detect the category and position of the illegal target specifically includes: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

[0045] It should be noted that, in this embodiment, the K-means algorithm is used instead of the clustering algorithm. The K-means algorithm is used to cluster the sizes of the illegal targets in the monitoring image to obtain candidate frame parameters, and then the monitored image is subjected to feature extraction and feature screening based on the candidate frame parameter set using the trained target detection model to obtain the model output, thereby obtaining the category and position of the illegal target based on the model output. Among them, the target detection model in this embodiment is a trained model, and the specific training steps will be described in the subsequent explanation. In this embodiment, it is directly applied to obtain the corresponding model output, that is, the type and position of the illegal target are obtained.

[0046] According to an embodiment of the present invention, the visual display and early warning based on the category of the violation target specifically include: Matching scheme levels based on the category and behavior of the violation target, wherein the scheme levels include at least a primary scheme, a secondary scheme, a tertiary scheme, and a quaternary scheme; Warnings are differentiated based on the plan level. The first-level plan warning method includes simultaneously enabling optical warnings, sound warnings, pop-up warnings and SMS warnings; the second-level plan warning method includes simultaneously enabling sound warnings, pop-up warnings and SMS warnings; the third-level plan warning method includes simultaneously enabling pop-up warnings and SMS warnings; the fourth-level plan warning method includes only enabling pop-up warnings.

[0047] It should be noted that in this embodiment, since the application includes dual detection of violation behaviors and violation categories, when issuing early warnings for violation targets, scheme level matching and early warnings can be performed based on violation behaviors and violation types. Specifically, the first-level scheme is classified into two or more of overloading, not wearing a helmet, and not wearing a seat belt; the second-level scheme is classified into one of overloading, not wearing a helmet, and not wearing a seat belt; the third-level scheme is not classified into any of overloading, not wearing a helmet, and not wearing a seat belt, and the violation is two or more of changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, and speeding; the fourth-level scheme is not classified into any of overloading, not wearing a helmet, and not wearing a seat belt, and the violation is one of changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, and speeding.

[0048] Furthermore, warnings are carried out based on warning schemes, specifically including: a first-level scheme, in which optical warnings, sound warnings, pop-up warnings and SMS warnings are enabled at the same time; a second-level scheme, in which sound warnings, pop-up warnings and SMS warnings are enabled at the same time; a third-level scheme, in which pop-up warnings and SMS warnings are enabled at the same time; a fourth-level scheme, in which only pop-up warnings are enabled; among which, the optical warning is to enable flashing lights for light flashing warnings, the sound warning is to enable loudspeakers for voice broadcasts, the pop-up warning is to analyze the computer pop-up categories and violations for text warnings, and the SMS warning is to send categories and violations to the mobile phones of relevant personnel via SMS for text warnings.

[0049] According to an embodiment of the present invention, the method further includes training the target detection model, specifically including: Acquire a training data set, wherein the training data set includes violation categories and violation locations; Classifying the training data set into a training set, a validation set and a test set based on a preset ratio; The initial model is loaded, trained based on the training set, and each training result is verified using the validation set, and finally the model that meets the preset conditions is tested with the test set to be used as the target detection model.

[0050] It should be noted that, in this embodiment, the acquired training data set may be a historically collected surveillance stream video or an image in a third-party database, and the training data set has corresponding annotation information, which specifically annotates information such as violation behavior, location, and size, and then the training data set is classified based on a preset ratio to obtain a training set, a verification set, and a test set, wherein the preset ratio may be "7:2:1", and the initial model is loaded, the initial model is trained based on the training set, and each training result is verified using the verification set, and finally the test set is used for testing to obtain a model that meets the preset conditions as the target detection model, wherein the preset conditions correspond to testing the optimal model with the test set.

[0051] According to an embodiment of the present invention, when the initial neural network model is trained using the training set, the method further includes preprocessing the images in the training data set, wherein the preprocessing method includes at least image translation, image scaling, and image distortion.

[0052] It should be noted that in this embodiment, for the images in the training data set, since they have been labeled in advance, preprocessing is required when applying them so that all images in the training data set meet the requirements of unified standardization. Therefore, at least image translation, image scaling and image distortion are required.

[0053] It is worth mentioning that the corresponding violation trajectory is generated based on the location of the violation target, including: Tracking the same traffic violation target that appears in the monitoring image in the monitoring video stream; Generate the corresponding violation track based on the position and size information at each frame.

[0054] It should be noted that, in this embodiment, when generating a judgment on whether a moving target is a traffic violation target, if so, the monitoring image is saved as a traffic violation image, and the traffic violation target is tracked, and the position and size information of the traffic violation target in the monitoring image is calculated, wherein the traffic violation behaviors of the traffic violation target include changing lanes across a solid line, not giving way to pedestrians, running a red light, the driver's hands leaving the steering wheel, speeding, etc.; the judgment condition for speeding is that the vehicle speed exceeds a certain threshold; the judgment condition for running a red light is that the car crosses the zebra crossing and does not stop immediately when the red light is detected; the judgment condition for not giving way to pedestrians is when the distance between the motor vehicle and the pedestrian is less than A certain threshold is reached, and pedestrians have a tendency to cross the road, and the car has no obvious deceleration or braking behavior; the judgment condition for changing lanes across the solid line is that the car changes lanes across the solid line in the solid line section; the judgment condition for the driver's hands leaving the steering wheel is that the driver's hands do not touch the car steering wheel within a certain time range. When a violation occurs, the violation behavior, occurrence time, subject snapshot photos, license plate number and other information are saved, and it is determined whether the position is between the initial position and the end position of the monitoring position. If so, go to the next frame of the monitoring image until the tracking is ended, thereby generating the corresponding violation trajectory based on the monitoring image of the violation target.

[0055] A third aspect of the present invention provides a computer-readable storage medium, which includes a traffic behavior visual detection method program. When the traffic behavior visual detection method program is executed by a processor, the steps of a traffic behavior visual detection method as described in any one of the above items are implemented.

[0056] The present invention discloses a traffic behavior visual detection method, system and readable storage medium, which can detect illegal behaviors based on monitoring videos and issue early warnings based on illegal behaviors, greatly reducing driving safety hazards.

[0057] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0058] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0059] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0060] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0061] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A traffic behavior visual detection method, characterized in that: The following steps are involved: Acquire monitoring images; Input the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; Generate a corresponding violation trajectory based on the location of the violation target, and visualize the violation trajectory; Visual display and warning are performed based on the categories of violation targets, wherein corresponding warning plans are configured based on the identified categories.

2. A traffic behavior visual detection method according to claim 1, characterized in that: The acquiring of monitoring images specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

3. A traffic behavior visual detection method according to claim 2, characterized in that: The step of inputting the monitoring image into a trained target detection model to extract features and screen features to detect the category and location of the illegal target specifically includes: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

4. A traffic behavior visual detection method according to claim 3, characterized in that: The visual display and early warning based on the category of violation targets specifically include: Matching scheme levels based on the category and behavior of the violation target, wherein the scheme levels include at least a primary scheme, a secondary scheme, a tertiary scheme, and a quaternary scheme; Warnings are differentiated based on the plan level. The first-level plan warning method includes simultaneously enabling optical warnings, sound warnings, pop-up warnings and SMS warnings; the second-level plan warning method includes simultaneously enabling sound warnings, pop-up warnings and SMS warnings; the third-level plan warning method includes simultaneously enabling pop-up warnings and SMS warnings; the fourth-level plan warning method includes only enabling pop-up warnings.

5. A traffic behavior visual detection method according to claim 1, characterized in that: The method further includes training the target detection model, specifically comprising: Acquire a training data set, wherein the training data set includes violation categories and violation locations; Classifying the training data set into a training set, a validation set and a test set based on a preset ratio; The initial model is loaded, trained based on the training set, and each training result is verified using the validation set, and finally the model that meets the preset conditions is tested with the test set to be used as the target detection model.

6. A traffic behavior visual detection method according to claim 5, characterized in that: When the initial neural network model is trained using the training set, the method further includes preprocessing the images in the training data set, wherein the preprocessing method at least includes image translation, image scaling and image distortion.

7. A traffic behavior visual detection system, characterized in that: The invention comprises a memory and a processor, wherein the memory comprises a traffic behavior visual detection method program, and when the traffic behavior visual detection method program is executed by the processor, the following steps are implemented: Acquire monitoring images; Input the monitoring image into a trained target detection model to perform feature extraction and feature screening to detect the category and location of the illegal target; Generate a corresponding violation trajectory based on the location of the violation target, and visualize the violation trajectory; Visual display and warning are performed based on the categories of violation targets, wherein corresponding warning plans are configured based on the identified categories.

8. A traffic behavior visual detection system according to claim 7, characterized in that: The acquiring of monitoring images specifically includes: Identify monitoring locations; Monitoring moving targets in images based on an image acquisition device disposed at a monitoring position; Determine whether the moving target is a violation target based on the moving target, wherein if the moving target is a violation target, the corresponding image is used as the monitoring image, otherwise, it is not used as the monitoring image; Judging whether the moving target is a violation target based on the moving target specifically includes judging violation behavior, and judging whether the moving target is a violation target based on the judgment result of whether there is violation behavior.

9. A traffic behavior visual detection system according to claim 8, characterized in that: The step of inputting the monitoring image into a trained target detection model to extract features and screen features to detect the category and location of the illegal target specifically includes: Based on the preset clustering algorithm, the sizes of the illegal targets in the monitoring image are clustered to obtain the candidate frame parameters; Based on the candidate box parameter set, the monitoring image uses a trained target detection model to perform feature extraction and feature screening to obtain a model output; The category and location of the violation target are obtained based on the model output.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a traffic behavior visual detection method program, and when the traffic behavior visual detection method program is executed by a processor, the steps of a traffic behavior visual detection method as described in any one of claims 1 to 6 are implemented.

Images