A method and device for auditing illegal activities and a storage medium

Abstract

The application discloses a kind of methods, device and storage medium of violating law, method includes: obtaining the image of violating law scene of violating law vehicle, and the position of violating law vehicle is positioned from violating law scene image, determine as first position;In the case where the violation category of violating law vehicle is preset category, utilize preset special vehicle detection model to carry out special vehicle detection to violating law scene image;In the case where special vehicle is detected from violating law scene image, the position of special vehicle is positioned from violating law scene image, determine as second position;In the case where the relative relationship of first position and second position meets preset condition, utilize preset vehicle state detection model and detect the working state of special vehicle from violating law scene image;In the case where working state is execution task, determine the reason of violating law vehicle for avoiding special vehicle for violating law. Through the above technical scheme, the efficiency and intelligence of violating law audit are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and storage medium for reviewing violations. Background Technology

[0002] Currently, with the increasing number of civilian cars, there are more and more vehicles participating in traffic, and consequently, more and more cases and incidents of traffic violations. However, the current method of processing a large number of traffic violation information by simply using corresponding detection devices or equipment can only determine the type of violation, not the specific scenario. As a result, violations that occur under special circumstances require the parties involved to go to the traffic police department to file a complaint after receiving the violation notice, which is time-consuming, labor-intensive, and lacks intelligence. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of the present invention aim to provide a method, apparatus, and storage medium for reviewing traffic violations. By detecting special vehicles and their operating status, the method determines whether a traffic violation was caused by the vehicle trying to avoid a special vehicle, thereby improving the efficiency and intelligence of traffic violation review.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a method for reviewing violations, the method comprising:

[0006] Obtain the violation scene image of the vehicle, and locate the position of the vehicle from the violation scene image, which is determined as the first position;

[0007] When the violation category of the offending vehicle is a preset category, a preset special vehicle detection model is used to detect special vehicles in the violation scene image;

[0008] If the special vehicle is detected from the violation scene image, the position of the special vehicle is located from the violation scene image and determined as the second position;

[0009] When the relative relationship between the first position and the second position meets the preset conditions, the working status of the special vehicle is detected from the violation scene image using a preset vehicle state detection model.

[0010] When the working state is performing a task, the reason for the violation by the offending vehicle is determined to be avoiding the special vehicle.

[0011] In the above method, the preset vehicle state detection model includes a preset indicator light detection model and a preset indicator light state discrimination model. The step of using the preset vehicle state detection model to detect the working state of the special vehicle from the violation scene image includes:

[0012] The vehicle image containing the special vehicle is cropped from the image of the traffic violation scene;

[0013] Using the preset indicator light detection model, the indicator lights of the specific vehicle are detected in the vehicle image;

[0014] If the indicator light is detected from the vehicle image, the operating status of the indicator light is detected from the vehicle image using the preset indicator light status discrimination model;

[0015] If the running status is flashing, the working status is determined to be executing a task.

[0016] In the above method, after detecting the operating state of the indicator light from the vehicle image using the preset indicator light state discrimination model, the method further includes:

[0017] If the running state is off, the working state is determined to be "no task executed".

[0018] In the above method, after detecting the working state of the special vehicle from the violation scene image using a preset vehicle state detection model, the method further includes:

[0019] If the working state is "not performing a task", the reason for the violation by the offending vehicle is determined to be "failure to yield to the special vehicle".

[0020] In the above method, after determining that the reason for the traffic violation by the offending vehicle is to avoid the special vehicle, the method further includes:

[0021] The traffic violation record corresponding to the aforementioned vehicle will be revoked.

[0022] In the above method, before performing special vehicle detection on the violation scene image using a preset special vehicle detection model, the method further includes:

[0023] Acquire scene sample images and use the special vehicle detection model to be trained to determine the probability information that the scene sample images contain special vehicles based on the scene sample images;

[0024] The first loss information is obtained by calculating the loss information between the probability information of the scene sample image containing a special vehicle and the first probability information preset for the scene sample image;

[0025] Based on the first loss information, the model parameters of the special vehicle detection model to be trained are adjusted to obtain the preset special vehicle detection model.

[0026] In the above method, before detecting the working state of the special vehicle from the violation scene image using a preset vehicle state detection model, the method further includes:

[0027] Acquire vehicle sample images and, using the vehicle state detection model to be trained, determine the probability information of indicator lights operating in the vehicle sample images based on the vehicle sample images;

[0028] The second loss information is obtained by calculating the loss information between the probability information of the indicator lights operating in the vehicle sample image and the second probability information preset for the vehicle sample image;

[0029] Based on the second loss information, the model parameters of the vehicle state detection model to be trained are adjusted to obtain the preset vehicle state detection model.

[0030] This invention provides a violation review device, comprising:

[0031] The acquisition module is used to acquire the violation scene image of the vehicle, locate the position of the vehicle from the violation scene image, and determine it as the first position;

[0032] The detection module is used to detect special vehicles in the violation scene image using a preset special vehicle detection model when the violation category of the vehicle is a preset category.

[0033] The positioning module is used to locate the position of the special vehicle from the violation scene image when the special vehicle is detected from the violation scene image, and determine it as a second position;

[0034] The processing module is used to detect the working status of the special vehicle from the violation scene image using a preset vehicle status detection model when the relative relationship between the first position and the second position meets preset conditions.

[0035] The determination module is used to determine, when the working state is performing a task, the reason for the violation of the vehicle is to avoid the special vehicle.

[0036] In the above-described device, the processing module is specifically used to crop out a vehicle image containing the special vehicle from the violation scene image; to detect the indicator lights of the special vehicle in the vehicle image using the preset indicator light detection model; and, if the indicator lights are detected in the vehicle image, to detect the operating status of the indicator lights in the vehicle image using the preset indicator light status discrimination model.

[0037] In the above-described device, the processing module is further configured to determine that the working state is "no task executed" when the operating state is "off".

[0038] In the aforementioned device, the determining module is further configured to determine, when the working state is "not performing a task," that the reason for the violation by the offending vehicle is "not yielding to the special vehicle."

[0039] In the aforementioned device, the processing module is also used to revoke the violation record corresponding to the violating vehicle.

[0040] The aforementioned device further includes a model training module, used to acquire scene sample images and, using a special vehicle detection model to be trained, determine the probability information that the scene sample images contain special vehicles based on the scene sample images; calculate the loss information between the probability information that the scene sample images contain special vehicles and a first probability information preset for the scene sample images to obtain first loss information; and, based on the first loss information, adjust the model parameters of the special vehicle detection model to be trained to obtain the preset special vehicle detection model.

[0041] In the above-mentioned device, the model training module is further configured to acquire vehicle sample images, and using the vehicle state detection model to be trained, determine the probability information of the indicator lights operating in the vehicle sample images based on the vehicle sample images; calculate the loss information between the probability information of the indicator lights operating in the vehicle sample images and the second probability information preset for the vehicle sample images to obtain the second loss information; and adjust the model parameters of the vehicle state detection model to be trained based on the second loss information to obtain the preset vehicle state detection model.

[0042] This invention provides a violation review device, comprising: a processor, a memory, and a communication bus;

[0043] The communication bus is used to realize the communication connection between the processor and the memory;

[0044] The processor is used to execute the violation review program stored in the memory to implement the above-mentioned violation review method.

[0045] The present invention provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the above-described violation review method.

[0046] This invention provides a method, apparatus, and storage medium for traffic violation review. The method includes: acquiring a traffic violation scene image of a vehicle, and locating the position of the vehicle from the scene image as a first position; if the violation category of the vehicle is a preset category, using a preset special vehicle detection model to detect special vehicles in the scene image; if a special vehicle is detected in the scene image, locating the position of the special vehicle from the scene image as a second position; if the relative relationship between the first and second positions satisfies preset conditions, using a preset vehicle state detection model to detect the working state of the special vehicle from the scene image; if the working state is performing a task, determining that the violation was caused by avoiding a special vehicle. The technical solution provided by this invention, by detecting special vehicles and their working states, determines whether a traffic violation was caused by avoiding a special vehicle, thus improving the efficiency and intelligence of traffic violation review. Attached Figure Description

[0047] Figure 1 A flowchart illustrating a violation review method provided in an embodiment of the present invention;

[0048] Figure 2 An exemplary regional convolutional neural network structure diagram provided for embodiments of the present invention;

[0049] Figure 3 This is a schematic diagram illustrating an exemplary model training dataset classification method provided in an embodiment of the present invention.

[0050] Figure 4 A schematic diagram illustrating an exemplary method for determining the relative position of vehicles provided in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of an exemplary violation review process provided for an embodiment of the present invention;

[0052] Figure 6 A schematic diagram of the structure of a violation review device provided in an embodiment of the present invention. Figure 1 ;

[0053] Figure 7 A schematic diagram of the structure of a violation review device provided in an embodiment of the present invention. Figure 2 . Detailed Implementation

[0054] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. It is to be understood that the specific embodiments described herein are merely for explaining the relevant application and are not intended to limit the application. Furthermore, it should be noted that, for ease of description, only the parts relevant to the relevant application are shown in the accompanying drawings.

[0055] This invention provides a method for reviewing violations, which is applied to a violation review device. Figure 1 This is a flowchart illustrating a violation review method provided in an embodiment of the present invention. Figure 1 As shown, the main steps include:

[0056] S101. Obtain the violation scene image of the vehicle, locate the position of the vehicle from the violation scene image, and determine it as the first position.

[0057] In an embodiment of the present invention, the violation review device acquires a violation scene image of a vehicle and locates the position of the vehicle from the violation scene image, determining it as the first position.

[0058] It should be noted that, in the embodiments of the present invention, the violation record information includes the violation scene image of the violating vehicle, the violation category of the violating vehicle, and the license plate information of the violating vehicle. The violation review device can directly obtain the violation scene image of the violating vehicle from the violation record information and locate the position of the violating vehicle from the violation scene image, i.e., the first position. The first position can be the coordinate position of the violating vehicle in the violation scene image, or it can be the coordinate position in the actual scene. The specific type of the first position can be set according to actual needs and application scenarios.

[0059] S102. When the violation category of the offending vehicle is a preset category, use the preset special vehicle detection model to detect special vehicles in the violation scene image.

[0060] In an embodiment of the present invention, when the violation category of the offending vehicle is a preset category, the violation review device uses a preset special vehicle detection model to perform special vehicle detection on the violation scene image.

[0061] It should be noted that, in the embodiments of the present invention, the preset category can be categories such as crossing solid lines, running red lights, not driving in the designated lane, and occupying bus lanes. The specific preset category can be set according to actual needs and application scenarios.

[0062] It should be noted that, in the embodiments of the present invention, before the violation review device performs special vehicle detection on the violation scene image using the preset special vehicle detection model, it will determine whether the violation category of the violating vehicle included in the violation record information belongs to the preset category. If the violation category of the violating vehicle belongs to the preset category, it is necessary to further determine whether there is a special vehicle in the violation scene image. If the violation category of the violating vehicle does not belong to the preset category, the violating vehicle will be directly penalized normally. For example, when the violation review device reviews whether the violation was caused by avoiding a special vehicle, the preset category will not include categories such as drunk driving and not wearing a seat belt, because drunk driving and not wearing a seat belt are not violation categories caused by avoiding a special vehicle.

[0063] It should be noted that, in the embodiments of the present invention, special vehicles include ambulances, fire trucks, police cars, emergency rescue vehicles, or other rescue vehicles, and the specific special vehicles can be set according to actual needs and application scenarios.

[0064] It should be noted that, in the embodiments of the present invention, if the violation category of the offending vehicle is a preset category, the violation review device will use a preset special vehicle detection model to further detect whether there is a special vehicle in the violation scene image.

[0065] Specifically, in the embodiments of the present invention, before the violation review device uses a preset special vehicle detection model to detect special vehicles in violation scene images, it may also perform the following steps: acquire scene sample images, and use the special vehicle detection model to be trained to determine the probability information that the scene sample images contain special vehicles based on the scene sample images; calculate the loss information between the probability information that the scene sample images contain special vehicles and the first probability information preset for the scene sample images to obtain the first loss information; and adjust the model parameters of the special vehicle detection model to be trained based on the first loss information to obtain the preset special vehicle detection model.

[0066] It should be noted that, in the embodiments of the present invention, before the violation review device further detects whether there are special vehicles in the violation scene image using the preset special vehicle detection model, it will use the scene sample image to train the special vehicle detection model to be trained.

[0067] It should be noted that, in the embodiments of the present invention, the scene sample images obtained by the violation review device may be from public datasets or downloaded from the Internet, stored on cloud platforms, or stored on mobile phones. The specific image source method is not limited in the present invention.

[0068] It should be noted that, in the embodiments of the present invention, after acquiring the scene sample image, the violation review device classifies the images of ambulances, fire trucks, police cars, emergency rescue vehicles, and ordinary vehicles in the scene sample image and assigns them to the corresponding categories. Then, the labeled scene sample image is input into the special vehicle detection model to be trained for feature extraction to obtain the probability information that the scene sample image contains special vehicles. The special vehicle detection model to be trained can be an improved region-convolutional neural network (Faster-RCNN).

[0069] It should be noted that, in the embodiments of the present invention, the violation review device uses the special vehicle detection model to be trained to extract features from the scene sample image to obtain the probability information that the scene sample image contains a special vehicle. Then, it calculates the loss information between the probability information that the scene sample image contains a special vehicle and the first probability information preset for the scene sample image to obtain the first loss information. Finally, based on the first loss information, it adjusts the model parameters of the special vehicle detection model to be trained to obtain the preset special vehicle detection model.

[0070] Figure 2 This is an exemplary regional convolutional neural network structure diagram provided for embodiments of the present invention. For example... Figure 2 As shown, the special vehicle detection model to be trained includes a pre-trained network, a region proposal network, a region of interest pooling, and a classifier. The specific training process can be as follows: the format of the scene sample image is converted to a preset resolution, input into the pre-trained network, feature extraction is performed to obtain a feature map, and then the feature map is passed through the region proposal network to generate region proposal boxes. The region proposal boxes and the feature map are used as inputs to the region of interest pooling to obtain the final feature map. Finally, the probability information of the scene sample image containing special vehicles is obtained through the classifier structure. The preset resolution can be 1024*1024 or other resolutions.

[0071] It should be noted that, in the embodiments of the present invention, the violation review device uses a large number of scene sample images during the training process of the vehicle detection model to be trained. Specifically, the large number of scene sample images can be divided into training set, validation set and test set according to the proportion. The training set and validation set can be used to train the special vehicle detection model to be trained. Then, the test set is used to test the detection effect of the preset special vehicle detection model.

[0072] Figure 3 This is a schematic diagram illustrating an exemplary model training dataset classification method provided in an embodiment of the present invention. Figure 3As shown, the training set and validation set are used to train the special vehicle detection model to be trained. Then, the test set is used to test the detection effect of the preset special vehicle detection model. The specific data ratio of the training set, validation set and test set can be 8:1:1, or other preset ratios. Figure 3 Different datasets are used to train different models. For example, the special vehicle dataset is used to train the special vehicle detection model to be trained, resulting in a preset special vehicle detection model. In this case, the special vehicle dataset includes images of ambulances, fire trucks, police cars, emergency rescue vehicles, and ordinary vehicles, as shown in Table 1. The indicator light detection dataset is used to train the indicator light detection model to be trained, resulting in a preset indicator light detection model. In this case, the indicator light detection dataset includes images of indicator lights of ambulances, fire trucks, police cars, emergency rescue vehicles, and ordinary vehicles, as shown in Table 1. The indicator light blinking discrimination dataset is used to train the indicator light state discrimination model to be trained, resulting in a preset indicator light state discrimination model. In this case, the indicator light blinking discrimination dataset includes images of blinking indicator lights and off indicator lights, as shown in Table 1. If the vehicle state detection model to be trained is directly trained, the indicator light detection dataset and the indicator light blinking discrimination dataset are used.

[0073] Table 1

[0074]

[0075] S103. If a special vehicle is detected from the violation scene image, locate the position of the special vehicle from the violation scene image and determine it as the second position.

[0076] In an embodiment of the present invention, when a special vehicle is detected from the violation scene image, the violation review device locates the position of the special vehicle from the violation scene image and determines it as a second position.

[0077] It should be noted that, in the embodiments of the present invention, when a special vehicle is detected from the violation scene image, the violation review device locates the position of the special vehicle from the violation scene image, namely the second position, and the type of the second position is consistent with the type of the first position. For example, if the first position is the coordinate position of the violating vehicle in the violation scene image, then the second position is the coordinate position of the special vehicle in the violation scene image; if the first position is the actual coordinate position of the violating vehicle, then the second position is the actual coordinate position of the special vehicle.

[0078] Figure 4 This is a schematic diagram illustrating an exemplary method for determining the relative position of vehicles, provided as an embodiment of the present invention. Figure 4As shown, the traffic violation review device identifies the license plate in the traffic violation scene image from the traffic violation record information. Then, it matches the license plate identified from the traffic violation scene image with the license plate in the traffic violation record information to determine the location information of the violating vehicle. After obtaining the location information of the violating vehicle, it uses a preset special vehicle detection model to detect special vehicles in the traffic violation scene image and locates the position of the special vehicle from the traffic violation scene image.

[0079] S104. When the relative relationship between the first position and the second position meets the preset conditions, the working status of the special vehicle is detected from the violation scene image using the preset vehicle state detection model.

[0080] In an embodiment of the present invention, when the relative relationship between the first position and the second position meets a preset condition, the violation review device uses a preset vehicle status detection model to detect the working status of a special vehicle from the violation scene image.

[0081] It should be noted that in the embodiments of the present invention, the preset condition is that the direction of vehicle travel is taken as the reference direction, and the position information of the special vehicle is behind the vehicle violating the traffic rules. If the special vehicle is in front of the vehicle violating the traffic rules, there will be no situation where a traffic violation is caused by avoiding the special vehicle.

[0082] It should be noted that, in the embodiments of the present invention, when the special vehicle is behind the vehicle violating the traffic rules, the traffic violation review device will further use a preset vehicle status detection device to detect the working status of the special vehicle, so as to avoid judging the reason for the violation of the vehicle violating the traffic rules as avoiding the special vehicle when the special vehicle is not performing its work, thus causing review errors.

[0083] Specifically, in the embodiments of the present invention, the preset vehicle state detection model includes a preset indicator light detection model and a preset indicator light state discrimination model. The violation review device uses the preset vehicle state detection model to detect the working state of special vehicles from violation scene images, including: cropping a vehicle image containing the special vehicle from the violation scene image; using the preset indicator light detection model to detect the indicator lights of the special vehicle in the vehicle image; when an indicator light is detected in the vehicle image, using the preset indicator light state discrimination model to detect the running state of the indicator light in the vehicle image; and when the running state is flashing, determining the working state as performing a task.

[0084] It should be noted that, in the embodiments of the present invention, the preset vehicle state detection model includes a preset indicator light detection model and a preset indicator light state discrimination model. The violation review device first crops out the vehicle image containing the special vehicle from the violation scene image, and then uses the preset indicator light detection model to determine whether there is an indicator light in the vehicle image. If the vehicle image includes an indicator light, the preset indicator light state discrimination model is also used to determine the operating state of the indicator light in order to determine the working state of the special vehicle. If the operating state of the indicator light is flashing, it means that the working state of the special vehicle in the violation scene image is performing a task. In this case, if the violating vehicle is in front of the special vehicle performing the task, it is a violation caused by avoiding the special vehicle.

[0085] Specifically, in an embodiment of the present invention, after the violation review device detects the running status of the indicator lights from the vehicle image using a preset indicator light status discrimination model, it can also perform the following steps: if the running status is off, determine that the working status is that the task has not been performed.

[0086] It should be noted that, in the embodiments of the present invention, when the indicator light is off, the special vehicle in the image representing the violation scene has not performed any task.

[0087] Specifically, in the embodiments of the present invention, after the violation review device detects the working status of a special vehicle from the violation scene image using a preset vehicle status detection model, it can also perform the following steps: when the working status is not performing a task, determine that the reason for the violation of the vehicle is not yielding to a special vehicle.

[0088] It should be noted that, in the embodiments of the present invention, if the violation review device detects that the working state of the special vehicle in the violation scene image is not performing a task when using the preset vehicle state detection model, then the reason for the violation is not due to avoiding the special vehicle.

[0089] Specifically, in an embodiment of the present invention, before the violation review device detects the working state of a special vehicle from a violation scene image using a preset vehicle state detection model, it may also perform the following steps: acquire a vehicle sample image, and use the vehicle state detection model to be trained to determine the probability information of the indicator light operating in the vehicle sample image based on the vehicle sample image; calculate the loss information between the probability information of the indicator light operating in the vehicle sample image and the second probability information preset for the vehicle sample image to obtain the second loss information; and adjust the model parameters of the vehicle state detection model to be trained based on the second loss information to obtain the preset vehicle state detection model.

[0090] It should be noted that, in the embodiments of the present invention, the images used by the violation review device to train the vehicle state detection model can be scene sample images directly, or they can only include vehicle sample images of special vehicles. The specific sample data is not limited in the present invention.

[0091] It should be noted that, in the embodiments of the present invention, the vehicle state detection model to be trained can be Faster-RCNN. Of course, since the vehicle detection model to be trained includes the indicator light detection model to be trained and the indicator light state discrimination model to be trained, the indicator light detection model to be trained and the indicator light state discrimination model to be trained can also be Faster-RCNN. Specifically, the indicator light detection model to be trained and the indicator light state discrimination model to be trained can be trained as a whole, that is, the vehicle detection model to be trained can be trained, or they can be trained separately. The training data can be vehicle sample images, indicator light sample images, or scene sample images. The appropriate sample data can be selected according to actual needs and application scenarios. Furthermore, the specific training method is consistent with the training process of the aforementioned preset special vehicle detection model.

[0092] It should be noted that, in the embodiments of the present invention, the steps of the violation review device training the preset vehicle state detection model can be as follows: acquiring vehicle sample images and inputting them into the vehicle state detection model to be trained, obtaining the probability information of the indicator lights operating in the vehicle sample images, then calculating the loss information between the probability information of the indicator lights operating in the vehicle sample images and the second probability information preset for the vehicle sample images, and finally, using the obtained second loss information to adjust the model parameters of the vehicle state detection model to be trained, thereby obtaining the preset vehicle state detection model.

[0093] S105. When the working status is performing a task, determine that the reason for the violation by the offending vehicle is to avoid a special vehicle.

[0094] In an embodiment of the present invention, when the working state is performing a task, the violation review device determines that the reason for the violation by the vehicle is to avoid a special vehicle.

[0095] It should be noted that, in the embodiments of the present invention, when the violation review device determines that the working state of the special vehicle in the violation scene image is performing a task, the violation reason of the vehicle in the violation scene image is to avoid a special vehicle.

[0096] Specifically, in an embodiment of the present invention, after the violation review device determines that the reason for the violation of a vehicle is to avoid a special vehicle, it may also perform the following steps: cancel the violation record corresponding to the vehicle.

[0097] It should be noted that, in the embodiments of the present invention, after determining that the reason for the violation of a vehicle in the violation scene image is to avoid a special vehicle, the violation review device cancels the violation record of the vehicle, that is, exempts the vehicle from punishment. The whole process does not require the participation of the violator and the violation review personnel, saving time for both parties. The violation review device directly uses a preset special vehicle monitoring model and a preset vehicle status detection model to determine the reason for the violation of the vehicle, and cancels the punishment of the vehicle when the reason for the violation is to avoid a special vehicle, thereby improving the efficiency and intelligence of violation review.

[0098] Figure 5 This is a schematic diagram illustrating an exemplary violation review process provided in an embodiment of the present invention. Figure 5 As shown, after obtaining the traffic violation record information, the violation review device determines whether it belongs to a preset category based on the violation category included in the record information. If it belongs to a preset category, it further uses a preset special vehicle detection model to check whether a special vehicle exists in the violation scene image included in the record information. If it does not belong to a preset category, the vehicle is directly penalized normally. If a special vehicle exists in the violation scene image, the device determines the location information of the special vehicle and locates its position from the violation scene image in the record information. Then, it determines whether the special vehicle is behind the violating vehicle. If the special vehicle is behind the violating vehicle... The system then crops the vehicle image from the traffic violation scene image. If the special vehicle is not behind the violating vehicle, the violating vehicle is penalized normally. After cropping the vehicle image, the violation review device uses a preset indicator light detection model to check if there are indicator lights in the vehicle image. If there are indicator lights, it further uses a preset indicator light status discrimination model to determine if the indicator light is functioning normally. If there are no indicator lights, the violating vehicle is penalized normally directly. If the indicator light is flashing, it is determined that the violation was caused by yielding to a special vehicle. If the indicator light is off, the violating vehicle is penalized normally directly.

[0099] This invention provides a method for reviewing traffic violations. The method includes: acquiring a scene image of a vehicle violating traffic rules, and locating the position of the vehicle from the scene image as a first position; if the violation category of the vehicle is a preset category, using a preset special vehicle detection model to detect special vehicles in the scene image; if a special vehicle is detected in the scene image, locating the position of the special vehicle from the scene image as a second position; if the relative relationship between the first and second positions satisfies preset conditions, using a preset vehicle state detection model to detect the working state of the special vehicle from the scene image; if the working state is performing a task, determining that the violation was caused by avoiding a special vehicle. The traffic violation review method provided by this invention, by detecting special vehicles and their working states, determines whether the violation was caused by avoiding a special vehicle, thus improving the efficiency and intelligence of traffic violation review.

[0100] This invention provides a violation review device. Figure 6 A schematic diagram of the structure of a violation review device provided in an embodiment of the present invention. Figure 1 .like Figure 6 As shown, it includes:

[0101] The acquisition module 601 is used to acquire the violation scene image of the vehicle, locate the position of the vehicle from the violation scene image, and determine it as the first position;

[0102] Detection module 602 is used to detect special vehicles in the violation scene image by using a preset special vehicle detection model when the violation category of the vehicle is a preset category.

[0103] The positioning module 603 is used to locate the position of the special vehicle from the violation scene image when the special vehicle is detected from the violation scene image, and determine it as a second position;

[0104] The processing module 604 is used to detect the working status of the special vehicle from the violation scene image using a preset vehicle status detection model when the relative relationship between the first position and the second position meets preset conditions.

[0105] The determination module 605 is used to determine, when the working state is performing a task, the reason for the violation of the vehicle is to avoid the special vehicle.

[0106] Optionally, the processing module 604 is specifically used to crop out a vehicle image containing the special vehicle from the violation scene image; to detect the indicator lights of the special vehicle in the vehicle image using the preset indicator light detection model; and, if the indicator lights are detected in the vehicle image, to detect the operating status of the indicator lights in the vehicle image using the preset indicator light status discrimination model.

[0107] Optionally, the processing module 604 is further configured to determine that the working state is "no task executed" when the running state is "off".

[0108] Optionally, the determining module 605 is further configured to determine, when the working state is "not performing a task", that the reason for the violation of the vehicle is not yielding to the special vehicle.

[0109] Optionally, the processing module 604 is also used to revoke the violation record corresponding to the violating vehicle.

[0110] Optionally, the violation review device further includes a model training module (not shown in the figure), used to acquire scene sample images, and using a special vehicle detection model to be trained, determine the probability information that the scene sample image contains a special vehicle based on the scene sample image; calculate the loss information between the probability information that the scene sample image contains a special vehicle and the first probability information preset for the scene sample image to obtain the first loss information; and adjust the model parameters of the special vehicle detection model to be trained based on the first loss information to obtain the preset special vehicle detection model.

[0111] Optionally, the model training module (not shown in the figure) is further configured to acquire vehicle sample images, and using the vehicle state detection model to be trained, determine the probability information of the indicator lights operating in the vehicle sample images based on the vehicle sample images; calculate the loss information between the probability information of the indicator lights operating in the vehicle sample images and the second probability information preset for the vehicle sample images to obtain the second loss information; and adjust the model parameters of the vehicle state detection model to be trained based on the second loss information to obtain the preset vehicle state detection model.

[0112] This invention provides a violation review device. Figure 7 A schematic diagram of the structure of a violation review device provided in an embodiment of the present invention. Figure 2 .like Figure 7 As shown, the device includes: a processor 701, a memory 702, and a communication bus 703;

[0113] The communication bus 703 is used to realize the communication connection between the processor 701 and the memory 702;

[0114] The processor 701 is used to execute the violation review program stored in the memory 702 to implement the above-mentioned violation review method.

[0115] This invention provides a traffic violation review device that acquires images of traffic violations by vehicles and locates the position of the violating vehicle from the image, determining it as a first position. If the violation category is a preset category, a preset special vehicle detection model is used to detect special vehicles in the violation scene image. If a special vehicle is detected in the violation scene image, its position is located and determined as a second position. If the relative relationship between the first and second positions satisfies preset conditions, a preset vehicle state detection model is used to detect the working state of the special vehicle from the violation scene image. If the working state is performing a task, the cause of the violation is determined to be avoiding a special vehicle. This traffic violation review device improves the efficiency and intelligence of traffic violation review by detecting special vehicles and their working states to determine whether the violation was caused by avoiding a special vehicle.

[0116] This invention provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the aforementioned violation review method. The computer-readable storage medium can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it can be a device comprising one or any combination of the above-mentioned memories, such as a mobile phone, computer, tablet device, personal digital assistant, etc.

[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0118] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this utility application should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for reviewing violations, characterized in that, The method includes: Obtain images of the traffic violation scene of the vehicle, and locate the position of the vehicle from the violation scene image, which is determined as the first position; When the violation category of the offending vehicle is a preset category, a preset special vehicle detection model is used to detect special vehicles in the violation scene image; If the special vehicle is detected from the violation scene image, the position of the special vehicle is located from the violation scene image and determined as the second position; When the relative relationship between the first position and the second position meets the preset conditions, the working status of the special vehicle is detected from the violation scene image using a preset vehicle state detection model. When the working state is performing a task, the reason for the violation of the vehicle is determined to be to avoid the special vehicle. The preset vehicle state detection model includes a preset indicator light detection model and a preset indicator light state discrimination model. The step of using the preset vehicle state detection model to detect the working state of the special vehicle from the violation scene image includes: The vehicle image containing the special vehicle is cropped from the image of the traffic violation scene; Using the preset indicator light detection model, the indicator lights of the specific vehicle are detected in the vehicle image; If the indicator light is detected from the vehicle image, the operating status of the indicator light is detected from the vehicle image using the preset indicator light status discrimination model; If the running status is flashing, the working status is determined to be executing a task.

2. The method according to claim 1, characterized in that, After detecting the operating status of the indicator lights from the vehicle image using the preset indicator light status discrimination model, the method further includes: If the running state is off, the working state is determined to be "no task executed".

3. The method according to claim 1, characterized in that, After detecting the working status of the special vehicle from the violation scene image using a preset vehicle status detection model, the method further includes: If the working state is "not performing a task", the reason for the violation by the offending vehicle is determined to be "failure to yield to the special vehicle".

4. The method according to claim 1, characterized in that, After determining that the reason for the traffic violation by the offending vehicle was to avoid the special vehicle, the method further includes: The traffic violation record corresponding to the aforementioned vehicle will be revoked.

5. The method according to claim 1, characterized in that, Before using a preset special vehicle detection model to detect special vehicles in the violation scene image, the method further includes: Acquire scene sample images and use the special vehicle detection model to be trained to determine the probability information that the scene sample images contain special vehicles based on the scene sample images; The first loss information is obtained by calculating the loss information between the probability information of the scene sample image containing a special vehicle and the first probability information preset for the scene sample image; Based on the first loss information, the model parameters of the special vehicle detection model to be trained are adjusted to obtain the preset special vehicle detection model.

6. The method according to claim 1, characterized in that, Before detecting the working status of the special vehicle from the violation scene image using a preset vehicle status detection model, the method further includes: Acquire vehicle sample images and, using the vehicle state detection model to be trained, determine the probability information of indicator lights operating in the vehicle sample images based on the vehicle sample images; The second loss information is obtained by calculating the loss information between the probability information of the indicator lights operating in the vehicle sample image and the second probability information preset for the vehicle sample image; Based on the second loss information, the model parameters of the vehicle state detection model to be trained are adjusted to obtain the preset vehicle state detection model.

7. A violation verification device, characterized in that, include: The acquisition module is used to acquire images of traffic violations by vehicles, locate the position of the vehicles from the images of traffic violations, and determine the position as the first position. The detection module is used to detect special vehicles in the violation scene image using a preset special vehicle detection model when the violation category of the vehicle is a preset category. The positioning module is used to locate the position of the special vehicle from the violation scene image when the special vehicle is detected from the violation scene image, and determine it as a second position; The processing module is used to detect the working status of the special vehicle from the violation scene image using a preset vehicle status detection model when the relative relationship between the first position and the second position meets preset conditions. The determination module is used to determine, when the working state is executing a task, the reason for the violation of the vehicle is to avoid the special vehicle; The preset vehicle state detection model includes a preset indicator light detection model and a preset indicator light state discrimination model. The processing module is further configured to: crop out a vehicle image containing the special vehicle from the violation scene image; use the preset indicator light detection model to detect the indicator lights of the special vehicle in the vehicle image; when the indicator light is detected in the vehicle image, use the preset indicator light state discrimination model to detect the operating state of the indicator light in the vehicle image; and when the operating state is flashing, determine that the working state is task execution.

8. A violation verification device, characterized in that, include: Processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is configured to execute the violation review program stored in the memory to implement the violation review method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the violation review method according to any one of claims 1-6.

Images