Park vehicle violation behavior monitoring method and device, electronic equipment and storage medium

Abstract

The invention discloses a park vehicle violation behavior monitoring method and device, electronic equipment and a storage medium, and the method comprises the steps: collecting video data of a management and control region through a camera, obtaining violation judgment conditions of the management and control region, inputting the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features, and carrying out the recognition of the vehicle features; and judging whether a vehicle violation judgment condition is met or not according to the vehicle characteristics, if so, determining that a vehicle violation behavior occurs in the management and control area, generating alarm information and sending the alarm information to the target terminal, thereby realizing identification of the vehicle violation behavior by extracting the vehicle characteristics and the violation judgment condition after collecting the video data. The vehicle violation behavior can be found without manual inspection or manual video viewing, the efficiency of finding the vehicle violation behavior is improved, the problem that the vehicle violation behavior is easily found by mistake and omission is avoided, the vehicle violation behavior can be prevented in time, corresponding vehicle violation judgment conditions can be set for different management and control areas in the park, and the management and control efficiency is improved. And vehicle violation monitoring can be flexibly carried out on each management and control area.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, device, electronic equipment and storage medium for monitoring illegal behaviors of vehicles in a park. Background Art

[0002] The enterprise park is an important production and operation area for enterprises. In order to ensure normal production and manufacturing, it is crucial to monitor vehicle violations on park roads and in and outside areas of the workshop.

[0003] At present, the monitoring of vehicle violations in enterprise parks is mainly carried out by inspectors regularly inspecting important areas such as roads, parking lots, manufacturing workshops, etc. in the park to manually confirm whether there are violations such as vehicles driving in the wrong direction, non-motor vehicles entering, illegal parking, parking in prohibited areas, etc., or by installing cameras in various areas to collect video data and send it to the monitoring room, where the above-mentioned vehicle violations are confirmed by manually viewing the video.

[0004] The existing method of discovering vehicle violations through manual on-site inspections or inspections using video data collected by cameras is inefficient and difficult to ensure 24-hour manual inspections. It is easy to miss vehicle violations and fail to stop vehicle violations in a timely manner. Summary of the invention

[0005] The present invention provides a method, device, electronic device and storage medium for monitoring violations of park vehicles, so as to solve the problem that manual inspection of violations of park vehicles is inefficient and easy to miss or discover violations of vehicles.

[0006] In a first aspect, the present invention provides a method for monitoring violations of park vehicles, comprising:

[0007] Collect video data of the controlled area through cameras;

[0008] Obtain vehicle violation judgment conditions in the control area;

[0009] Inputting the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features;

[0010] Determining whether the vehicle violation determination condition is met according to the vehicle characteristics;

[0011] If so, it is determined that a vehicle violation has occurred in the control area, and an alarm message is generated and sent to the target terminal.

[0012] Optionally, the video data is input into a pre-trained vehicle detection and tracking model to obtain vehicle features, including:

[0013] Inputting multiple frames of video images in the video data into the vehicle detection and tracking model to obtain target detection and tracking results;

[0014] When the target detection and tracking result indicates that a vehicle exists, extracting the license plate color and / or the license plate number from the multiple frames of video images to obtain the license plate features;

[0015] The driving trajectory of the vehicle is generated based on the vehicle detection frames of the multiple frames of video images in the target detection and tracking results.

[0016] Optionally, the target detection and tracking result includes a vehicle detection frame, and the license plate color and / or license plate number are extracted from the multiple frames of video images to obtain the license plate features, including:

[0017] Capturing a vehicle detection frame area from multiple frames of video images to obtain a vehicle image;

[0018] identifying a license plate area from the vehicle image;

[0019] Binarizing the license plate area to obtain a binary image;

[0020] Extracting a character region from the binary image, and matching the character region with a character template to obtain a license plate number;

[0021] Pixel values ​​of an area other than the character area in the license plate area are extracted, and a license plate color of the license plate is determined based on the pixel values.

[0022] Optionally, the control area includes a road area, the violation judgment condition includes that a non-motor vehicle is traveling on a motor vehicle road, and the vehicle movement direction is opposite to a preset direction, and judging whether the vehicle violation judgment condition is met according to the vehicle characteristics includes:

[0023] Determining whether the vehicle is a non-motor vehicle based on the license plate color and / or license plate number;

[0024] If so, determining whether the vehicle is traveling in a motor vehicle lane based on the vehicle's driving trajectory;

[0025] Determining that a non-lane driving violation occurred in the road area when the vehicle was traveling in the motor vehicle lane;

[0026] Determining a driving direction vector of the vehicle based on a driving trajectory of the vehicle;

[0027] The vector angle is calculated based on the following formula:

[0028]

[0029] Among them, (x t ,y t) represents the vector of the driving direction of the road area, and represents the coordinate point T1(x 1 ,y 1 ) to the coordinate point T2(x 2 ,y 2 ), (x p ,y p ) represents the vehicle's driving direction vector, and represents the adjacent coordinate point P3(x 3 ,y 3 ) to the coordinate point P4(x 4 ,y 4 ) vector;

[0030] Determining whether the vector angle is greater than a preset angle threshold;

[0031] If so, it is determined that the vehicle is traveling in the wrong direction in the road area, and wrong direction behavior of the vehicle occurs in the road area.

[0032] Optionally, the control area includes a prohibited parking area, the violation judgment condition includes that the vehicle is parked in the prohibited parking area, and the determining whether the vehicle violation judgment condition is met according to the vehicle characteristics includes:

[0033] Determining whether the vehicle has stopped driving according to the driving trajectory;

[0034] If so, determining a parking position of the vehicle based on the driving trajectory;

[0035] When the parking position is within the no-parking zone, it is determined that a no-parking violation occurs in the no-parking zone.

[0036] Optionally, the control area further includes a parking area, the violation judgment condition includes that the vehicle is parked outside the parking area, and the determining whether the vehicle violation judgment condition is met according to the vehicle characteristics includes:

[0037] When the parking position is outside the parking area, it is determined that illegal parking occurs in the parking area.

[0038] Optionally, determining that a vehicle violation occurs in the control area and generating an alarm message to send to a target terminal includes:

[0039] Determine a target terminal associated with the control area, or determine a target terminal associated with the vehicle;

[0040] The illegal image is intercepted from the video data, and alarm information including the illegal image is generated and sent to the target terminal.

[0041] In a second aspect, the present invention provides a device for monitoring violations of park vehicles, comprising:

[0042] Video data acquisition module, used to collect video data of the control area through cameras;

[0043] A violation condition acquisition module, used to obtain vehicle violation judgment conditions in the control area;

[0044] A vehicle feature extraction module, used for inputting the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features;

[0045] A violation judgment module, used to judge whether the vehicle violation judgment condition is met according to the vehicle characteristics, and if so, execute the alarm module;

[0046] The alarm module is used to determine whether a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal.

[0047] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0048] at least one processor; and

[0049] a memory communicatively connected to the at least one processor; wherein,

[0050] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for monitoring violations of vehicles in a park as described in the first aspect of the present invention.

[0051] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the method for monitoring violations of park vehicles described in the first aspect of the present invention when executed.

[0052] The embodiment of the present invention collects video data of the control area in the park through a camera, and obtains the violation judgment condition of the control area, inputs the video data into the vehicle detection and tracking model to extract vehicle features, and further judges whether the vehicle violation judgment condition of the control area is met according to the vehicle features. If so, it is determined that a vehicle violation occurs in the control area, and an alarm information is generated and sent to the target terminal, thereby realizing the identification of vehicle violations by extracting vehicle features and violation judgment conditions after collecting video data, without manual inspection or manual video review to discover vehicle violations, thereby improving the efficiency of discovering vehicle violations, and since no manual participation is required, the problem of manual error in discovering vehicle violations is avoided, and vehicle violations can be prevented in time. Furthermore, different vehicle violation judgment conditions can be set for different control areas in the park, and vehicle violations in each control area in the park can be flexibly monitored.

[0053] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 This is a flow chart of a method for monitoring illegal behaviors of vehicles in a park provided by Embodiment 1 of the present invention;

[0056] Figure 2 This is a flow chart of a method for monitoring illegal behaviors of vehicles in a park provided by Embodiment 2 of the present invention;

[0057] Figure 3 This is a schematic diagram for retrograde judgment;

[0058] Figure 4 This is an example flow chart of judging the violation of park vehicles in the present invention;

[0059] Figure 5 It is a structural schematic diagram of a device for monitoring illegal behavior of vehicles in a park provided by Embodiment 3 of the present invention;

[0060] Figure 6 It is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0062] Embodiment 1

[0063] Figure 1 This is a flow chart of a method for monitoring vehicle violations in a park provided in the first embodiment of the present invention. This embodiment can be applied to monitoring vehicle violations in various areas within a park. The method can be executed by a vehicle violation monitoring device in a park. The vehicle violation monitoring device in a park can be implemented in the form of hardware and / or software and can be configured in an electronic device, such as an edge terminal or server set up in the park. Figure 1 As shown, the monitoring methods for vehicle violations in the park include:

[0064] S101. Collect video data of the controlled area through cameras.

[0065] In this embodiment, the park can be an activity place for enterprise production and manufacturing, and the control area can be the area within the park where vehicle violation monitoring is performed. By way of example, the control area can include road areas, such as the roads in the production workshop where engineering transfer vehicles travel, or the roads outside the production workshop where social vehicles or engineering transfer vehicles travel. The control area can also include no-parking areas and allowed parking areas. By way of example, no-parking areas can include areas near fire-fighting equipment, areas for special workstations, etc., and allowed parking areas can be areas where parking spaces are located in parking lots, etc.

[0066] In actual applications, the camera can collect video data of the controlled area at a fixed or dynamic frame rate, and the video data can be sent to the terminal device in the form of a video stream. For example, cameras can be set up in different controlled areas, and video data collected by the cameras can be sent to the terminal device. One terminal device can be connected to multiple cameras, and the video data collected by multiple cameras can be processed to identify vehicle violations. Of course, each camera can be connected to a terminal device, or a vehicle violation recognition algorithm can be integrated in the camera.

[0067] S102. Obtain vehicle violation judgment conditions in the control area.

[0068] In this embodiment, different violation judgment conditions can be set for different control areas within the park, where the violation judgment condition can be a condition for judging whether a vehicle within the control area has violated the regulations. The violation judgment condition can be one or more combinations. Technical personnel in this field can set corresponding violation judgment conditions according to the needs of different control areas within the park. The embodiment of the present invention does not limit the violation judgment conditions of the control area.

[0069] In one example, when a camera collects video data of a control area and is connected to only one terminal device, the vehicle violation judgment conditions of the control area can be stored on the terminal device. When a terminal device is connected to multiple cameras, the vehicle violation judgment conditions of the control area collected by multiple cameras can be stored on the terminal device. When the terminal device receives the video data, it can obtain the stored vehicle violation judgment conditions through the source information of the video data. By way of example, after binding the camera number with the corresponding control area, the vehicle violation judgment conditions can be directly read through the camera number.

[0070] S103: Input the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features.

[0071] In this embodiment, a vehicle detection and tracking model may be pre-trained, and the vehicle detection and tracking model may be used to detect and track the vehicle to obtain a detection and tracking result, and the vehicle characteristics may be further determined by the detection and tracking result. Exemplarily, the vehicle characteristics may include motion characteristics and attribute characteristics, and the attribute characteristics may include the vehicle's license plate color, license plate number, vehicle type, etc., and the motion characteristics may include the vehicle's driving trajectory, speed, driving direction, parking behavior, and other characteristics.

[0072] In one embodiment, when the vehicle detection and tracking model detects a vehicle, the vehicle image can be captured from the video data, and the attribute features such as license plate color, license plate number, and vehicle type can be identified through the vehicle image. When the vehicle is detected and tracked, the vehicle's driving trajectory can be obtained, and the vehicle's speed, driving direction, parking behavior and other motion characteristics can be determined through the driving trajectory.

[0073] S104. Determine whether vehicle violation determination conditions are met based on vehicle characteristics.

[0074] After obtaining the vehicle characteristics, it can be determined whether the vehicle characteristics meet the vehicle violation judgment conditions of the control area. If so, it is determined that there are vehicles with violations in the control area, and S105 can be executed. If not, it is determined that there are no vehicles with violations in the control area, and the process can return to S103 to continue inputting the collected video data into the vehicle detection and tracking model.

[0075] S105: Determine if a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal.

[0076] When it is determined that a vehicle violation has occurred in a control area, the target terminal associated with the control area can be determined, or the target terminal associated with the vehicle can be determined, and the violation image can be intercepted from the video data, and an alarm message including the violation image can be generated and sent to the target terminal. Exemplarily, the control area is associated with the mobile phone of the patrol officer, and the video graphics or video clips including the vehicle violation can be intercepted from the video data, and the alarm message including the video image or video clip can be sent to the mobile phone of the patrol officer. Of course, the license plate of the vehicle can also be identified to obtain the license plate number, and the owner's mobile phone can be determined by the mobile phone number associated with the license plate number, and the alarm message can be sent to the owner's mobile phone.

[0077] The embodiment of the present invention collects video data of the control area in the park through a camera, and obtains the violation judgment condition of the control area, inputs the video data into the vehicle detection and tracking model to extract vehicle features, and further judges whether the vehicle violation judgment condition of the control area is met according to the vehicle features. If so, it is determined that a vehicle violation occurs in the control area, and an alarm information is generated and sent to the target terminal, thereby realizing the identification of vehicle violations by extracting vehicle features and violation judgment conditions after collecting video data, without manual inspection or manual video viewing to discover vehicle violations, thereby improving the efficiency of discovering vehicle violations, and since no manual participation is required, the problem of manual error in discovering vehicle violations is avoided, and vehicle violations can be prevented in time. Furthermore, different vehicle violation judgment conditions can be set for different control areas in the park, and vehicle violations in each control area in the park can be flexibly monitored.

[0078] Embodiment 2

[0079] Figure 2 This is a flow chart of a method for monitoring violations of park vehicles provided in the second embodiment of the present invention. The embodiment of the present invention is optimized on the basis of the above-mentioned first embodiment, such as Figure 2 As shown, the monitoring methods for vehicle violations in the park include:

[0080] S201. Collect video data of the controlled area through cameras.

[0081] In this embodiment, cameras and edge terminals can be set up in various control areas of the park. One edge terminal is connected to one or more cameras. When it is necessary to monitor vehicle violations in various control areas in the park, video data of the control areas can be collected through the cameras in each control area. The video data may include the identity information of the camera. For example, the camera transmits the video data and the camera number to the edge terminal.

[0082] S202: Obtain vehicle violation judgment conditions in the control area.

[0083] After receiving the video data, the edge terminal can determine the camera number from the video data. Specifically, the video data can be transmitted to the edge terminal in the form of a data packet. When the edge terminal receives one channel of video data, it can unpack it to obtain the camera number. The target control area to which the camera belongs can be further determined by the camera number, so as to request the vehicle violation judgment conditions pre-configured for the target control area from the memory or the server.

[0084] S203: Input multiple frames of video images in the video data into a vehicle detection and tracking model to obtain target detection and tracking results.

[0085] This embodiment can pre-train a vehicle detection and tracking model. Exemplarily, the vehicle detection and tracking model can be a combination of a YOLO series neural network and a DeepSORT neural network, wherein YOLO is used to perform target detection on a video image and output detection frame information (x, y, w, h), where x and y are the coordinates of the center point of the detection frame, w is the width of the detection frame, and h is the height of the detection frame. After outputting the detection frame, the DeepSORT neural network tracks the target corresponding to each detection frame.

[0086] During training, a large number of images of vehicles such as cars, electric bicycles, and ordinary bicycles can be collected as training images, and the categories, positions, widths, heights, etc. of the vehicles in the training images can be annotated, and the training images can be divided into training sets, test sets, and validation sets. The training set is used to train the vehicle detection and tracking model, and the test set is used to test the model to determine whether the detection and tracking results output by the trained vehicle detection and tracking model are close enough to the pre-annotated information (for example, the loss rate can be calculated, and the smaller the loss rate, the closer the detection and tracking results are to the annotated information). After multiple iterations of training, a vehicle detection and tracking model with high accuracy is obtained, and then the validation set is used to verify the vehicle detection and tracking model to ensure that the model has good generalization ability. The training process can refer to the training method of the target detection and tracking model in the prior art, which will not be described in detail here.

[0087] After the vehicle detection and tracking model is trained and deployed on the edge terminal, multiple frames of video images in the video data can be input into the vehicle detection and tracking model to obtain a detection box for each frame of the video image, which includes category and location information.

[0088] S204: When the target detection and tracking result indicates that a vehicle exists, extracting the license plate color and / or the license plate number from the multiple frames of video images to obtain license plate features.

[0089] Specifically, since the target detection and tracking result includes the target detection frame of multiple video frames, when the category of the target detection frame is a vehicle, it means that the vehicle is detected, and the vehicle detection frame area can be intercepted from any one frame or multiple frames of the multiple video images to obtain a vehicle image, and the vehicle image is further converted into a grayscale image and then processed by Gaussian blur to remove noise in the image, and then the license plate area is identified from the denoised vehicle image. For example, edge detection is performed using edge detection algorithms such as Canny to obtain the license plate area in the image. For example, multiple areas are obtained after edge detection, and the license plate area is matched from the multiple areas based on the rectangular shape and aspect ratio of the license plate, and further The license plate area is binarized to obtain a binary image, the character area is extracted from the binary image, and the character area is matched with the character template to obtain the license plate number. Specifically, the character area can be segmented using connected component analysis, and then valid characters are extracted according to the aspect ratio of the characters. The Euclidean distance is calculated one by one with the character database template for matching, and finally the result with the highest matching degree is selected as the recognition result to obtain the license plate number. Of course, the license plate number can also be obtained by directly using ORC to recognize multiple character areas. The validity of the recognized license plate number can also be further verified according to the license plate rules in the traffic regulations to obtain a valid and accurate license plate number.

[0090] For the license plate area, the pixel values ​​of the area outside the character area in the license plate area can be extracted, and the license plate color of the license plate can be determined based on the pixel values. For example, the color of the license plate can be determined based on the threshold color segmentation method. The license plate color can include blue, yellow, green, white, black, etc. Different colors represent different types of vehicles.

[0091] S205 , generating a driving trajectory of the vehicle based on the vehicle detection frames of the multiple frames of video images in the target detection and tracking results.

[0092] The target detection and tracking results may include target detection frames of multiple frames of video images. Targets classified as vehicles are tracked to obtain the driving trajectory of each vehicle. The driving trajectory may be fitted to the center position of the vehicle's detection frame (the detection frame of the same ID), that is, the detection results include the driving trajectory of each vehicle (if the video data includes the vehicle).

[0093] S206: Determine whether vehicle violation determination conditions are met based on the license plate features and the driving trajectory.

[0094] In one embodiment, the controlled area may be a road area, and the violation judgment conditions include a non-motor vehicle driving on a motor vehicle road, and the direction of vehicle movement is opposite to a preset direction. Whether the vehicle is a non-motor vehicle can be determined based on the license plate color and / or license plate number. If so, whether the vehicle is driving on a motor vehicle lane is determined based on the vehicle's driving trajectory. When the vehicle is driving on a motor vehicle lane, it is determined that a violation of not driving in the lane has occurred in the road area.

[0095] Exemplarily, different license plate colors and license plate number coding rules represent different vehicle types. For example, a white license plate may be an electric bicycle. The license plate color and / or license plate number can be used to determine whether the vehicle is a non-motor vehicle. If so, it can be further determined whether the vehicle's driving trajectory is within the boundary of the motor vehicle lane. If so, it is determined that a violation of driving in a lane has occurred within the control area. The motor vehicle lane identification method can be: after the camera is fixed, a reference image is captured, the motor vehicle lane boundary is pre-marked in the reference image, and the driving trajectory is superimposed on the reference image for coordinate comparison to determine whether the driving trajectory is within the motor vehicle lane boundary.

[0096] In another embodiment, the driving direction vector of the vehicle can also be determined based on the driving trajectory of the vehicle. For example, two coordinate points P3 (x 3 ,y 3 ) and the coordinate point P4(x 4 ,y 4 ), the vector from P3 to P4 is the vehicle's driving direction vector (x p ,y p ), and calculate the vector angle based on the following formula:

[0097]

[0098] Among them, (x t ,y t ) represents the vector of the driving direction of the road area, and represents the coordinate point T1(x 1 ,y 1 ) to the coordinate point T2(x 2 ,y 2 ), (x p ,y p ) represents the vehicle's driving direction vector, and represents the adjacent coordinate point P3(x 3 ,y 3 ) to the coordinate point P4(x 4 ,y 4 ) and determine whether the vector angle is greater than a preset angle threshold. If so, it is determined that the vehicle is traveling in the wrong direction in the road area and that the vehicle is traveling in the wrong direction in the road area.

[0099] like Figure 3 As shown, the driving direction specified by the lane is T1(x 1 ,y 1 ) to T2(x 2 ,y 2 ), recorded as the driving direction vector (x t ,y t ), the vehicle moves from coordinate point P3(x 3 ,y 3 ) to the coordinate point P4(x 4 ,y 4 ), and its driving direction vector is (x p ,y p ), vector (x t ,y t ) and vector (x p ,y p ) is θ, from Figure 3 It can be seen that if the angle θ is greater than a preset angle threshold (such as 90°), it can be determined that the driving direction of the vehicle is opposite to the driving direction specified by the road, and it can be determined that the vehicle is driving in the wrong direction.

[0100] For road areas, this embodiment can determine non-motor vehicles by license plate color or license plate number, and then determine whether a violation of driving out of lane has occurred through the driving trajectory and the boundary of the motor vehicle lane. In addition, by calculating the angle between the vehicle's driving direction vector and the direction vector specified by the road, it can be judged whether the vehicle is driving in the wrong direction through the angle, and the road area can be accurately and quickly checked for violations of driving out of lane and driving in the wrong direction.

[0101] In one embodiment, the control area may include a prohibited parking area, and the violation judgment condition includes that the vehicle is parked in the prohibited parking area. It can be determined whether the vehicle has stopped driving based on the driving trajectory. For example, if the end point of the driving trajectory remains unchanged within a preset time, it can be determined that the vehicle has stopped driving. The parking position of the vehicle can be determined based on the driving trajectory. The parking position can be the coordinates of the center point of the vehicle detection frame corresponding to the end point of the driving trajectory. When the parking position is in the prohibited parking area, it is determined that a prohibited parking violation has occurred in the prohibited parking area. For example, if the coordinates of the center point of the vehicle detection frame are in the prohibited parking area, it is determined that a prohibited parking behavior has occurred in the prohibited parking area. For example, an engineering transfer vehicle is parked for a long time in the area near the consumer equipment in the manufacturing workshop, or a social vehicle is parked on the consumer channel on the road outside the workshop. In this embodiment, the parking position of the vehicle is determined after the vehicle stops by the driving trajectory of the vehicle, so as to determine whether the vehicle is in the prohibited parking area by the parking position, and the prohibited parking violation of the vehicle can be monitored in real time and accurately in the prohibited parking area, and the prohibited parking behavior can be discovered and prevented in time.

[0102] In one embodiment, the control area also includes a parking area, and the violation judgment condition includes that the vehicle is parked outside the parking area. After determining that the vehicle has stopped driving and the parking position is determined, when the parking position is outside the parking area, it is determined that illegal parking has occurred in the parking area. For example, a dedicated parking space for engineering transfer vehicles is set in the workshop. When the engineering transfer vehicle is monitored to be parked near the dedicated parking space, but the center point coordinates of the detection frame of the engineering transfer vehicle are outside the dedicated parking space, it can be determined that illegal parking has occurred. Alternatively, a parking space for social vehicles is set on the road outside the workshop. When a social vehicle is monitored to be parked, when the center point coordinates of the detection frame of the social vehicle are outside the parking space, it can be determined that illegal parking has occurred. This realizes illegal parking monitoring of the parking area, and can promptly detect and prevent illegal parking.

[0103] S207: Determine if a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal.

[0104] When it is determined that a vehicle violation has occurred in a control area, the target terminal associated with the control area can be determined, or the target terminal associated with the vehicle can be determined, and the violation image can be intercepted from the video data, and an alarm message including the violation image can be generated and sent to the target terminal. Exemplarily, the control area is associated with the mobile phone of the patrol officer, and the video graphics or video clips including the vehicle violation can be intercepted from the video data, and the alarm message including the video image or video clip can be sent to the mobile phone of the patrol officer. Of course, the license plate of the vehicle can also be identified to obtain the license plate number, and the owner's mobile phone can be determined by the mobile phone number associated with the license plate number, and the alarm message can be sent to the owner's mobile phone.

[0105] like Figure 4 The flowchart shown is an example of monitoring vehicle violations in the park. After acquiring the video image data, the vehicle target is first identified and tracked, the license plate information is identified, and then it is determined whether the vehicle is a non-motor vehicle. If so, it is determined whether the non-motor vehicle is driving in the motor vehicle lane. If so, it is confirmed that a violation of not driving in the lane has occurred, and further an abnormal alarm is issued. At the same time, the vehicle's driving direction can be identified to determine whether the vehicle's driving direction is opposite to the set direction. If so, an abnormal alarm is issued. In addition, the vehicle's movement state can be identified to determine whether the vehicle has stopped driving. When the vehicle stops driving, it is determined whether the vehicle is in a prohibited parking area. If the vehicle is parked in a prohibited parking area, an abnormal alarm is issued. If the vehicle is not parked in a prohibited parking area, it is determined whether it is parked in the parking area. If not, an abnormal alarm is issued.

[0106] The embodiment of the present invention collects video data of the control area through a camera, obtains the vehicle violation judgment condition of the control area, inputs multiple frames of video images in the video data into the vehicle detection and tracking model to obtain the target detection and tracking result, and when the target detection and tracking result is that there is a vehicle, extracts the license plate color and / or license plate number from the multiple frames of video images to obtain the license plate feature, and generates the vehicle's driving trajectory based on the vehicle detection frame of the multiple frames of video images in the target detection and tracking result, and determines whether the vehicle violation judgment condition is met based on the license plate feature and the driving trajectory. If so, a vehicle violation occurs in the control area, and generates an alarm message and sends it to the target terminal. It is realized that the vehicle violation is determined by extracting the license plate color, license plate number and driving trajectory after collecting video data, and then combining the vehicle violation judgment conditions of different control areas, without manual inspection or manual video viewing to find vehicle violations, which improves the efficiency of finding vehicle violations, and because no manual participation is required, it avoids the problem of manual error in finding vehicle violations, and can prevent vehicle violations in time. Further, different vehicle violation judgment conditions can be set for different control areas in the park, and vehicle violations can be flexibly monitored in various control areas in the park.

[0107] Embodiment 3

[0108] Figure 4 This is a schematic diagram of the structure of a device for monitoring violations of park vehicles provided in Embodiment 3 of the present invention. Figure 4 As shown, the vehicle violation monitoring device in the park includes:

[0109] The video data acquisition module 501 is used to collect video data of the control area through a camera;

[0110] A violation condition acquisition module 502, used to obtain vehicle violation judgment conditions in the control area;

[0111] The vehicle feature extraction module 503 is used to input the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features;

[0112] A violation judgment module 504 is used to judge whether the vehicle violation judgment condition is met according to the vehicle characteristics, and if so, execute an alarm module 505;

[0113] The alarm module 505 is used to determine whether a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal.

[0114] Optionally, the vehicle feature extraction module 503 includes:

[0115] A vehicle detection and tracking unit, used for inputting multiple frames of video images in the video data into the vehicle detection and tracking model to obtain target detection and tracking results;

[0116] A license plate feature extraction unit, configured to extract the license plate color and / or license plate number from the multiple frames of video images to obtain license plate features when the target detection and tracking result indicates that a vehicle exists;

[0117] A driving trajectory generating unit is used to generate the driving trajectory of the vehicle based on the vehicle detection frames of multiple frames of video images in the target detection and tracking results.

[0118] Optionally, the target detection and tracking result includes a vehicle detection frame, and the license plate feature extraction unit includes:

[0119] The vehicle image capture subunit is used to capture the vehicle detection frame area from the multi-frame video image to obtain the vehicle image;

[0120] A license plate area recognition subunit, used to recognize the license plate area from the vehicle image;

[0121] An image binarization subunit, used for binarizing the license plate area to obtain a binary image;

[0122] A license plate character matching subunit, used for extracting a character region from the binary image, and matching the character region with a character template to obtain a license plate number;

[0123] The license plate color extraction subunit is used to extract pixel values ​​of an area other than the character area in the license plate area, and determine the license plate color of the license plate based on the pixel values.

[0124] Optionally, the control area includes a road area, the violation judgment condition includes that a non-motor vehicle is traveling on a motor vehicle road, and the vehicle movement direction is opposite to a preset direction, and the violation judgment module 504 includes:

[0125] A non-motor vehicle judging unit, used to judge whether the vehicle is a non-motor vehicle according to the license plate color and / or license plate number; if so, executing a driving judging unit;

[0126] a driving judgment unit, configured to judge whether the vehicle is driving in a motor vehicle lane based on the driving trajectory of the vehicle, and if so, to execute a non-lane driving determination unit;

[0127] A lane-failure driving determination unit, configured to determine that a lane-failure driving violation occurs in the road area when the vehicle is driving in the motor vehicle lane;

[0128] A driving direction vector determining unit, configured to determine a driving direction vector of the vehicle based on a driving trajectory of the vehicle;

[0129] Angle calculation unit, used to calculate the vector angle based on the following formula:

[0130]

[0131] Among them, (x t ,y t ) represents the vector of the driving direction of the road area, and represents the coordinate point T1(x 1 ,y 1 ) to the coordinate point T2(x 2 ,y 2 ), (x p ,y p ) represents the vehicle's driving direction vector, and represents the adjacent coordinate point P3(x 3 ,y 3 ) to the coordinate point P4(x 4 ,y 4 ) vector;

[0132] An angle determination unit, used to determine whether the vector angle is greater than a preset angle threshold; if so, execute the retrograde determination unit;

[0133] The reverse driving determination unit is used to determine that the vehicle is reverse driving in the road area and that the vehicle is reverse driving in the road area.

[0134] Optionally, the control area includes a prohibited parking area, the violation judgment condition includes that the vehicle is parked in the prohibited parking area, and the violation judgment module 504 includes:

[0135] a stop driving judgment unit, used to judge whether the vehicle has stopped driving according to the driving trajectory; if so, execute a parking position determination unit;

[0136] a parking position determining unit, configured to determine a parking position of the vehicle based on the driving trajectory;

[0137] The no-parking violation determination unit is used to determine that a no-parking violation occurs in the no-parking area when the parking position is within the no-parking area.

[0138] Optionally, the control area further includes a parking area, the violation judgment condition includes that the vehicle is parked outside the parking area, and the violation judgment module 504 includes:

[0139] The illegal parking behavior determination unit is used to determine that illegal parking behavior occurs in the parking area when the parking position is outside the parking area.

[0140] Optionally, the alarm module 505 includes:

[0141] A target terminal determining unit, used to determine a target terminal associated with the control area, or to determine a target terminal associated with the vehicle;

[0142] The alarm information generating and sending unit is used to intercept the illegal image from the video data, generate the alarm information including the illegal image and send it to the target terminal.

[0143] The device for monitoring violations of park vehicles provided in the embodiment of the present invention can execute the method for monitoring violations of park vehicles provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0144] Embodiment 4

[0145] Figure 6 A schematic diagram of the structure of an electronic device 60 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0146] like Figure 6 As shown, the electronic device 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0147] A number of components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0148] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 61 executes the various methods and processes described above, such as a method for monitoring violations of park vehicles.

[0149] In some embodiments, the park vehicle violation monitoring method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the park vehicle violation monitoring method described above can be performed. Alternatively, in other embodiments, the processor 61 can be configured to execute the park vehicle violation monitoring method in any other appropriate manner (e.g., by means of firmware).

[0150] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0152] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0155] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0156] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0157] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring illegal behaviors of vehicles in a park, characterized in that: include: Collect video data of the controlled area through cameras; Obtain vehicle violation judgment conditions in the control area; Inputting the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features; Determining whether the vehicle violation determination condition is met according to the vehicle characteristics; If so, it is determined that a vehicle violation has occurred in the control area, and an alarm message is generated and sent to the target terminal.

2. The method for monitoring violations of park vehicles according to claim 1, characterized in that: The video data is input into a pre-trained vehicle detection and tracking model to obtain vehicle features, including: Inputting multiple frames of video images in the video data into the vehicle detection and tracking model to obtain target detection and tracking results; When the target detection and tracking result indicates that a vehicle exists, extracting the license plate color and / or the license plate number from the multiple frames of video images to obtain the license plate features; The driving trajectory of the vehicle is generated based on the vehicle detection frames of the multiple frames of video images in the target detection and tracking results.

3. The method for monitoring violations of park vehicles according to claim 2, characterized in that: The target detection and tracking result includes a vehicle detection frame, and the license plate color and / or license plate number are extracted from the multiple frames of video images to obtain the license plate features, including: Capturing a vehicle detection frame area from multiple frames of video images to obtain a vehicle image; identifying a license plate area from the vehicle image; Binarizing the license plate area to obtain a binary image; Extracting a character region from the binary image, and matching the character region with a character template to obtain a license plate number; Pixel values ​​of an area other than the character area in the license plate area are extracted, and a license plate color of the license plate is determined based on the pixel values.

4. The method for monitoring violations of park vehicles according to claim 2, characterized in that: The control area includes a road area, the violation judgment condition includes that a non-motor vehicle is traveling on a motor vehicle road, and the vehicle movement direction is opposite to a preset direction, and the determination of whether the vehicle violation judgment condition is met based on the vehicle characteristics includes: Determining whether the vehicle is a non-motor vehicle based on the license plate color and / or license plate number; If so, determining whether the vehicle is traveling in a motor vehicle lane based on the vehicle's driving trajectory; Determining that a non-lane driving violation occurred in the road area when the vehicle was traveling in the motor vehicle lane; Determining a driving direction vector of the vehicle based on a driving trajectory of the vehicle; The vector angle is calculated based on the following formula: Among them, (x t ,y t ) represents the vector of the driving direction of the road area, and represents the vector from the coordinate point T1(x1,y1) to the coordinate point T2(x2,y2) in the preset target driving direction of the road area, (x p ,y p ) represents the driving direction vector of the vehicle, and represents the vector from the adjacent coordinate point P3 (x3, y3) to the coordinate point P4 (x4, y4) on the driving trajectory; Determining whether the vector angle is greater than a preset angle threshold; If so, it is determined that the vehicle is traveling in the wrong direction in the road area, and wrong direction behavior of the vehicle occurs in the road area.

5. The method for monitoring illegal behaviors of vehicles in a park according to claim 2, characterized in that: The control area includes a prohibited parking area, the violation judgment condition includes that the vehicle is parked in the prohibited parking area, and the determining whether the vehicle violation judgment condition is met according to the vehicle characteristics includes: Determining whether the vehicle has stopped driving according to the driving trajectory; If so, determining a parking position of the vehicle based on the driving trajectory; When the parking position is within the no-parking zone, it is determined that a no-parking violation occurs in the no-parking zone.

6. The method for monitoring illegal behaviors of vehicles in a park according to claim 5, characterized in that: The control area also includes a parking area, the violation judgment condition includes that the vehicle is parked outside the parking area, and the determining whether the vehicle violation judgment condition is met according to the vehicle characteristics includes: When the parking position is outside the parking area, it is determined that illegal parking occurs in the parking area.

7. The method for monitoring illegal behaviors of vehicles in a park according to any one of claims 1 to 6, characterized in that: Determine that a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal, including: Determine a target terminal associated with the control area, or determine a target terminal associated with the vehicle; The illegal image is intercepted from the video data, and alarm information including the illegal image is generated and sent to the target terminal.

8. A device for monitoring illegal behaviors of vehicles in a park, characterized in that: include: Video data acquisition module, used to collect video data of the control area through cameras; A violation condition acquisition module, used to obtain vehicle violation judgment conditions in the control area; A vehicle feature extraction module, used for inputting the video data into a pre-trained vehicle detection and tracking model to obtain vehicle features; A violation judgment module, used to judge whether the vehicle violation judgment condition is met according to the vehicle characteristics, and if so, execute the alarm module; The alarm module is used to determine whether a vehicle violation occurs in the control area, generate an alarm message and send it to the target terminal.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for monitoring violations of vehicles in a park according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for monitoring violations of park vehicles according to any one of claims 1 to 7 when executed.

Images