Vehicle illegal parking detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle illegal parking detection method and device, electronic equipment and a storage medium. The method comprises the steps that a to-be-detected video image is acquired; performing feature extraction on the to-be-detected video image, and determining semantic features and position features; determining a detection frame feature vector based on the semantic feature and the position feature; performing classification based on the to-be-detected video image and the detection frame feature vector, and determining an illegal parking probability; and determining an illegal parking vehicle based on the illegal parking probability. According to the embodiment of the invention, the vehicle target identification and tracking accuracy can be improved, so that the illegal parking judgment response is accelerated.

Description

Technical Field

[0001] The present invention relates to the field of smart transportation technology, and in particular to a vehicle illegal parking detection method, a vehicle illegal parking detection device, an electronic device and a storage medium. Background Art

[0002] With the continuous innovation and development of science and technology, new energy and the sharing economy are increasingly permeating every aspect of our lives. Cars and shared bikes are becoming mainstream transportation in an increasing number of cities. This rapid increase in traffic has led to a high incidence of illegal parking. This not only tarnishes a city's image but also leads to road congestion and increases the likelihood of traffic accidents.

[0003] Currently, illegal parking detection primarily relies on automated algorithmic detection combined with manual review. However, due to the high volume of vehicles on the road, existing methods are prone to inaccuracies and mismatches in vehicle detection and location information, leading to inaccurate or delayed parking violation determinations. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention are proposed to provide a vehicle illegal parking detection method, a vehicle illegal parking detection device, an electronic device and a storage medium that overcome the above problems or at least partially solve the above problems.

[0005] In order to solve the above problems, in a first aspect of the present invention, an embodiment of the present invention discloses a method for detecting illegally parked vehicles, comprising:

[0006] Acquire the video image to be detected;

[0007] Extracting features from the video image to be detected to determine semantic features and position features;

[0008] Determining a detection box feature vector based on the semantic feature and the position feature;

[0009] Classify the video image to be detected and the detection frame feature vector to determine the probability of illegal parking;

[0010] Illegally parked vehicles are determined based on the illegal parking probability.

[0011] Optionally, the step of extracting features from the video image to be detected and determining semantic features and position features includes:

[0012] Performing semantic recognition on the video image to be detected to determine a target vehicle detection frame;

[0013] Determining the target vehicle detection frame and the corresponding target vehicle as semantic features;

[0014] Position encoding is performed on the target vehicle detection frame to determine position features.

[0015] Optionally, the step of determining a detection box feature vector based on the semantic feature and the position feature includes:

[0016] Combining the position feature with a preset learnable parameter to obtain an intermediate feature;

[0017] The intermediate features are combined with the semantic features to obtain a detection box feature vector.

[0018] Optionally, the step of performing classification based on the detection frame feature vector to determine the probability of illegal parking includes:

[0019] Preprocessing the video image to be detected to determine a detection frame for an illegal parking area;

[0020] Determine a vehicle history detection frame based on the detection frame feature vector;

[0021] The vehicle history detection frame is compared with the illegal parking area detection frame to determine the illegal parking probability.

[0022] Optionally, the step of comparing the vehicle history detection frame with the illegal parking area detection frame to determine the illegal parking probability includes:

[0023] The coordinate data of the vehicle history detection frame is compared with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability.

[0024] Optionally, the step of comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability includes:

[0025] The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame are input into a preset pointer network. The preset pointer network is used to fit the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

[0026] Optionally, the step of determining illegally parked vehicles based on the illegal parking probability includes:

[0027] In response to the illegal parking probability being greater than a preset probability threshold, the vehicle corresponding to the illegal parking probability is determined as an illegally parked vehicle.

[0028] In a second aspect of the present invention, an embodiment of the present invention discloses a vehicle illegal parking detection device, comprising:

[0029] An acquisition module, used for acquiring the video image to be detected;

[0030] A feature extraction module is used to extract features from the video image to be detected and determine semantic features and position features;

[0031] A feature combination module, configured to determine a detection box feature vector based on the semantic feature and the position feature;

[0032] A classification module, configured to perform classification based on the video image to be detected and the feature vector of the detection frame to determine the probability of illegal parking;

[0033] The illegal parking identification module is used to determine the illegally parked vehicle based on the illegal parking probability.

[0034] In a third aspect of the present invention, an embodiment of the present invention discloses an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the system interface generation method as described above.

[0035] In a fourth aspect of the present invention, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the system interface generation method described above are implemented.

[0036] The embodiments of the present invention include the following advantages:

[0037] The embodiment of the present invention obtains a video image to be detected; performs feature extraction on the video image to be detected to determine semantic features and position features; determines a detection frame feature vector based on the semantic features and the position features; performs classification based on the video image to be detected and the detection frame feature vector to determine the probability of illegal parking; and determines the illegally parked vehicle based on the illegal parking probability. By fusing the vehicle position information into the deep feature extraction, semantic features and the position features are obtained, and feature differences are enhanced, so that the deep features of vehicles with similar appearances will also be different. Using multimodal feature fusion and utilizing the detection frame feature vector to determine whether a vehicle is illegally parked can reduce the number of changes in vehicle identification, improve the accuracy of vehicle target recognition and tracking, and thus speed up the response to illegal parking judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flowchart of a method for detecting illegal parking of a vehicle according to an embodiment of the present invention;

[0039] Figure 2 is a flowchart of another embodiment of a method for detecting illegal parking of a vehicle according to the present invention;

[0040] Figure 3 It is a feature extraction schematic diagram of the present invention;

[0041] Figure 4It is a pointer network processing schematic diagram of the present invention;

[0042] Figure 5 This is a flowchart of an example of a method for detecting illegal parking of a vehicle according to the present invention;

[0043] Figure 6 This is a structural block diagram of an embodiment of a vehicle illegal parking detection device of the present invention;

[0044] Figure 7 This is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0045] Figure 8 This is a structural block diagram of a storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] With the continuous innovation and development of science and technology, new energy and the sharing economy are increasingly permeating every aspect of our lives. Cars and shared bikes have become the mainstream means of transportation in more and more cities. With the rapid growth of traffic, illegal parking is becoming more and more common, which not only leads to road congestion but also increases the probability of traffic accidents.

[0048] At present, illegal parking detection mainly adopts the method of automatic detection by algorithm combined with manual review. However, due to the large number of vehicles on the road, the vehicle detection and position information judgment in the existing methods are prone to inaccuracy or matching errors, which leads to inaccurate or delayed judgment of illegal parking. There is no illegal parking detection method for non-motor vehicles in the current methods. Although the detection method of illegal parking of motor vehicles can be used in theory, since non-motor vehicles may carry people, this will lead to inaccurate vehicle position information, thereby affecting the accuracy and timeliness of the judgment of illegal parking of non-motor vehicles. In addition, most traditional illegal parking judgment methods determine whether the vehicle is in the illegal parking area by calculating the overlapping area, but this method requires manual adjustment of the threshold, so it is less flexible.

[0049] Multi-target tracking algorithms play a vital role in detecting illegally parked vehicles. They can distinguish different vehicle instances and quickly issue parking violation alerts. The Deepsort (target tracking) multi-target tracking algorithm is one of the most commonly used fast tracking algorithms in projects. Although the Deepsort algorithm combines vehicle location information and depth feature information, depth features primarily reflect the local characteristics of a vehicle. Therefore, when two vehicles have similar appearances, the depth feature information may not be able to effectively distinguish them, resulting in matching errors.

[0050] In summary, the prior art has the following problems:

[0051] In the Deepsort algorithm, the deep feature information of vehicles with similar appearances is relatively similar. At this time, the deep features cannot distinguish vehicle instances, resulting in ID (identity) changes, which in turn leads to slow response to illegal parking alarms. Traditional illegal parking judgment involves the adjustment and adaptation of various parameters and has poor flexibility.

[0052] When a non-motor vehicle carries passengers, the detection frame changes significantly, which may cause the motor vehicle illegal parking judgment method to be unable to detect illegal parking.

[0053] Based on the above problems, embodiments of the present invention are proposed.

[0054] Reference Figure 1 , shows a flowchart of a method for detecting illegal parking of a vehicle according to an embodiment of the present invention, wherein the method for detecting illegal parking of a vehicle may specifically include the following steps:

[0055] Step 101: Acquire a video image to be detected;

[0056] In an embodiment of the present invention, when a certain area needs to be monitored for illegal parking of vehicles, a video image to be detected can be obtained.

[0057] Step 102: extracting features from the video image to be detected to determine semantic features and position features;

[0058] The vehicles in the video image to be detected can be identified, the features of each vehicle in the video image to be detected can be extracted, and the semantic features and position features can be determined.

[0059] The vehicles in the video images to be detected include motor vehicles and non-motor vehicles. Motor vehicles refer to vehicles that use an engine as a power source. These vehicles are generally capable of high-speed travel and require drivers with certain driving skills and qualifications. Motor vehicles include various types, such as cars, motorcycles, tractors, battery-powered vehicles, wheeled special-purpose mechanical vehicles, and trailers. Most of these vehicles use internal combustion engines or electric motors as their power source, driving the wheels through a transmission system to achieve travel. Non-motor vehicles refer to vehicles that are driven by human or animal power and travel on the road. These vehicles generally travel at a slower speed and have a relatively simple structure, and do not require the driver to have special driving skills. Non-motor vehicles include bicycles, tricycles, rickshaws, animal-powered vehicles, motorized wheelchairs for the disabled, and electric bicycles.

[0060] Step 103: determining a detection box feature vector based on the semantic feature and the position feature;

[0061] After obtaining the semantic features and position features, the semantic features and position features can be combined to achieve multimodal fusion and determine the detection box feature vector.

[0062] Step 104: Classify the video image to be detected and the feature vector of the detection frame to determine the probability of illegal parking;

[0063] Based on the processing of the video image to be detected, the detection frame feature vector is compared and classified to determine the probability of the currently identified vehicle being illegally parked, that is, the illegal parking probability. The greater the illegal parking probability, the more likely the vehicle is to have parked illegally; conversely, the smaller the illegal parking probability, the less likely the vehicle is to have parked illegally.

[0064] Step 105: Determine illegally parked vehicles based on the illegal parking probability.

[0065] Based on the probability of illegal parking, it is determined whether the vehicle is illegally parked, and then the illegally parked vehicle is determined.

[0066] The embodiment of the present invention obtains a video image to be detected; performs feature extraction on the video image to be detected to determine semantic features and position features; determines a detection frame feature vector based on the semantic features and the position features; performs classification based on the video image to be detected and the detection frame feature vector to determine the probability of illegal parking; and determines the illegally parked vehicle based on the illegal parking probability. By fusing the vehicle position information into the deep feature extraction, semantic features and the position features are obtained, and feature differences are enhanced, so that the deep features of vehicles with similar appearances will also be different. Using multimodal feature fusion and utilizing the detection frame feature vector to determine whether a vehicle is illegally parked can reduce the number of changes in vehicle identification, improve the accuracy of vehicle target recognition and tracking, and thus speed up the response to illegal parking judgment.

[0067] Reference Figure 2 , shows a flowchart of a method for detecting illegal parking of a vehicle according to an embodiment of the present invention, wherein the method for detecting illegal parking of a vehicle may specifically include the following steps:

[0068] Step 201: Acquire a video image to be detected;

[0069] A video image to be detected may be obtained, and the video image to be detected may be a continuous video image in a video stream.

[0070] Step 202: extracting features from the video image to be detected to determine semantic features and position features;

[0071] Feature extraction can be performed on the vehicle in the image to be detected, and semantic features and position features corresponding to the vehicle can be determined in the image to be detected.

[0072] In an optional embodiment of the present invention, the step of extracting features from the video image to be detected and determining semantic features and position features includes: performing semantic recognition on the video image to be detected to determine a target vehicle detection frame; determining the target vehicle detection frame and the corresponding target vehicle as semantic features; and performing position encoding on the target vehicle detection frame to determine position features.

[0073] Semantic recognition can be performed on the video image to be detected, and the road, vehicle, background, etc. can be divided, and a target vehicle detection frame corresponding to the vehicle can be determined from the image. The target vehicle detection frame and the target vehicle corresponding to the target vehicle detection frame can be determined as semantic features.

[0074] The corner points of the target vehicle detection frame are then position-encoded, and the resulting position coordinates are the position features. To reduce the amount of data processing, only the top-left and bottom-right corner points of the target vehicle detection frame can be position-encoded and determined as position features.

[0075] Step 203: determining a detection box feature vector based on the semantic feature and the position feature;

[0076] In an embodiment of the present invention, semantic features and position features may be fused to obtain a detection box feature vector.

[0077] In an optional embodiment of the present invention, the step of determining the detection box feature vector based on the semantic feature and the position feature includes: combining the position feature with a preset learnable parameter to obtain an intermediate feature; and combining the intermediate feature with the semantic feature to obtain a detection box feature vector.

[0078] In practical applications, you can first combine the positional features with the preset learnable parameters to obtain intermediate features. The intermediate features are the features obtained by combining the positional features with the preset learnable parameters. Then, the intermediate features are combined with the semantic features to obtain the detection box feature vector.

[0079] For feature extraction and fusion, please refer to Figure 3 After the vehicle is instantiated, the accumulated illegal parking time of each vehicle is promptly alerted. This is used to count illegally parked vehicles and to vote on the license plate numbers of illegally parked vehicles to improve the accuracy of the alert.

[0080] like Figure 3As shown, a multimodal feature fusion model can be used to extract features. The CNN (convolutional neural network) model extracts semantic features, while the positional encoding extracts positional features. The addition of positional encoding increases the global information about the vehicle in the image. To enhance the fusion of positional and semantic features, a preset learnable parameter wi is added to automatically learn the importance of features during fusion. The final feature vector is composed of the semantic features + wi * the positional features Fi, enhancing feature differentiation and ensuring that the deep features of similar-looking vehicles are distinct.

[0081] Step 204: pre-process the video image to be detected to determine a detection frame of the illegal parking area;

[0082] In the embodiment of the present invention, the detected video image may be pre-processed to determine the illegal parking area, and then determine the illegal parking area detection frame corresponding to the illegal parking area in the image.

[0083] Step 205: determining a vehicle history detection frame based on the detection frame feature vector;

[0084] The vehicle's historical detection frame can be determined from the position of the vehicle's detection frame after position fusion in the detection frame feature vector. Since the video image to be detected can be a continuous multi-frame image, the corresponding vehicle historical detection frame can also be multiple vehicle detection frames corresponding to the same vehicle.

[0085] Step 206: Compare the vehicle history detection frame with the illegal parking area detection frame to determine the illegal parking probability;

[0086] The positions of the illegal parking area detection frame and the vehicle history detection frame can be compared to determine the illegal parking probability corresponding to the vehicle.

[0087] In an optional embodiment of the present invention, the step of comparing the vehicle history detection frame with the illegal parking area detection frame to determine the probability of illegal parking includes: comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the probability of illegal parking.

[0088] The coordinate data of the vehicle history detection frame may be compared with the coordinate data of the illegal parking area detection frame to determine the probability of illegal parking. The coordinate data of the vehicle history detection frame may be the coordinate data of the upper left corner and the lower right corner.

[0089] Specifically, the step of comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the probability of illegal parking includes: inputting the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame into a preset pointer network, and the preset pointer network is used to fit the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

[0090] The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame can be used as model input parameters and input into the preset pointer network. Pointer network models are mainly used to solve combinatorial optimization problems. The number of output points can be changed according to the number of input points, and the network output is a subset of the input set. The network output of the pointer network can be adjusted, and the original probability output can be changed to softmax (normalized exponential function), and the output is the probability value of illegal parking, so that the illegal parking judgment model can automatically learn whether it is an illegally parked vehicle.

[0091] After the preset pointer network receives the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame, it can compare and classify the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame, and output the illegal parking probability through a normalized exponential function.

[0092] For example, you can refer to Figure 4 The vehicle history detection frame and illegal parking area detection frame of each ID (representation) can be used as input. The pointer network input is the coordinate point x1, y1. You can input (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left point of the history detection frame, and (x2, y2) is the coordinate of the lower right point of the history detection frame. The input coordinate point is regarded as a token in NLP (natural language processing), and the illegally parked vehicle coordinate point is modeled as an NLP token. Finally, the model outputs the illegal parking probability through softmax.

[0093] Step 207: Determine illegally parked vehicles based on the illegal parking probability.

[0094] The illegally parked vehicles are determined based on the illegal parking probability. The number of illegally parked vehicles in one or more frames of the video stream to be detected may be empty, that is, there are no illegally parked vehicles in the corresponding monitoring area.

[0095] In an optional embodiment of the present invention, the step of determining an illegally parked vehicle based on the illegal parking probability includes: in response to the illegal parking probability being greater than a preset probability threshold, determining the vehicle corresponding to the illegal parking probability as an illegally parked vehicle.

[0096] In practical applications, it is possible to determine whether the illegal parking probability is greater than a preset probability threshold. If the illegal parking probability is greater than the preset probability threshold, it indicates that the vehicle has illegally parked. In response to the illegal parking probability being greater than the preset probability threshold, the vehicle corresponding to the illegal parking probability can be determined as an illegally parked vehicle.

[0097] When the illegal parking probability is not greater than the preset probability threshold, it means that the vehicle has not engaged in illegal parking.

[0098] The preset probability threshold can be selected according to actual conditions, and the embodiment of the present invention does not impose any specific limitation on this.

[0099] The embodiment of the present invention uses the tracking algorithm of the multimodal feature fusion model to fuse the vehicle position information into the deep feature extraction, enhance the feature difference, and make the deep features of vehicles with similar appearances different. Using the deep features extracted by multimodal feature fusion to replace the feature extraction in deepsort (target tracking) can reduce ID conversion, improve tracking accuracy, and thus speed up the response of illegal parking judgment. In addition, the illegal parking problem is modeled, and the illegal parking detection model is determined based on the pointer network. For each vehicle, it is judged whether it is illegally parked. It only needs to set the minimum confidence level to complete the automated vehicle illegal parking judgment, thereby increasing the flexibility of implementation. The illegal parking detection model focuses on the rigid body target, reduces the impact of the non-rigid body characteristics of people on non-motor vehicles on the illegal parking judgment, and enhances the robustness of the model. The illegal parking detection model solves the problem of resource waste and application limitations caused by the inconsistent illegal parking methods of motor vehicles and non-motor vehicles.

[0100] In order to make the embodiments of the present invention clear to those skilled in the art, reference may be made to Figure 5 , the following is an example:

[0101] It can be divided into two parts: pre-processing and real-time processing. The pre-processing process mainly includes the use of panoramic segmentation technology to assist in the selection of illegal parking areas. The real-time processing process mainly includes: using target detection technology and panoramic segmentation technology to obtain vehicle location information and illegal parking areas; target tracking based on multimodal feature fusion model and using pointer network for illegal parking judgment to determine whether the vehicle is illegally parked.

[0102] It should be noted that for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0103] Reference Figure 6, shows a structural block diagram of an embodiment of a vehicle illegal parking detection device of the present invention, wherein the vehicle illegal parking detection device may specifically include the following modules:

[0104] An acquisition module 601 is used to acquire a video image to be detected;

[0105] A feature extraction module 602 is used to extract features from the video image to be detected and determine semantic features and position features;

[0106] A feature combination module 603 is configured to determine a detection box feature vector based on the semantic feature and the position feature;

[0107] A classification module 604 is configured to perform classification based on the video image to be detected and the feature vector of the detection frame to determine a probability of illegal parking;

[0108] The illegal parking identification module 605 is configured to determine illegally parked vehicles based on the illegal parking probability.

[0109] In an optional embodiment of the present invention, the feature extraction module 602 includes:

[0110] A semantic recognition submodule is used to perform semantic recognition on the video image to be detected and determine a target vehicle detection frame;

[0111] A semantic feature determination submodule, configured to determine the target vehicle detection frame and the corresponding target vehicle as semantic features;

[0112] The position feature determination submodule is used to perform position encoding on the target vehicle detection frame and determine the position features.

[0113] In an optional embodiment of the present invention, the steps of the feature combination module 603 include:

[0114] A first combining submodule, configured to combine the position feature with a preset learnable parameter to obtain an intermediate feature;

[0115] The second combining submodule is used to combine the intermediate features with the semantic features to obtain a detection box feature vector.

[0116] In an optional embodiment of the present invention, the classification module 604 includes:

[0117] A preprocessing submodule, configured to preprocess the video image to be detected and determine a detection frame for an illegal parking area;

[0118] A vehicle history detection frame determination submodule, configured to determine a vehicle history detection frame based on the detection frame feature vector;

[0119] The comparison submodule is used to compare the vehicle history detection frame with the illegal parking area detection frame to determine the probability of illegal parking.

[0120] In an optional embodiment of the present invention, the comparison submodule includes:

[0121] The comparison unit is used to compare the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the probability of illegal parking.

[0122] In an optional embodiment of the present invention, the comparison unit includes:

[0123] The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame are input into a preset pointer network. The preset pointer network is used to fit the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

[0124] In an optional embodiment of the present invention, the illegal parking identification module 605 includes:

[0125] The identification submodule is configured to, in response to the illegal parking probability being greater than a preset probability threshold, determine a vehicle corresponding to the illegal parking probability as an illegally parked vehicle.

[0126] The embodiment of the present invention obtains a video image to be detected; performs feature extraction on the video image to be detected to determine semantic features and position features; determines a detection frame feature vector based on the semantic features and the position features; performs classification based on the video image to be detected and the detection frame feature vector to determine the probability of illegal parking; and determines the illegally parked vehicle based on the illegal parking probability. By fusing the vehicle position information into the deep feature extraction, semantic features and the position features are obtained, and feature differences are enhanced, so that the deep features of vehicles with similar appearances will also be different. Using multimodal feature fusion and utilizing the detection frame feature vector to determine whether a vehicle is illegally parked can reduce the number of changes in vehicle identification, improve the accuracy of vehicle target recognition and tracking, and thus speed up the response to illegal parking judgment.

[0127] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0128] Reference Figure 7 , an embodiment of the present invention further provides an electronic device, including:

[0129] A processor 701 and a storage medium 702, wherein the storage medium 702 stores a computer program executable by the processor 701. When the electronic device is running, the processor 701 executes the computer program to perform the vehicle illegal parking detection method as described in any one of the embodiments of the present invention.

[0130] The vehicle illegal parking detection method comprises:

[0131] Acquire the video image to be detected;

[0132] Extracting features from the video image to be detected to determine semantic features and position features;

[0133] Determining a detection box feature vector based on the semantic feature and the position feature;

[0134] Classify the video image to be detected and the detection frame feature vector to determine the probability of illegal parking;

[0135] Illegally parked vehicles are determined based on the illegal parking probability.

[0136] Optionally, the step of extracting features from the video image to be detected and determining semantic features and position features includes:

[0137] Performing semantic recognition on the video image to be detected to determine a target vehicle detection frame;

[0138] Determining the target vehicle detection frame and the corresponding target vehicle as semantic features;

[0139] Position encoding is performed on the target vehicle detection frame to determine position features.

[0140] Optionally, the step of determining a detection box feature vector based on the semantic feature and the position feature includes:

[0141] Combining the position feature with a preset learnable parameter to obtain an intermediate feature;

[0142] The intermediate features are combined with the semantic features to obtain a detection box feature vector.

[0143] Optionally, the step of performing classification based on the detection frame feature vector to determine the probability of illegal parking includes:

[0144] Preprocessing the video image to be detected to determine a detection frame for an illegal parking area;

[0145] Determine a vehicle history detection frame based on the detection frame feature vector;

[0146] The vehicle history detection frame is compared with the illegal parking area detection frame to determine the illegal parking probability.

[0147] Optionally, the step of comparing the vehicle history detection frame with the illegal parking area detection frame to determine the illegal parking probability includes:

[0148] The coordinate data of the vehicle history detection frame is compared with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability.

[0149] Optionally, the step of comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability includes:

[0150] The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame are input into a preset pointer network. The preset pointer network is used to fit the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

[0151] Optionally, the step of determining illegally parked vehicles based on the illegal parking probability includes:

[0152] In response to the illegal parking probability being greater than a preset probability threshold, the vehicle corresponding to the illegal parking probability is determined as an illegally parked vehicle.

[0153] The embodiment of the present invention obtains a video image to be detected; performs feature extraction on the video image to be detected to determine semantic features and position features; determines a detection frame feature vector based on the semantic features and the position features; performs classification based on the video image to be detected and the detection frame feature vector to determine the probability of illegal parking; and determines the illegally parked vehicle based on the illegal parking probability. By fusing the vehicle position information into the deep feature extraction, semantic features and the position features are obtained, and feature differences are enhanced, so that the deep features of vehicles with similar appearances will also be different. Using multimodal feature fusion and utilizing the detection frame feature vector to determine whether a vehicle is illegally parked can reduce the number of changes in vehicle identification, improve the accuracy of vehicle target recognition and tracking, and thus speed up the response to illegal parking judgment.

[0154] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0155] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0156] Reference Figure 8 An embodiment of the present invention further provides a computer-readable storage medium 801, on which a computer program is stored. When the computer program is run by a processor, the vehicle parking violation detection method as described in any one of the embodiments of the present invention is executed.

[0157] The vehicle illegal parking detection method comprises:

[0158] Acquire the video image to be detected;

[0159] Extracting features from the video image to be detected to determine semantic features and position features;

[0160] Determining a detection box feature vector based on the semantic feature and the position feature;

[0161] Classify the video image to be detected and the detection frame feature vector to determine the probability of illegal parking;

[0162] Illegally parked vehicles are determined based on the illegal parking probability.

[0163] Optionally, the step of extracting features from the video image to be detected and determining semantic features and position features includes:

[0164] Performing semantic recognition on the video image to be detected to determine a target vehicle detection frame;

[0165] Determining the target vehicle detection frame and the corresponding target vehicle as semantic features;

[0166] Position encoding is performed on the target vehicle detection frame to determine position features.

[0167] Optionally, the step of determining a detection box feature vector based on the semantic feature and the position feature includes:

[0168] Combining the position feature with a preset learnable parameter to obtain an intermediate feature;

[0169] The intermediate features are combined with the semantic features to obtain a detection box feature vector.

[0170] Optionally, the step of performing classification based on the detection frame feature vector to determine the probability of illegal parking includes:

[0171] Preprocessing the video image to be detected to determine a detection frame for an illegal parking area;

[0172] Determine a vehicle history detection frame based on the detection frame feature vector;

[0173] The vehicle history detection frame is compared with the illegal parking area detection frame to determine the illegal parking probability.

[0174] Optionally, the step of comparing the vehicle history detection frame with the illegal parking area detection frame to determine the illegal parking probability includes:

[0175] The coordinate data of the vehicle history detection frame is compared with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability.

[0176] Optionally, the step of comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability includes:

[0177] The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame are input into a preset pointer network. The preset pointer network is used to fit the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

[0178] Optionally, the step of determining illegally parked vehicles based on the illegal parking probability includes:

[0179] In response to the illegal parking probability being greater than a preset probability threshold, the vehicle corresponding to the illegal parking probability is determined as an illegally parked vehicle.

[0180] The embodiment of the present invention obtains a video image to be detected; performs feature extraction on the video image to be detected to determine semantic features and position features; determines a detection frame feature vector based on the semantic features and the position features; performs classification based on the video image to be detected and the detection frame feature vector to determine the probability of illegal parking; and determines the illegally parked vehicle based on the illegal parking probability. By fusing the vehicle position information into the deep feature extraction, semantic features and the position features are obtained, and feature differences are enhanced, so that the deep features of vehicles with similar appearances will also be different. Using multimodal feature fusion and utilizing the detection frame feature vector to determine whether a vehicle is illegally parked can reduce the number of changes in vehicle identification, improve the accuracy of vehicle target recognition and tracking, and thus speed up the response to illegal parking judgment.

[0181] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0182] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, apparatus, or computer program products. Thus, embodiments of the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware. Furthermore, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0186] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0187] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.

[0188] The above is a detailed introduction to a vehicle illegal parking detection method, a vehicle illegal parking detection device, an electronic device and a storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A vehicle illegal parking detection method, characterized in that: include: Acquire the video image to be detected; Extracting features from the video image to be detected to determine semantic features and position features; Determine a detection box feature vector based on the semantic feature and the position feature; Classification is performed based on the video image to be detected and the feature vector of the detection frame to determine the probability of illegal parking; An illegally parked vehicle is determined based on the illegal parking probability.

2. The method according to claim 1, characterized in that The step of extracting features from the video image to be detected and determining semantic features and position features comprises: Performing semantic recognition on the video image to be detected to determine a target vehicle detection frame; Determining the target vehicle detection frame and the corresponding target vehicle as semantic features; Position encoding is performed on the target vehicle detection frame to determine position features.

3. The method according to claim 2, characterized in that The step of determining the detection box feature vector based on the semantic feature and the position feature comprises: Combining the position feature with a preset learnable parameter to obtain an intermediate feature; The intermediate feature is combined with the semantic feature to obtain a detection box feature vector.

4. The method according to claim 3, characterized in that The step of classifying based on the detection frame feature vector and determining the probability of illegal parking includes: Preprocessing the video image to be detected to determine a detection frame of an illegal parking area; Determine a vehicle history detection frame based on the detection frame feature vector; The vehicle history detection frame and the illegal parking area detection frame are compared to determine the illegal parking probability.

5. The method according to claim 4, characterized in that The step of comparing the vehicle history detection frame with the illegal parking area detection frame to determine the illegal parking probability includes: The coordinate data of the vehicle history detection frame is compared with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability.

6. The method according to claim 4, characterized in that The step of comparing the coordinate data of the vehicle history detection frame with the coordinate data of the illegal parking area detection frame to determine the illegal parking probability includes: The coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame are input into a preset pointer network. The preset pointer network is used to fit the coordinate data of the vehicle history detection frame and the coordinate data of the illegal parking area detection frame through a normalized exponential function, and output the probability of illegal parking.

7. The method according to claim 1, characterized in that The step of determining the illegally parked vehicle based on the illegal parking probability comprises: In response to the illegal parking probability being greater than a preset probability threshold, the vehicle corresponding to the illegal parking probability is determined as an illegally parked vehicle.

8. A vehicle illegal parking detection device, characterized in that: include: An acquisition module, used for acquiring a video image to be detected; A feature extraction module is used to extract features from the video image to be detected and determine semantic features and position features; A feature combination module, used to determine a detection box feature vector based on the semantic feature and the position feature; A classification module, used for classifying based on the video image to be detected and the feature vector of the detection frame to determine the probability of illegal parking; The illegal parking identification module is used to determine the illegally parked vehicle based on the illegal parking probability.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the vehicle illegal parking detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle illegal parking detection method as described in any one of claims 1 to 7 are implemented.

Images