Sidewalk parking violation detection method based on target detection and semantic segmentation

Abstract

The application discloses a sidewalk illegal parking detection method based on target detection and semantic segmentation, and applies to the technical field of data processing.The method comprises the following steps: image information of a monitoring area is obtained by image acquisition on a target area; edge identification information is determined by performing edge identification on the image information of the monitoring area; a sidewalk area division node is determined based on the edge identification information; a monitoring target feature is obtained by obtaining an illegal vehicle monitoring target; an illegal monitoring identification model is constructed based on the sidewalk area division node, the monitoring target feature and a semantic segmentation model structure; real-time monitoring area image information of an image monitoring device is input into the illegal monitoring identification model to obtain a vehicle parking result; and illegal parking feedback information is sent when the vehicle parking result is yes.The technical problem that the manpower cost is large, the monitoring cost is high and the monitoring often has missed judgment in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of data processing, and more particularly to a method for detecting illegal parking on sidewalks based on object detection and semantic segmentation. Background Technology

[0002] With the increase in car ownership, illegal parking on sidewalks occurs frequently. Current technology for monitoring illegal parking on sidewalks relies heavily on manual methods. Monitoring illegal parking in multiple areas requires significant manpower, resulting in high costs and frequent instances of missed detections.

[0003] Therefore, in the existing technology, the monitoring of illegal parking on sidewalks is labor-intensive and costly, and there are often technical problems such as missed detections. Summary of the Invention

[0004] This application provides a method for detecting illegal parking on sidewalks based on object detection and semantic segmentation, which solves the technical problems of high manpower costs, high monitoring costs, and frequent missed detections in existing technologies for monitoring illegal parking on sidewalks.

[0005] This application provides a method for detecting illegal parking on sidewalks based on target detection and semantic segmentation, comprising: acquiring images of a target area using an image monitoring device to obtain image information of the monitoring area; performing edge recognition on the image information of the monitoring area to determine edge recognition information; determining sidewalk area division nodes based on the edge recognition information; obtaining the illegal vehicle monitoring target, performing feature analysis on the illegal vehicle, and constructing monitoring target features; constructing an illegal parking monitoring and recognition model based on the sidewalk area division nodes, monitoring target features, and semantic segmentation model structure; interacting with the real-time monitoring area image information of the image monitoring device, inputting the real-time monitoring area image information into the illegal parking monitoring and recognition model to obtain the vehicle parking result; and sending illegal parking feedback information when the vehicle parking result is positive.

[0006] This application also provides a pedestrian crossing illegal parking detection system based on target detection and semantic segmentation, comprising: an image acquisition module for acquiring images of a target area through an image monitoring device to obtain image information of the monitoring area; an edge recognition information determination module for performing edge recognition on the image information of the monitoring area to determine edge recognition information; a segmentation node acquisition module for determining pedestrian crossing area segmentation nodes based on the edge recognition information; a monitoring target feature construction module for obtaining illegal vehicle monitoring targets, performing feature analysis on illegal vehicles, and constructing monitoring target features; a model construction module for constructing an illegal parking monitoring and recognition model based on the pedestrian crossing area segmentation nodes, monitoring target features, and semantic segmentation model structure; a monitoring and recognition module for interacting with the real-time monitoring area image information of the image monitoring device, inputting the real-time monitoring area image information into the illegal parking monitoring and recognition model to obtain vehicle parking results; and an illegal parking feedback module for sending illegal parking feedback information when the vehicle parking result is positive.

[0007] This application also provides an electronic device, including:

[0008] Memory, used to store executable instructions;

[0009] The processor, when executing the executable instructions stored in the memory, implements the pedestrian crossing illegal parking detection method based on object detection and semantic segmentation provided in this application.

[0010] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the sidewalk illegal parking detection method based on object detection and semantic segmentation provided in this application.

[0011] This application proposes a method for detecting illegal parking on sidewalks based on object detection and semantic segmentation. The method acquires images of the target area to obtain monitoring area image information. Edge recognition is performed on the monitoring area image information to determine edge recognition information. Based on the edge recognition information, sidewalk area division nodes are determined. The illegal vehicle monitoring target is obtained, and monitoring target features are constructed. Based on the sidewalk area division nodes, monitoring target features, and semantic segmentation model structure, an illegal parking monitoring and recognition model is constructed. Real-time monitoring area image information from an interactive image monitoring device is input into the illegal parking monitoring and recognition model to obtain the vehicle parking result. When the vehicle parking result is positive, illegal parking feedback information is sent. This method achieves real-time monitoring of illegal parking behavior on sidewalks, further reducing monitoring costs and improving monitoring accuracy. It solves the technical problems of high manpower costs, high monitoring costs, and frequent missed detections in existing sidewalk illegal parking monitoring technologies.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments of this disclosure will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0014] Figure 1 A flowchart illustrating the pedestrian crossing violation detection method based on object detection and semantic segmentation provided in this application embodiment;

[0015] Figure 2 A schematic diagram illustrating the process of determining edge recognition information in the sidewalk illegal parking detection method based on object detection and semantic segmentation provided in the embodiments of this application;

[0016] Figure 3 A flowchart illustrating the process of constructing a violation monitoring and identification model for the sidewalk violation detection method based on object detection and semantic segmentation provided in this application embodiment;

[0017] Figure 4 A schematic diagram of the system structure for the sidewalk illegal parking detection method provided in the embodiments of this application;

[0018] Figure 5 This is a schematic diagram of the system electronic device for the sidewalk illegal parking detection method based on object detection and semantic segmentation provided in an embodiment of the present invention.

[0019] Explanation of reference numerals in the attached figures: Image acquisition module 11, Edge recognition information determination module 12, Node segmentation acquisition module 13, Target feature construction module 14, Model construction module 15, Monitoring and recognition module 16, Illegal parking feedback module 17, Processor 31, Memory 32, Input device 33, Output device 34. Detailed Implementation

[0020] Example 1

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0023] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only.

[0025] While this application makes various references to certain modules of the system according to embodiments of this application, any number of different modules may be used and run on user terminals and / or servers. These modules are merely illustrative, and different aspects of the system and method may use different modules.

[0026] This application uses flowcharts to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0027] like Figure 1 As shown in the embodiments of this application, a method for detecting illegal parking on sidewalks based on object detection and semantic segmentation is provided. The method includes:

[0028] S10: Acquire images of the target area using image monitoring equipment to obtain image information of the monitored area;

[0029] S20: Perform edge recognition on the image information of the monitored area to determine the edge recognition information;

[0030] S30: Based on the edge recognition information, determine the pedestrian walkway area division nodes;

[0031] Specifically, with the increase in car ownership, illegal parking on sidewalks occurs frequently. Current technologies for monitoring illegal parking on sidewalks are costly and often result in missed detections. To address these issues, image monitoring equipment is used to acquire images of the target area, specifically the sidewalk requiring parking monitoring, obtaining image information of the monitoring area. Subsequently, based on a watershed segmentation algorithm, edge recognition is performed on the monitoring area image information to determine edge recognition information. Based on this edge recognition information, the sidewalk area is further determined, and then the sidewalk area segmentation nodes, i.e., the nodes defining the sidewalk area's range, are identified. Obtaining these sidewalk area range nodes facilitates the construction of segmentation training data, thus providing data support for subsequent identification of the sidewalk area.

[0032] like Figure 2 As shown, the method S20 provided in this application embodiment further includes:

[0033] S21: Set a starting point based on the image information of the monitored area;

[0034] S22: Set a distance constraint threshold based on the grayscale value distribution of the image information of the monitoring area;

[0035] S23: Fit the liquid level rise by starting point, and generate a watershed line when the liquid levels corresponding to any starting point intersect;

[0036] S24: When the liquid level rises to the maximum gray value of the image, stop fitting, perform image segmentation based on all watershed lines, and determine the edge recognition information.

[0037] Specifically, when performing edge recognition on the image information of the monitored area and determining the edge recognition information, a watershed segmentation algorithm is used. Based on the image information of the monitored area, a starting point is set, where the starting point is the pixel with the minimum gray value of the grayscale image, and there can be multiple pixels with minimum values. A distance constraint threshold is also set, with the specific pixel distance value set according to the actual application environment. Liquid level rise fitting is performed using the starting point, that is, rising fitting is performed based on the grayscale data. When the liquid surfaces corresponding to any starting point (i.e., different minimum values) intersect, a watershed line is generated. Fitting stops when the liquid level rises to the maximum grayscale value of the image. Image segmentation is then performed based on all watershed lines to determine the edge recognition information.

[0038] S40: Obtain the monitoring targets of illegal vehicles, conduct feature analysis on illegal vehicles, and construct the monitoring target features;

[0039] S50: Based on the pedestrian area division nodes, monitoring target features, and semantic segmentation model structure, construct a violation monitoring and identification model;

[0040] S60: Real-time monitoring area image information of the interactive image monitoring device; input the real-time monitoring area image information into the violation monitoring and identification model to obtain the vehicle parking result;

[0041] S70: When the vehicle parking result is yes, send illegal parking feedback information.

[0042] Specifically, the process involves acquiring monitoring targets for illegally parked vehicles, performing feature analysis on these vehicles, and constructing monitoring target features. These features include various labeled features of multiple illegally parked vehicle monitoring targets. Feature analysis of the illegally parked vehicles is conducted through manual labeling to obtain the monitoring target features. Subsequently, based on the pedestrian zone division nodes, monitoring target features, and semantic segmentation model structure, a violation monitoring and identification model is constructed. Further, real-time image information from the interactive image monitoring device is input into the violation monitoring and identification model. This model determines whether the pedestrian zone division nodes within the real-time monitoring area contain vehicle features, thus obtaining the vehicle parking result. When the vehicle parking result is yes, meaning the pedestrian zone division nodes within the real-time monitoring area contain vehicle features, a violation parking feedback message is sent; otherwise, no violation parking behavior exists in the corresponding pedestrian zone within the monitoring area. This achieves real-time monitoring of illegal parking behavior on pedestrian walkways, further reducing monitoring costs and improving monitoring accuracy.

[0043] like Figure 3 As shown, the method S50 provided in this application embodiment further includes:

[0044] S51: Use the pedestrian area to divide nodes, construct segmentation training data, train the neural network sub-model using the segmentation training data, and construct the target area segmentation sub-model.

[0045] S52: Based on the features of the monitored target, construct identification training data, use the identification training data to train the semantic segmentation model structure, and construct a semantic segmentation sub-model;

[0046] S53: Connect the target region segmentation sub-model and the semantic segmentation sub-model to construct the violation monitoring and identification model.

[0047] Specifically, a violation monitoring and identification model is constructed. Using the pedestrian walkway area segmentation nodes, segmentation training data is built. This segmentation training data is then used to train a neural network sub-model, constructing a target area segmentation sub-model. That is, using the pedestrian walkway area segmentation nodes, pedestrian walkway images and segmentation training data containing the corresponding pedestrian walkway area segmentation nodes are acquired. Supervised training of the neural network sub-model is performed using this segmentation training data until the model's output area segmentation nodes meet a predetermined accuracy rate. The trained model, the target area segmentation sub-model, is then obtained. Subsequently, based on the monitoring target features, i.e., the obtained violation vehicle monitoring targets, feature analysis is performed on the violation vehicles. In the actual feature analysis process, image annotation can be used to obtain the vehicle's identification features, and identification training data is constructed. This identification training data is used to train the semantic segmentation model structure until the model can accurately identify and classify the vehicle's identification features, and the annotation results meet a predetermined accuracy rate. The semantic segmentation sub-model is then constructed. Furthermore, the target area segmentation sub-model and the semantic segmentation sub-model are connected to construct the violation monitoring and identification model. Finally, the target area segmentation sub-model and the semantic segmentation sub-model are connected to identify the vehicle identification features within the pedestrian area segmentation nodes, thereby enabling the monitoring of illegal vehicles and completing the construction of the violation monitoring and identification model.

[0048] The method S50 provided in this embodiment further includes:

[0049] S54: Define background target features and construct background training data based on the background target features;

[0050] S55: The background training data is used as incremental training data to perform incremental learning on the semantic segmentation sub-model to obtain an incremental violation monitoring and identification model. The incremental violation monitoring and identification model is used to segment and identify vehicles and background buildings.

[0051] Specifically, background target features are defined, which include specific building features, such as the features of a public toilet. Background training data is then constructed based on these background target features. Subsequently, the background training data is used as incremental training data to incrementally learn the semantic segmentation sub-model until the model can accurately identify and classify the background building features, and the labeling results meet a predetermined accuracy rate. This completes the incremental learning process, resulting in an incremental violation monitoring and identification model, which is used for segmenting and identifying vehicles and background buildings.

[0052] The method S55 provided in this application embodiment further includes:

[0053] S551: When the background target feature recognition result and the monitoring target feature recognition result output by the incremental violation monitoring and identification model both meet the preset conditions, the real-time monitoring area image information is marked using the background target feature recognition result and the monitoring target feature recognition result.

[0054] S552: Input the real-time monitoring area image information after the labeling process into the preset SlowFast model. The preset SlowFast model includes a fast processing recognition channel and a slow processing recognition channel. The fast processing recognition channel has a first video acquisition step size, and the slow processing recognition channel has a second video acquisition step size. The first video acquisition step size is smaller than the second video acquisition step size.

[0055] S553: ​​The fast processing recognition channel and the slow processing recognition channel in the SlowFast model are used to extract video information from the real-time monitoring area image information and identify vehicle-related dynamic people. The fast processing recognition channel is used to recognize people's actions, and the slow processing recognition channel is used to recognize people.

[0056] S554: Based on the recognition results of human motion recognition and human recognition, determine whether a human has entered the background target;

[0057] S555: When a person is confirmed to have entered the background target, set the interval recognition time;

[0058] S556: Based on the interval recognition time, the image monitoring device is activated to collect images of the target vehicle, obtain real-time monitoring area image information, and perform violation monitoring and identification to determine the vehicle parking result.

[0059] Specifically, when the background target feature recognition result and the monitoring target feature recognition result output by the incremental violation monitoring and recognition model both meet preset conditions, that is, when a specific background is identified and illegal parking behavior exists, the real-time monitoring area image information is marked using the background target feature recognition result and the monitoring target feature recognition result. Subsequently, the marked real-time monitoring area image information is input into a preset SlowFast model. The SlowFast model is a dual-path recognition model for video recognition. The preset SlowFast model includes a fast processing recognition channel and a slow processing recognition channel. The fast processing recognition channel has a first video acquisition step size, and the slow processing recognition channel has a second video acquisition step size, wherein the first video acquisition step size is smaller than the second video acquisition step size. The fast processing recognition channel is used to acquire the image at a high refresh rate according to the first video acquisition step size. The slow processing recognition channel is used to acquire the image at a low refresh rate according to the second video acquisition step size. Subsequently, the fast processing and slow processing recognition channels in the SlowFast model are used to extract video information from the real-time monitoring area image information and identify vehicle-related dynamic figures. The fast processing recognition channel is used for figure action recognition, and the slow processing recognition channel is used for figure identification. Vehicle-related dynamic figures refer to those entering and exiting vehicles. Further, based on the figure action recognition and figure identification results from the SlowFast model, it is determined whether a figure has entered the background target. Then, when a figure is confirmed to have entered the background target, an interval recognition time is set, which is the interval between the next detection of the same vehicle's violation. The specific interval recognition time can be set according to actual conditions, such as 30 minutes or less. Based on the interval recognition time, when the interval recognition time is reached, the image monitoring device is activated to acquire images of the monitored target vehicle, obtain real-time monitoring area image information at the interval, perform violation detection and identification, and determine the vehicle's parking result. That is, when the interval recognition time is reached, real-time monitoring area image information after the interval time is acquired, and violation detection and identification are performed to determine the vehicle's parking result, thereby determining whether the vehicle is illegally parked. This enables intelligent monitoring of illegally parked vehicles based on the background buildings of the sidewalk.

[0060] The method S10 provided in this application embodiment further includes:

[0061] S11: Extract the historical illegal parking dataset corresponding to the monitoring area of ​​the image monitoring equipment;

[0062] S12: Based on the historical illegal parking dataset, perform illegal parking time cycle feature analysis to determine the illegal parking time cycle;

[0063] S13: Set the monitoring call step size according to the aforementioned illegal parking time period;

[0064] S14: The monitoring start-up cycle is set using the monitoring call step size. The monitoring start-up cycle is used to control the start-up of the image monitoring device according to the call time step size.

[0065] Specifically, before acquiring real-time monitoring area image information using the interactive image monitoring device, historical illegal parking datasets for the corresponding monitoring area are extracted. These datasets contain specific illegal parking times. Based on these historical datasets, illegal parking time period feature analysis is performed to obtain illegal parking data within different time periods, determining the illegal parking time period. Different illegal parking time periods have preset monitoring call step lengths. The more illegal parkings within a time period, the shorter the monitoring call step length and the higher the monitoring start frequency; conversely, the fewer illegal parkings within a time period, the longer the monitoring call step length and the lower the monitoring start frequency. The specific preset correspondence can be set according to actual needs. Further, the monitoring call step length is set according to the illegal parking time period. Finally, the monitoring start cycle is set using the monitoring call step length. This monitoring start cycle is used to control the start of the image monitoring device according to the call time step.

[0066] The technical solution provided by this invention acquires image information of the target area by image acquisition. Edge recognition is performed on the image information to determine edge recognition information. Based on the edge recognition information, pedestrian crossing area division nodes are determined. Illegal vehicle monitoring targets are obtained, and monitoring target features are constructed. Based on the pedestrian crossing area division nodes, monitoring target features, and semantic segmentation model structure, a violation monitoring and recognition model is constructed. Real-time monitoring area image information from an interactive image monitoring device is input into the violation monitoring and recognition model to obtain vehicle parking results. When the vehicle parking result is positive, violation feedback information is sent. This solves the technical problems of high manpower costs, high monitoring costs, and frequent missed detections in existing pedestrian crossing violation monitoring technologies. It achieves real-time monitoring of pedestrian crossing violation behavior, further reducing monitoring costs and improving monitoring accuracy.

[0067] Example 2

[0068] Based on the same inventive concept as the sidewalk illegal parking detection method based on object detection and semantic segmentation in the foregoing embodiments, this invention also provides a system for the sidewalk illegal parking detection method based on object detection and semantic segmentation. The system can be implemented in hardware and / or software, and is generally integrated into an electronic device to execute the methods provided in any embodiment of this invention. For example... Figure 4 As shown, the system includes:

[0069] Image acquisition module 11 is used to acquire images of the target area through image monitoring equipment and obtain image information of the monitoring area;

[0070] The edge recognition information determination module 12 is used to perform edge recognition on the image information of the monitored area and determine the edge recognition information.

[0071] The node acquisition module 13 is used to determine the pedestrian area division nodes based on the edge recognition information;

[0072] The monitoring target feature construction module 14 is used to obtain monitoring targets of illegal vehicles, perform feature analysis on illegal vehicles, and construct monitoring target features;

[0073] Model building module 15 is used to build a violation monitoring and identification model based on the node division of the pedestrian area, the monitoring target features, and the semantic segmentation model structure.

[0074] The monitoring and identification module 16 is used to input the real-time monitoring area image information of the interactive image monitoring device into the violation monitoring and identification model to obtain the vehicle parking result;

[0075] The illegal parking feedback module 17 is used to send illegal parking feedback information when the vehicle parking result is yes.

[0076] Furthermore, the edge recognition information determination module 12 is also used for:

[0077] Based on the image information of the monitored area, a starting point is set;

[0078] Based on the grayscale value distribution of the image information in the monitored area, a distance constraint threshold is set;

[0079] By fitting the liquid level rise through the starting point, a watershed line is generated when the liquid levels corresponding to any starting point intersect.

[0080] When the liquid level rises to the maximum gray value of the image, the fitting stops, and image segmentation is performed based on all watershed lines to determine the edge recognition information.

[0081] Furthermore, the model building module 15 is also used for:

[0082] The pedestrian area is divided into nodes to construct segmentation training data. The neural network sub-model is trained using the segmentation training data to construct a target area segmentation sub-model.

[0083] Based on the features of the monitored target, identification training data is constructed, and the semantic segmentation model structure is trained using the identification training data to construct a semantic segmentation sub-model;

[0084] The target region segmentation sub-model and the semantic segmentation sub-model are connected to construct the violation monitoring and identification model.

[0085] Furthermore, the model building module 15 is also used for:

[0086] Define background target features, and construct background training data based on the background target features;

[0087] The background training data is used as incremental training data to incrementally learn the semantic segmentation sub-model to obtain an incremental violation monitoring and identification model, which is used to segment and identify vehicles and background buildings.

[0088] Furthermore, the model building module 15 is also used for:

[0089] When the background target feature recognition result and the monitoring target feature recognition result output by the incremental violation monitoring and identification model both meet the preset conditions, the real-time monitoring area image information is marked using the background target feature recognition result and the monitoring target feature recognition result.

[0090] The real-time monitoring area image information after labeling is input into a preset SlowFast model. The preset SlowFast model includes a fast processing recognition channel and a slow processing recognition channel. The fast processing recognition channel has a first video acquisition step size, and the slow processing recognition channel has a second video acquisition step size. The first video acquisition step size is smaller than the second video acquisition step size.

[0091] The fast processing recognition channel and the slow processing recognition channel in the SlowFast model are used to extract video information from the real-time monitoring area image information and identify vehicle-related dynamic people. The fast processing recognition channel is used for person action recognition, and the slow processing recognition channel is used for person recognition.

[0092] Based on the results of human motion recognition and human recognition, determine whether a human has entered the background target;

[0093] When a person is confirmed to have entered the background target, a set interval recognition time is set;

[0094] Based on the interval recognition time, the image monitoring device is activated to acquire images of the target vehicle, obtain real-time monitoring area image information, and perform violation monitoring and identification to determine the vehicle parking result.

[0095] Furthermore, the image acquisition module 11 is also used for:

[0096] Extract historical illegal parking datasets for the monitoring areas corresponding to the image monitoring equipment;

[0097] Based on the historical illegal parking dataset, the characteristics of illegal parking time cycles are analyzed to determine the illegal parking time cycles.

[0098] Based on the aforementioned illegal parking time period, the monitoring call step size is set;

[0099] The monitoring start-up cycle is set using the monitoring call step size, which is used to control the start-up of the image monitoring device according to the call time step size. The various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; furthermore, the specific names of each functional unit are only for easy differentiation and are not intended to limit the scope of protection of this invention.

[0100] Example 3

[0101] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention, showing a block diagram of an exemplary electronic device suitable for implementing the embodiments of the present invention. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. Figure 5 As shown, the electronic device includes a processor 31, a memory 32, an input device 33, and an output device 34; the number of processors 31 in the electronic device can be one or more. Figure 5 Taking a processor 31 as an example, the processor 31, memory 32, input device 33, and output device 34 in an electronic device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0102] The memory 32, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the pedestrian crossing illegal parking detection method based on object detection and semantic segmentation in this embodiment of the invention. The processor 31 executes various functional applications and data processing of the computer device by running the software programs, instructions, and modules stored in the memory 32, thereby implementing the aforementioned pedestrian crossing illegal parking detection method based on object detection and semantic segmentation.

[0103] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for detecting illegal parking on sidewalks based on object detection and semantic segmentation, characterized in that, include: Image monitoring equipment is used to acquire images of the target area to obtain image information of the monitored area; Edge recognition is performed on the image information of the monitored area to determine the edge recognition information; Based on the edge recognition information, the pedestrian walkway area division nodes are determined; Obtain the monitoring targets of illegal vehicles, conduct feature analysis on illegal vehicles, and construct the monitoring target features; Based on the pedestrian area division nodes, monitoring target features, and semantic segmentation model structure, a violation monitoring and identification model is constructed. The real-time monitoring area image information of the interactive image monitoring device is input into the violation monitoring and identification model to obtain the vehicle parking result; When the vehicle parking result is yes, send a parking violation feedback message; Specifically, based on the pedestrian area division nodes, monitoring target features, and semantic segmentation model structure, a violation monitoring and identification model is constructed, including: The pedestrian area is divided into nodes to construct segmentation training data. The neural network sub-model is trained using the segmentation training data to construct a target area segmentation sub-model. Based on the features of the monitored target, identification training data is constructed, and the semantic segmentation model structure is trained using the identification training data to construct a semantic segmentation sub-model; The target region segmentation sub-model and the semantic segmentation sub-model are connected to construct the violation detection and identification model; Define background target features, and construct background training data based on the background target features; The background training data is used as incremental training data to perform incremental learning on the semantic segmentation sub-model to obtain an incremental violation detection and recognition model. The incremental violation detection and recognition model is used to segment and recognize vehicles and background buildings. When the background target feature recognition result and the monitoring target feature recognition result output by the incremental violation monitoring and identification model both meet the preset conditions, the real-time monitoring area image information is marked using the background target feature recognition result and the monitoring target feature recognition result. The real-time monitoring area image information after labeling is input into a preset SlowFast model. The preset SlowFast model includes a fast processing recognition channel and a slow processing recognition channel. The fast processing recognition channel has a first video acquisition step size, and the slow processing recognition channel has a second video acquisition step size. The first video acquisition step size is smaller than the second video acquisition step size. The fast processing recognition channel and the slow processing recognition channel in the SlowFast model are used to extract video information from the real-time monitoring area image information and identify vehicle-related dynamic people. The fast processing recognition channel is used for person action recognition, and the slow processing recognition channel is used for person recognition. Based on the results of human motion recognition and human recognition, determine whether a human has entered the background target; When a person is confirmed to have entered the background target, a set interval recognition time is set; Based on the interval recognition time, the image monitoring device is activated to acquire images of the target vehicle, obtain real-time monitoring area image information, and perform violation monitoring and identification to determine the vehicle parking result.

2. The method as described in claim 1, characterized in that, Edge recognition is performed on the image information of the monitored area to determine edge recognition information, including: Based on the image information of the monitored area, a starting point is set; Based on the grayscale value distribution of the image information in the monitored area, a distance constraint threshold is set; By fitting the liquid level rise through the starting point, a watershed line is generated when the liquid levels corresponding to any starting point intersect. When the liquid level rises to the maximum gray value of the image, the fitting stops, and image segmentation is performed based on all watershed lines to determine the edge recognition information.

3. The method as described in claim 1, characterized in that, Before the real-time monitoring area image information of the interactive image monitoring device, it includes: Extract historical illegal parking datasets for the monitoring areas corresponding to the image monitoring equipment; Based on the historical illegal parking dataset, the characteristics of illegal parking time cycles are analyzed to determine the illegal parking time cycles. Based on the aforementioned illegal parking time period, the monitoring call step size is set; The monitoring start-up cycle is set using the monitoring call step size, and the monitoring start-up cycle is used to control the start-up of the image monitoring device according to the call time step size.

4. A pedestrian crossing illegal parking detection system based on object detection and semantic segmentation, characterized in that, The system is used to execute the sidewalk illegal parking detection method based on object detection and semantic segmentation as described in any one of claims 1 to 3, the system comprising: The image acquisition module is used to acquire images of the target area through image monitoring equipment to obtain image information of the monitored area; An edge recognition information determination module is used to perform edge recognition on the image information of the monitored area and determine edge recognition information; The node division acquisition module is used to determine the pedestrian area division nodes based on the edge recognition information; The monitoring target feature construction module is used to obtain monitoring targets of illegal vehicles, perform feature analysis on illegal vehicles, and construct monitoring target features; The model building module is used to construct a violation monitoring and identification model based on the node division of the pedestrian area, the monitoring target features, and the semantic segmentation model structure. The monitoring and identification module is used to input the real-time monitoring area image information of the interactive image monitoring device into the violation monitoring and identification model to obtain the vehicle parking result; The illegal parking feedback module is used to send illegal parking feedback information when the vehicle is parked correctly.

5. An electronic device, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the pedestrian crossing illegal parking detection method based on object detection and semantic segmentation as described in any one of claims 1 to 3.

6. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the sidewalk illegal parking detection method based on object detection and semantic segmentation as described in any one of claims 1-3.

Images