Traffic violation identification method and related device

Abstract

The invention relates to a traffic violation identification method, which is used for identifying violation behaviors of pedestrians or non-motor vehicles in a traffic scene. According to the method, an actual residence area or an actual passing area of pedestrians or non-motor vehicles is determined based on the condition of trace points in a traffic scene in a historical time period; and the target position of the pedestrian or the non-motor vehicle at a certain moment is compared with the position relationship between the preset residence area and the actual residence area or the position relationship between the preset passing area and the actual passing area, so that whether the pedestrian or the non-motor vehicle has the illegal behavior or not and the illegal behavior type of the pedestrian or the non-motor vehicle can be determined. And the recognition efficiency of various types of illegal behaviors is improved. According to the scheme, the illegal behavior can be identified only by acquiring the space-time information of the pedestrians or the non-motor vehicles and comparing whether the positions of the pedestrians or the non-motor vehicles fall into the predetermined area, so that the illegal behavior of the pedestrians or the non-motor vehicles can still be effectively identified in a complex traffic scene.

Description

Technical Field

[0001] This application relates to the technical field of traffic safety, and in particular, to a traffic violation recognition method and related devices. Background Art

[0002] With the acceleration of the urbanization process, the scale of urban traffic has been continuously expanding, and the volume of population flow has increased sharply, resulting in frequent traffic accidents. At present, most urban road traffic accidents occur at intersection locations. The important inducement for slow traffic accidents at intersections lies in the illegal behaviors of pedestrians and non-motor vehicles. Therefore, the accurate recognition of the illegal behaviors of pedestrians and non-motor vehicles is an effective means to reduce safety risks and improve traffic efficiency.

[0003] At present, with the development of artificial intelligence technology, deep learning-based models are often used for the illegal analysis of pedestrians and non-motor vehicles. Specifically, by pre-collecting a large number of videos with pedestrians and marking the pedestrians with illegal behaviors in the videos, a behavior recognition model with time-series analysis ability is trained. Then, the videos collected in the actual traffic scenario are input into the trained behavior recognition model to identify whether there are illegal behaviors of pedestrians.

[0004] However, the model-based illegal behavior analysis method in the related technology can only achieve good results in some small scenarios with low traffic flow. For example, in some complex intersection environments, the traffic flow of pedestrians, non-motor vehicles, and motor vehicles is large and they block each other. In the actual application process, it is difficult to obtain effective pose data of pedestrians or non-motor vehicles, resulting in the behavior recognition model being unable to effectively analyze and obtain the illegal behaviors of pedestrians or non-motor vehicles.

[0005] Therefore, there is an urgent need for a method that can recognize the illegal behaviors of pedestrians and non-motor vehicles in complex traffic scenarios. Summary of the Invention

[0006] This application provides a traffic violation recognition method, which can effectively recognize the illegal behaviors of pedestrians or non-motor vehicles in complex traffic scenarios.

[0007] The first aspect of the present application provides a traffic violation recognition method, which is applied to a violation behavior analysis device such as a server or an edge device. The method includes: First, based on the distribution of multiple trace points in a traffic scenario in a historical time period, determine the actual residence area and / or the actual passing area. The multiple trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments, that is, the multiple trace points in the historical time period are actually a set composed of the trace points obtained at each moment in the multiple moments of the historical time period. Among them, the actual residence area is the area where pedestrians or non-motor vehicles actually reside, and the actual residence area includes a preset residence area, and the preset residence area includes a preset area for pedestrians or non-motor vehicles to reside (such as a safety island), and the actual passing area is the area where pedestrians or non-motor vehicles actually pass.

[0008] Then, obtain a first trace point, where the first trace point is used to indicate the target position of a pedestrian or a non-motor vehicle at a first moment, and the first trace point is a point on a continuous trajectory or a point on a discontinuous trajectory. That is, in practical applications, the first trace point can be any trace point containing spatio-temporal information collected in a traffic scenario. If the first trace point also includes the corresponding user identity information, the first trace point can be a point on a continuous trajectory, and if the first trace point does not include the corresponding user identity information, the first trace point can be a point on a discontinuous trajectory.

[0009] Finally, based on the first position relationship and / or the second position relationship, output the violation behavior corresponding to the first trace point. The first position relationship is the position relationship between the target position and the preset residence area and the actual residence area, and the second position relationship is the position relationship between the target position and the preset passing area and the actual passing area. The preset passing area includes a preset area for pedestrians or non-motor vehicles to pass, and the violation behavior includes illegal residence or illegal passing.

[0010] That is to say, based on the position relationship between the target position and the preset residence area and the actual residence area, and / or the position relationship between the target position and the preset passing area and the actual passing area, the violation behavior corresponding to the first trace point can be determined.

[0011] In this solution, after determining the actual residence area or actual passing area of pedestrians or non-motor vehicles based on the situation of trace points in the traffic scene during a historical time period, by comparing the target position of pedestrians or non-motor vehicles at a certain moment with the preset residence area and the actual residence area, or with the preset passing area and the actual passing area, it is possible to determine whether there are any violations of pedestrians or non-motor vehicles and the types of violations of pedestrians or non-motor vehicles, thereby improving the recognition efficiency of various types of violations. Since this solution only needs to obtain the spatio-temporal information of pedestrians or non-motor vehicles, without the need to obtain the posture data and identity information of pedestrians or non-motor vehicles, and only needs to compare whether the position of pedestrians or non-motor vehicles falls into a pre-determined area to achieve the recognition of violations, this solution can still effectively identify the violations of pedestrians or non-motor vehicles in a complex traffic scene.

[0012] In a possible implementation manner, outputting the violation behavior corresponding to the first trace point specifically includes: when the target position is not within the target area and is within the actual residence area, outputting that the violation behavior corresponding to the first trace point is a violation of residence behavior or a violation of passing behavior; or, when the target position is not within the target area and is within the actual passing area, outputting that the violation behavior of the first trace point is a violation of passing behavior; where the target area includes the preset residence area and the passable area of the preset passing area at the first moment.

[0013] In a possible implementation manner, outputting that the violation behavior corresponding to the first trace point is a violation of residence behavior or a violation of passing behavior specifically includes: when the target position is not within the target area and the preset passing area and is within the actual residence area, outputting that the violation behavior of the first trace point is a violation of residence behavior in the spatial dimension. Specifically, the target position not being within the target area and being within the actual residence area means that the target position is at the outer edge of the preset residence area; and the target position not being within the preset passing area means that the target position is actually outside the preset residence area and in a non-passable area (for example, the target position is on the lane outside the safety island). Therefore, regardless of whether the traffic signal is red or green at the current moment, the area where the target position is located is non-passable, that is, the first trace point belongs to a violation of residence behavior in the spatial dimension.

[0014] Or, when the target position is outside the actual residence area and the preset passing area, outputting that the violation behavior of the first trace point is a violation of passing behavior in the spatial dimension.

[0015] Alternatively, when the target position is not within the target area but within the actual residence area and the preset passage area, the violation behavior of the first trace point is an illegal residence behavior or an illegal passage behavior in the time dimension. That is, at the first moment (i.e., the red light period), the target position where the first trace point is located is not allowed to stay or pass; however, during the green light period, it is not a violation if there are pedestrians or non-motor vehicles at the target position where the first trace point is located.

[0016] In this solution, by comparing the target position of the first trace point with the position relationship between the actual residence area and the preset passage area, it is possible to further determine whether the violation behavior to which the first trace point belongs is a violation behavior in the spatial dimension or a violation behavior in the time dimension, realizing a further distinction of violation behaviors and improving the recognition efficiency of various types of violation behaviors.

[0017] In a possible implementation manner, based on the distribution of multiple trace points in the traffic scenario, the actual residence area and the actual passage area are determined, including: obtaining a traffic map in the traffic scenario, and this traffic map is used to indicate the setting conditions of multiple traffic facilities.

[0018] Then, based on the traffic map, multiple sub-regions are divided, and the multiple sub-regions do not overlap; moreover, the multiple sub-regions obtained by division are used to constitute the entire traffic area indicated in the traffic map.

[0019] Secondly, based on the distribution of multiple trace points in multiple sub-regions, the actual residence area and the actual passage area are determined. The actual residence area is determined based on the time length of the sub-regions having trace points in the historical time period, and the actual passage area is determined based on the situation of the sub-regions around the preset passage area having trace points.

[0020] In this solution, by dividing the traffic map into multiple sub-regions and determining the actual residence area and the actual passage area based on the situation of each sub-region in the multiple sub-regions having trace points in the historical time period, it is possible to further identify the type of violation behavior where the trace points appear based on the determined actual residence area and actual passage area, and improve the recognition efficiency of various types of violation behaviors.

[0021] In a possible implementation manner, determining the actual residence area specifically includes: based on multiple trace points, determining the usage characteristics of each sub-region in the multiple sub-regions, and the usage characteristics are used to indicate the ratio of the duration of trace points existing in the sub-region to the historical time period. Since the duration of the historical time period is fixed, the longer the duration of trace points existing in the sub-region, the higher the usage characteristics of the sub-region, indicating that pedestrians or non-motor vehicles stay or pass more frequently in the sub-region.

[0022] Based on usage features, multiple resident sub-regions for constituting an actual residence area are determined in multiple sub-regions, and the multiple resident sub-regions belong to the multiple sub-regions. That is, based on the usage features of each sub-region, some sub-regions with higher usage features on the outer edge of the preset residence area are screened out and combined with the preset residence area, so that multiple resident sub-regions for constituting the actual residence area can be determined in the multiple sub-regions, and then the determination of the actual residence area is realized.

[0023] In this solution, based on the duration of the trace points appearing in the sub-region to determine the usage feature of the sub-region, and then combining the size of the usage feature of the sub-region and the preset residence area to determine the actual residence area, it can effectively determine the area where pedestrians or non-motor vehicles often stay in the entire traffic area, ensure that an accurate actual residence area can be obtained, and is beneficial to improving the recognition accuracy of subsequent violation behaviors.

[0024] In a possible implementation manner, determining multiple resident sub-regions for constituting an actual residence area in multiple sub-regions based on usage features includes: performing edge detection on the multiple sub-regions based on the usage features of each sub-region in the multiple sub-regions to obtain at least one sub-region located at the edge of the actual residence area; performing spatial clustering on the at least one sub-region to obtain the residence area type corresponding to each sub-region in the at least one sub-region; performing connection processing on the sub-regions under each residence area type based on the convex hull construction method to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-regions.

[0025] In this solution, based on the usage features of the sub-region, by combining edge detection, spatial clustering, and convex hull construction methods, it can effectively realize the determination of the actual residence area in various scenarios, ensure that an accurate actual residence area can be obtained, and is beneficial to improving the recognition accuracy of subsequent violation behaviors.

[0026] In a possible implementation manner, the trajectory coverage rate of the actual traffic area is greater than a preset threshold, and the trajectory coverage rate is the ratio of the number of sub-regions with trace points in the actual traffic area in the historical time period to the total number of sub-regions.

[0027] In this solution, based on the characteristics of pedestrians or non-motor vehicles walking straight when passing through, by setting the condition that the trajectory coverage rate of the actual traffic area needs to be greater than the preset threshold, it can effectively determine the actual traffic area, which is beneficial to improving the recognition accuracy of subsequent violation behaviors.

[0028] In a possible implementation manner, the sizes of different sub-regions in the multiple sub-regions are the same, and the size of the sub-regions in the multiple sub-regions is related to the floor area of pedestrians or non-motor vehicles.

[0029] In a possible implementation, the method further includes: First, determining a target passage area connected to the actual residence area, where the target passage area belongs to the actual passage area. Then, based on multiple trace points, determining the usage characteristics of the target passage area, where the usage characteristics of the target passage area are obtained based on the usage characteristics of the sub-areas included in the target passage area, and the usage characteristics of the sub-areas are used to indicate the ratio of the duration with trace points in the sub-areas to the historical time period. In this way, in response to the first trace point being within the actual residence area, based on the passage probability and residence probability of the sub-area where the first trace point is located, determining the probabilities that the violation behavior of the first trace point belongs to the violation residence behavior and the violation passage behavior respectively, where the passage probability and residence probability are determined based on the usage characteristics of the target passage area.

[0030] Specifically, there are usually two behaviors in the actual residence area, namely residence and passage. That is, the usage characteristics of the actual residence area are composed of the superposition of the passage characteristics and the residence characteristics. Therefore, by using the characteristic that the usage characteristics of the target passage area will be transmitted to the actual residence area, the passage characteristics of the actual residence area can be determined. Furthermore, based on the actual usage characteristics and passage characteristics of the actual residence area, the residence characteristics of the actual residence area can be determined, and finally the passage probability and residence probability of the sub-areas in the actual residence area can be determined.

[0031] In this solution, by using the transmission characteristic of the passage characteristics in the passage area and the characteristic that the usage characteristics of the residence area are obtained by the superposition of the passage characteristics and the residence characteristics, based on the passage characteristics in the passage area, the passage characteristics and residence characteristics in the adjacent actual residence area are obtained, and then the passage probability and residence probability in the actual residence area are further determined, ensuring that the passage probability and residence probability can be determined for the trace points in the actual residence area.

[0032] In a possible implementation, when the first trace point is in the extended area of the target passage area, the passage probability of the sub-area where the first trace point is located is the ratio of the usage characteristics of the target passage area to the usage characteristics of the sub-area where the first trace point is located.

[0033] In a possible implementation, the first trace point is collected by any one or more of the following devices: camera, lidar, and millimeter wave radar.

[0034] In a possible implementation, the preset residence area is located at the intersection position.

[0035] The second aspect of the present application provides a traffic violation recognition device, including: a processing module, configured to determine an actual residence area and / or an actual passage area based on the distribution of a plurality of trace points in a traffic scene during a historical time period, where the plurality of trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments, the actual residence area is the area where pedestrians or non-motor vehicles actually reside and the actual residence area includes a preset residence area, the preset residence area includes a preset area for pedestrians or non-motor vehicles to reside, and the actual passage area is the area where pedestrians or non-motor vehicles actually pass through; an acquisition module, configured to acquire a first trace point, where the first trace point is used to indicate the target position of a pedestrian or a non-motor vehicle at a first moment, and the first trace point is a point on a continuous trajectory or a point on a discontinuous trajectory; the processing module is further configured to output a violation behavior corresponding to the first trace point based on a first position relationship and / or a second position relationship, where the first position relationship is the position relationship between the target position and the preset residence area and the actual residence area, and the second position relationship is the position relationship between the target position and the preset passage area and the actual passage area, the preset passage area includes a preset area for pedestrians or non-motor vehicles to pass through, and the violation behavior includes a violation of residence or a violation of passage.

[0036] In a possible implementation manner, the processing module is specifically configured to: when the target position is not within the target area and is within the actual residence area, output that the violation behavior corresponding to the first trace point is a violation of residence behavior or a violation of passage behavior; or, when the target position is not within the target area and is within the actual passage area, output that the violation behavior of the first trace point is a violation of passage behavior; where the target area includes the preset residence area and the area that can be passed through at the first moment in the preset passage area.

[0037] In a possible implementation manner, the processing module is specifically configured to: when the target position is not within the target area and the preset passage area and is within the actual residence area, output that the violation behavior of the first trace point is a violation of residence behavior in the spatial dimension; when the target position is outside the actual residence area and the preset passage area, output that the violation behavior of the first trace point is a violation of passage behavior in the spatial dimension; or, when the target position is not within the target area and is within the actual residence area and within the preset passage area, output that the violation behavior of the first trace point is a violation of residence behavior or a violation of passage behavior in the time dimension.

[0038] In a possible implementation, the processing module is specifically configured to: obtain a traffic map in a traffic scenario, where the traffic map is used to indicate the settings of multiple traffic facilities; divide the traffic map into multiple sub-regions based on the traffic map, and the multiple sub-regions do not overlap; determine an actual residence area and an actual passage area based on the distribution of multiple trace points in the multiple sub-regions, where the actual residence area is determined based on the time length of the sub-regions having trace points in a historical time period, and the actual passage area is determined based on the situation of the sub-regions around a preset passage area having trace points.

[0039] In a possible implementation, the processing module is specifically configured to: determine the usage characteristics of each sub-region in the multiple sub-regions based on the multiple trace points, where the usage characteristics are used to indicate the ratio of the duration of trace points existing in the sub-region to the historical time period; determine multiple residence sub-regions for forming the actual residence area in the multiple sub-regions based on the usage characteristics, and the multiple residence sub-regions belong to the multiple sub-regions.

[0040] In a possible implementation, the processing module is specifically configured to: perform edge detection on the multiple sub-regions based on the usage characteristics of each sub-region in the multiple sub-regions to obtain at least one sub-region located at the edge of the actual residence area; perform spatial clustering on the at least one sub-region to obtain the residence area type corresponding to each sub-region in the at least one sub-region; perform connection processing on the sub-regions under each residence area type based on the convex hull construction method to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-regions.

[0041] In a possible implementation, the trajectory coverage rate of the actual passage area is greater than a preset threshold, and the trajectory coverage rate is the ratio of the number of sub-regions having trace points in the actual passage area in a historical time period to the total number of sub-regions.

[0042] In a possible implementation, the sizes of different sub-regions in the multiple sub-regions are the same, and the size of the sub-regions in the multiple sub-regions is related to the floor area of pedestrians or non-motor vehicles.

[0043] In a possible implementation, the processing module is further configured to: determine a target passage area connected to the actual residence area, where the target passage area belongs to the actual passage area; determine the usage characteristics of the target passage area based on the multiple trace points, where the usage characteristics of the target passage area are obtained based on the usage characteristics of the sub-regions included in the target passage area, and the usage characteristics of the sub-region are used to indicate the ratio of the duration of trace points existing in the sub-region to the historical time period; in response to a first trace point being located within the actual residence area, output the probabilities that the violation behavior of the first trace point belongs to a violation residence behavior and a violation passage behavior respectively based on the passage probability and the residence probability of the sub-region where the first trace point is located, where the passage probability and the residence probability are determined based on the usage characteristics of the target passage area.

[0044] In a possible implementation, when the first trace point is in the extended area of the target passage area, the passage probability of the sub-area where the first trace point is located is the ratio of the usage characteristics of the target passage area to the usage characteristics of the sub-area where the first trace point is located.

[0045] In a possible implementation, the first trace point is collected by any one or more of the following devices: camera, lidar, and millimeter wave radar.

[0046] In a possible implementation, the preset residence area is located at the intersection position.

[0047] A third aspect of this application provides a traffic violation identification device, which may include a processor. The processor is coupled to a memory, and the memory stores program instructions. When the program instructions stored in the memory are executed by the processor, the method of the first aspect or any implementation manner of the first aspect is implemented. For the steps executed by the processor in each possible implementation manner of the first aspect, reference may be specifically made to the first aspect, and details are not described herein again.

[0048] A fourth aspect of this application provides a computer-readable storage medium, in which a computer program is stored. When it runs on a computer, the computer is enabled to execute the method of the first aspect or any implementation manner of the first aspect.

[0049] A fifth aspect of this application provides a circuit system, which includes a processing circuit configured to execute the method of the first aspect or any implementation manner of the first aspect.

[0050] A sixth aspect of this application provides a computer program product, which when running on a computer enables the computer to execute the method of the first aspect or any implementation manner of the first aspect.

[0051] A seventh aspect of this application provides a chip system, which includes a processor for supporting an electronic device to implement the functions involved in the first aspect or any implementation manner of the first aspect. For example, processing the data and / or information involved in the above method. In a possible design, the chip system further includes a memory for storing the necessary program instructions and data of the electronic device. The chip system may be composed of chips or may include chips and other discrete devices.

[0052] The beneficial effects of the second to seventh aspects described above can refer to the introduction of the first aspect, and details are not described herein again. Description of the Drawings

[0053] Figure 1Schematic diagram of an application scenario of a traffic violation recognition method provided by an embodiment of this application;

[0054] Figure 2 Schematic diagram of the structure of an electronic device 201 provided by an embodiment of this application;

[0055] Figure 3 Schematic diagram of the process of a traffic violation recognition method provided by an embodiment of this application;

[0056] Figure 4 Schematic diagram of an intersection scenario provided by an embodiment of this application;

[0057] Figure 5A Schematic diagram of a target area of an intersection scenario at a certain moment provided by an embodiment of this application;

[0058] Figure 5B Schematic diagram of a target area of an intersection scenario at another moment provided by an embodiment of this application;

[0059] Figure 6 Schematic diagram of the division of an actual residence area and an actual passing area provided by an embodiment of this application;

[0060] Figure 7 Schematic diagram of the recognition of a violation behavior provided by an embodiment of this application;

[0061] Figure 8 Another schematic diagram of the recognition of a violation behavior provided by an embodiment of this application;

[0062] Figure 9 Schematic diagram of determining a target passing area provided by an embodiment of this application;

[0063] Figure 10 Schematic diagram of the process of an intersection violation behavior recognition method provided by an embodiment of this application;

[0064] Figure 11 Schematic diagram of an intersection semantic space coupling model provided by an embodiment of this application;

[0065] Figure 12 Schematic diagram of the process of dividing an actual residence area provided by an embodiment of this application;

[0066] Figure 13 Schematic diagram of determining an actual passing area based on a region growing algorithm provided by an embodiment of this application;

[0067] Figure 14 Schematic diagram of the superposition property of spatial usage features provided by an embodiment of this application;

[0068] Figure 15 Schematic diagram of the transitivity of the space characteristics of passing behaviors provided by an embodiment of the present application;

[0069] Figure 16 Schematic diagram of two actual passing areas with connected actual residence areas provided by an embodiment of the present application;

[0070] Figure 17 Schematic diagram of the transmission of passing behavior characteristics provided by an embodiment of the present application;

[0071] Figure 18 Schematic diagram of the process for identifying illegal behaviors provided by an embodiment of the present application;

[0072] Figure 19 Schematic diagram of the structure of a traffic violation identification device provided by an embodiment of the present application;

[0073] Figure 20 Another schematic diagram of the structure of a traffic violation identification device provided by an embodiment of the present application;

[0074] Figure 21 Schematic diagram of the structure of a computer-readable storage medium provided by an embodiment of the present application. Detailed implementation manners

[0075] The embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Those of ordinary skill in the art will understand that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0076] Terms such as "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0077] In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules does not have to be limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or are inherent to these processes, methods, products, or devices. The naming or numbering of steps that appear in this application does not mean that the steps in the method flow must be executed in the chronological / logical order indicated by the naming or numbering. The named or numbered process steps can be changed in the order of execution according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved.

[0078] For ease of understanding, the technical terms related to the embodiments of this application will be introduced below.

[0079] (1) Transportation facilities

[0080] Transportation facilities refer to the necessary tools (including vehicles, ships, airplanes), mechanical equipment, sites, lines, communication equipment, signal signs, buildings (including stations, warehouses, waiting areas, ticket offices), etc. in transportation. The transportation facilities related to this embodiment will be mainly introduced below.

[0081] Lane: The general term for the part of a road where various vehicles drive, including the express lane and the slow lane.

[0082] Lane: The part of a lane where a single column of vehicles drives.

[0083] Non-motor vehicle lane: The lane on a road specifically for non-motor vehicles to pass through.

[0084] Footpath: The part of a road that is separated by a curbstone, guardrail, or other similar facilities and is specifically for pedestrians to pass through.

[0085] Crosswalk: The walking area marked by zebra lines or other methods on a lane and designated for pedestrians to cross the lane.

[0086] Safety island: A pedestrian crossing facility, which is actually various island-shaped facilities set on the road surface to facilitate pedestrians to stop when crossing the street. Safety islands are generally located in the middle of the road or between different lanes for pedestrians to temporarily stop and avoid vehicles when crossing the road.

[0087] Non-motor vehicle dedicated lane: A passage that is only for non-motor vehicles to drive, and pedestrians and other vehicles are not allowed to pass. When there is a crosswalk at an intersection, the non-motor vehicle dedicated lane should be parallel to the crosswalk.

[0088] (2) Topological relationship

[0089] The topological relationship in this embodiment refers to the relationship that describes the accessibility between traffic facilities and the relationship between directions. Generally speaking, the topological relationships of traffic facilities include the following multiple relationships.

[0090] Connectivity relationship: Describes the accessibility between traffic facilities. If one traffic facility can reach another traffic facility after passing through several intermediate facilities, it is said that there is a connectivity between these two traffic facilities.

[0091] Connection relationship: Describes the relationship that traffic facilities are in spatial contact and accessible, that is, there is no hard or soft isolation between traffic facilities. When the connected traffic facilities all have directions, if the directions of the connected traffic facilities are the same and parallel to each other, it is called an adjacency relationship, such as the adjacent lanes in the same direction have an adjacency relationship.

[0092] Conflict relationship: Describes the relationship that traffic facilities intersect in space and have different directions, such as the conflict relationship between lanes and crosswalks.

[0093] (3) Crossing behavior

[0094] The behavior taken by pedestrians or non-motor vehicles to complete crossing the road is called crossing behavior. Crossing behavior is generally divided into staying behavior and passing behavior.

[0095] (4) Signal control data

[0096] The control scheme data of traffic signal lights, such as the time periods when each traffic signal light is green, yellow, and red respectively.

[0097] Currently, in complex and large-scale traffic scenarios (such as intersection scenarios), fixed single-view video monitoring can often only cover a local area and has low detection accuracy for distant areas. At the same time, the intersection environment is complex, the traffic flow of pedestrians, non-motor vehicles, and motor vehicles is large and they block each other. In addition, the individual characteristics of pedestrians and non-motor vehicles are not obvious, which further limits the tracking of individuals of pedestrians and non-motor vehicles. In addition, the quality of the video taken in the traffic scenario is also affected by the performance of the video shooting equipment and the external environment (such as weather). All these reasons directly lead to the loss of perception of pedestrians and non-motor vehicles, making it difficult to obtain the pose data of pedestrians and non-motor vehicles and the trajectory data with continuous identities. In most cases, only the spatial point data with time information of pedestrians or non-motor vehicles (that is, there are pedestrians or non-motor vehicles with unknown identities at a certain position at a certain moment) can be obtained.

[0098] In view of this, an embodiment of the present application provides a traffic violation recognition method. After determining the actual residence area or actual passing area of a pedestrian or non-motor vehicle based on the situation of the trace points in the traffic scene in a historical time period, by comparing the target position of the pedestrian or non-motor vehicle at a certain moment with the preset residence area and the actual residence area, or with the preset passing area and the actual passing area, it is possible to determine whether there is a violation behavior of the pedestrian or non-motor vehicle and the type of the violation behavior of the pedestrian or non-motor vehicle, thereby improving the recognition efficiency of various types of violation behaviors. Since this solution only needs to obtain the spatio-temporal information of the pedestrian or non-motor vehicle, without obtaining the posture data and identity information of the pedestrian or non-motor vehicle, and only needs to compare whether the position of the pedestrian or non-motor vehicle falls into a pre-determined area to achieve the recognition of the violation behavior, this solution can still effectively recognize the violation behavior of the pedestrian or non-motor vehicle in a complex traffic scene.

[0099] Exemplarily, please refer to Figure 1 , Figure 1 which is a schematic diagram of an application scenario of a traffic violation recognition method provided by an embodiment of the present application. As Figure 1 shown, the application scenario of the traffic violation recognition method includes a data acquisition module, a violation behavior analysis device, and a database. Among them, the traffic violation recognition method provided in this embodiment can be applied to the violation behavior analysis device.

[0100] The data acquisition module may include a lidar, a millimeter-wave radar, and a camera, and is used to collect the trace points of pedestrians or non-motor vehicles in the traffic scene. Specifically, the lidar and the millimeter-wave radar are responsible for collecting the three-dimensional point cloud data in the traffic scene, and the camera is responsible for collecting the two-dimensional images in the traffic scene. By fitting the three-dimensional point cloud data collected by the lidar or the millimeter-wave radar and the two-dimensional images collected by the camera, the trace points of pedestrians or non-motor vehicles can be obtained. Among them, the trace points of pedestrians or non-motor vehicles are essentially a kind of scattered point data that contains spatio-temporal information but loses the identity label and cannot form an ordered trajectory of pedestrians or non-motor vehicles. That is, the trace points can only indicate that there is a pedestrian or non-motor vehicle at a certain position at a certain moment, but cannot give the identity information of the pedestrian or non-motor vehicle.

[0101] Optionally, the lidar, the millimeter-wave radar, and the camera can also work independently to obtain the trace points of pedestrians or non-motor vehicles. For example, the camera deployed in the traffic scene is a three-dimensional camera (i.e., a depth camera), which can detect the distance information of the shooting space, and thus can obtain the position information of the pedestrians or non-motor vehicles captured in the traffic scene to generate the trace points of pedestrians or non-motor vehicles.

[0102] After the data acquisition module acquires trace points, it can send the acquired trace points to the illegal behavior analysis device, and the illegal behavior analysis device can identify the illegal behaviors of pedestrians or non-motor vehicles based on the traffic violation recognition method provided in this embodiment.

[0103] Among them, the data acquisition module can send the acquired trace points to the illegal behavior analysis device; the data acquisition module can also directly send the acquired three-dimensional point cloud data and / or two-dimensional images to the illegal behavior analysis device, and the illegal behavior analysis device can obtain trace points based on the three-dimensional point cloud data and / or two-dimensional images. Among them, the illegal behavior analysis device can be, for example, a server deployed in the cloud or an edge device. When the device in the data acquisition module has high computing power, the illegal behavior analysis device can also be a device in the data acquisition device (such as a three-dimensional camera). That is, the device in the data acquisition module can be used to collect trace points and analyze illegal behaviors of trace points at the same time.

[0104] In addition, after analyzing the illegal behaviors of pedestrians or non-motor vehicles, the illegal behavior analysis device can obtain the images or video clips of the occurrence of illegal behaviors from the data acquisition module, and save these images or video clips and the analysis results of the illegal behaviors to the database for users to view the images or video clips of the occurrence of illegal behaviors at any time.

[0105] Reference can be made to Figure 2 , Figure 2 which is a schematic structural diagram of an electronic device 201 provided in an embodiment of the present application. Among them, Figure 2 the electronic device 201 shown in Figure 1 can be the above-mentioned

[0106] As Figure 2 shown, the electronic device 201 includes a processor 203, and the processor 203 is coupled to the system bus 205. The processor 203 can be one or more processors, and each processor can include one or more processor cores. A display adapter 207, which can drive a display 209, and the display 209 is coupled to the system bus 205. The system bus 205 is coupled to an input / output (I / O) bus through a bus bridge 211. An I / O interface 215 is coupled to the I / O bus. The I / O interface 215 communicates with a variety of I / O devices, such as an input device 217 (such as a touch screen, etc.), an external memory 221 (for example, a hard disk, a floppy disk, an optical disc, or a USB flash drive, a multimedia interface, etc.), a transceiver 223 (which can send and / or receive radio communication signals), a camera 255 (which can capture static and dynamic digital video images), and an external USB port 225. Among them, optionally, the interface connected to the I / O interface 215 can be a USB interface.

[0107] Among them, the processor 203 can be any conventional processor, including a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, or a combination of the above. Optionally, the processor can be a dedicated device such as an ASIC.

[0108] The electronic device 201 can communicate with other electronic devices through the network interface 229 to obtain trace points.

[0109] The processor 203 can communicate with the memory 235 through the system bus 205, and fetch instructions and data in the application program from the memory 235, so as to implement the execution of the program.

[0110] It should be understood that Figure 2 the structure of the shown electronic device does not constitute a limitation on the electronic device in the embodiments of the present application. In practical applications, the electronic device may include more or fewer components than Figure 2 those shown, or combine some components, such as at least one output device such as a speaker, a vibration mechanism, a lamp, etc.; at least one input device such as a keyboard, a mouse, a pickup, etc. The embodiments of the present application will not list them one by one here.

[0111] Please refer to Figure 3 , Figure 3 which is a schematic flowchart of a traffic violation recognition method provided by an embodiment of the present application. As Figure 3 shown, the process of the traffic violation recognition method provided in this embodiment includes the following steps 301-303.

[0112] Step 301, based on the distribution of multiple trace points in a traffic scene in a historical time period, determine an actual residence area and / or an actual passing area. The multiple trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments. The actual residence area is the area where pedestrians or non-motor vehicles actually reside, and the actual residence area includes a preset residence area. The preset residence area includes a preset area for pedestrians or non-motor vehicles to reside. The actual passing area is the area where pedestrians or non-motor vehicles actually pass.

[0113] In this embodiment, the actual residence area is the area where pedestrians or non-motor vehicles actually reside, and the actual residence area includes the preset residence area. The actual residence area can be determined based on the time length of the trace points in multiple sub-areas in the historical time period. The actual passage area is the area where pedestrians or non-motor vehicles actually pass, and the actual passage area can be determined based on the situation of the sub-areas around the preset passage area having trace points. Briefly speaking, the area where pedestrians or non-motor vehicles stay for a long time around the preset residence area can be determined as the actual residence area; the area where pedestrians or non-motor vehicles often pass during the green light period around the preset passage area can be determined as the actual passage area.

[0114] Step 302: Obtain a first trace point, which is used to indicate the target position of a pedestrian or a non-motor vehicle at a first moment, and the first trace point is a point on a continuous trajectory or a point on a discontinuous trajectory.

[0115] In practical applications, the first trace point can be any trace point containing spatio-temporal information collected in a traffic scenario. If the first trace point also includes the corresponding user identity information, the first trace point can be a point on a continuous trajectory. If the first trace point does not include the corresponding user identity information, the first trace point can be a point on a discontinuous trajectory. That is to say, in this embodiment, only the spatio-temporal information of pedestrians or non-motor vehicles is required to identify the illegal behaviors of pedestrians or non-motor vehicles, without the need to obtain other information such as the identity information and posture information of pedestrians or non-motor vehicles.

[0116] Optionally, the first trace point can be collected by any one or more of the following devices: camera, lidar, and millimeter-wave radar. For example, the first trace point is obtained by fitting the two-dimensional image collected by the camera and the three-dimensional point cloud data collected by the lidar; or, the first trace point is obtained by fitting the two-dimensional image collected by the camera and the three-dimensional point cloud data collected by the millimeter-wave radar; or, the first trace point is obtained based on the two-dimensional image collected by the three-dimensional camera and the depth information of each pixel point in the two-dimensional image. Specifically, the method for obtaining the trace point can refer to the existing related technologies and will not be elaborated here.

[0117] Step 303: Output the illegal behavior corresponding to the first trace point based on the first position relationship and / or the second position relationship. The first position relationship is the position relationship between the target position and the preset residence area and the actual residence area, and the second position relationship is the position relationship between the target position and the preset passage area and the actual passage area. The preset passage area includes the preset area for pedestrians or non-motor vehicles to pass. The illegal behaviors include illegal residence or illegal passage.

[0118] In this embodiment, based on the positional relationship between the target position and the preset residence area and the actual residence area, and / or the positional relationship between the target position and the preset passage area and the actual passage area, the violation behavior corresponding to the first trace point can be determined.

[0119] It should be noted that in practical applications, an electronic device that executes the traffic violation recognition method provided in this embodiment can obtain a large number of trace points, and each trace point has corresponding time information and position information, which are respectively used to indicate the positions of different pedestrians or non-motor vehicles at specific moments. In this embodiment, the first trace point among a large number of trace points is taken as an example to introduce the process of the electronic device executing the traffic violation recognition method. In practical applications, the electronic device can identify whether each trace point has a violation behavior based on the spatio-temporal information of the obtained trace points and the preset residence area and preset passage area obtained in step 302.

[0120] In addition, the preset residence area and preset passage area in this embodiment can be determined based on a traffic map indicating the setting conditions of multiple traffic facilities. That is, the traffic map refers to the map of the site where traffic violation recognition needs to be performed. For example, when the site where traffic violation recognition needs to be performed is an intersection, the traffic map is the map of the intersection. And the traffic map is drawn based on the traffic facilities under the actual site. For example, the traffic facilities under the actual site are scaled down proportionally to draw the traffic map. Therefore, the traffic map includes multiple traffic facilities under the actual site and indicates the setting conditions of each traffic facility among the multiple traffic facilities (that is, the position where each traffic facility is respectively set). In addition, the size relationship and topological relationship between different traffic facilities can also be presented in the traffic map. For example, the longitude and latitude where each traffic facility is located are marked in the traffic map; or for another example, in the traffic map, a two-dimensional coordinate system can be constructed with a certain point as the origin, and the coordinates of each traffic facility are marked.

[0121] In this way, after determining the setting positions of multiple traffic facilities, the areas where pedestrians or non-motor vehicles can stay or pass can be determined based on the type of each traffic facility, and then the preset residence area and preset passage area can be determined. Among them, the preset residence area refers to the area where pedestrians or non-motor vehicles are pre-set to stay in the traffic scene, and the preset passage area refers to the area where pedestrians or non-motor vehicles are pre-set to pass in the traffic scene.

[0122] Specifically, in the traffic scenario at intersections, a safety island is a traffic facility for pedestrians or non-motor vehicles to stay. Therefore, the area where the safety island is located among multiple traffic facilities belongs to the preset staying area, that is, the preset staying area is located at the intersection. Roads such as sidewalks, crosswalks, and non-motor vehicle lanes are traffic facilities for pedestrians or non-motor vehicles to pass through. Therefore, the areas where sidewalks, crosswalks, non-motor vehicle lanes, etc. are located among multiple traffic facilities belong to the preset passing area.

[0123] It should be noted that the preset staying area can include a continuous area or multiple discontinuous areas. For example, when the area for pedestrians or non-motor vehicles to stay in the traffic map only includes one safety island, the preset staying area only includes the area where the safety island is located; for another example, when the area for pedestrians or non-motor vehicles to stay in the traffic map includes multiple separate safety islands, the preset staying area includes the areas where multiple safety islands are located (i.e., multiple discontinuous areas).

[0124] Similarly, the preset passing area can be a continuous area or include multiple discontinuous areas. For example, when the area for pedestrians or non-motor vehicles to pass through in the traffic map only includes one crosswalk, the preset passing area only includes the area where the crosswalk is located; for another example, when the area for pedestrians or non-motor vehicles to pass through in the traffic map includes multiple roads such as sidewalks, crosswalks, and non-motor vehicle lanes, the preset passing area includes the areas where multiple roads such as sidewalks, crosswalks, and non-motor vehicle lanes are located (i.e., multiple discontinuous areas).

[0125] It can be understood that since the preset staying area is an area for pedestrians or non-motor vehicles to stay, pedestrians or non-motor vehicles appearing in the preset staying area at any time do not belong to violations. For the preset passing area, since the preset passing area may have areas that can only be passed by pedestrians or non-motor vehicles at specific times (such as a crosswalk that can only be passed during the green light period), or may have areas that can be passed by pedestrians or non-motor vehicles at any time (such as a sidewalk), pedestrians or non-motor vehicles do not necessarily not belong to violations when appearing in the preset passing area at any time.

[0126] Specifically, by combining the preset residence area and the traffic facilities corresponding to each part of the preset passage area, it is possible to determine the target area where pedestrians or non-motor vehicles appear and do not violate the regulations at the first moment. That is, the target area includes the preset residence area and the areas in the preset passage area that can be passed through at the first moment. Among them, the areas in the preset passage area that can be passed through at the first moment can be all areas in the preset passage area, or can be some areas in the preset passage area. This embodiment does not make specific limitations here. In addition, the areas in the preset passage area that can be passed through at the first moment may also not exist, that is, there are no areas in the preset passage area at the first moment that can be used for pedestrians or non-motor vehicles to pass through.

[0127] Exemplarily, please refer to Figure 4 , Figure 4 which is a schematic diagram of an intersection scenario provided by an embodiment of the present application. As Figure 4 shown, in the intersection scenario, the preset residence area includes Figure 4 the safety island indicated by the dotted line in Figure 4 , and the preset passage area includes the crosswalk and the non-motor vehicle lane indicated by the solid line in

[0128] . Moreover, pedestrians or non-motor vehicles do not violate the regulations when appearing in the non-motor vehicle lane at any moment; for the crosswalk, pedestrians or non-motor vehicles do not violate the regulations only when appearing on the crosswalk during the period when the green light is on.

[0129] Exemplarily, please refer to Figure 5A , Figure 5A which is a schematic diagram of the target area of an intersection scenario at a moment provided by an embodiment of the present application. As Figure 5A shown, assuming that at the first moment, the traffic light of the crosswalk is green, then the crosswalk is in a passable state. Therefore, Figure 5A the target area in Figure 5A includes the crosswalk, the non-motor vehicle lane, and the safety island, that is, it is not a violation for the first trace point to appear in the crosswalk, the non-motor vehicle lane, and the safety island. It should be noted that

[0130] in Figure 5B , Figure 5B it is described by taking the traffic lights of the two crosswalks connecting the safety island being green at the same time as an example. In actual situations, the traffic lights of the two crosswalks connecting the safety island may not be green at the same time. Figure 5BAs shown, assume that at the first moment, the traffic light at the crosswalk is red, so the crosswalk is in a non-passable state. Therefore, Figure 5A the target area in Figure 5A includes the non-motor vehicle lane and the safety island and no longer includes the crosswalk. That is, only when the first trace point appears in the non-motor vehicle lane and the safety island is it not a violation. It should be noted that Figure 5B in Figure 5B , the example is given where the traffic lights of the two crosswalks connecting the safety island are both red at the same time. In actual situations, the traffic lights of the two crosswalks connecting the safety island may not be red at the same time.

[0131] Generally speaking, in this solution, only the position information of pedestrians or non-motor vehicles at a certain moment needs to be obtained, and by comparing whether the position of pedestrians or non-motor vehicles falls into the target area determined based on that moment, the identification of violation behaviors can be achieved. Therefore, this solution can still effectively identify the violation behaviors of pedestrians or non-motor vehicles in complex traffic scenarios. In this way, based on this solution, the violation behaviors of a large number of trace points obtained in each time period are identified, and the violation behaviors that occur in each time period can be identified, so as to facilitate traffic responsible personnel to take corresponding measures based on the occurring violation behaviors to reduce the occurrence of violation behaviors.

[0132] In the process introduced above, by comparing the trace points with the determined target area, it can be determined whether the trace points have violation behaviors. The following will introduce how to further identify the types of violation behaviors corresponding to the trace points by determining the actual residence area and the actual passing area.

[0133] Optionally, the above traffic violation identification method further includes the following steps 1 - 3.

[0134] Step 1, obtain multiple trace points in a historical time period, where the multiple trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments.

[0135] Specifically, the historical time period includes multiple moments. At each moment, trace points in the traffic scenario can be obtained through acquisition devices such as lidar, millimeter-wave radar, and cameras. Moreover, the multiple trace points in the historical time period are all collected in the same traffic area (i.e., the traffic area indicated by the above traffic map). It should be noted that in the historical time period, there may be one or more trace points at some moments (such as the moments during the morning and evening rush hours), and there may be no trace points at some moments (such as the moments during noon or early morning). Generally speaking, the multiple trace points in the historical time period are actually a set composed of the trace points obtained at each moment in the multiple moments of the historical time period.

[0136] Exemplarily, the historical time period is, for example, one day under historical time, and the moments in the historical time period are divided in seconds as the time unit, that is, one moment per second. Then, there are a total of 24 * 60 * 60 = 86,400 seconds in the historical time period, that is, there are a total of 86,400 moments in the historical time period. The multiple trace points under the historical time period refer to the trace points collected at these 86,400 moments. In addition, the length of the historical time period can also be other durations, such as 12 hours, two days, or one week under historical time; the time unit for dividing multiple moments in the historical time period can also be other time units, such as dividing one moment every 30 seconds or every 1 minute under the historical time period. This embodiment does not limit the duration of the historical time period and the time unit for dividing multiple moments in the historical time period.

[0137] Step 2: Based on the traffic map, multiple sub-regions are divided, and the multiple sub-regions do not overlap.

[0138] In this embodiment, in the case of having a traffic map, the traffic map can be subjected to regional division, thereby dividing multiple non-overlapping sub-regions. Moreover, the multiple sub-regions obtained by division are used to constitute the entire traffic area indicated in the traffic map.

[0139] Optionally, the sizes of different sub-regions among the multiple sub-regions divided based on the traffic map are the same, and the size of the sub-regions in the multiple sub-regions is related to the floor area occupied by a pedestrian or a non-motor vehicle, so as to ensure that each sub-region can accommodate one pedestrian or one non-motor vehicle. For example, assuming that the traffic area indicated by the traffic map is 500 * 500 (unit: meter) in size, each sub-region can be set as a 1 * 1 square region, and then the 500 * 500 traffic area indicated by the traffic map is divided into 250,000 sub-regions of 1 * 1 size.

[0140] Step 3: Based on the distribution of the multiple trace points in the multiple sub-regions, the actual residence area and / or the actual passage area is determined.

[0141] It can be understood that the preset residence area is generally an area that is pre-specified and demarcated for pedestrians and non-motor vehicles to stay in the traffic scenario, such as a bounded safety island area. However, in the case of a large flow of people (such as during peak commuting hours) or an improper traffic signal cycle setting (such as an overly long red light time), it may occur that the preset residence area cannot accommodate all the current pedestrians and non-motor vehicles that need to stay. For example, assuming that the preset residence area is a safety island, due to too many pedestrians waiting for the green light or an overly long red light duration resulting in an excessive accumulation of pedestrians, it may lead to the situation that all the pedestrians waiting for the green light cannot be accommodated in the safety island, and then some pedestrians may stay on the outer edge of the safety island (such as the crosswalk adjacent to the safety island). That is to say, in the actual traffic scenario, it often occurs that pedestrians or non-motor vehicles stay outside the preset residence area due to various factors, that is, the area where pedestrians or non-motor vehicles actually stay does not match the preset residence area. Moreover, since pedestrians or non-motor vehicles usually stay within the preset residence area or at the outer edge of the preset residence area when staying, and pedestrians or non-motor vehicles often stay for a certain period of time until the traffic signal turns into a passable green light, therefore, in this embodiment, the actual residence area can be determined based on the time length of the trace points in multiple sub-areas in the historical time period.

[0142] Specifically, pedestrians or non-motor vehicles mostly stay within the preset residence area and pass within the preset passing area in the traffic area, and rarely appear in other areas. Moreover, when the preset residence area is difficult to accommodate all the pedestrians or non-motor vehicles waiting to pass, pedestrians or non-motor vehicles also tend to stay in some areas at the outer edge of the preset residence area. Therefore, by determining the time length of the trace points in each of the multiple sub-areas in the historical time period, it is possible to determine which sub-areas are the areas where trace points often appear (i.e., the areas where pedestrians or non-motor vehicles often stay), so as to further determine the actual residence area.

[0143] Exemplarily, please refer to Figure 6 , Figure 6 which is a schematic diagram of the division of the actual residence area and the actual passing area provided by the embodiment of the present application. As Figure 6 shown, in the intersection scenario, the area where the safety island is located is divided into the preset residence area, and the area where the safety island is located and some areas at the outer edge of the safety island are divided into the actual residence area.

[0144] Similarly, a preset passing area is an area that is pre-specified and demarcated for pedestrians and non-motor vehicles in a traffic scenario, such as a bounded crosswalk. However, in the case of a large flow of people, there may be a situation where a large number of pedestrians and non-motor vehicles pass simultaneously, resulting in the preset passing area being unable to accommodate too many pedestrians and non-motor vehicles. As a result, some pedestrians and non-motor vehicles pass on the outer edge of the preset passing area. For example, when there are a large number of pedestrians and non-motor vehicles staying on a safety island, when the traffic signal corresponding to the crosswalk turns green, a large number of pedestrians and non-motor vehicles will pass on the crosswalk at the same time. And due to the limited area of the crosswalk, it often cannot accommodate too many pedestrians and non-motor vehicles crossing the street at the same time, resulting in some pedestrians and non-motor vehicles passing outside the crosswalk. That is to say, in the actual traffic scenario, there will often be a situation where pedestrians or non-motor vehicles pass outside the preset passing area due to various factors, that is, the area where pedestrians or non-motor vehicles actually pass does not match the preset passing area.

[0145] Based on this, in this embodiment, it can be determined based on the situation that there are trace points in the sub-areas around the preset passing area. For example, if trace points often appear in a part of the sub-areas around the preset passing area, it means that this part of the sub-areas is the area where pedestrians or non-motor vehicles pass, so it can be determined that this part of the sub-areas belongs to the actual passing area. If trace points never appear or rarely appear in a part of the sub-areas around the preset passing area, it means that this part of the sub-areas is the area where there are no pedestrians or non-motor vehicles passing, so it can be determined that this part of the sub-areas does not belong to the actual passing area.

[0146] Exemplarily, as Figure 6 shown, in the intersection scenario, the three crosswalks connected to the safety island are demarcated as the preset passing area, and these three crosswalks and the partial areas parallel to the crosswalks outside the crosswalks (i.e., Figure 6 the area surrounded by the dotted line in

[0147] are demarcated as the actual passing area. It should be noted that since some areas in the actual staying area may belong to the preset passing area, that is, some areas in the preset passing area are covered by the actual staying area, when determining the actual passing area, the areas in the preset passing area covered by the actual staying area need to be removed, so as to ensure that there is no overlapping area between the actual passing area and the actual staying area.

[0148] Specifically, after determining the actual residence area and the actual passage area, the type of the violation behavior corresponding to the first trace point can be further determined based on the positional relationship between the target position indicated by the first trace point and the actual residence area and the actual passage area.

[0149] Exemplarily, when the target position indicated by the first trace point is not within the target area and is within the actual residence area, it can be determined that the violation behavior of the first trace point is a violation of residence behavior or a violation of passage behavior.

[0150] Since the actual residence area actually includes the preset residence area and a partial area of the outer periphery of the preset residence area, when the target position indicated by the first trace point is not within the target area and is within the actual residence area, it can be determined that the target position is actually in a partial area of the outer periphery of the preset residence area. In addition, since the preset residence area often has a connection relationship with the preset passage area, for example, the safety island belonging to the preset residence area is connected to the crosswalk belonging to the preset passage area, when a pedestrian or a non-motor vehicle appears in a partial area of the outer periphery of the preset residence area, it may be a violation of residence (for example, staying outside the safety island during a red light), or it may be a violation of passage (for example, passing on the crosswalk outside the safety island during a red light).

[0151] Alternatively, when the target position indicated by the first trace point is not within the target area and is within the actual passage area, it can be determined that the violation behavior of the first trace point is a violation of passage behavior. Since a pedestrian or a non-motor vehicle will not stay in the actual passage area, when the target position indicated by the first trace point is not within the target area and is within the actual passage area, it can be determined that the violation behavior of the first trace point is a violation of passage behavior, rather than a violation of residence behavior.

[0152] In this solution, by dividing the traffic map into multiple sub-areas and determining the actual residence area and the actual passage area based on the occurrence of trace points in each sub-area in the historical time period, the type of the violation behavior where the trace point appears can be further identified based on the determined actual residence area and the actual passage area, improving the recognition efficiency of various types of violation behaviors.

[0153] Exemplarily, please refer to Figure 7 , Figure 7 which is a schematic diagram of the identification of a violation behavior provided by an embodiment of the present application. As Figure 7As shown, when the traffic signal at the crosswalk is red, the position of trace point 1 is outside the safety island and on the crosswalk. It can be determined that the position of trace point 1 is outside the preset staying area and within the actual staying area and the preset passing area. Therefore, it is determined that the violation of trace point 1 is a violation of staying or a violation of passing. In addition, the position of trace point 2 is in the middle of the crosswalk and far from the safety island. It can be determined that the position of trace point 2 is outside the preset staying area and within the actual passing area. Therefore, it is determined that the violation of trace point 2 is a violation of passing.

[0154] Optionally, when the target position is not within the target area and is within the actual staying area, in addition to being able to determine that the violation of the first trace point is a violation of staying or a violation of passing, it is also possible to further determine in which dimension (time dimension or space dimension) the violation of the first trace point occurs. Among them, the violation in the time dimension means that it is illegal for a pedestrian or non-motor vehicle to be at the position of the trace point at the current moment, but it may not be illegal for a pedestrian or non-motor vehicle to be at the same position of the trace point at other moments. The violation in the space dimension means that it is illegal for a pedestrian or non-motor vehicle to be at the position of the trace point at any moment.

[0155] Exemplarily, in response to the target position not being within the target area and the preset passing area and being within the actual staying area, it is determined that the violation of the first trace point is a violation of staying in the space dimension.

[0156] Specifically, the target position not being within the target area and being within the actual staying area means that the target position is at the outer edge of the preset staying area; and the target position not being within the preset passing area means that the target position is actually outside the preset staying area and in an area where passage is not allowed (for example, the target position is on the lane outside the safety island). Therefore, regardless of whether the traffic signal is red or green at the current moment, the area where the target position is located is not passable, that is, the first trace point belongs to a violation of staying in the space dimension.

[0157] Alternatively, in response to the target position not being within the target area but being within the actual residence area and within the preset passage area, it is determined that the violation of the first trace point is an illegal residence behavior or an illegal passage behavior in the time dimension. The target position not being within the target area but being within the actual residence area means that the target position is at the outer edge of the preset residence area; moreover, the target position being within the preset passage area means that the target position is actually in an area where passage is not allowed at the current moment but is allowed during the green light period. Therefore, the first trace point belongs to an illegal residence behavior or an illegal passage behavior in the time dimension. That is, at the first moment, the target position where the first trace point is located is not allowed to be occupied or passed through; however, it is not a violation if pedestrians or non-motor vehicles appear at the target position where the first trace point is located during the green light period.

[0158] Generally speaking, in this solution, by comparing the position relationship between the target position of the first trace point and the actual residence area and the preset passage area, it is possible to further determine whether the violation behavior to which the first trace point belongs is a violation behavior in the spatial dimension or a violation behavior in the time dimension, realizing further differentiation of violation behaviors and improving the recognition efficiency of various types of violation behaviors.

[0159] Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic diagram of another violation behavior recognition provided by an embodiment of the present application. As Figure 8 shown, at the moment when the traffic signal of the crosswalk is red, the position where trace point 1 is located is at the outer edge of the safety island and on the lane. Then, it can be determined that the position where trace point 1 is located is outside the preset residence area and the preset passage area and within the actual residence area. Therefore, it is determined that the violation behavior of trace point 1 is an illegal residence behavior in the spatial dimension.

[0160] The position where trace point 2 is located is at the outer edge of the safety island and on the crosswalk. Then, it can be determined that the position where trace point 2 is located is outside the preset residence area and within the actual residence area and the preset passage area. Therefore, it is determined that the violation behavior of trace point 2 is an illegal residence behavior or an illegal passage behavior in the time dimension.

[0161] The position where trace point 3 is located is on the lane and far from the safety island. Then, it can be determined that the position where trace point 3 is located is outside the actual residence area and outside the preset passage area. Therefore, it is determined that the violation behavior of trace point 3 is an illegal residence behavior in the spatial dimension.

[0162] The process of further determining the type of the violation behavior corresponding to the first trace point based on the actual residence area and the actual passage area is introduced above. Next, how to determine the actual residence area and the actual passage area will be introduced.

[0163] Optionally, during the process of determining the actual residence area, the usage characteristics of each sub-area in multiple sub-areas can be determined based on multiple trace points in a historical time period. Among them, the usage characteristics of a sub-area are used to indicate the ratio of the duration of trace points in the sub-area to the historical time period. Since the duration of the historical time period is fixed, the longer the duration of trace points in the sub-area, the higher the usage characteristics of the sub-area, indicating that pedestrians or non-motor vehicles stay or pass through the sub-area more frequently. Therefore, the usage characteristics of a sub-area can be used to indicate the frequency of use of the sub-area by pedestrians or non-motor vehicles.

[0164] Then, based on the usage characteristics of each sub-area, multiple residence sub-areas that make up the actual residence area are determined among the multiple sub-areas, and the multiple residence sub-areas belong to the multiple sub-areas.

[0165] It can be understood that since pedestrians or non-motor vehicles usually stay in the preset residence area and some areas on the outer edge of the preset residence area, it can be considered that some sub-areas with higher usage characteristics on the outer edge of the preset residence area belong to the areas where pedestrians or non-motor vehicles often stay. In this way, based on the usage characteristics of each sub-area, some sub-areas with higher usage characteristics on the outer edge of the preset residence area are screened out and combined with the preset residence area, and multiple residence sub-areas that make up the actual residence area can be determined among the multiple sub-areas, thereby realizing the determination of the actual residence area.

[0166] In this solution, the usage characteristics of a sub-area are determined based on the duration of trace points in the sub-area, and then the actual residence area is determined by combining the size of the usage characteristics of the sub-area and the preset residence area, which can effectively determine the areas where pedestrians or non-motor vehicles often stay in the entire traffic area, ensure that an accurate actual residence area can be obtained, and is beneficial to improving the recognition accuracy of subsequent illegal behaviors.

[0167] Optionally, there are various ways to determine the actual residence area based on the usage characteristics of the sub-area, and the specific way to determine the actual residence area is not limited in this embodiment.

[0168] In a possible implementation, during the process of determining the actual residence area, first, based on the usage characteristics of each sub-area among the multiple sub-areas, edge detection is performed on the multiple sub-areas to obtain at least one sub-area located at the edge of the actual residence area.

[0169] Among them, edge detection is a commonly used method in image processing for detecting edges with obvious brightness changes in an image. In this embodiment, edge detection is used to detect the edges of the actual residence area. In image processing, edge detection is achieved based on the brightness of each pixel in the image; in this embodiment, edge detection is achieved based on the magnitudes of the usage characteristics of each sub-region among multiple sub-regions. Specifically, since the usage characteristics of the sub-regions within the actual residence area are relatively high, while those of the sub-regions outside the actual residence area are relatively low, that is, the change in the usage characteristics of the sub-regions inside and outside the actual residence area is relatively obvious. Therefore, through the edge detection method, the sub-regions located at the edge of the actual residence area can be detected. That is, the working principle of edge detection is actually to detect the regions with obvious changes in usage characteristics, and the sub-regions detected by edge detection can be considered as the sub-regions at the edge of the actual residence area.

[0170] Then, after obtaining at least one sub-region located at the edge of the actual residence area, spatial clustering is performed on the at least one sub-region to obtain the residence area type corresponding to each sub-region among the at least one sub-region. It can be understood that in the traffic area indicated by the traffic map, there may be multiple separated areas that all belong to the preset residence area, that is, the preset residence area may include multiple separated areas. For example, in Figure 4 the shown traffic area, there are a total of 4 separated safety islands, and these 4 separated safety islands all belong to the preset residence area. Therefore, in this embodiment, after identifying three sub-regions located at the edge of the actual residence area, spatial clustering is performed on these sub-regions to determine the residence area type corresponding to each sub-region, that is, to determine the actual area to which each sub-region belongs, ensuring that the sub-regions under each area among multiple separated areas can be identified. Among them, the specific implementation method of spatial clustering can refer to the existing related technologies, and the specific implementation method of spatial clustering will not be elaborated here.

[0171] Finally, connection processing is performed on the sub-regions under each residence area type based on the convex hull construction method to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-regions. Specifically, the convex hull construction method is a method of screening the points on the edge of a polygon among multiple points and connecting the points on the edge of the polygon to obtain the polygon surrounded by the connected points. Since the sub-regions under each determined residence area type are actually located at the edge of the actual residence area, based on the convex hull construction method, the sub-regions located at the edge position under the same residence area type are connected, and then the area surrounded by the connected sub-regions can be obtained, thereby obtaining the actual residence area.

[0172] In this solution, based on the usage characteristics of sub-regions, by combining edge detection, spatial clustering, and convex hull construction methods, it is possible to effectively determine the actual residence area in various scenarios, ensuring an accurate actual residence area, which is beneficial to improving the recognition accuracy of subsequent violations.

[0173] In another possible implementation, when the preset residence area is obtained, sub-regions with a distance less than a specific distance threshold from the preset residence area can be determined first around the preset residence area, and then sub-regions with usage characteristics greater than a certain threshold can be further screened out from these determined sub-regions. In this way, the area composed of the finally screened sub-regions and the preset residence area can be determined as the actual residence area.

[0174] Optionally, in the process of determining the actual passage area, multiple parallel regions can be divided first based on the preset passage area and the areas around the preset passage area. The multiple parallel regions are parallel to the preset passage area, and each parallel region of the multiple parallel regions includes multiple sub-regions. Generally speaking, pedestrians or non-motor vehicles pass within the preset passage area when passing, and in the case of a large flow of people, some pedestrians or non-motor vehicles may pass in the areas around the preset passage area. Moreover, since the passage routes of pedestrians or non-motor vehicles are usually straight (for example, pedestrians or non-motor vehicles walk straight on the crosswalk), in this embodiment, multiple parallel regions are divided based on the preset passage area and the areas around the preset passage area, and each parallel region is used to indicate the possible passage routes of pedestrians or non-motor vehicles.

[0175] Then, based on the distribution of multiple trace points in multiple sub-regions in a historical time period, at least one parallel region for constituting the actual passage area is determined among the multiple parallel regions, so that the trajectory coverage rate of the actual passage area is greater than a preset threshold. The trajectory coverage rate is the ratio of the number of sub-regions with trace points in the actual passage area to the total number of sub-regions. Among them, the specific value of the preset threshold, such as 90% or 95%, can be adjusted according to the actual situation, and the value of the preset threshold is not limited here. Specifically, since the passage routes of pedestrians or non-motor vehicles are usually straight, it is set that the actual passage area is composed of multiple parallel regions. And if the trajectory coverage rate in a parallel region is high, it means that there are pedestrians or non-motor vehicles passing straight in this parallel region; if the trajectory coverage rate in a parallel region is low, it means that there are no pedestrians or non-motor vehicles passing straight in this parallel region. Therefore, it can be set that the trajectory coverage rate of the actual passage area is not lower than the preset threshold, and the parallel regions constituting the actual passage area can be determined among the divided multiple parallel regions.

[0176] In this solution, based on the characteristic that pedestrians or non-motor vehicles walk straight when passing through, by setting the condition that the trajectory coverage rate of the actual passing area needs to be greater than a preset threshold, the actual passing area can be effectively determined, which is beneficial to improving the recognition accuracy of subsequent violation behaviors.

[0177] Based on the content introduced in the above embodiments, by judging the positional relationship between the position of the trace point and the actual residence area and the actual passing area, the type of violation behavior corresponding to the trace point can be further identified. However, when the trace point is located in the actual residence area, due to the lack of continuous route information of the trace point, it is actually difficult to uniquely determine whether the violation behavior to which the trace point belongs is a violation residence behavior or a violation passing behavior, and it can only be determined that the trace point belongs to one of the violation residence behavior and the violation passing behavior. Based on this, this embodiment provides a method for determining the probability that the behavior corresponding to the trace point belongs to a residence behavior or a passing behavior, so that when it is impossible to uniquely determine whether the violation behavior to which the trace point belongs is a violation residence behavior or a violation passing behavior, the probabilities that the behavior corresponding to the trace point belongs to the violation residence behavior and the violation passing behavior respectively can be determined, ensuring that when a large number of trace points for which the violation behavior type cannot be uniquely determined are obtained, the proportions of the trace points that belong to the violation residence behavior and the violation passing behavior respectively can be further determined.

[0178] Specifically, the trace points for which the violation behavior type cannot be determined are usually located in the actual residence area, and the passing behavior has transitivity in space, that is, the passing route of pedestrians or non-motor vehicles is a continuous route, and different sub-regions on this passing route will have the same passing characteristics. Therefore, in this embodiment, the target passing area connected to the actual residence area is first determined, where the target passing area belongs to the actual passing area. If there are pedestrians or non-motor vehicles passing through the target passing area, then the pedestrians or non-motor vehicles will necessarily pass through the actual residence area connected to the target passing area. Among them, the target passing area can include, for example, one or more discontinuous areas. Exemplarily, please refer to Figure 9 , Figure 9 which is a schematic diagram for determining the target passing area provided by an embodiment of the present application. As Figure 9 shown, the target passing area includes three areas connected to the actual residence area, and these three areas respectively include different crosswalks.

[0179] Based on multiple trace points, the usage characteristics of the target passing area are determined. The usage characteristics of the target passing area are obtained based on the usage characteristics of the sub-regions included in the target passing area, and the usage characteristics of the sub-regions are used to indicate the ratio of the duration of the existence of trace points in the sub-regions to the historical time period. For example, the usage characteristics of the target passing area are the average of the usage characteristics of multiple sub-regions included in the target passing area.

[0180] In response to the first trace point being located within the actual residence area, based on the passing probability and residence probability of the sub - area where the first trace point is located, determine the probabilities that the violation behavior of the first trace point belongs to the violation residence behavior and the violation passing behavior respectively. Among them, the passing probability and residence probability of the sub - area where the first trace point is located are determined based on the usage characteristics of the target passing area.

[0181] Specifically, there are usually two behaviors, residence and passing, in the actual residence area, that is, the usage characteristics of the actual residence area are composed of the superposition of passing characteristics and residence characteristics. Therefore, by using the characteristic that the usage characteristics of the target passing area will be transmitted to the actual residence area, the passing characteristics of the actual residence area can be determined. Furthermore, based on the actual usage characteristics and passing characteristics of the actual residence area, the residence characteristics of the actual residence area can be determined, and finally the passing probability and residence probability of the sub - areas in the actual residence area can be determined.

[0182] Exemplarily, in the case where the first trace point is in the extended area of the target passing area (that is, the area obtained by extending the two boundaries of the target passing area), it can be considered that the sub - area where the first trace point is located will transmit the usage characteristics of the target passing area. Therefore, the passing probability of the sub - area where the first trace point is located is the ratio of the usage characteristics of the target passing area to the usage characteristics of the sub - area where the first trace point is located. In addition, the residence probability of the sub - area where the first trace point is located is the difference between 1 and the passing probability of the sub - area where the first trace point is located.

[0183] In this solution, by using the transmission characteristic of the passing characteristics in the passing area and the characteristic that the usage characteristics of the residence area are obtained by the superposition of passing characteristics and residence characteristics, based on the passing characteristics in the passing area, the passing characteristics and residence characteristics in the adjacent actual residence area are obtained, and then the passing probability and residence probability in the actual residence area are further determined, ensuring that the passing probability and residence probability can be determined for all trace points in the actual residence area.

[0184] The above introduces the traffic violation recognition method provided in this embodiment. For the convenience of understanding, the following will introduce the actual application process of the traffic violation recognition method provided in this embodiment in detail with specific examples.

[0185] Please refer to Figure 10 , Figure 10 which is a schematic flow chart of a method for identifying intersection violation behaviors provided in an embodiment of the present application. As Figure 10 shown, taking the traffic area as an intersection as an example, the process of identifying the violation behaviors of pedestrians or non - motor vehicles at the intersection mainly includes four stages, namely: establishing a spatial digital model of the intersection, dividing the actual residence area and the actual passing area based on the usage characteristics of the area, parsing the behaviors of pedestrians or non - motor vehicles in the intersection space, and identifying violation behaviors.

[0186] Phase 1, establish a spatial digital model of the intersection.

[0187] In this phase, by modeling the traffic facilities and spaces at the intersection and establishing the association relationships between the traffic facilities and spaces, a digital scenario with complete facilities, consistent spaces, and complete relationships is formed, so as to support the machine understanding of the intersection scenario and the spatio-temporal matching of pedestrians and non-motor vehicles. Exemplarily, please refer to Figure 11 , Figure 11 which is a schematic diagram of a cross-intersection semantic space coupling model provided by an embodiment of the present application. As Figure 11 shown, the model is divided into two levels, namely the facility layer and the space layer, and the facility layer and the space layer can be coupled through two methods of spatial association and semantic association to obtain the coupling model.

[0188] In this embodiment, the spatial digital model of the intersection is established from two aspects of the facility level and the space level, realizing the construction of various traffic facilities at the intersection and the spatial division for different traffic objects, and meeting the traffic behavior recognition requirements of various traffic objects at different granularities.

[0189] Specifically, the process of establishing the spatial digital model of the intersection can be divided into the following three steps:

[0190] Step 1, semantic expression.

[0191] Semantic expression refers to, after obtaining the traffic map related to the intersection, classifying the traffic facilities in the intersection by identifying the traffic facilities in the traffic map and constructing the spatial topological relationships between the traffic facilities. As Figure 11 shown, the traffic facilities can be divided into, for example, road passage facilities and slow crossing facilities. Among them, the road passage facilities include lanes, non-motor vehicle lanes, and sidewalks; the slow crossing facilities include crosswalks, safety islands, and non-motor vehicle dedicated lanes. And the relationships between various traffic facilities include, for example, connectivity, connection, adjacency, and intersection.

[0192] Step 2, spatial expression.

[0193] Spatial expression refers to using the grid technology to discretize the intersection into a grid set adapted to the individual sizes of pedestrians, non-motor vehicles, and motor vehicles. Generally speaking, for the intersection area indicated in the traffic map, the same intersection area can be grid-divided from two aspects respectively. One aspect is to divide the entire intersection area into multiple motor vehicle grids of the same size from the perspective of motor vehicles; the other aspect is to divide the entire intersection area into multiple pedestrian and non-motor vehicle grids of the same size (i.e., Figure 11The pedestrian and non-motor vehicle grid shown in [figure] (not shown). Among them, the pedestrian and non-motor vehicle grid corresponds to the sub-region introduced in the above embodiment.

[0194] That is to say, the spatial expression actually divides the intersection space into finer discrete spatial objects through grids, and endows the grids with semantic attributes according to the correspondence between the spatial grids and the infrastructure. Since motor vehicles, pedestrians, and non-motor vehicles can pass through the intersection, a motor vehicle grid space and a pedestrian and non-motor vehicle grid space are respectively constructed. Based on the basic floor areas of motor vehicle and pedestrian and non-motor vehicle objects, the size of the motor vehicle grid is defined as 4×4m 2 and the size of the pedestrian and non-motor vehicle grid is defined as 1×1m. 2 .

[0195] The relationship between motor vehicle grids is an adjacent relationship, and the relationship between pedestrian and non-motor vehicle grids is an adjacent relationship. In addition, the area of the motor vehicle grid is larger than that of the pedestrian and non-motor vehicle grid, so there is a relationship where the motor vehicle grid contains the pedestrian and non-motor vehicle grid. Specifically, the grid space can be expressed as: C k ={c i}, c i =(type,width). Where k represents a motor vehicle, a pedestrian, or a non-motor vehicle; type represents the semantic attribute of the grid, such as a safety island or a crosswalk, etc.; width represents the side length of the grid, such as the width of the pedestrian and non-motor vehicle grid is 1m.

[0196] Exemplarily, the topological relationship between traffic facilities and grids is shown in Table 1.

[0197] Table 1

[0198]

[0199] Among them, the reversibility in Table 1 refers to whether the relationship between object 1 and object 2 can be converted into the relationship between object 2 and object 1. For example, when object 1 is a motor vehicle grid and object 2 is a pedestrian and non-motor vehicle grid, the motor vehicle grid contains the pedestrian and non-motor vehicle grid, while the pedestrian and non-motor vehicle grid cannot contain the motor vehicle grid.

[0200] Step 3, language-space coupling.

[0201] Language-space coupling refers to using a spatial matching method to couple and associate traffic facilities with semantics and grids. Specifically, there are spatial and semantic association relationships between traffic facilities and grids: The traffic facilities themselves are composed of a certain space, and the relationship between traffic facilities and grids can be established from the spatial position and geometry, that is, which grids the traffic facilities are composed of. In addition, the space itself also carries the semantic information of the traffic facilities, that is, the grid belongs to a certain traffic facility.

[0202] In stage 2, the actual residence area and the actual passage area are divided based on the usage characteristics of the area.

[0203] Exemplarily, please refer to Figure 12 , Figure 12 , which is a schematic flowchart of a process for dividing the actual residence area provided by an embodiment of the present application. As Figure 12 shown, in the spatially digitized model of the intersection constructed, first, the trace points of pedestrians or non-motor vehicles are matched with the pedestrian and non-motor vehicle grids obtained by division based on the spatial position to determine the grid where each trace point is located. That is, based on the position indicated by the trace points obtained at each moment, on the premise that the spatial coordinate systems are aligned, the grid including the position indicated by the trace points is determined, and then the grid where each trace point is located is determined.

[0204] After determining the grid where each trace point is located, based on the situation where trace points appear in each grid at each moment, the usage characteristics of each grid in the intersection area are determined. Among them, the usage characteristics of the grid refer to the proportion of the duration during which there are trace points in the grid in the total statistical time interval (for example, the historical time period introduced in the above embodiment). Specifically, the usage characteristics of the grid can be represented by the following formula.

[0205]

[0206] Among them, G T (c i ) refers to the spatial usage characteristics of grid c i in the statistical time interval T, T = T 1 -T 0 , T represents the length of the statistical time interval, t ∈ [T 0 , T 1 , and x t represents whether there are pedestrians or non-motor vehicles in the grid at time t.

[0207] Secondly, based on the edge detection algorithm, the grids located at the edge of the actual residence space are extracted, and the edge detection algorithm realizes the grid extraction based on the usage characteristics of the grid during the use process. That is to say, taking the usage characteristics of the grid as the detection target of edge detection, the position where the usage characteristics of the grid change significantly is detected through the edge detection algorithm, so as to determine the grids located at the edge of the actual residence space.

[0208] Then, based on the spatial clustering algorithm, the grids located at the edge of the actual residence space are classified. Since the grids extracted based on the edge detection algorithm may belong to multiple separate regions respectively, the spatial clustering algorithm can be used to divide each grid into a unique category, so that the grids under each category correspond to one region.

[0209] Finally, based on the convex hull construction algorithm, the boundary of the actual residence space is constructed, that is, the connection process is performed on the grids under each category based on the convex hull construction algorithm, so as to obtain the boundary formed by the connection lines between the grids. In this way, after determining the boundary of the actual residence space, all regions inside the boundary can be determined to belong to the actual residence space, and the determination of the actual residence space is realized.

[0210] In addition, the process of determining the actual passage area can be realized based on a preset passage area and using a region growing algorithm.

[0211] First, the effective space coverage rate is defined as the ratio of the number of used grids in the region to the total number of grids in the region. The effective space coverage rate can be specifically expressed by the following formula.

[0212]

[0213] Among them, f represents the effective space coverage rate; C P ={c i} represents the grid set of the actual passage space; C′ P ={c i |G(c i )>0, c i ∈C P} represents the set of used grids in the actual passage space.

[0214] Exemplarily, please refer to Figure 13 , Figure 13 , which is a schematic diagram of determining the actual passage area based on the region growing algorithm provided by the embodiment of the present application. As Figure 13 shown, first, a preset passage area is selected, and the left and right boundaries of the preset passage area are obtained is the boundary slope, b 1 is the intercept, k i is the offset. Then, in the preset passage area, the grid c i with the largest use feature is selected as the starting point. Secondly, the center coordinates of the grid c i are used to fit the boundary curve to obtain the initial boundary And the initial boundary is translated by a unit step (one grid) respectively, and the effective utilization space coverage rate f in the grid space formed by the translated boundary and the initial boundary at this time is calculated. If the effective space coverage rate f of the grid space formed by the translated boundary and the initial boundary is less than the threshold F (the threshold F is, for example, the preset threshold in the above embodiment, and the threshold F is, for example, set to 0.95), then this movement is abandoned; if the effective space coverage rate f of the grid space formed by the translated boundary and the initial boundary is less than the threshold F (the threshold F is, for example, the preset threshold in the above embodiment, and the threshold F is, for example, set to 0.95), then this movement is abandoned. And so on, by gradually moving the boundary left and right, an actual passage space with as large an area as possible and a coverage rate not lower than F can be obtained.

[0215] Phase 3, analysis of pedestrian or non-motor vehicle behavior in the intersection space.

[0216] Generally speaking, the grids in the intersection space have two characteristics: superposition of space utilization characteristics and transitivity of passage behavior space characteristics. As Figure 14 shown, Figure 14 is a schematic diagram of superposition of space utilization characteristics provided by an embodiment of the present application. The superposition of space utilization characteristics means that the space utilization characteristics in the grid are linearly superimposed by the space utilization characteristics of passage behavior and the space utilization characteristics of residence behavior, and characterize the mixing situation of individual behaviors in a single space. Exemplarily, the space utilization characteristics in the grid are shown in the following formula.

[0217] G T (c i ) = G T,P (c i ) + G T,W (c i )

[0218] Among them, G T (c i ) refers to the space utilization characteristics of grid c i within the statistical time interval T; G T,P (c i ) refers to the space utilization characteristics generated by passage behavior in grid c i within the statistical time interval T; G T,W (c i ) refers to the space utilization characteristics generated by residence behavior in grid c i within the statistical time interval T.

[0219] As Figure 15 shown, Figure 15Schematic diagram of the transitivity of the space characteristics of passing behaviors provided by the embodiments of the present application. The transitivity of the space characteristics of passing behaviors means that in the grids on the movement routes of pedestrians or non-motor vehicles, the space usage characteristics of passing behaviors in the upstream grids can be transmitted to the downstream grids.

[0220] Due to the superposition of the space characteristics of the intersection grids, as long as the space usage characteristics of one of the passing or staying behaviors in the grid are known, the space usage characteristics of the other behavior can be known. The transitivity of the space characteristics of the passing behaviors of the grid indicates that as long as the space usage characteristics of the passing behaviors of a certain grid on the passing path (hereinafter referred to as passing behavior characteristics) are known, the characteristics of other grids can be known. Moreover, the grids in the passing area are only used for passing, that is, G T (c i ) = G T,P (c i );while the grids in the staying area can be used for both passing and staying, that is, G T,P (c i ) > 0, G T,W (c i ) > 0. Therefore, the calculation steps of the usage characteristics of the space behaviors in the grid can include, for example, the following steps 1-step 5.

[0221] Step 1, extract the actual staying area and two actual passing areas that are topologically connected to the actual staying area. Among them, the two actual passing areas are respectively defined as C Pu1 = {c i}, C Pu2 = {c i}; the actual staying area is defined as C Wu = {c i}.

[0222] Exemplarily, please refer to Figure 16 , Figure 16 which is a schematic diagram of two actual passing areas connected to the actual staying area provided by the embodiments of the present application. As Figure 16 shown, in the intersection area, the actual staying area includes the safety island and part of the area around the safety island. One actual passing area includes the crosswalk connected to the actual staying area and part of the area around the crosswalk, and the other actual passing area includes another crosswalk connected to the actual staying area and part of the area around the crosswalk.

[0223] Step 2, calculate the usage characteristics of the actual passing area.

[0224] Among them, the usage characteristics of the actual passing area can be obtained by calculating the average usage characteristics of all grids in the actual passing area. Specifically, the usage characteristics of the two actual passing areas are respectively expressed as: GT (C Pu1 ) and G T (C Pu2 ), and the usage feature calculation formulas for the two actual passage areas are as follows.

[0225]

[0226]

[0227] Among them, are respectively the number of grids of the actual passage areas C Pu1 and C Pu2 .

[0228] Step 3, based on the transitivity of the space usage features, calculate the passage behavior features of the grids in the actual residence area. Due to the transitivity of the passage behavior, the grids in the actual residence area exhibit not only residence features but also passage features. That is, the passage behaviors of the actual passage areas C Pu1 and C Pu2 are transmitted to the space sets formed by the actual residence area C Wu which are C Wu1 and C Wu2 respectively. Therefore, the passage behavior features of C Wu are obtained by calculating the passage behavior features of C Wu1 and C Wu2 .

[0229] Exemplarily, please refer to Figure 17 , Figure 17 which is a schematic diagram of the transmission of the passage behavior features provided by the embodiment of the present application. As Figure 17 shown, assume that x i = (x, y) represents the coordinate vector of the grid c Wu in the actual residence area C i . It is known that the boundaries of the actual passage area C Pu1 are respectively shown by the following formulas:

[0230] Left boundary:

[0231] Right boundary:

[0232] Among them, k′ 1 <k′ 2 . Then, substituting the coordinate vector of the grid c i into the boundary function, we get:

[0233]

[0234] If k′ 1 <k < k′2 , then grid c i belongs to C Wu1 , that is, G T,P (c i ) > 0; otherwise, the passage behavior space usage feature G i of c T,P (c i ) = 0.

[0235] Similarly, it is possible to determine whether grid c i belongs to C Wu2 in a similar manner to the above.

[0236] Therefore, the passage behavior feature of grid c Wu in C i is as follows:

[0237] G T,P (c i ) = G T,P1 (c i ) + G T,P2 (c i )

[0238] where G T,P1 (c i ), G T,P2 (c i ) are the passage behavior features of C Wu1 , C Wu2 respectively.

[0239] Step 4, according to the additivity of the space usage feature, calculate the residence behavior feature of the actual residence area, as shown in the following formula.

[0240] G T,W (c i ) = G T (c i ) - G T,P (c i )

[0241] That is, the residence usage feature of each grid in the actual residence area is the difference between the usage feature of the grid and the passage behavior feature of the grid.

[0242] Step 5, after obtaining the passage behavior feature G T,P (c i ) and the residence behavior feature G T,W (c i ) of the grid, it is possible to construct a grid behavior probability map based on this, and the passage behavior probability and residence behavior probability of each grid are as shown in the following formula.

[0243]

[0244]

[0245] Among them, Pr P (c i ) is the probability of grid passing behavior; Pr W (c i ) is the probability of grid staying behavior.

[0246] Phase 4, violation behavior recognition.

[0247] Specifically, in this embodiment, the violation behaviors of pedestrians or non-motor vehicles are divided into four types according to three dimensions of space, time, and behavior, namely: illegal staying in the space dimension, illegal passing in the space dimension, illegal stopping in the time dimension, and illegal passing in the time dimension.

[0248] Exemplarily, please refer to Figure 18 , Figure 18 which is a schematic flow chart of a violation behavior recognition provided by an embodiment of the present application. As Figure 18 shown, the specific process of violation behavior recognition takes the trace points containing spatio-temporal information as the basic input, that is, the trace points can indicate the position of a pedestrian or a non-motor vehicle at a certain moment. Among them, on the one hand, the spatial information of the trace points is matched with the spatial digital model constructed in Phase 1 to identify the grid where the trace points are located, so as to determine whether the trace points belong to spatial violations; on the other hand, a spatial behavior judgment is made on the trace points, that is, the probability of grid passing behavior and the probability of staying behavior obtained in Phase 4 are combined to determine the most likely behavior of the grid where the trace points are located. In this way, by combining the time information of the trace points and the spatial matching result, it can be determined whether the traffic signal corresponding to the grid where the trace points are currently located is red. Finally, the violation behavior is identified by comprehensively considering the spatial violation judgment, the red light running judgment, and the behavior of the trace points.

[0249] Specifically, first, it is judged whether the grid where the trace points are located is stayable or passable in space, that is, whether the grid where the trace points are located belongs to the preset staying area or the preset passing area. If the grid where the trace points are located belongs to the preset staying area, it means that the trace points do not have violation behaviors; if the grid where the trace points are located belongs to the preset passing area, it is necessary to further identify whether the trace points have violation behaviors by combining the time information of the trace points. If the grid where the trace points are located does not belong to the preset staying area and the preset passing area, the behavior corresponding to the trace points is further determined based on the grid where the trace points are located, so as to facilitate the subsequent determination of the type of violation behavior of the trace points.

[0250] If the grid where the trace point is located belongs to the preset passing area, and the traffic signal light corresponding to the grid where the trace point is located is red, it means that the preset passing area to which the grid where the trace point is located belongs cannot be passed at the current moment. Therefore, the violation behavior of the trace point is a violation behavior in the time dimension. And, if the grid where the trace point is located belongs to both the preset passing area and the actual staying area (for example, pedestrians appear on the crosswalk during a red light), the passing behavior probability and staying behavior probability of the grid where the trace point is located can be combined to further determine the probability that the trace point belongs to a violation of passing in the time dimension and the probability that the trace point belongs to a violation of staying in the time dimension. If the grid where the trace point is located belongs to the preset passing area but does not belong to the actual staying area, it can be determined that the violation behavior of the trace point is a violation of passing in the time dimension.

[0251] If the grid where the trace point is located does not belong to the preset staying area and the preset passing area, and the grid where the trace point is located belongs to the actual staying area, the passing behavior probability and staying behavior probability of the grid where the trace point is located can be combined to further determine the probability that the trace point belongs to a violation of passing in the space dimension and the probability that the trace point belongs to a violation of staying in the space dimension. If the grid where the trace point is located does not belong to the preset staying area and the preset passing area, and the grid where the trace point is located also does not belong to the actual staying area, it can be determined that the violation behavior of the trace point is a violation of passing in the space dimension.

[0252] The above introduces a traffic violation recognition method provided by an embodiment of the present application. Next, a traffic violation recognition device for executing the above traffic violation recognition method will be introduced.

[0253] Please refer to Figure 19 , Figure 19 which is a schematic structural diagram of a traffic violation recognition device provided by an embodiment of the present application. As Figure 19 shown, the traffic violation recognition device includes: an acquisition module 1901, configured to acquire a traffic map, where the traffic map is used to indicate the setting conditions of multiple traffic facilities; a processing module 1902, configured to determine a preset staying area and a preset passing area based on the setting positions of the multiple traffic facilities in the traffic map, where the preset staying area includes a preset area for pedestrians or non-motor vehicles to stay, and the preset passing area includes a preset area for pedestrians or non-motor vehicles to pass; the acquisition module 1901 is further configured to acquire a first trace point, where the first trace point is used to indicate the target position of a pedestrian or a non-motor vehicle at a first moment; the processing module 1902 is further configured to determine that the first trace point has a violation behavior in response to the target position not being located within the target area; where the target area includes the preset staying area and the area that can be passed at the first moment in the preset passing area.

[0254] In a possible implementation, the obtaining module 1901 is further configured to obtain a plurality of trace points in a historical time period, where the plurality of trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments; the processing module 1902 is further configured to divide, based on a traffic map, to obtain a plurality of sub-regions, and there is no overlap between the plurality of sub-regions; the processing module 1902 is further configured to determine an actual residence area and / or an actual passing area based on the distribution of the plurality of trace points in the plurality of sub-regions, where the actual residence area is the area where pedestrians or non-motor vehicles actually reside, and the actual residence area includes a preset residence area, and the actual residence area is determined based on the time length of the sub-regions having trace points in the historical time period, and the actual passing area is the area where pedestrians or non-motor vehicles actually pass, and the actual passing area is determined based on the situation of the sub-regions around the preset passing area having trace points; where the actual residence area and / or the actual passing area are used to determine the type of the violation behavior corresponding to the first trace point.

[0255] In a possible implementation, the processing module 1902 is further configured to: in response to the target position not being within the target area and being within the actual residence area, determine that the violation behavior of the first trace point is a violation of residence behavior or a violation of passing behavior; or, in response to the target position not being within the target area and being within the actual passing area, determine that the violation behavior of the first trace point is a violation of passing behavior.

[0256] In a possible implementation, the processing module 1902 is further configured to: in response to the target position not being within the target area and the preset passing area and being within the actual residence area, determine that the violation behavior of the first trace point is a violation of residence behavior in the spatial dimension; or, in response to the target position not being within the target area and being within the actual residence area and within the preset passing area, determine that the violation behavior of the first trace point is a violation of residence behavior or a violation of passing behavior in the time dimension.

[0257] In a possible implementation, the processing module 1902 is further configured to: determine the usage characteristics of each sub-region in the plurality of sub-regions based on the plurality of trace points, where the usage characteristics are used to indicate the ratio of the duration of the existence of trace points in the sub-region to the historical time period; and determine, based on the usage characteristics, a plurality of residence sub-regions used to form the actual residence area in the plurality of sub-regions, and the plurality of residence sub-regions belong to the plurality of sub-regions.

[0258] In a possible implementation, the processing module 1902 is further configured to: perform edge detection on multiple sub-regions based on the usage characteristics of each sub-region in the multiple sub-regions to obtain at least one sub-region located at the edge of the actual residence area; perform spatial clustering on the at least one sub-region to obtain the residence area type corresponding to each sub-region in the at least one sub-region; perform connection processing on the sub-regions of each residence area type based on the convex hull construction method to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-regions.

[0259] In a possible implementation, the trajectory coverage rate of the actual passage area is greater than a preset threshold, and the trajectory coverage rate is the ratio of the number of sub-regions with trace points in the actual passage area in the historical time period to the total number of sub-regions.

[0260] In a possible implementation, the sizes of different sub-regions in the multiple sub-regions are the same, and the size of the sub-regions in the multiple sub-regions is related to the floor area of pedestrians or non-motor vehicles.

[0261] In a possible implementation, the processing module 1902 is further configured to: determine a target passage area connected to the actual residence area, and the target passage area belongs to the actual passage area; determine the usage characteristics of the target passage area based on multiple trace points, and the usage characteristics of the target passage area are obtained based on the usage characteristics of the sub-regions included in the target passage area, and the usage characteristics of the sub-regions are used to indicate the ratio of the duration with trace points in the sub-regions to the historical time period; in response to the first trace point being located within the actual residence area, determine the probabilities that the violation behavior of the first trace point belongs to the violation residence behavior and the violation passage behavior respectively based on the passage probability and the residence probability of the sub-region where the first trace point is located, where the passage probability and the residence probability are determined based on the usage characteristics of the target passage area.

[0262] In a possible implementation, when the first trace point is in the extended area of the target passage area, the passage probability of the sub-region where the first trace point is located is the ratio of the usage characteristics of the target passage area to the usage characteristics of the sub-region where the first trace point is located.

[0263] In a possible implementation, the first trace point is collected by any one or more of the following devices: camera, lidar, and millimeter wave radar.

[0264] In a possible implementation, the preset residence area is located at the intersection position.

[0265] Please refer to Figure 20 , Figure 20Another structural schematic diagram of a traffic violation recognition device provided by an embodiment of the present application. The traffic violation recognition device 2000 may specifically include a Microcontroller Unit (MCU), a chip, a chip system, a circuit system, etc., which are not limited herein. Specifically, the traffic violation recognition device 2000 includes a transceiver 2001, a processor 2002, and a memory 2003 (where the number of processors 2002 in the traffic violation recognition device 2000 may be one or more, Figure 20 and one processor is taken as an example herein). Among them, the processor 2002 may include an application processor 20021 and a communication processor 20022. In some embodiments of the present application, the transceiver 2001, the processor 2002, and the memory 2003 may be connected through a bus or other means.

[0266] The memory 2003 may include a read-only memory and a random access memory, and provide instructions and data to the processor 2002. A part of the memory 2003 may also include a non-volatile random access memory (NVRAM). The memory 2003 stores processor and operation instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, where the operation instructions may include various operation instructions for implementing various operations.

[0267] The processor 2002 controls the operation of path planning. In a specific application, the various components of the traffic violation recognition device are coupled together through a bus system, where the bus system may include a power bus, a control bus, a status signal bus, etc. in addition to a data bus. However, for the sake of clarity, all kinds of buses are referred to as a bus system in the figure.

[0268] The method disclosed in the embodiments of the present application can be applied to or implemented by the processor 2002. The processor 2002 can be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method can be completed by the integrated logic circuit in hardware or instructions in software form in the processor 2002. The above-mentioned processor 2002 can be a general-purpose processor, a digital signal processor (DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor 2002 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 2003, and the processor 2002 reads the information in the memory 2003 and combines its hardware to complete the steps of the above method.

[0269] The transceiver 2001 (such as a network card) can be used to receive input digital or character information, and generate signal inputs related to the relevant settings and function control of the traffic violation recognition device. The transceiver 2001 can also be used to output digital or character information through the first interface; and send instructions to the disk array through the first interface to modify the data in the disk array.

[0270] The traffic violation recognition device provided by the embodiment of the present application may specifically include a chip, and the chip includes: a processing unit and a communication unit. The processing unit may be a processor, for example, and the communication unit may be an input / output interface, a pin, a circuit, etc. The processing unit can execute the computer execution instructions stored in the storage unit to enable the chip in the processing module to execute the road surface detection method described in the above embodiments. Optionally, the storage unit is a storage unit inside the chip, such as a register, a cache, etc. The storage unit may also be a storage unit outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0271] Reference may be made to Figure 21 , Figure 21 which is a schematic structural view of a computer-readable storage medium provided by an embodiment of the present application. The present application also provides a computer-readable storage medium. In some embodiments, the above-disclosed method may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles.

[0272] Figure 21 Schematically shown is a conceptual partial view of an example computer-readable storage medium arranged according to at least some of the embodiments presented here. The example computer-readable storage medium includes a computer program for executing a computer process on a computing device.

[0273] In one embodiment, the computer-readable storage medium 2100 is provided using a signal-bearing medium 2101. The signal-bearing medium 2101 may include one or more program instructions 2102, which when run by one or more processors can provide the functions or partial functions described above for the embodiments. In addition, Figure 21 the program instructions 2102 in

[0274] In some examples, the signal-bearing medium 2101 may include a computer-readable medium 2103, such as, but not limited to, a hard disk drive, a compact disc (CD), a digital video disc (DVD), a digital tape, a memory, a ROM, or a RAM, etc.

[0275] In some embodiments, the signal-bearing medium 2101 may include a computer-readable medium 2104, such as but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, etc. In some embodiments, the signal-bearing medium 2101 may include a communication medium 2105, such as but not limited to, a digital and / or analog communication medium (e.g., an optical fiber cable, a waveguide, a wired communication link, a wireless communication link, etc.). Thus, for example, the signal-bearing medium 2101 may be conveyed by a wireless form of the communication medium 2105 (e.g., a wireless communication medium compliant with the IEEE 802 standard or other transmission protocols).

[0276] One or more program instructions 2102 may be, for example, computer-executable instructions or logic-implemented instructions. In some examples, a computing device of the computing device may be configured to provide various operations, functions, or actions in response to the program instructions 2102 communicated to the computing device through one or more of the computer-readable medium 2103, the computer-readable medium 2104, and / or the communication medium 2105.

[0277] It should be further noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationships between the modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines.

[0278] Through the description of the above embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for this application, software program implementation is a better embodiment in more cases. Based on such an understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disc of a computer, and includes several instructions to enable a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of this application.

[0279] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0280] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device, or data center to another website, computer, training device, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

Claims

1. A traffic violation recognition method, characterized in that, it includes: Based on the distribution of multiple trace points in a traffic scenario over a historical time period, determine the actual residence area and / or the actual passing area. The multiple trace points are used to indicate the positions of pedestrians or non-motor vehicles at multiple moments. The actual residence area is the area where pedestrians or non-motor vehicles actually reside, and the actual residence area includes a preset residence area. The preset residence area includes a preset area for pedestrians or non-motor vehicles to reside. The actual passing area is the area where pedestrians or non-motor vehicles actually pass through; Obtain a first trace point, which is used to indicate the target position of a pedestrian or non-motor vehicle at a first moment, and the first trace point is a point on a continuous trajectory or a point on a discontinuous trajectory; Based on a first positional relationship and / or a second positional relationship, output the violation behavior corresponding to the first trace point. The first positional relationship is the positional relationship between the target position and the preset residence area and the actual residence area. The second positional relationship is the positional relationship between the target position and the preset passing area and the actual passing area. The preset passing area includes a preset area for pedestrians or non-motor vehicles to pass through. The violation behavior includes illegal residence or illegal passing.

2. The method according to claim 1, characterized in that, the output of the violation behavior corresponding to the first trace point includes: In the case where the target position is not within the target area and is within the actual residence area, output that the violation behavior corresponding to the first trace point is an illegal residence behavior or an illegal passing behavior; Or, in the case where the target position is not within the target area and is within the actual passing area, output that the violation behavior of the first trace point is an illegal passing behavior; wherein, the target area includes the preset residence area and the area that can be passed through at the first moment in the preset passing area.

3. The method according to claim 2, characterized in that, the output that the violation behavior corresponding to the first trace point is an illegal residence behavior or an illegal passing behavior includes: In the case where the target position is not within the target area and the preset passing area and is within the actual residence area, output that the violation behavior of the first trace point is an illegal residence behavior in the spatial dimension; Or, in the case where the target position is outside the actual residence area and the preset passing area, output that the violation behavior of the first trace point is an illegal passing behavior in the spatial dimension; Or, in the case where the target position is not within the target area and is within the actual residence area and within the preset passing area, output that the violation behavior of the first trace point is an illegal residence behavior or an illegal passing behavior in the time dimension.

4. The method according to any one of claims 1-3, characterized in that, the determination of the actual residence area and the actual passing area based on the distribution of the multiple trace points in the traffic scenario includes: Obtain a traffic map of the traffic scenario, and the traffic map is used to indicate the setting conditions of multiple traffic facilities; Based on the traffic map, a plurality of sub-regions are divided, and there is no overlap between the plurality of sub-regions; Based on the distribution of the plurality of trace points in the plurality of sub-regions, the actual residence area and the actual passing area are determined. The actual residence area is determined based on the time length of the sub-regions having trace points in the historical time period, and the actual passing area is determined based on the situation of the sub-regions around the preset passing area having trace points.

5. The method according to claim 4, wherein, the determining the actual residence area includes: determining the usage characteristics of each sub-region in the plurality of sub-regions based on the plurality of trace points, where the usage characteristics are used to indicate the ratio of the duration of the existence of trace points in the sub-region to the historical time period; based on the usage characteristics, determining the plurality of residence sub-regions for constituting the actual residence area in the plurality of sub-regions, and the plurality of residence sub-regions belong to the plurality of sub-regions.

6. The method according to claim 5, wherein, the determining the plurality of residence sub-regions for constituting the actual residence area in the plurality of sub-regions based on the usage characteristics includes: performing edge detection on the plurality of sub-regions based on the usage characteristics of each sub-region in the plurality of sub-regions to obtain at least one sub-region located at the edge of the actual residence area; performing spatial clustering on the at least one sub-region to obtain the residence area type corresponding to each sub-region in the at least one sub-region; performing connection processing on the sub-regions of each residence area type based on the convex hull construction method to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-regions.

7. The method according to any one of claims 4-6, wherein, the trajectory coverage rate of the actual passing area is greater than a preset threshold, and the trajectory coverage rate is the ratio of the number of sub-regions having trace points in the historical time period in the actual passing area to the total number of sub-regions.

8. The method according to any one of claims 4-7, wherein, the sizes of different sub-regions in the plurality of sub-regions are the same, and the size of the sub-regions in the plurality of sub-regions is related to the floor area of pedestrians or non-motor vehicles.

9. The method according to any one of claims 4-8, wherein, the method further includes: determining a target passing area connected to the actual residence area, and the target passing area belongs to the actual passing area; determining the usage characteristics of the target passing area based on the plurality of trace points, where the usage characteristics of the target passing area are obtained based on the usage characteristics of the sub-regions included in the target passing area, and the usage characteristics of the sub-regions are used to indicate the ratio of the duration of the existence of trace points in the sub-region to the historical time period; In response to the first trace point being located within the actual residence area, based on the passing probability and residence probability of the sub-region where the first trace point is located, output the probabilities that the violation behavior of the first trace point belongs to the violation residence behavior and the violation passing behavior respectively, where the passing probability and the residence probability are determined based on the usage characteristics of the target passing area.

10. The method according to claim 9, wherein, when the first trace point is in the extended area of the target passing area, the passing probability of the sub-region where the first trace point is located is the ratio of the usage characteristics of the target passing area to the usage characteristics of the sub-region where the first trace point is located.

11. The method according to any one of claims 1-10, wherein, the first trace point is collected by any one or more of the following devices: camera, lidar, and millimeter wave radar.

12. The method according to any one of claims 1-11, wherein, the preset residence area is located at the intersection position.

13. A traffic violation recognition device, wherein, comprising: a processing module, configured to determine an actual residence area and / or an actual passing area based on the distribution of multiple trace points in a traffic scenario in a historical time period, the multiple trace points being used to indicate the positions of pedestrians or non-motor vehicles at multiple moments, the actual residence area being the area where pedestrians or non-motor vehicles actually reside and the actual residence area including a preset residence area, the preset residence area including a preset area for pedestrians or non-motor vehicles to reside, and the actual passing area being the area where pedestrians or non-motor vehicles actually pass; an acquisition module, configured to acquire a first trace point, the first trace point being used to indicate the target position of a pedestrian or a non-motor vehicle at a first moment, and the first trace point being a point on a continuous trajectory or a point on a discontinuous trajectory; the processing module is further configured to output the violation behavior corresponding to the first trace point based on a first position relationship and / or a second position relationship, the first position relationship being the position relationship between the target position and the preset residence area and the actual residence area, and the second position relationship being the position relationship between the target position and a preset passing area and the actual passing area, the preset passing area including a preset area for pedestrians or non-motor vehicles to pass, and the violation behavior including violation residence or violation passing.

14. The device according to claim 13, wherein, the processing module is specifically configured to: when the target position is not located within the target area and is located within the actual residence area, output that the violation behavior corresponding to the first trace point is a violation residence behavior or a violation passing behavior; or, when the target position is not located within the target area and is located within the actual passing area, output that the violation behavior of the first trace point is a violation passing behavior; wherein, the target area includes the area that can be passed at the first moment in the preset residence area and the preset passing area.

15. The device according to claim 14, wherein, The processing module is specifically configured to: When the target position is not within the target area and the preset passage area and is within the actual residence area, output that the violation behavior of the first trace point is an illegal residence behavior in the spatial dimension; When the target position is outside the actual residence area and the preset passage area, output that the violation behavior of the first trace point is an illegal passage behavior in the spatial dimension; Alternatively, when the target position is not within the target area and is within the actual residence area and the preset passage area, output that the violation behavior of the first trace point is an illegal residence behavior or an illegal passage behavior in the time dimension.

16. The device according to any one of claims 13-15, wherein, The processing module is specifically configured to: Obtain a traffic map of the traffic scenario, where the traffic map is used to indicate the setting conditions of multiple traffic facilities; Based on the traffic map, divide to obtain multiple sub-areas, and the multiple sub-areas do not overlap; Based on the distribution of the multiple trace points in the multiple sub-areas, determine the actual residence area and the actual passage area. The actual residence area is determined based on the time length of the sub-areas having trace points in the historical time period, and the actual passage area is determined based on the situation of the sub-areas around the preset passage area having trace points.

17. The device according to claim 16, wherein, The processing module is specifically configured to: Based on the multiple trace points, determine the usage characteristics of each sub-area in the multiple sub-areas, where the usage characteristics are used to indicate the ratio of the duration of trace points in the sub-area to the historical time period; Based on the usage characteristics, determine the multiple residence sub-areas for constituting the actual residence area in the multiple sub-areas, and the multiple residence sub-areas belong to the multiple sub-areas.

18. The device according to claim 17, wherein, The processing module is specifically configured to: Based on the usage characteristics of each sub-area in the multiple sub-areas, perform edge detection on the multiple sub-areas to obtain at least one sub-area located at the edge of the actual residence area; Perform spatial clustering on the at least one sub-area to obtain the residence area type corresponding to each sub-area in the at least one sub-area; Based on the convex hull construction method, perform connection processing on the sub-areas under each residence area type to obtain the actual residence area, and the actual residence area is composed of the area surrounded by the connected sub-areas.

19. The device according to any one of claims 16-18, wherein, The trajectory coverage rate of the actual passage area is greater than a preset threshold, and the trajectory coverage rate is the ratio of the number of sub-areas having trace points in the historical time period in the actual passage area to the total number of sub-areas.

20. The device according to any one of claims 16-19, wherein, The sizes of different sub-areas in the multiple sub-areas are the same, and the size of the sub-areas in the multiple sub-areas is related to the floor area of pedestrians or non-motor vehicles.

21. The device according to any one of claims 16 - 20, wherein, the processing module is further configured to: determine a target passage area connected to the actual residence area, and the target passage area belongs to the actual passage area; based on the multiple trace points, determine the usage characteristics of the target passage area, where the usage characteristics of the target passage area are obtained based on the usage characteristics of the sub - areas included in the target passage area, and the usage characteristics of the sub - area are used to indicate the ratio of the duration with trace points in the sub - area to the historical time period; in response to the first trace point being within the actual residence area, output the probabilities that the violation behavior of the first trace point belongs to the violation residence behavior and the violation passage behavior respectively based on the passage probability and the residence probability of the sub - area where the first trace point is located, where the passage probability and the residence probability are determined based on the usage characteristics of the target passage area.

22. The device according to claim 21, wherein, when the first trace point is in the extended area of the target passage area, the passage probability of the sub - area where the first trace point is located is the ratio of the usage characteristics of the target passage area to the usage characteristics of the sub - area where the first trace point is located.

23. A traffic violation identification device, wherein, it includes a memory and a processor; the memory stores code, and the processor is configured to execute the code, and when the code is executed, the device performs the method according to any one of claims 1 to 12.

24. A computer storage medium, wherein, the computer storage medium stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

25. A computer program product, wherein, the computer program product stores instructions, and when the instructions are executed by a computer, the computer implements the method according to any one of claims 1 to 12.

Images