Parking recognition method and system based on multi-modal fusion recognition curbstone machine

Abstract

The application discloses a parking identification method and system based on a multi-modal fusion identification curb machine, and belongs to the technical field of parking management. The method comprises the following steps: detecting a vehicle entering event through a built-in sensor of the curb machine and triggering perception data collection; when adjacent curb machines detect vehicles in similar time, a collaborative perception network is automatically established; each curb machine broadcasts the collected stereoscopic image and point cloud data combined with the geographical position code to the collaborative perception network and performs fusion, reconstructs a three-dimensional scene model, and determines the actual number of vehicle units through connected domain analysis; if it is a single vehicle, the best recognition unit is selected according to the three-dimensional contour and the spatial position relationship of the curb machine to perform a license plate recognition task. The application effectively solves the recognition problem of vehicle cross-position parking, avoids repeated billing or missed detection, and improves the accuracy of license plate recognition and the reliability of billing in complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of parking management, in particular to a parking identification method and system based on multi-modal fusion identification curb machine. BACKGROUND

[0002] In urban parking management, curb machines have become an important equipment for intelligent management of roadside parking spaces. The existing curb machine system usually adopts a fixed management mode of "one machine for one position", that is, each curb machine is responsible for monitoring and managing a fixed standard parking space, and independently judges the parking space occupation state and attempts to identify the license plate information through its own integrated sensors (such as radar, camera, etc.). However, this mode has obvious limitations when facing large vehicles (such as freight trucks, vans, long-distance buses, etc.).

[0003] Due to the long body and large volume of large vehicles, they often need to cross multiple consecutive parking spaces when parking in public parking areas, or occupy non-standard space due to the protruding front and rear of the vehicle body. First, large vehicles may only partially cover a parking space, causing the curb machine corresponding to the parking space to fail to accurately generate an effective occupation signal due to incomplete detection target or inconsistent signal characteristics, or even completely miss the report; second, the license plate of a large vehicle is usually installed at a higher position at the front or rear of the vehicle, when the vehicle is parked across the parking spaces, its license plate may be far away from the front recognition area of any fixed curb machine, but is located in the monitoring blind area of the adjacent curb machine or beyond the effective focal length, causing the system to fail to capture a clear license plate image, resulting in identification failure.

[0004] More seriously, the same large vehicle may trigger multiple adjacent parking spaces of curb machines, and due to the lack of effective coordination and information fusion between the curb machines, the central management system is easy to misjudge it as multiple small vehicles parked at the same time, thereby causing incorrect charging behavior (such as repeated charging or confusion of charging subjects) and confusion of on-site management instructions (such as false alarms or dispatch conflicts).

[0005] Therefore, the existing curb machine system based on fixed parking space management cannot effectively cope with the precise identification and management challenges brought by large vehicles parked across parking spaces, and there is an urgent need for a parking management method that can realize intelligent coordination of multiple devices, accurately distinguish the number and identity of vehicles, and dynamically optimize resource allocation, to improve the identification accuracy, management reliability and overall energy efficiency of the system in complex scenarios. SUMMARY

[0006] The purpose of the present application is to provide a parking identification method and system based on multi-modal fusion identification curb machine, which solves the following technical problems:

[0007] Therefore, the existing curb machine system based on fixed parking space management cannot effectively cope with the precise identification and management challenges brought by large vehicles parking across spaces, and there is an urgent need for a parking management method that can realize multi-device intelligent collaboration, accurately distinguish vehicle quantity and identity, and dynamically optimize resource allocation to improve the identification accuracy, management reliability and overall energy efficiency of the system in complex scenarios.

[0008] The object of the application can be achieved by the following technical solutions:

[0009] The parking identification method based on multi-modal fusion identification curb machine comprises the following steps:

[0010] S1, the built-in geomagnetic sensor of the curb machine collects real-time geomagnetic signals in the preset parking area, and compares them with the preset geomagnetic reference value. If the comparison is inconsistent, a vehicle entry event is generated and the time of the event is recorded. At the same time, the sensing unit of the curb machine corresponding to the parking area is activated and sensing data is obtained;

[0011] S2, when any curb machine generates a vehicle entry event, the event stream data recorded by the adjacent curb machine in the past preset time period is obtained. If there is a vehicle entry event in the event stream data, the curb machine and the adjacent curb machine are adaptively networked to construct a collaborative sensing network;

[0012] S3, each curb machine broadcasts the sensing data recorded in the vehicle entry event together with its own preset geographic location code to the collaborative sensing network. According to the geographic location code of each curb machine, the sensing data reported by all curb machines is three-dimensionally registered and a three-dimensional scene model is generated. The actual number of vehicle units is determined according to the three-dimensional scene model;

[0013] S4, if the number of vehicle units is not one, each curb machine independently performs a license plate recognition task based on the sensing data it collects. If the number of vehicle units is one, the best curb machine is selected according to the three-dimensional scene model to authorize it to perform the license plate recognition task.

[0014] As a further scheme of the application: in S1, the specific process of simultaneously activating the sensing unit of the curb machine corresponding to the parking area and obtaining sensing data is:

[0015] Send a start instruction to the built-in binocular vision sensing module and laser radar ranging module of the curb machine. The binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate. At the same time, the laser radar ranging module emits a laser beam and receives the echo to generate high-precision three-dimensional point cloud data in the preset parking area.

[0016] As a further scheme of the application: in S2, the specific process of constructing a collaborative sensing network is:

[0017] The curb machine that generates the vehicle entering event is taken as a coordination initiation node, the identification of other curb machines within a preset distance threshold of its physical location is acquired, and a networking request signal containing its event occurrence time and geographic location information is sent to all curb machines corresponding to the identification;

[0018] The curb machines receiving the request signal compare whether there is a vehicle entering event with a time difference within a preset tolerance range in the event flow data recorded by themselves; if there is, a networking confirmation response is returned to the coordination initiation node;

[0019] The coordination initiation node lists the corresponding curb nodes in the coordination member list according to all returned confirmation responses, and allocates communication resources and synchronous clock references to each member node based on the list to generate a collaborative perception network.

[0020] As a further scheme of the application: in S2, if there is no vehicle entering event in the event flow data, a license plate recognition task is performed based on the perception data collected by itself, and the recognition result is uploaded to the cloud platform to generate a single vehicle billing record.

[0021] As a further scheme of the application: in S3, the specific process of determining the actual number of vehicle units is:

[0022] According to the geographic location code of each curb machine, all stereoscopic image data and surface three-dimensional point cloud data received are spatio-temporally aligned and the coordinate systems are unified, and different curb machine perception data is spliced into a complete scene point cloud model through a three-dimensional point cloud registration algorithm;

[0023] The scene point cloud model is subjected to voxel-based spatial segmentation and clustering analysis, if all point cloud data constitutes a single connected component, the point cloud data is determined to be one vehicle unit; if it constitutes multiple independent point cloud components that are not connected, the number of independent point cloud components is counted and output as the actual number of vehicle units.

[0024] As a further scheme of the application: in S4, the specific process of selecting the best curb machine is:

[0025] The geographic location coordinates of each curb machine in the collaborative perception network are acquired and a reference plane is constructed, the projection area of the three-dimensional scene model on the reference plane is acquired, if the geographic location coordinates of any curb machine are not within the projection area, the curb machine is marked as a pending curb machine;

[0026] Determine the head and tail end points of the projection according to the maximum length direction of the projection area, and construct a head section plane and a tail section plane perpendicular to the reference plane according to the head and tail end points, respectively, wherein the head section plane passes through the head end point and is perpendicular to the reference plane, and the tail section plane passes through the tail end point and is perpendicular to the reference plane; obtain the collection axis of each to-be-determined curb machine perception unit and calculate the spatial included angle with the head section plane and the tail section plane, and take the minimum value as the reference included angle of the to-be-determined curb machine;

[0027] Obtain the reference included angles corresponding to all to-be-determined curb machines, select the to-be-determined curb machine corresponding to the minimum value of the reference included angle as the best curb machine, and authorize it to perform the license plate recognition task.

[0028] As a further scheme of the application, in S4, the actual covered equivalent standard parking spaces of the vehicle unit are calculated based on the projection area, the recognition result of the best curb machine performing the license plate recognition task is obtained, and the equivalent standard parking spaces and the recognition result are uploaded to the cloud platform to generate a single vehicle billing record.

[0029] The parking recognition system for identifying curb machines based on multi-modal fusion is used to implement the above-mentioned parking recognition method for identifying curb machines based on multi-modal fusion, and comprises:

[0030] A vehicle recognition module is configured to continuously monitor the magnetic field changes in a preset parking area through the magnetic field sensor built in the curb machine, and calculate the total volume of metal objects in the preset parking area according to the magnetic field sensing data, so that a vehicle entering event is generated and the time when the event occurs is recorded when the total volume of the metal objects exceeds a preset volume threshold, and the perception unit of the curb machine corresponding to the parking area is activated and perception data is obtained;

[0031] A cooperative perception module is configured to obtain the event stream data recorded by the adjacent curb machines in a past preset time period when any curb machine generates a vehicle entering event, and if there is a vehicle entering event in the event stream data, the curb machine and the adjacent curb machines are adaptively networked to construct a cooperative perception network.

[0032] A vehicle judgment module is configured to broadcast the perception data recorded in the vehicle entering event and the preset geographic location code of each curb machine to the cooperative perception network, perform three-dimensional space registration on the perception data reported by all curb machines according to the geographic location codes of the curb machines, and generate a three-dimensional scene model, and determine the actual number of vehicle units according to the three-dimensional scene model.

[0033] A task execution module is configured to independently execute a license plate recognition task based on the perception data collected by each curb machine if the number of vehicle units is not one, and select the best curb machine according to the three-dimensional scene model if the number of vehicle units is one, and authorize it to perform the license plate recognition task.

[0034] Advantages of the present application:

[0035] 1) The present application effectively solves the problem that a single road stud cannot completely perceive and misjudge as multiple vehicles when a large vehicle parks across parking spaces by establishing a multi-road stud cooperative perception and dynamic networking mechanism. When a road stud detects a vehicle entering event, the system automatically queries the event stream of adjacent road studs within a preset time period and accordingly adaptively networks the relevant road studs to build a cooperative perception network. It can be understood that when multiple consecutive road studs detect the target object entering within a preset time, either a large vehicle or multiple vehicles exist and are parking, so a cooperative perception network is built, the perception data reported by multiple road studs is three-dimensionally registered to generate a three-dimensional scene model accurately reflecting the actual spatial distribution of vehicles, and the actual number of parked vehicle units is determined accordingly. This process ensures that even if a vehicle parks across multiple parking spaces, the system can still correctly identify it as a whole vehicle unit, fundamentally avoiding billing errors and management instruction confusion caused by misjudgment, and significantly improving the accuracy and reliability of parking management.

[0036] 2) The present application significantly improves the success rate of large vehicle license plate recognition by introducing a best road stud selection mechanism based on a three-dimensional scene model. To address the problem that the position of a large vehicle license plate may exceed the best recognition range of any fixed road stud, the system intelligently selects the road stud unit with the best current perspective and most appropriate distance based on the spatial information provided by the three-dimensional scene model after determining a single vehicle unit, and authorizes it to perform license plate recognition tasks. This ensures that the system can call the perception unit most likely to capture a clear, directly facing license plate image, overcoming the defect that the license plate is easily in the recognition blind area or distortion area in the traditional fixed management mode, thereby significantly improving the accuracy of license plate recognition and the overall performance of the system.

[0037] 3) The present application adopts an event-triggered on-demand perception and cooperative computing mode, which optimizes the system energy efficiency. The system normally only monitors metal objects volume through low-power magnetic field sensors, and only activates the corresponding perception unit and builds a cooperative network for complex calculations when it is determined that a vehicle has entered and meets the conditions. Once the vehicle quantity determination and license plate recognition tasks are completed, the system can return to a low-power state. This dynamic resource scheduling strategy avoids the huge energy consumption caused by the continuous high-frequency work of all road studs in the prior art, effectively reducing the overall power consumption, data transmission volume, and computing burden of the system, and provides convenience for large-scale and long-term deployment. BRIEF DESCRIPTION OF DRAWINGS

[0038] The present application will be further described below with reference to the accompanying drawings.

[0039] Figure 1It is a parking recognition method flow diagram of the curb machine based on multi-modal fusion recognition of the application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the application.

[0041] Please refer to Figure 1 The application is a parking recognition method based on multi-modal fusion recognition of the curb machine, which comprises the following steps:

[0042] S1, the preset parking area is collected by the geomagnetic sensor built in the curb machine in real time, and compared with the preset geomagnetic reference value. If the comparison is inconsistent, a vehicle entering event is generated and the time of event occurrence is recorded, and the sensing unit of the curb machine corresponding to the parking area is activated and sensing data is obtained;

[0043] S2, when any curb machine generates a vehicle entering event, the event flow data recorded by the adjacent curb machine in the past preset time period is obtained. If there is a vehicle entering event in the event flow data, the curb machine and the adjacent curb machine are adaptively networked to construct a collaborative sensing network;

[0044] S3, each curb machine broadcasts the sensing data recorded in the vehicle entering event and its own preset geographic position code to the collaborative sensing network. According to the geographic position code of each curb machine, the sensing data reported by all curb machines is three-dimensionally registered and a three-dimensional scene model is generated. The actual number of vehicle units is determined according to the three-dimensional scene model.

[0045] S4, if the number of vehicle units is not one, each curb machine independently performs a license plate recognition task based on the sensing data collected by itself. If the number of vehicle units is one, the best curb machine is selected according to the three-dimensional scene model, which is authorized to perform the license plate recognition task.

[0046] It is worth noting that, considering that in actual process, there will also be large vehicles and small vehicles parking at the same time, or multiple small cars parking at the same time, the application also sets up a corresponding verification mechanism, which analyzes each independent point cloud component formed after spatial registration in the three-dimensional scene model, projects each independent point cloud component onto the reference plane constructed by the geographical position coordinates of the coordinated networking of curb machine units, and calculates the geometric characteristics of the projection area, if the length of the projection area of a certain independent point cloud component in its main dimension is greater than the preset length of a single standard parking area, it is determined that the component corresponds to a large vehicle, and the cross-space cooperative identification process for the large vehicle is triggered; at the same time, the system determines other components with projection area size conforming to the standard parking space characteristics as small vehicles, and continues to perform independent license plate recognition according to the original process.

[0047] In a preferred embodiment of the application, in S1, the specific process of simultaneously activating the sensing unit of the curb machine corresponding to the parking area and acquiring sensing data is:

[0048] Send a start instruction to the binocular vision sensing module and the laser radar ranging module built in the curb machine, the binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate; synchronously, the laser radar ranging module emits laser beams and receives echoes to generate high-precision three-dimensional point cloud data in the preset parking area.

[0049] The control unit built in the curb machine sends a start command to the binocular vision sensing module and the laser radar ranging module through the internal signal transmission line. This command is an electrical signal that can trigger the two modules to switch from a low-power standby state to a working state and start performing sensing tasks. The binocular vision sensing module includes two cameras spaced a certain distance apart (similar to the distance between human eyes) and a preset frame rate such as 20 frames per second. The two cameras will simultaneously capture images of the preset parking area. Due to the different angles of view, two sets of images of the same area will be obtained. These two sets of images have slight positional differences, i.e., parallax, just like the slightly different pictures seen by the left and right eyes when looking at the same object. Using this parallax, the module can calculate the depth information of each point in the image, thereby forming a multi-view stereoscopic image. For example, when a truck parked in the area is photographed, the position of the truck head photographed by the left camera and the position of the truck head photographed by the right camera have a slight offset. Through this offset, the distance between the truck head and the camera can be determined, and the three-dimensional profile of the truck can be constructed. At the same time, the laser radar ranging module will start working. It will emit a series of continuous laser pulses to the preset parking area. These laser pulses will cover a certain range like a flashlight beam. When the laser encounters objects such as the vehicle body, the ground, the curb, etc., it will be reflected back. The receiver on the module will capture these reflected echoes. Since the propagation speed of the laser is known (the speed of light), the module will calculate the time difference between the emission and reception of the laser. Multiply this time difference by the speed of light and divide by 2 (because the laser has traveled a round trip), and the distance from the laser emission point to the reflecting object can be obtained. The laser radar will emit laser light at certain angular intervals (such as every 0.1 degree in a direction). After the distance data of different directions is collected, a large number of points with spatial coordinates are formed. These points collectively constitute high-precision three-dimensional point cloud data in the preset parking area. For example, the laser reflects to different parts of the truck roof, side, wheels, etc., and the reflected echoes are calculated to different distances. These distance data combined with the emission angle can mark the positions of different parts of the truck in three-dimensional space and form a three-dimensional point cloud model of the truck.

[0050] By simultaneously enabling two different principle perception modules, complementary perception data is obtained, thereby providing more comprehensive and accurate information for subsequent vehicle identification. The binocular vision sensing module can capture detailed information such as the color, texture, and license plate of the vehicle, which is crucial for identifying the identity of the vehicle (such as the license plate number). The three-dimensional point cloud data generated by the laser radar ranging module can accurately reflect the three-dimensional size, position, and shape of the vehicle and is not affected by light conditions, and can work stably even in the dark or strong light environment. The synchronous operation of the two can keep the image data and point cloud data consistent in time, facilitating subsequent fusion processing of the two types of data, such as using point cloud data to determine the actual size and position of the vehicle and using visual image data to identify the license plate. This multi-modal data fusion method can effectively compensate for the shortcomings of a single sensor, such as the difficulty of identifying clearly in poor light using only vision or the difficulty of obtaining license plate details using only laser radar. The combination of the two can provide a reliable data basis for constructing an accurate three-dimensional scene model, judging the number of vehicles, and identifying large vehicle cross-position parking, and thus help to achieve accurate identification and management of vehicles in the parking area and solve the identification problem of large vehicles in existing systems.

[0051] In another preferred embodiment of the present application, the specific process of constructing a cooperative perception network in S2 is as follows:

[0052] The kerb machine that generates the vehicle entry event is used as the coordination initiation node, the identification of other kerb machines within a preset distance threshold of its physical location is obtained, and a networking request signal containing its own event occurrence time and geographic location information is sent to all kerb machines corresponding to the identification.

[0053] The kerb machine receiving the request signal compares whether there is a vehicle entry event with a time difference within the preset tolerance range from the event occurrence time in its own recorded event flow data. If there is, it returns a networking confirmation response to the coordination initiation node.

[0054] The coordination initiation node lists the corresponding kerb nodes in the cooperative member list according to all returned confirmation responses, and allocates communication resources and synchronous clock references to each member node based on the list to generate a cooperative perception network.

[0055] When a certain curbstone detects a vehicle entering event, the system automatically queries the event stream of the adjacent curbstone within a preset time period, and accordingly adaptively groups the relevant curbstones to construct a collaborative perception network. It can be understood that when multiple consecutive curbstones detect the target object entering within a preset time, either a large vehicle exists, or multiple vehicles exist simultaneously, thus constructing a collaborative perception network, performing three-dimensional space registration on the perception data reported by multiple curbstones, generating a three-dimensional scene model accurately reflecting the actual spatial distribution of the vehicle, and determining the actual number of vehicle units parked accordingly. This process ensures that even if the vehicle is parked across multiple parking spaces, the system can still correctly identify it as a whole vehicle unit, fundamentally avoiding billing errors and management instruction confusion caused by misjudgment, and significantly improving the accuracy and reliability of parking management.

[0056] In another preferred embodiment of the application, in S2, if there is no vehicle entering event in the event stream data, a license plate recognition task is performed based on the perception data collected by the curbstone, and the recognition result is uploaded to the cloud platform to generate a single vehicle billing record.

[0057] When there is no vehicle entering event in the event stream data of the adjacent curbstone, it indicates that the currently entering vehicle may only occupy a single standard parking space, and multiple curbstones are not needed for collaborative perception. At this time, the curbstone that generates the event performs a license plate recognition task based on the perception data collected by itself and uploads the result to generate a billing record, which is to simplify the process in the single parking space scenario, avoid unnecessary collaborative networking operations, and reduce the consumption of system resources. Because there is no need for networking communication and data collaboration between curbstones, the identification and billing of a single parking space vehicle can be quickly completed, improving the efficiency of small vehicle parking management. The purpose is to enable the system to flexibly switch processing modes according to the actual parking situation, and to complete the management task in a more concise way when the vehicle is not across the parking space. It can solve the problem of large vehicle cross-parking identification while ensuring efficient management of regular small vehicles, so that the system has the ability to handle complex scenarios and can maintain high efficiency in simple scenarios, thereby improving the adaptability and overall efficiency of the parking management system.

[0058] In another preferred embodiment of the application, in S3, the specific process of determining the actual number of vehicle units is as follows:

[0059] According to the geographical position code of each curbstone, the received all stereoscopic image data and surface three-dimensional point cloud data are spatio-temporally aligned and the coordinate systems are unified, and the perception data of different curbstones is spliced into a complete scene point cloud model through a three-dimensional point cloud registration algorithm;

[0060] The voxel-based spatial segmentation and clustering analysis are performed on the scene point cloud model, if all point cloud data form a single connected component, it is determined that the point cloud data is one vehicle unit; if multiple independent point cloud components which are not connected with each other are formed, the number of independent point cloud components is counted and output as the actual number of vehicle units.

[0061] Firstly, each curb machine has a preset geographic location code, which contains its own spatial position information, such as specific latitude and longitude or coordinates in a relative coordinate system. According to these codes, the system will first perform spatio-temporal alignment on the stereo image data and three-dimensional point cloud data received from different curb machines, because different curb machines may not collect data at exactly the same time, so all data needs to be adjusted to a unified time reference, just like when multiple cameras capture the same event, the pictures need to be synchronized in time; at the same time, the coordinate system is unified, because the sensing unit of each curb machine may establish a local coordinate system with itself as the origin, so these local coordinates need to be converted to a unified global coordinate system, such as converting the local coordinates (x1, y1, z1) of curb machine A and the local coordinates (x2, y2, z2) of curb machine B to the global coordinates with the entrance of the parking lot as the origin, so that the data of different curb machines can be corresponded in space. Then, the perception data is spliced through a three-dimensional point cloud registration algorithm, because when different curb machines shoot the same area, the point clouds of each may have angle or position deviation, the registration algorithm will find the overlapping parts between these point clouds and adjust the positions, such as curb machine A shoots the left side point cloud of the truck, and curb machine B shoots the right side point cloud of the truck, the algorithm will accurately splice the two side point clouds by identifying the common features of the truck body in the two point clouds (such as the door edge, tire position), forming a complete truck point cloud model, and finally obtaining a complete scene point cloud model. Then the scene point cloud model is subjected to voxel-based spatial segmentation, voxel is a small cubic unit in three-dimensional space, the system will divide the entire point cloud model into multiple voxels of the same size, each voxel contains the point cloud data in that space, which can simplify data processing; then clustering analysis is performed, that is, to determine which voxels of point cloud are connected with each other, the point clouds of the same vehicle will be in connected voxels because the vehicle body is a continuous whole, forming a connected component, such as the point clouds of a car will be concentrated in a group of connected voxels, and if there are two cars in the parking area, their point clouds will be in two groups of independent voxels, forming two independent components. If all point clouds form a single connected component, it means that it is one vehicle unit, such as a truck that spans two parking spaces, its point cloud covers a large range but is connected as a whole, and is determined as one unit; if there are multiple independent components which are not connected with each other, the number is counted as the number of vehicle units.

[0062] By integrating the perception data of multiple curb machines and performing accurate analysis, the actual number of vehicles in the parking area is accurately determined, avoiding misjudgment caused by single curb machine perspective limitations or large vehicle cross-position parking. Spatio-temporal alignment and coordinate unification ensure the compatibility of data from different sources, allowing the spliced point cloud model to accurately reflect the spatial relationship of the scene; voxel-based segmentation and clustering analysis can clearly distinguish the point clouds of different vehicles, even if the vehicles are close together, the system can accurately identify whether they are the same unit. This effectively solves the problem of large vehicles being misjudged as multiple vehicles in existing systems, while accurately identifying multiple vehicles coexisting, providing the correct number of vehicles for subsequent license plate recognition and billing, thereby improving the accuracy of parking management, avoiding repeated billing or missed billing, and ensuring the reliability of management.

[0063] In another preferred embodiment of the present application, in S4, the specific process of selecting the best curb machine is:

[0064] Obtain the geographic position coordinates of each curb machine in the collaborative perception network and construct a reference plane, obtain the projection area of the three-dimensional scene model on the reference plane, if the geographic position coordinates of any curb machine are not within the projection area, then mark the curb machine as a pending curb machine;

[0065] Determine the projection head and tail points according to the maximum length direction of the projection area, construct a head section plane and a tail section plane perpendicular to the reference plane according to the projection head and tail points, respectively, wherein the head section plane passes through the head end point and is perpendicular to the reference plane, and the tail section plane passes through the tail end point and is perpendicular to the reference plane; obtain the collection axis of each pending curb machine perception unit and calculate the spatial angle with the head section plane and the tail section plane, and take the minimum value as the reference angle of the pending curb machine;

[0066] Obtain the reference angles corresponding to all pending curb machines, select the pending curb machine corresponding to the minimum reference angle as the best curb machine, which is used to authorize it to perform license plate recognition tasks.

[0067] Firstly, the geographical position coordinates of all participating kerb machines in the cooperative perception network are obtained, which are precise position information stored in advance, such as kerb machine A at (x1, y1, z1), kerb machine B at (x2, y2, z2), kerb machine C at (x3, y3, z3), etc., and then a reference plane is constructed according to these coordinates. The construction method is similar to that of determining a plane with three points not on the same straight line. A unified reference plane is formed by spatial fitting of multiple kerb machine coordinates. This plane is usually close to the ground or parallel to the ground of the parking area. Then, a three-dimensional scene model (such as a three-dimensional point cloud model of a large vehicle parked across the position) is projected onto this reference plane to obtain a projection area, just like the shadow range of the vehicle on the ground under the sun. This projection area can reflect the coverage range of the vehicle on the reference plane. Then, check whether the geographical position coordinates of each kerb machine are within this projection area. If the coordinates of a kerb machine are not within the area, such as kerb machine A whose position is in front of the vehicle projection area and is not covered by the projection, it is marked as a pending kerb machine. It can be understood that these kerb machines not within the projection area are in front of and behind the vehicle, and can capture the license plate. The kerb machines within the projection area may be blocked by the vehicle body or have poor viewing angle, so they are directly excluded.

[0068] According to the shape of the projection area, find its maximum length direction, such as the projection of the truck is a rectangle, the longer side is the maximum length direction, and the two endpoints of this direction are the projection head and tail endpoints, respectively. Assuming they are the head direction endpoint and the tail direction endpoint. Then, construct a head section plane with the head endpoint as the reference, which passes through the head endpoint and is perpendicular to the reference plane, just like a board perpendicular to the ground is erected at the head projection endpoint. Similarly, construct a tail section plane with the tail endpoint as the reference, which passes through the tail endpoint and is perpendicular to the reference plane, similar to a board perpendicular to the ground is erected at the tail projection endpoint. Each pending kerb machine's perception unit (such as a binocular vision module) has a collection axis, which is the main collection direction of the perception unit, such as the direction of the camera lens axis. Calculate the spatial angle between the collection axis and the head section plane and the tail section plane. The angle size reflects the inclination degree of the collection axis to the plane. The smaller the angle, the closer the collection axis is to being perpendicular to the plane, that is, the more directly it faces the head and tail directions of the vehicle. Take the minimum value of the two angles as the reference angle of the pending kerb machine, such as the collection axis of kerb machine A has an angle of 30 degrees with the head section plane and an angle of 60 degrees with the tail section plane, and its reference angle is 30 degrees. Finally, compare the reference angles of all pending kerb machines and select the kerb machine corresponding to the minimum value as the best kerb machine, because the collection axis of this kerb machine is closest to directly facing the head and tail of the vehicle, and it is more likely to clearly capture the license plate.

[0069] Through the above selection process, the unit closest to the vertical direction of the vehicle length direction can be automatically selected from the multiple side curb machines, so as to ensure that the selected curb machine has the most correct view, the smallest geometric distortion and the highest effective pixel ratio, which is beneficial to improve the accuracy and reliability of subsequent license plate character recognition. Based on the objective calculation of the three-dimensional scene model and the spatial position relationship of the curb machine, the problem of unstable image quality caused by subjective selection or random assignment is avoided, especially for large vehicles with long body and crossing multiple parking spaces, which can effectively solve the recognition difficulties caused by view angle deviation, such as partial license plate occlusion, deformation or insufficient resolution, and improve the overall success rate of license plate recognition task in the cooperative perception network from the data source, and enhance the accuracy and practicality of the multi-space cooperative charging system.

[0070] In another preferred embodiment of the application, in S4, the actual covered equivalent standard parking space number of the vehicle unit is calculated based on the projection area, and the recognition result of the best curb machine performing license plate recognition task is obtained, and the equivalent standard parking space number and the recognition result are uploaded to the cloud platform to generate a single vehicle charging record.

[0071] When calculating the actual covered equivalent standard parking space number of the vehicle unit based on the projection area, first, the size parameters of the preset standard parking space are determined, such as the fixed length and width of the standard parking space (for example, 5 meters long and 2.5 meters wide), which are the pre-set reference. Then, the actual size of the projection area of the three-dimensional scene model on the reference plane is measured, such as a large truck parked across the space, which may present a rectangle, the length of which is 10 meters and the width of which is 2.5 meters. Then, the area of the projection area is divided by the area of a single standard parking space (5 meters x 2.5 meters = 12.5 square meters), i.e. (10 meters x 2.5 meters) ÷ 12.5 square meters = 2, so that the equivalent standard parking space number covered by the vehicle unit is 2; because the projection area directly reflects the actual space occupied by the vehicle on the ground, and the size of the standard parking space is the reference for measuring the parking space, the area ratio of the two can reasonably convert the equivalent number of parking spaces. Then, the recognition result of the best curb machine performing license plate recognition task is obtained, such as the best curb machine capturing a clear license plate image, and then analyzing the character information on the license plate through its own image recognition function. Finally, the calculated equivalent standard parking space number and the recognized license plate information are sent to the cloud platform through the communication module of the curb machine, and the cloud platform receives these information, combines the entry time of the vehicle and other records to generate a charging record for this vehicle, which reflects the vehicle identity and the actual occupied equivalent parking space number, which serves as the basis for charging.

[0072] In order to realize accurate charging for large vehicles parked across positions, ensure that the charging amount matches the actual occupied parking resource, avoid the problem of mismatch between resource occupation and fee caused by charging large vehicles according to a single parking space, and avoid repeated charging caused by misjudgment of large vehicles as multiple vehicles, the license plate recognition result is combined with the equivalent parking space quantity for uploading, which can ensure that the charging record accurately corresponds to the specific vehicle, ensure clear responsibility, solve the problem of inaccurate charging of large vehicles parked across positions in the existing system, perfect the charging link of parking management, make the whole system not only accurately identify vehicles but also reasonably charge when dealing with complex parking scenes, and thus improve the fairness, accuracy and practicality of the parking management system.

[0073] The application also includes a parking identification system based on multi-modal fusion identification of curb machines, which is used for the above-mentioned multi-modal fusion identification of curb machines, and includes:

[0074] A vehicle identification module is used to continuously monitor the magnetic field change in the preset parking area through the magnetic field sensor built in the curb machine, calculate the total volume of metal objects in the preset parking area according to the magnetic field sensing data, generate a vehicle entry event and record the event occurrence time when the total volume of the metal objects exceeds the preset volume threshold, and activate the sensing unit of the curb machine corresponding to the parking area and obtain the sensing data;

[0075] A cooperative sensing module is used to obtain the event stream data recorded by the adjacent curb machine in the past preset time period when any curb machine generates a vehicle entry event, and if there is a vehicle entry event in the event stream data, the curb machine and the adjacent curb machine are adaptively networked to construct a cooperative sensing network;

[0076] A vehicle judgment module is used to broadcast the sensing data recorded in the vehicle entry event and the preset geographic location code of each curb machine to the cooperative sensing network, perform three-dimensional space registration on the sensing data reported by all curb machines according to the geographic location code of each curb machine, and generate a three-dimensional scene model, and determine the actual number of vehicle units according to the three-dimensional scene model;

[0077] A task execution module is used to independently execute a license plate recognition task based on the sensing data collected by each curb machine if the number of vehicle units is not one, and select the best curb machine according to the three-dimensional scene model if the number of vehicle units is one, for authorizing it to execute the license plate recognition task.

[0078] The above has been described in detail one embodiment of the present application, but the content is only the preferred embodiment of the present application, cannot be considered for limiting the scope of the present application. Any equivalent changes and improvements made in the scope of the present application, should still belong to the scope of the present application.

Claims

1. A parking recognition method for a curbstone machine based on multi-modal fusion recognition, characterized in that, The method comprises the following steps: S1, collecting real-time geomagnetic signals of a preset parking area through a geomagnetic sensor built in a curbstone machine, and comparing the signals with a preset geomagnetic reference value, if the signals are inconsistent with the reference value, generating a vehicle entering event and recording the time when the event occurs, and simultaneously activating a sensing unit of the curbstone machine corresponding to the parking area and obtaining sensing data; S2, when any curbstone machine generates a vehicle entering event, obtaining event stream data recorded by curbstone machines adjacent to the curbstone machine within a preset time period, if there is a vehicle entering event in the event stream data, adaptively networking the curbstone machine and the adjacent curbstone machines, and constructing a collaborative sensing network; S3, each curbstone machine broadcasts the sensing data recorded in the vehicle entering event and its own preset geographical position code to the collaborative sensing network, performs three-dimensional space registration on the sensing data reported by all curbstone machines according to the geographical position codes of the curbstone machines, and generates a three-dimensional scene model, and determines the actual number of vehicle units according to the three-dimensional scene model; S4, if the number of vehicle units is not one, each curbstone machine independently performs a license plate recognition task based on the sensing data collected by itself; if the number of vehicle units is one, a best curbstone machine is selected according to the three-dimensional scene model, which is authorized to perform a license plate recognition task; The specific process of selecting the best curbstone machine is as follows: obtain the geographical position coordinates of each curbstone machine in the collaborative sensing network and construct a reference plane, obtain the projection area of the three-dimensional scene model on the reference plane, if the geographical position coordinates of any curbstone machine are not in the projection area, the curbstone machine is marked as a pending curbstone machine; determine the projection head and tail points according to the maximum length direction of the projection area, and construct a head section plane and a tail section plane perpendicular to the reference plane according to the projection head and tail points, wherein the head section plane passes through the head point and is perpendicular to the reference plane, and the tail section plane passes through the tail point and is perpendicular to the reference plane; obtain the collection axis of the sensing unit of each pending curbstone machine, calculate the spatial included angle with the head section plane and the tail section plane, and take the minimum value as the reference included angle of the pending curbstone machine; obtain the reference included angles of all pending curbstone machines, select the pending curbstone machine corresponding to the minimum value of the reference included angle as the best curbstone machine, which is authorized to perform a license plate recognition task. 2.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In the S1, the specific process of simultaneously activating the sensing unit of the curbstone machine corresponding to the parking area and obtaining the sensing data is as follows: sending a start instruction to a binocular vision sensing module and a laser radar ranging module built in the curbstone machine, the binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate; simultaneously, the laser radar ranging module emits a laser beam and receives a return wave to generate high-precision three-dimensional point cloud data in the preset parking area. 3.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In the S2, the specific process of constructing the collaborative sensing network is as follows: taking the curbstone machine generating the vehicle entering event as a coordination initiation node, obtaining device identifiers of other curbstone machines within a preset distance threshold of the physical location of the coordination initiation node, and sending a networking request signal to all curbstone machines corresponding to the identifiers, the signal containing the time when the event occurs and the geographical position information of the coordination initiation node; The curb machine receiving the request signal compares whether there is a vehicle entering event in the event stream data recorded by itself within a preset tolerance range of the event occurrence time; If so, return a network confirmation response to the coordination initiation node; The coordination initiation node lists the corresponding curb machine node in the collaborative member list according to all returned confirmation responses, and assigns communication resources and synchronous clock reference to each member node based on the list, and generates a collaborative sensing network. 4.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S2, if there is no vehicle entering event in the event stream data, the curb machine performs license plate recognition based on the sensing data collected by itself, and uploads the recognition result to the cloud platform to generate a single vehicle billing record. 5.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 2, characterized in that, In S3, the specific process of determining the actual number of vehicle units is as follows: According to the geographical position code of each curb machine, the received all stereoscopic image data and surface three-dimensional point cloud data are spatio-temporally aligned and the coordinate system is unified, and the three-dimensional point cloud registration algorithm is used to splice the sensing data of different curb machines into a complete scene point cloud model; The scene point cloud model is subjected to voxel-based spatial segmentation and clustering analysis, and if all point cloud data form a single connected component, the point cloud data is determined as one vehicle unit; if it forms multiple independent point cloud components, the number of independent point cloud components is counted and output as the actual number of vehicle units. 6.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S4, the actual number of equivalent standard parking spaces covered by the vehicle unit is calculated based on the projection area, and the recognition result of the curb machine performing the license plate recognition task is obtained, and the number of equivalent standard parking spaces and the recognition result are uploaded to the cloud platform to generate a single vehicle billing record.

7. The parking identification system based on multi-modal fusion identification of curbstone machine, for implementing the parking identification method based on multi-modal fusion identification of curbstone machine according to any one of claims 1-6, characterized in that, It includes: A vehicle recognition module for continuously monitoring the magnetic field changes in a preset parking area through the magnetic field sensor built-in the curb machine, and calculating the total volume of metal objects in the preset parking area according to the magnetic field sensing data, and generating a vehicle entering event and recording the event occurrence time when the total volume of metal objects exceeds a preset volume threshold, and activating the sensing unit of the curb machine corresponding to the parking area and obtaining sensing data; A collaborative sensing module for generating a vehicle entering event when any curb machine generates a vehicle entering event, obtaining the event stream data recorded by the adjacent curb machines within a preset time period, and if there is a vehicle entering event in the event stream data, adaptively networking the curb machine and the adjacent curb machines to build a collaborative sensing network; A vehicle judgment module for each curb machine to broadcast the sensing data recorded in the vehicle entering event and its own preset geographical position code to the collaborative sensing network, perform three-dimensional space registration on all reported sensing data of the curb machines according to the geographical position codes of the curb machines, and generate a three-dimensional scene model, and determine the actual number of vehicle units according to the three-dimensional scene model; A task execution module for performing license plate recognition based on the sensing data collected by itself if the number of vehicle units is not one; if the number of vehicle units is one, the best curb machine is selected according to the three-dimensional scene model to authorize it to perform the license plate recognition task.

Images