Parking identification method and system based on multi-modal fusion identification curbstone machine

Abstract

The invention discloses a parking identification method and system based on a multi-modal fusion identification curbstone machine, and belongs to the technical field of parking management, and the method comprises the following steps: detecting a vehicle entry event through a built-in sensor of the curbstone machine, and triggering the collection of sensing data; when the adjacent curbstone machines detect vehicles within similar time, a collaborative sensing network is automatically established; each tooth machine broadcasts the acquired stereo image and point cloud data to a collaborative awareness network in combination with geographic position codes, carries out fusion, reconstructs a three-dimensional scene model, and analyzes and judges the number of actual vehicle units through a connected domain; and if the vehicle is a single vehicle, selecting an optimal recognition unit to execute a license plate recognition task according to the relationship between the three-dimensional contour of the vehicle and the spatial position of the curbstone machine. According to the invention, the identification problem during cross-position parking of the vehicle is effectively solved, repeated charging or missing detection is avoided, and the accuracy of license plate identification and the charging reliability in a complex scene are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of parking management, in particular to a parking identification method and system based on multi-modal fusion identification curb machine. BACKGROUND

[0002] In urban parking management, curb machines have become an important equipment for intelligent management of roadside parking spaces. The existing curb machine system usually adopts a fixed management mode of "one machine for one position", that is, each curb machine is responsible for monitoring and managing a fixed standard parking space, and independently judges the parking space occupation state and attempts to identify the license plate information through its integrated sensors (such as radar, camera, etc.). However, this mode has obvious limitations when facing large vehicles (such as freight trucks, vans, long-distance buses, etc.).

[0003] Due to the long body and large volume of large vehicles, they often need to cross multiple consecutive parking spaces when parking in public parking areas, or occupy non-standard space due to the protruding front and rear of the vehicle body. First, large vehicles may only partially cover a parking space, causing the curb machine corresponding to the parking space to fail to accurately generate an effective occupation signal due to incomplete detection target or inconsistent signal characteristics, or even completely miss the report; second, the license plate of a large vehicle is usually installed at a higher position at the front or rear of the vehicle, when the vehicle is parked across the parking spaces, its license plate may be far away from the front recognition area of any fixed curb machine, but is located in the monitoring blind area of the adjacent curb machine or beyond the effective focal length, causing the system to fail to capture a clear license plate image, resulting in identification failure.

[0004] More seriously, the same large vehicle may trigger multiple adjacent parking spaces of curb machines, and due to the lack of effective coordination and information fusion between the curb machines, the central management system is easy to misjudge it as multiple small vehicles parked at the same time, thereby causing incorrect charging behavior (such as repeated charging or confusion of charging subjects) and confusion of on-site management instructions (such as false alarms or dispatch conflicts).

[0005] Therefore, the existing curb machine system based on fixed parking space management cannot effectively cope with the precise identification and management challenges brought by large vehicles parked across parking spaces, and there is an urgent need for a parking management method that can realize intelligent coordination of multiple devices, accurately distinguish the number and identity of vehicles, and dynamically optimize resource allocation, to improve the identification accuracy, management reliability and overall energy efficiency of the system in complex scenarios. SUMMARY

[0006] The purpose of the present application is to provide a parking identification method and system based on multi-modal fusion identification curb machine, which solves the following technical problems: Therefore, the existing curb machine system based on fixed parking space management cannot effectively cope with the precise identification and management challenges brought by large vehicles parking across spaces, and there is an urgent need for a parking management method that can realize multi-device intelligent collaboration, accurately distinguish vehicle quantity and identity, and dynamically optimize resource allocation to improve the identification accuracy, management reliability and overall energy efficiency of the system in complex scenarios.

[0007] The object of the application can be achieved by the following technical solutions: The parking identification method based on multi-modal fusion identification curb machine comprises the following steps: S1, the built-in geomagnetic sensor of the curb machine collects real-time geomagnetic signals in the preset parking area, and compares with the preset geomagnetic reference value, if the comparison is inconsistent, a vehicle entering event is generated and the event occurrence time is recorded, and the sensing unit of the curb machine corresponding to the parking area is activated and sensing data is obtained; S2, when any curb machine generates a vehicle entering event, the event stream data recorded by the adjacent curb machine in the past preset time period is obtained, if there is a vehicle entering event in the event stream data, the curb machine and the adjacent curb machine are adaptively networked to construct a collaborative sensing network; S3, each curb machine broadcasts the sensing data recorded in the vehicle entering event and its own preset geographic position code to the collaborative sensing network, according to the geographic position code of each curb machine, the sensing data reported by all curb machines is three-dimensionally registered and a three-dimensional scene model is generated, and the actual vehicle unit quantity is determined according to the three-dimensional scene model; S4, if the vehicle unit quantity is not one, each curb machine independently performs license plate recognition task based on the sensing data collected by itself; if the vehicle unit quantity is one, the best curb machine is selected according to the three-dimensional scene model, which is authorized to perform license plate recognition task.

[0008] As a further scheme of the application: in S1, the specific process of simultaneously activating the sensing unit of the curb machine corresponding to the parking area and obtaining sensing data is: Send a start instruction to the built-in binocular vision sensing module and laser radar ranging module of the curb machine, the binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate; at the same time, the laser radar ranging module emits laser beams and receives echoes to generate high-precision three-dimensional point cloud data in the preset parking area.

[0009] As a further scheme of the application: in S2, the specific process of constructing a collaborative sensing network is: The curb machine generating the vehicle entering event is taken as a coordination initiating node, the identification of other curb machines within a preset distance threshold of the physical position thereof is acquired, and a networking request signal is sent to all curb machines corresponding to the identification, the signal containing the event occurrence time and geographic position information of the self; The curb machine receiving the request signal compares whether a vehicle entering event with a time difference within a preset tolerance range from the event occurrence time exists in the event flow data recorded by the self, and if so, returns a networking confirmation response to the coordination initiating node; The coordination initiating node lists the corresponding curb machine nodes in a cooperative member list according to all returned confirmation responses, and allocates communication resources and a synchronous clock reference to each member node based on the list, to generate a cooperative perception network.

[0010] As a further scheme of the application, in S2, if no vehicle entering event exists in the event flow data, a license plate recognition task is performed based on the perception data collected by the self, and the recognition result is uploaded to a cloud platform to generate a single vehicle billing record.

[0011] As a further scheme of the application, in S3, the specific process of determining the actual number of vehicle units is as follows: According to the geographic position codes of the curb machines, all stereoscopic image data and surface three-dimensional point cloud data received are subjected to spatio-temporal alignment and coordinate system unification, and three-dimensional point cloud registration algorithm is used to splice the perception data of different curb machines into a complete scene point cloud model; The scene point cloud model is subjected to voxel-based spatial segmentation and clustering analysis, if all point cloud data constitutes a single connected component, the point cloud data is determined as one vehicle unit, if it constitutes multiple independent point cloud components that are not connected to each other, the number of independent point cloud components is counted and output as the actual number of vehicle units.

[0012] As a further scheme of the application, in S4, the specific process of selecting the best curb machine is as follows: The geographic position coordinates of the curb machines in the cooperative perception network are acquired and a reference plane is constructed, the projection area of the three-dimensional scene model on the reference plane is acquired, if the geographic position coordinates of any curb machine are not within the projection area, the curb machine is marked as a pending curb machine; The projection head and tail points are determined according to the maximum length direction of the projection area, the head section plane and the tail section plane perpendicular to the reference plane are respectively constructed according to the projection head and tail points, wherein the head section plane passes through the head end point and is perpendicular to the reference plane, and the tail section plane passes through the tail end point and is perpendicular to the reference plane, the collection axis of each pending curb machine perception unit is acquired and the spatial included angle with the head section plane and the tail section plane is calculated, and the minimum value thereof is taken as the reference included angle of the pending curb machine. Obtain the reference included angle corresponding to all pending curbs, select the pending curb corresponding to the minimum value of the reference included angle as the best curb for authorizing it to perform the license plate recognition task.

[0013] As a further scheme of the application: in S4, further comprising calculating the actual covered equivalent standard parking space number of the vehicle unit based on the projection area, uploading the equivalent standard parking space number and the recognition result of the license plate recognition task performed by the best curb to the cloud platform to generate a single vehicle billing record.

[0014] The parking recognition system for identifying the curb based on multi-modal fusion is used to implement the above-mentioned parking recognition method for identifying the curb based on multi-modal fusion, comprising: The vehicle recognition module is used to continuously monitor the magnetic field change in the preset parking area through the magnetic field sensor built-in the curb, and calculate the total volume of metal objects in the preset parking area according to the magnetic field sensing data, and when the total volume of the metal objects exceeds the preset volume threshold, a vehicle entering event is generated and the event occurrence time is recorded, and the perception unit of the curb corresponding to the parking area is activated and the perception data is obtained; The cooperative perception module is used to generate a vehicle entering event when any curb generates a vehicle entering event, and then obtain the event stream data recorded by the adjacent curb in the past preset time period, and if the vehicle entering event exists in the event stream data, the curb and the adjacent curb are adaptively networked to construct a cooperative perception network; The vehicle judgment module is used to broadcast the perception data recorded in the vehicle entering event and the preset geographic location code of each curb to the cooperative perception network, perform three-dimensional space registration on the perception data reported by all curbs according to the geographic location code of each curb, and generate a three-dimensional scene model, and determine the actual number of vehicle units according to the three-dimensional scene model. The task execution module is used to independently perform the license plate recognition task based on the perception data collected by each curb if the number of vehicle units is not one, and select the best curb according to the three-dimensional scene model if the number of vehicle units is one, for authorizing it to perform the license plate recognition task.

[0015] The beneficial effects of the application are: 1) The present application effectively solves the problem that a single road curb cannot completely perceive and misjudge as multiple vehicles when a large vehicle parks across parking spaces by establishing a multi-road curb cooperative perception and dynamic networking mechanism. When a road curb detects a vehicle entering event, the system automatically queries the event stream of adjacent road curbs within a preset time period, and accordingly adaptively networks the relevant road curbs to build a cooperative perception network. It can be understood that when multiple consecutive road curbs detect the target object entering within a preset time, either a large vehicle or multiple vehicles exist and are parking, therefore a cooperative perception network is built, the perception data reported by multiple road curbs is registered in three-dimensional space, a three-dimensional scene model accurately reflecting the actual spatial distribution of vehicles is generated, and the actual number of parked vehicle units is determined accordingly. This process ensures that even if a vehicle parks across multiple parking spaces, the system can still correctly identify it as a whole vehicle unit, fundamentally avoiding billing errors and management instruction confusion caused by misjudgment, and significantly improving the accuracy and reliability of parking management.

[0016] 2) The present application significantly improves the success rate of large vehicle license plate recognition by introducing a best road curb selection mechanism based on a three-dimensional scene model. In view of the problem that the position of the license plate of a large vehicle may be beyond the optimal recognition range of any fixed road curb, after determining a single vehicle unit, the system intelligently selects the road curb unit with the best current perspective and the most appropriate distance based on the spatial information provided by the three-dimensional scene model, and authorizes it to perform license plate recognition tasks. This ensures that the system can call the perception unit that is most likely to capture a clear and directly facing license plate image, overcoming the defect that the license plate is easily in the recognition blind area or distortion area in the traditional fixed management mode, thereby significantly improving the accuracy of license plate recognition and the overall performance of the system.

[0017] 3) The present application adopts an event-triggered on-demand perception and cooperative computing mode, which optimizes the energy efficiency of the system. The system normally only monitors metal objects through low-power magnetic field sensors, and only activates the corresponding perception unit and forms a cooperative network for complex calculation when it is determined that a vehicle has entered and meets the conditions. Once the vehicle quantity determination and license plate recognition tasks are completed, the system can return to a low-power state. This dynamic resource scheduling strategy avoids the huge energy consumption caused by the continuous high-frequency work of all road curbs in the prior art, effectively reduces the overall power consumption, data transmission volume and computing burden of the system, and provides convenience for large-scale and long-term deployment. BRIEF DESCRIPTION OF DRAWINGS

[0018] The present application will be further described below with reference to the accompanying drawings.

[0019] Figure 1 is a parking recognition method flowchart of the present application based on multi-modal fusion recognition road curb. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] Please refer to Figure 1 The parking identification method of the curb machine based on multi-modal fusion recognition provided by the present application comprises the following steps: S1, collecting real-time geomagnetic signals of a preset parking area through a geomagnetic sensor built in the curb machine, and comparing the signals with a preset geomagnetic reference value, if the signals are inconsistent, generating a vehicle entering event and recording the time when the event occurs, and activating the sensing unit of the curb machine corresponding to the parking area and obtaining sensing data; S2, when any curb machine generates a vehicle entering event, obtaining event stream data recorded by adjacent curb machines of the curb machine within a preset time period, if there is a vehicle entering event in the event stream data, adaptively networking the curb machine and the adjacent curb machines, and constructing a collaborative sensing network; S3, each curb machine broadcasts the sensing data recorded in the vehicle entering event and its own preset geographic position code to the collaborative sensing network, according to the geographic position codes of the curb machines, performs three-dimensional space registration on the sensing data reported by all curb machines and generates a three-dimensional scene model, and determines the actual number of vehicle units according to the three-dimensional scene model; S4, if the number of vehicle units is not one, each curb machine independently performs a license plate recognition task based on the sensing data collected by itself; if the number of vehicle units is one, the best curb machine is selected according to the three-dimensional scene model, which is authorized to perform the license plate recognition task.

[0022] It is worth noting that, considering that in actual process, there may also be a situation that a large vehicle and a small vehicle simultaneously perform parking behavior, or multiple small vehicles simultaneously perform parking, the present application also sets a corresponding verification mechanism, which analyzes each independent point cloud component formed after space registration in the three-dimensional scene model, projects each independent point cloud component onto a reference plane constructed by the geographic position coordinates of the curb machine units in the collaborative network, and calculates the geometric features of the projection area, if the length of the projection area of a certain independent point cloud component in its main dimension is greater than the preset length of a single standard parking area, it is determined that the component corresponds to a large vehicle, and a cross-parking collaborative recognition process for the large vehicle is triggered; at the same time, the system determines the components with projection area size conforming to the standard parking characteristics as small vehicles, and continues to perform independent license plate recognition according to the original process.

[0023] In a preferred embodiment of the present application, the specific process of simultaneously activating the sensing unit of the curb machine corresponding to the parking area and obtaining sensing data in S1 is as follows: Send a start instruction to the binocular vision sensing module and the laser radar ranging module built in the curb machine, the binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate; synchronously, the laser radar ranging module emits laser beams and receives echoes to generate high-precision three-dimensional point cloud data in the preset parking area.

[0024] The control unit built in the curb machine sends a start instruction to the binocular vision sensing module and the laser radar ranging module through the internal signal transmission line, which is an electrical signal that can trigger the two modules to switch from a low-power standby state to a working state and start performing sensing tasks. The binocular vision sensing module includes two cameras spaced apart by a certain distance (similar to the distance between human eyes), and the preset frame rate is, for example, 20 frames per second. The two cameras will simultaneously capture the preset parking area. Due to the different angles of view, two sets of images of the same area will be obtained, and there will be a slight positional difference between the two sets of images, that is, parallax, just like the slightly different pictures seen by the left and right eyes when looking at the same object. Using this parallax, the module can calculate the depth information of each point in the picture to form multi-view stereoscopic images. For example, when a truck parked in the area is photographed, the position of the truck head photographed by the left camera and the position of the truck head photographed by the right camera will have a slight offset. Through this offset, the distance of the truck head from the camera can be determined, and the three-dimensional contour of the truck can be constructed. Synchronously, the laser radar ranging module will start working. It will emit a series of continuous laser pulses to the preset parking area. These laser pulses will cover a certain range like the beam of a flashlight. When the laser encounters objects such as the vehicle body, the ground, the curb, etc., it will be reflected back. The receiver on the module will capture these reflected echoes. Since the propagation speed of the laser is known (the speed of light), the module will calculate the time difference between the emission and reception of the laser. By multiplying this time difference by the speed of light and dividing by 2 (because the laser has traveled a round trip), the distance from the laser emission point to the reflecting object can be determined. The laser radar will emit laser light at certain angular intervals (such as every 0.1 degree in a direction). After the distance data of different directions are combined, a large number of points with spatial coordinates are formed, which together constitute the high-precision three-dimensional point cloud data in the preset parking area. For example, the laser reflects to different parts of the truck roof, side, wheels, etc., and the distances calculated from the reflected echoes are different. These distance data combined with the emission angle can mark the positions of various parts of the truck in three-dimensional space to form a three-dimensional point cloud model of the truck.

[0025] By simultaneously enabling two different principle perception modules, complementary perception data is obtained, thereby providing more comprehensive and accurate information for subsequent vehicle identification. The binocular vision sensing module can capture detailed information such as the color, texture, and license plate of the vehicle, which is crucial for identifying the identity of the vehicle (such as the license plate number). The three-dimensional point cloud data generated by the laser radar ranging module accurately reflects the three-dimensional size, position, and shape of the vehicle and is not affected by lighting conditions, and can work stably even in the dark or strong light environment. The synchronous operation of the two modules ensures that the image data and point cloud data obtained are consistent in time, facilitating subsequent fusion processing of the two types of data, such as using point cloud data to determine the actual size and position of the vehicle and using visual image data to identify the license plate. This multi-modal data fusion approach can effectively compensate for the shortcomings of a single sensor, such as the difficulty of identifying a license plate in poor lighting conditions or the difficulty of obtaining detailed information using only laser radar. The combination of the two can provide a reliable data foundation for constructing an accurate three-dimensional scene model, judging the number of vehicles, and identifying large vehicle cross-position parking, and thus help achieve accurate identification and management of vehicles in the parking area and solve the identification problem of large vehicles in existing systems.

[0026] In another preferred embodiment of the present application, the specific process of constructing a cooperative perception network in S2 is as follows: The kerb machine that generates the vehicle entry event is used as the coordination initiation node, the identification of other kerb machines within a preset distance threshold of its physical location is obtained, and a networking request signal containing its own event occurrence time and geographic location information is sent to all kerb machines corresponding to the identification. The kerb machine receiving the request signal compares whether there is a vehicle entry event with a time difference within the preset tolerance range from the event occurrence time in its own recorded event flow data. If there is, it returns a networking confirmation response to the coordination initiation node. The coordination initiation node lists the corresponding kerb nodes in the cooperative member list according to all returned confirmation responses, and allocates communication resources and synchronous clock references to each member node based on the list to generate a cooperative perception network.

[0027] When a certain curbstone detects a vehicle entering event, the system automatically queries the event stream of the adjacent curbstone within a preset time period, and accordingly adaptively groups the relevant curbstones to construct a collaborative perception network. It can be understood that when multiple consecutive curbstones detect the target object entering within a preset time, either a large vehicle exists, or multiple vehicles exist simultaneously, thus constructing a collaborative perception network, performing three-dimensional space registration on the perception data reported by multiple curbstones, generating a three-dimensional scene model accurately reflecting the actual spatial distribution of the vehicle, and determining the actual number of vehicle units parked accordingly. This process ensures that even if a vehicle is parked across multiple parking spaces, the system can still correctly identify it as a whole vehicle unit, fundamentally avoiding billing errors and management instruction confusion caused by misjudgment, and significantly improving the accuracy and reliability of parking management.

[0028] In another preferred embodiment of the present application, in S2, if there is no vehicle entering event in the event stream data, a license plate recognition task is performed based on the perception data collected by the curbstone, and the recognition result is uploaded to the cloud platform to generate a single vehicle billing record.

[0029] When there is no vehicle entering event in the event stream data of the adjacent curbstone, it indicates that the currently entering vehicle may only occupy a single standard parking space, and multiple curbstones are not needed for collaborative perception. At this time, the curbstone that generates the event performs a license plate recognition task based on the perception data collected by itself and uploads the result to generate a billing record, which is to simplify the process in the single parking space scenario, avoid unnecessary collaborative networking operations, and reduce the consumption of system resources. Because there is no need for networking communication and data collaboration between curbstones, the identification and billing of a single parking space vehicle can be quickly completed, improving the efficiency of small vehicle parking management. The purpose is to enable the system to flexibly switch processing modes according to the actual parking situation, and to complete the management task in a more concise way when the vehicle does not cross the parking space. It can solve the problem of large vehicle cross-parking identification while ensuring efficient management of regular small vehicles, so that the system has the ability to handle complex scenarios and can maintain high efficiency in simple scenarios, thereby improving the adaptability and overall efficiency of the parking management system.

[0030] In another preferred embodiment of the present application, in S3, the specific process of determining the actual number of vehicle units is: According to the geographical position code of each curbstone, the received all stereoscopic image data and surface three-dimensional point cloud data are time and space aligned and the coordinate system is unified, and the perception data of different curbstones is spliced into a complete scene point cloud model by a three-dimensional point cloud registration algorithm. The voxel-based spatial segmentation and clustering analysis are performed on the scene point cloud model, if all point cloud data form a single connected component, it is determined that the point cloud data is one vehicle unit; if multiple independent point cloud components which are not connected with each other are formed, the number of independent point cloud components is counted and output as the actual number of vehicle units.

[0031] Firstly, each curb machine has a preset geographic position code, which contains its own spatial position information, such as specific latitude and longitude or coordinates in a relative coordinate system. According to these codes, the system will first perform spatio-temporal alignment on the stereo image data and three-dimensional point cloud data received from different curb machines, because different curb machines may not collect data at exactly the same time, so all data needs to be adjusted to a unified time reference, just like when multiple cameras capture the same event, the pictures need to be synchronized in time; at the same time, the coordinate system is unified, because the sensing unit of each curb machine may establish a local coordinate system with itself as the origin, so these local coordinates need to be converted to a unified global coordinate system, such as converting the local coordinates (x1, y1, z1) of curb machine A and the local coordinates (x2, y2, z2) of curb machine B to the global coordinates with the entrance of the parking lot as the origin, so that the data of different curb machines can be corresponded in space. Then, the perception data is spliced through a three-dimensional point cloud registration algorithm, because when different curb machines shoot the same area, the point clouds of each may have angle or position deviation, the registration algorithm will find the overlapping parts between these point clouds and adjust the positions, such as curb machine A shoots the left side point cloud of the truck, and curb machine B shoots the right side point cloud of the truck, the algorithm will accurately splice the two side point clouds by identifying the common features of the truck body in the two point clouds (such as the door edge, tire position), forming a complete truck point cloud model, and finally obtaining a complete scene point cloud model. Then the scene point cloud model is subjected to voxel-based spatial segmentation, voxel is a small cubic unit in three-dimensional space, the system will divide the entire point cloud model into multiple voxels of the same size, each voxel contains the point cloud data in that space, which can simplify data processing; then clustering analysis is performed, that is, to determine which voxels of point cloud are connected with each other, the point clouds of the same vehicle will be in connected voxels because the vehicle body is a continuous whole, forming a connected component, such as the point clouds of a car will be concentrated in a group of connected voxels, and if there are two cars in the parking area, their point clouds will be in two groups of independent voxels, forming two independent components. If all point clouds form a single connected component, it means that it is one vehicle unit, such as a truck that spans two parking spaces, its point cloud covers a large range but is connected as a whole, and is determined as one unit; if there are multiple independent components which are not connected with each other, the number is counted as the number of vehicle units.

[0032] By integrating the perception data of multiple curb machines and performing accurate analysis, the actual number of vehicles in the parking area is accurately determined, avoiding misjudgment caused by single curb machine perspective limitations or large vehicle cross-position parking. Spatio-temporal alignment and coordinate unification ensure the compatibility of data from different sources, allowing the spliced point cloud model to accurately reflect the spatial relationship of the scene; voxel-based segmentation and clustering analysis can clearly distinguish the point clouds of different vehicles, even if the vehicles are close together, the system can accurately identify whether they are the same unit. This effectively solves the problem of large vehicles being misjudged as multiple vehicles in existing systems, while accurately identifying multiple vehicles coexisting, providing the correct number of vehicles for subsequent license plate recognition and billing, thereby improving the accuracy of parking management, avoiding repeated billing or missed billing, and ensuring the reliability of management.

[0033] In another preferred embodiment of the present application, in S4, the specific process of selecting the best curb machine is as follows: Obtain the geographic position coordinates of each curb machine in the collaborative perception network and construct a reference plane, obtain the projection area of the three-dimensional scene model on the reference plane, if the geographic position coordinates of any curb machine are not within the projection area, then mark the curb machine as a pending curb machine; Determine the projection head and tail points according to the maximum length direction of the projection area, construct the head segment plane and the tail segment plane perpendicular to the reference plane according to the projection head and tail points, respectively, wherein the head segment plane passes through the head point and is perpendicular to the reference plane, and the tail segment plane passes through the tail point and is perpendicular to the reference plane; obtain the collection axis of each pending curb machine perception unit and calculate the spatial angle with the head segment plane and the tail segment plane, and take the minimum value as the reference angle of the pending curb machine; Obtain the reference angles corresponding to all pending curb machines, select the pending curb machine corresponding to the minimum reference angle as the best curb machine, which is used to authorize it to perform license plate recognition tasks.

[0034] Firstly, the geographical position coordinates of all participating kerb machines in the cooperative perception network are obtained, which are precise position information stored in advance, such as kerb machine A at (x1, y1, z1), kerb machine B at (x2, y2, z2), kerb machine C at (x3, y3, z3), etc., and then a reference plane is constructed according to these coordinates. The construction method is similar to that of determining a plane with three points not on the same straight line. A unified reference plane is formed by spatial fitting of multiple kerb machine coordinates. This plane is usually close to the ground or parallel to the ground of the parking area. Next, a three-dimensional scene model (such as a three-dimensional point cloud model of a large vehicle parked across the position) is projected onto this reference plane to obtain a projection area, just like the shadow range of the vehicle on the ground under the sun. This projection area can reflect the coverage range of the vehicle on the reference plane. Then, check whether the geographical position coordinates of each kerb machine are within this projection area. If the coordinates of a kerb machine are not within the area, such as kerb machine A whose position is in front of the vehicle projection area and is not covered by the projection, it is marked as a pending kerb machine. It can be understood that these kerb machines not within the projection area are located in front of and behind the vehicle, and can capture the license plate. However, the kerb machines within the projection area may be blocked by the vehicle body or have poor viewing angle, so they are directly excluded.

[0035] According to the shape of the projection area, find its maximum length direction, such as the projection of the truck is a rectangle, the longer side is the maximum length direction, and the two endpoints of this direction are the projection head and tail endpoints, respectively. Assuming they are the head direction endpoint and the tail direction endpoint. Then, construct a head section plane with the head endpoint as the reference, which passes through the head endpoint and is perpendicular to the reference plane, just like a board perpendicular to the ground is erected at the head projection endpoint. Similarly, construct a tail section plane with the tail endpoint as the reference, which passes through the tail endpoint and is perpendicular to the reference plane, similar to a board perpendicular to the ground is erected at the tail projection endpoint. Each pending kerb machine's perception unit (such as a binocular vision module) has a collection axis, which is the main collection direction of the perception unit, such as the direction of the camera lens axis. Calculate the spatial angle between the collection axis and the head section plane and the tail section plane. The angle size reflects the inclination degree of the collection axis to the plane. The smaller the angle, the closer the collection axis is to being perpendicular to the plane, that is, the more directly it faces the head and tail directions of the vehicle. Take the minimum value of the two angles as the reference angle of the pending kerb machine, such as the collection axis of kerb machine A has an angle of 30 degrees with the head section plane and an angle of 60 degrees with the tail section plane, and its reference angle is 30 degrees. Finally, compare the reference angles of all pending kerb machines and select the kerb machine corresponding to the minimum value as the best kerb machine, because the collection axis of this kerb machine is closest to directly facing the head and tail of the vehicle, and it is more likely to clearly capture the license plate.

[0036] Through the above selection process, the unit closest to the vertical direction of the vehicle length direction can be automatically selected from the multiple side curb machines, so as to ensure that the selected curb machine has the most correct view, the smallest geometric distortion and the highest effective pixel ratio, which is beneficial to improve the accuracy and reliability of subsequent license plate character recognition. Based on the three-dimensional scene model and the spatial position relationship of the curb machine, objective calculation is carried out, which avoids the problem of unstable image quality caused by subjective selection or random assignment, and is especially suitable for large vehicles with long body and crossing multiple parking spaces. It can effectively solve the recognition difficulties caused by the deviation of the view angle, such as the local shielding, deformation or insufficient resolution of the license plate, and improve the overall success rate of the license plate recognition task in the cooperative perception network from the data source, and enhance the accuracy and practicality of the multi-parking space cooperative charging system.

[0037] In another preferred embodiment of the application, in the S4, the actual covered equivalent standard parking space number of the vehicle unit is calculated based on the projection area, and the recognition result of the best curb machine performing the license plate recognition task is obtained, and the equivalent standard parking space number and the recognition result are uploaded to the cloud platform to generate a single vehicle charging record.

[0038] When the actual covered equivalent standard parking space number of the vehicle unit is calculated based on the projection area, first, the size parameters of the preset standard parking space are determined, such as the fixed length and width of the standard parking space (for example, 5 meters long and 2.5 meters wide), which are the pre-set reference. Then, the actual size of the projection area of the three-dimensional scene model on the reference plane is measured, such as a large truck parked across the parking space, which may present a rectangle, and the length is 10 meters and the width is 2.5 meters. Then, the area of the projection area is divided by the area of a single standard parking space (5 meters x 2.5 meters = 12.5 square meters), that is, (10 meters x 2.5 meters) ÷ 12.5 square meters = 2, so that the equivalent standard parking space number covered by the vehicle unit is 2; Because the projection area directly reflects the actual space range occupied by the vehicle on the ground, and the size of the standard parking space is the reference for measuring the parking space, the equivalent number of parking spaces can be reasonably converted through the area ratio of the two. Then, the recognition result of the best curb machine performing the license plate recognition task is obtained, such as the best curb machine shooting a clear license plate image, and the character information on the license plate is parsed through the image recognition function of the curb machine. Finally, the calculated equivalent standard parking space number and the recognized license plate information are sent to the cloud platform through the communication module of the curb machine, and the cloud platform receives these information, and generates a charging record for the vehicle based on the driving time and other records of the vehicle, which reflects the vehicle identity and the actual occupied equivalent parking space number, which is used as the basis for charging.

[0039] In order to realize accurate charging for large vehicles parked across positions, ensure that the charging amount matches the actual occupied parking resource, avoid the problem that the resource occupation does not match the fee caused by charging large vehicles according to a single parking space, and avoid the problem of repeated charging caused by misjudgment of multiple vehicles across multiple parking spaces, the license plate recognition result and the equivalent parking space quantity are combined and uploaded, so that the charging record can be accurately matched with the specific vehicle, the responsibility is clear, the problem of inaccurate charging of large vehicles parked across positions in the existing system is solved, the charging link of parking management is improved, the entire system can accurately identify vehicles and reasonably charge when dealing with complex parking scenes, and the fairness, accuracy and practicality of the parking management system are improved.

[0040] The application also includes a parking identification system based on multi-modal fusion identification of curb machines, which is used for the above-mentioned multi-modal fusion identification of curb machines, and includes: A vehicle identification module is configured to continuously monitor the magnetic field change in a preset parking area through the magnetic field sensor built in the curb machine, calculate the total volume of metal objects in the preset parking area according to the magnetic field sensing data, generate a vehicle entry event and record the time when the event occurs when the total volume of the metal objects exceeds a preset volume threshold, and activate the sensing unit of the curb machine corresponding to the parking area and obtain sensing data. A cooperative sensing module is configured to obtain the event stream data recorded by the adjacent curb machine in a past preset time period when any curb machine generates a vehicle entry event, and if the vehicle entry event exists in the event stream data, the curb machine and the adjacent curb machine are adaptively networked to construct a cooperative sensing network. A vehicle judgment module is configured to broadcast the sensing data recorded in the vehicle entry event and the preset geographic location code of each curb machine to the cooperative sensing network, perform three-dimensional space registration on the sensing data reported by all curb machines according to the geographic location code of each curb machine, and generate a three-dimensional scene model to determine the actual number of vehicle units. A task execution module is configured to independently execute a license plate recognition task based on the sensing data collected by each curb machine if the number of vehicle units is not one, and select the best curb machine according to the three-dimensional scene model if the number of vehicle units is one, to authorize it to execute the license plate recognition task.

[0041] The above describes one embodiment of the application in detail, but the content described is only a preferred embodiment of the application and cannot be considered as limiting the scope of the application. Any equivalent changes and improvements made within the scope of the application should still belong to the patent coverage of the application.

Claims

1. A parking recognition method for a curbstone machine based on multi-modal fusion recognition, characterized in that, The method comprises the following steps: S1, collecting real-time geomagnetic signals of a preset parking area through a geomagnetic sensor built in a curbstone machine, and comparing with a preset geomagnetic reference value, if the comparison is inconsistent, a vehicle entering event is generated and the time of event occurrence is recorded, at the same time, the sensing unit of the curbstone machine corresponding to the parking area is activated and sensing data is obtained; S2, when any curbstone machine generates a vehicle entering event, the event stream data recorded by the adjacent curbstone machine in the past preset time period is obtained, if there is a vehicle entering event in the event stream data, the curbstone machine and the adjacent curbstone machine are adaptively networked, and a collaborative sensing network is constructed; S3, each curbstone machine broadcasts the sensing data recorded in the vehicle entering event and its own preset geographical position code to the collaborative sensing network, according to the geographical position code of each curbstone machine, the sensing data reported by all curbstone machines is three-dimensionally spatially registered and a three-dimensional scene model is generated, and the actual number of vehicle units is determined according to the three-dimensional scene model; S4, if the number of vehicle units is not one, each curbstone machine independently performs a license plate recognition task based on the sensing data collected by itself; if the number of vehicle units is one, the best curbstone machine is selected according to the three-dimensional scene model, which is used to authorize it to perform the license plate recognition task. 2.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S1, the specific process of activating the sensing unit of the curbstone machine corresponding to the parking area and obtaining the sensing data at the same time is: sending a start instruction to a binocular vision sensing module and a laser radar ranging module built in the curbstone machine, the binocular vision sensing module collects multi-view stereoscopic images in the preset parking area at a preset frame rate; at the same time, the laser radar ranging module emits a laser beam and receives a return wave to generate high-precision three-dimensional point cloud data in the preset parking area. 3.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S2, the specific process of constructing a collaborative sensing network is: taking the curbstone machine generating the vehicle entering event as a coordination initiation node, obtaining the device identifiers of other curbstone machines within a preset distance threshold of its physical location, and sending a networking request signal to all curbstone machines corresponding to the identifiers, the signal containing its event occurrence time and geographical position information; the curbstone machines receiving the request signal compare whether there is a vehicle entering event with a time difference within a preset tolerance range from the event occurrence time in the event stream data recorded by themselves; if there is, return a networking confirmation response to the coordination initiation node; the coordination initiation node lists the corresponding curbstone nodes in the collaborative member list according to all returned confirmation responses, and allocates communication resources and synchronous clock references to each member node based on the list, and generates a collaborative sensing network. 4.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S2, if there is no vehicle entering event in the event stream data, a license plate recognition task is performed based on the sensing data collected by itself, and the recognition result is uploaded to a cloud platform to generate a single vehicle billing record. 5.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 2, characterized in that, In S3, the specific process of determining the actual number of vehicle units is: According to the geographical position code of each curb machine, all received stereoscopic image data and surface three-dimensional point cloud data are spatio-temporally aligned and the coordinate systems are unified, and different curb machine perception data are spliced into a complete scene point cloud model through a three-dimensional point cloud registration algorithm; The scene point cloud model is subjected to voxel-based spatial segmentation and cluster analysis, if all point cloud data form a single connected component, the point cloud data is determined as a vehicle unit; if it forms multiple independent point cloud components, the number of independent point cloud components is counted and output as the actual number of vehicle units. 6.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 1, characterized in that, In S4, the specific process of selecting the best curb machine is as follows: Obtain the geographical position coordinates of each curb machine in the collaborative perception network and construct a reference plane, obtain the projection area of the three-dimensional scene model on the reference plane, if the geographical position coordinates of any curb machine are not in the projection area, the curb machine is marked as a pending curb machine; Determine the projection head and tail points according to the maximum length direction of the projection area, and construct the head section plane and the tail section plane perpendicular to the reference plane according to the projection head and tail points, respectively, wherein the head section plane passes through the head end point and is perpendicular to the reference plane, and the tail section plane passes through the tail end point and is perpendicular to the reference plane; Obtain the collection axis of each pending curb machine perception unit and calculate the spatial included angle with the head section plane and the tail section plane, and take the minimum value as the reference included angle of the pending curb machine; Obtain the corresponding reference included angle of all pending curb machines, select the pending curb machine corresponding to the minimum reference included angle as the best curb machine, which is used to authorize it to perform the license plate recognition task. 7.The parking identification method based on the multi-modal fusion identification curbstone machine according to claim 6, characterized in that, In S4, the actual covered equivalent standard parking space number of the vehicle unit is calculated based on the projection area, the recognition result of the best curb machine performing the license plate recognition task is obtained, and the equivalent standard parking space number and the recognition result are uploaded to the cloud platform to generate a single vehicle billing record.

8. The parking identification system based on multi-modal fusion identification of curbstone machine, for implementing the parking identification method based on multi-modal fusion identification of curbstone machine according to any one of claims 1-7, characterized in that, It includes: A vehicle recognition module for continuously monitoring the magnetic field changes in a preset parking area through the magnetic field sensor built-in the curb machine, and calculating the total volume of metal objects in the preset parking area according to the magnetic field sensing data, when the total volume of metal objects exceeds a preset volume threshold, a vehicle entry event is generated and the event occurrence time is recorded, and the perception unit of the curb machine corresponding to the parking area is activated and the perception data is obtained; A collaborative perception module for generating a vehicle entry event when any curb machine generates a vehicle entry event, obtaining the event stream data recorded by the adjacent curb machines of the curb machine in a past preset time period, if there is a vehicle entry event in the event stream data, the curb machine and the adjacent curb machines are adaptively networked to construct a collaborative perception network; A vehicle judgment module for each curb machine to broadcast the perception data recorded in the vehicle entry event and its own preset geographical position code to the collaborative perception network, to perform three-dimensional space registration on all reported perception data of the curb machines according to the geographical position code of each curb machine, and generate a three-dimensional scene model, and determine the actual number of vehicle units according to the three-dimensional scene model; The task execution module is configured to: if the number of vehicle units is not one, each curb machine independently executes a license plate recognition task based on the perception data collected by the curb machine; and if the number of vehicle units is one, a best curb machine is selected according to the three-dimensional scene model, and the curb machine is authorized to execute the license plate recognition task.

Images