Intelligent parking lock control method and system based on multi-modal perception and authority game

Abstract

The invention provides an intelligent parking lock control method and system based on multi-modal perception and permission game, and relates to the field of intelligent parking management. The method comprises the following steps: scanning a warehousing intention track by a millimeter wave radar, judging whether there is a warehousing intention, and if yes, collecting license plate recognition features; the millimeter wave radar scans the width and height of the vehicle, and the pressure sensor array measures the tire contact area to obtain the attribute characteristics of the vehicle; summarizing the warehousing intention track, the license plate recognition feature and the vehicle attribute feature to form a fusion feature; calculating a first permission score based on the fused feature; dividing the vehicles into long-term renting users, temporary visitors and emergency vehicles, and calculating a second permission score based on the identity matching degree and the real-time permission validity; fusing the first permission score and the second permission score to obtain a final permission score, and judging whether unlocking is executed or not based on the final permission score. The control logic of the intelligent parking lock is optimized, and the parking experience of a public parking area is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent parking management, and in particular to an intelligent ground lock control method and system based on multimodal perception and authority game. Background Art

[0002] A parking ground lock is a mechanical device installed on the ground. Traditional mechanical ground locks require manual unlocking and release before parking, which is very inconvenient. This has led to the development of automatic parking ground locks. Currently, some public parking areas have installed these ground locks, enabling intelligent parking management and control.

[0003] However, at present, such parking space locks are usually linked to license plate number recognition. The essence of license plate number recognition is OCR recognition technology, which can be easily bypassed (for example, it can be successfully unlocked through pictures or even handwritten license plates).

[0004] Therefore, there is an urgent need for a reliable control method for smart ground locks to improve the control logic of existing smart ground locks in public parking areas to improve safety and optimize the parking experience. Summary of the Invention

[0005] Purpose of the invention: To propose an intelligent ground lock control method and system based on multimodal perception and authority game to solve the above-mentioned problems existing in the prior art.

[0006] The first aspect of the present invention proposes an intelligent ground lock control method based on multimodal perception and authority game, which includes the following steps: Arrange cameras, millimeter-wave radars, and pressure sensor arrays in the parking area where the ground locks are located; Before the vehicle enters the parking area, the ground lock is in a raised pre-judgment state; when the vehicle approaches the parking area, the millimeter-wave radar is triggered to scan the vehicle's entry intention trajectory to determine whether the current vehicle intends to enter the parking area. If so, the camera is triggered to collect the vehicle license plate and obtain the license plate recognition feature; After obtaining the license plate recognition feature, the millimeter wave radar scans the width and height of the vehicle, and the pressure sensor array measures the tire contact area to obtain the vehicle attribute feature; Summarize the entry intention trajectory, license plate recognition features, and vehicle attribute features to form a fusion feature; Calculating a first authority score based on the fusion feature; Classify vehicles into long-term rental users, temporary visitors, and emergency vehicles, and calculate the second permission score based on identity matching and real-time permission validity; The first authority score and the second authority score are integrated to calculate a final authority score, and whether to perform unlocking is determined based on the final authority score.

[0007] In a further embodiment of the first aspect, the millimeter-wave radar scans the vehicle's entry intention trajectory and constructs a feature vector of the entry intention trajectory; the entry intention trajectory is a movement trajectory of the vehicle 3 to 8 meters away from the ground lock.

[0008] Construct the feature vector of the storage intention trajectory, including: The coordinate sequence of the vehicle's motion trajectory at a distance of 3 to 8 meters from the ground lock is: ; Calculate the trajectory directionality based on the cosine value of the angle between the vehicle's movement direction and the parking space's center axis : ; Where, is the direction vector of the central axis of the parking space; is the displacement vector of the vehicle motion; The average speed change rate of three consecutive sampling points is used to determine whether the vehicle has the intention to slow down when entering the parking lot, and the deceleration trend is used to determine whether the vehicle has the intention to slow down when entering the parking lot. express: ; Where, represents the instantaneous speed of the vehicle corresponding to the kth sampling point; Indicates the The instantaneous speed of the vehicle corresponding to each sampling point; Indicates the timestamp of the kth sampling point; Indicates the The timestamp of each sampling point; Calculate the rate of change of the distance between the vehicle and the ground lock over time and normalize it to , with the distance convergence rate express: ; Where, is the initial distance, that is, the vehicle is 8 meters away from the ground lock; is the current distance; Constructing the feature vector of the storage intention trajectory : ; Where, To slow down the trend The function for non-negative processing, when hour, ,when hour, ; The preset maximum deceleration rate threshold.

[0009] In a further embodiment of the first aspect, the camera captures the vehicle license plate, uses OCR to recognize characters, and obtains the license plate recognition feature vector ,in Indicates the nth character.

[0010] In a further embodiment of the first aspect, the license plate recognition feature vector is obtained Then, the width W and height H of the vehicle are scanned by the millimeter wave radar, and the tire contact area is measured by the pressure sensor array. , construct the normalized vehicle attribute feature vector : ; Where, 、 、 They are respectively the maximum width, maximum height, and maximum tire contact area of ​​the preset family car model.

[0011] In a further embodiment of the first aspect, the feature vector of the storage intention trajectory is summarized , license plate recognition feature vector , vehicle attribute feature vector , forming a fusion feature vector .

[0012] In a further embodiment of the first aspect, based on the fusion feature vector , calculate the first authority score : ; Where, 、 、 are the dynamically adjusted weight coefficients; is the license plate matching score, that is, the license plate recognition feature vector The ratio of the number of successfully matched characters to the total number of characters in the license plate; Score the inbound intent match; Score vehicle attribute matching; Based on the warehousing intention trajectory feature vector Take the average of the three elements to get the storage intention matching score : ; Storage intention matching score The higher the value, the clearer the vehicle's intention to enter the warehouse; Vehicle attribute matching score The calculation formula is as follows: ; Where, Represents the normalized vehicle attribute feature vector The elements; Represents the normalized value of the authorized vehicle attribute.

[0013] In a further embodiment of the first aspect, vehicles are divided into long-term rental users, temporary visitors, and emergency vehicles, a second permission score is calculated based on the identity matching degree and the real-time permission validity, and a normalized result is output; Among them, if the license plate recognition feature vector and normalized vehicle attribute feature vector If the information matches the reserved information of the parking space, the current vehicle is considered to be a long-term rental user and the second authority score is directly assigned. ; If no long-term rental user is matched, the user is further judged to be a temporary visitor. In this case: If the license plate recognition feature vector of the current vehicle is in the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1, otherwise the authorization validity is assigned a value of 0; if the current time is within the authorized period, the time matching degree is assigned a value of 1, otherwise the time exceeding the period is reduced proportionally; Use the camera to identify the emergency license plate and / or emergency lights. If the current vehicle is identified as an emergency vehicle, the second authority score is directly assigned. .

[0014] In a further embodiment of the first aspect, the first authority score is integrated and the second authority score , calculate the final authority score ;in is the fusion weight; Determine whether to unlock based on the final authority score S: ; Where T is the preset threshold. The preset threshold for long-term rental user parking spaces is the highest, the preset threshold for community public parking spaces is medium, and the preset threshold for temporary parking lots is the lowest.

[0015] A second aspect of the present invention discloses an intelligent ground lock control system based on multimodal perception and authority game, the system comprising: at least one camera, a plurality of millimeter-wave radars, a plurality of pressure sensors arranged in an array, and an execution module; the camera, millimeter-wave radar, and pressure sensor are all arranged in the parking area where the ground lock is located; The execution module establishes communication with the camera, millimeter wave radar, and pressure sensor; The execution module can automatically execute the intelligent ground lock control method based on multimodal perception and authority game described in the first aspect.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This technical solution adopts multimodal perception fusion (camera to recognize license plates, millimeter-wave radar to detect vehicle attributes, and pressure sensors to collect tire features). By fusing feature vectors, the first authority score is calculated, and the legitimacy is verified from multiple dimensions such as vehicle identity, size, and warehousing intention, avoiding misjudgment of a single feature.

[0017] Based on the fusion of multimodal perception, vehicles are divided into long-term users, temporary visitors, and emergency vehicles, and dynamic authority games are introduced to take into account daily use, temporary visits, emergency situations and other scenarios, with stronger dynamic management capabilities.

[0018] In summary, the present invention takes into account the convenience and scenario adaptability of various types of users through "multi-dimensional perception and type-based authority game", and solves the problems of "low anti-scratch accuracy and cumbersome authorization management" of traditional ground locks. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flowchart of the intelligent lock control method in the embodiment. DETAILED DESCRIPTION

[0020] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art have not been described to avoid confusion with the present invention. Example 1:

[0021] This embodiment discloses a smart ground lock control method based on multimodal perception and authority game. The prerequisite is that a camera, millimeter wave radar, and pressure sensor array are arranged in the parking area where the ground lock is located. Before the vehicle enters the garage, the ground lock is in the raised pre-judgment state. Figure 1 The specific steps are as follows: S1. When a vehicle approaches a parking area, the millimeter-wave radar is triggered to scan the vehicle's parking intention trajectory to determine whether the current vehicle has the intention to enter the parking area. If so, the camera is triggered to collect the vehicle license plate and obtain the license plate recognition feature.

[0022] S2. After obtaining the license plate recognition features, the width and height of the vehicle are scanned by the millimeter wave radar, and the tire contact area is measured by the pressure sensor array to obtain the vehicle attribute features.

[0023] S3. Summarize the storage intention trajectory, license plate recognition features, and vehicle attribute features to form a fusion feature.

[0024] S4. Calculate a first authority score based on the fused features.

[0025] S5. Classify the vehicles into long-term rental users, temporary visitors, and emergency vehicles, and calculate the second permission score based on identity matching and real-time permission validity.

[0026] S6. The first authority score and the second authority score are combined to calculate a final authority score, and whether to perform unlocking is determined based on the final authority score. Example 2:

[0027] Based on the solution disclosed in the above embodiment 1, the entry intention trajectory can be the movement trajectory of the vehicle 3 to 8 meters away from the ground lock, and the entry intention trajectory feature vector is constructed using the entry intention trajectory. The specific implementation steps are as follows: The coordinate sequence of the vehicle's motion trajectory at a distance of 3 to 8 meters from the ground lock is: ; Calculate the trajectory directionality based on the cosine value of the angle between the vehicle's movement direction and the parking space's center axis , see the following formula (1): ; Where, is the direction vector of the central axis of the parking space; is the displacement vector of the vehicle motion; The average speed change rate of three consecutive sampling points is used to determine whether the vehicle has the intention to slow down when entering the parking lot, and the deceleration trend is used to determine whether the vehicle has the intention to slow down when entering the parking lot. Expressed as formula (2): ; Where, represents the instantaneous speed of the vehicle corresponding to the kth sampling point; Indicates the The instantaneous speed of the vehicle corresponding to each sampling point; Indicates the timestamp of the kth sampling point; Indicates the The timestamp of each sampling point; Calculate the rate of change of the distance between the vehicle and the ground lock over time and normalize it to , with the distance convergence rate Expressed as formula (3): ; Where, is the initial distance, that is, the vehicle is 8 meters away from the ground lock; is the current distance; Constructing the feature vector of the storage intention trajectory , see formula (4) ; Where, To slow down the trend The function for non-negative processing, when hour, ,when hour, ; The preset maximum deceleration rate threshold. Example 3:

[0028] Based on the solution disclosed in the aforementioned embodiment 1, this embodiment discloses in detail a feasible solution for constructing vehicle attribute features.

[0029] The camera collects the vehicle license plate, uses OCR to recognize the characters, and obtains the license plate recognition feature vector ,in Indicates the nth character. Get the license plate recognition feature vector Then, the millimeter-wave radar scans the width W and height H of the vehicle, and the pressure sensor array measures the tire contact area. , construct the normalized vehicle attribute feature vector , see formula (5): ; Where, 、 、 They are respectively the maximum width, maximum height, and maximum tire contact area of ​​the preset family car model. Example 4:

[0030] Based on the solutions disclosed in the aforementioned embodiments 1 to 3, this embodiment further provides a feasible way to calculate the first authority score, the second authority score, and the final authority score.

[0031] Summarize the feature vector of the storage intention trajectory , license plate recognition feature vector , vehicle attribute feature vector , forming a fusion feature vector .

[0032] Based on the fusion feature vector , calculate the first authority score according to formula (6) : ; Where, 、 、 are the dynamically adjusted weight coefficients; is the license plate matching score, that is, the license plate recognition feature vector The ratio of the number of successfully matched characters to the total number of characters in the license plate; Score the inbound intent match; Score vehicle attribute matching; According to the following formula (7), based on the warehousing intention trajectory feature vector Take the average of the three elements to get the storage intention matching score : ; Inbound intent matching score The higher the value, the clearer the vehicle's intention to enter the warehouse; Vehicle attribute matching score The calculation is as follows (8): ; Where, Represents the normalized vehicle attribute feature vector The elements; Represents the normalized value of the authorized vehicle attribute.

[0033] Classify vehicles into long-term rental users, temporary visitors, and emergency vehicles, calculate the second permission score based on identity matching and real-time permission validity, and output the normalized result; Among them, if the license plate recognition feature vector and normalized vehicle attribute feature vector If the information matches the reserved information of the parking space, the current vehicle is considered to be a long-term rental user and the second authority score is directly assigned. ; If no long-term rental user is matched, further determination is made as to whether the user is a temporary visitor. .

[0034] If the license plate recognition feature vector of the current vehicle is in the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1, otherwise the authorization validity is assigned a value of 0; if the current time is within the authorized period, the time matching degree is assigned a value of 1, otherwise the time exceeding the authorized period is reduced proportionally; Use the camera to identify the emergency license plate and / or emergency lights. If the current vehicle is identified as an emergency vehicle, the second authority score is directly assigned. .

[0035] Fusion First Authority Score and the second authority score , calculate the final authority score ;in is the fusion weight; The final authority score S is used to determine whether to unlock the car. If the final authority score S exceeds the preset threshold T, unlocking is performed; otherwise, unlocking is denied. The preset threshold T can be adjusted. For example, the preset threshold for long-term rental parking spaces is the highest, the preset threshold for community public parking spaces is medium, and the preset threshold for temporary parking lots is the lowest. Example 5:

[0036] This embodiment discloses an intelligent ground lock control system based on multimodal sensing and authority game. The system includes at least one camera, several millimeter-wave radars, several arrayed pressure sensors, and an execution module. The camera, millimeter-wave radar, and pressure sensor are all located within the parking area where the ground lock is located; the execution module establishes communication with the camera, millimeter-wave radar, and pressure sensor. The execution module automatically executes the intelligent ground lock control methods based on multimodal sensing and authority game described in Examples 1 to 4 above. Specifically, the execution module consists of a trajectory prediction unit, a license plate recognition unit, a vehicle attribute recognition unit, a first calculation unit, a second calculation unit, and an unlocking execution unit. Before a vehicle enters the parking area, the ground lock is in a raised pre-judgment state. When the vehicle approaches the parking area, the trajectory prediction unit triggers the millimeter-wave radar to scan the vehicle's intended entry trajectory to determine whether the vehicle intends to enter the parking area. If so, the system sends feedback to the license plate recognition unit, triggering the camera to capture the vehicle's license plate and obtain the license plate recognition features. After obtaining the license plate recognition features, the vehicle attribute recognition unit triggers the millimeter-wave radar to scan the vehicle's width and height. The pressure sensor array measures the tire contact area to obtain the vehicle attribute features. The first calculation unit aggregates the storage intention trajectory, license plate recognition features, and vehicle attribute features to form a fused feature, and calculates a first permission score based on the fused feature. The second calculation unit categorizes vehicles as long-term renters, temporary visitors, and emergency vehicles, and calculates a second permission score based on identity matching and real-time permission validity. The unlocking execution unit fuses the first and second permission scores to calculate a final permission score, and uses this final permission score to determine whether to unlock the vehicle.

[0037] The logic behind the methods disclosed in the above embodiments can be implemented in whole or in part through software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0038] When a computer instruction or computer program is loaded or executed on a computer, the process or function according to the embodiments of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired method (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0039] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0040] The technical features of the above embodiments may be combined in any manner. To simplify the description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification. Although the present invention has been shown and described with reference to specific preferred embodiments, it should not be interpreted as limiting the invention itself. Various changes in form and details may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. An intelligent ground lock control method based on multimodal perception and authority game, characterized in that: The steps include: Arrange cameras, millimeter-wave radars, and pressure sensor arrays in the parking area where the ground locks are located; Before the vehicle enters the parking area, the ground lock is in a raised pre-judgment state; when the vehicle approaches the parking area, the millimeter-wave radar is triggered to scan the vehicle's entry intention trajectory to determine whether the current vehicle intends to enter the parking area. If so, the camera is triggered to collect the vehicle license plate and obtain the license plate recognition feature; After obtaining the license plate recognition feature, the millimeter wave radar scans the width and height of the vehicle, and the pressure sensor array measures the tire contact area to obtain the vehicle attribute feature; Summarize the entry intention trajectory, license plate recognition features, and vehicle attribute features to form a fusion feature; Calculating a first authority score based on the fusion feature; Classify vehicles into long-term rental users, temporary visitors, and emergency vehicles, and calculate the second permission score based on identity matching and real-time permission validity; The first authority score and the second authority score are integrated to calculate a final authority score, and whether to perform unlocking is determined based on the final authority score.

2. The intelligent ground lock control method based on multimodal perception and authority game according to claim 1 is characterized in that: The millimeter-wave radar scans the vehicle's entry intention trajectory and constructs an entry intention trajectory feature vector; The warehousing intention trajectory is the movement trajectory of the vehicle from 3 to 8 meters away from the ground lock.

3. The intelligent ground lock control method based on multimodal perception and authority game according to claim 2 is characterized in that: Construct the feature vector of the storage intention trajectory, including: The coordinate sequence of the vehicle's motion trajectory at a distance of 3 to 8 meters from the ground lock is: ; Calculate the trajectory directionality based on the cosine value of the angle between the vehicle's movement direction and the parking space's center axis : Where, is the direction vector of the central axis of the parking space; is the displacement vector of the vehicle motion; The average speed change rate of three consecutive sampling points is used to determine whether the vehicle has the intention to slow down when entering the parking lot, and the deceleration trend is used to determine whether the vehicle has the intention to slow down when entering the parking lot. express: Where, represents the instantaneous speed of the vehicle corresponding to the kth sampling point; Indicates the The instantaneous speed of the vehicle corresponding to each sampling point; Indicates the timestamp of the kth sampling point; Indicates the The timestamp of each sampling point; Calculate the rate of change of the distance between the vehicle and the ground lock over time and normalize it to , with the distance convergence rate express: Where, is the initial distance, that is, the vehicle is 8 meters away from the ground lock; is the current distance; Constructing the feature vector of the storage intention trajectory : Where, To slow down the trend The function for non-negative processing, when hour, ,when hour, ; The preset maximum deceleration rate threshold.

4. The intelligent ground lock control method based on multimodal perception and authority game according to claim 3 is characterized in that: The camera collects the vehicle license plate, uses OCR to recognize the characters, and obtains the license plate recognition feature vector ,in Indicates the nth character.

5. The intelligent ground lock control method based on multimodal perception and authority game according to claim 4 is characterized in that: Get the license plate recognition feature vector Then, the width W and height H of the vehicle are scanned by the millimeter wave radar, and the tire contact area is measured by the pressure sensor array. , construct the normalized vehicle attribute feature vector : Where, 、 、 They are respectively the maximum width, maximum height, and maximum tire contact area of ​​the preset family car model.

6. The intelligent ground lock control method based on multimodal perception and authority game according to claim 5 is characterized in that: Summarize the feature vector of the storage intention trajectory , license plate recognition feature vector , vehicle attribute feature vector , forming a fusion feature vector .

7. The intelligent ground lock control method based on multimodal perception and authority game according to claim 6 is characterized in that: Based on the fusion feature vector , calculate the first authority score : Where, 、 、 are the dynamically adjusted weight coefficients; is the license plate matching score, that is, the license plate recognition feature vector The ratio of the number of successfully matched characters to the total number of characters in the license plate; Score the inbound intent match; Score vehicle attribute matching; Based on the warehousing intention trajectory feature vector Take the average of the three elements to get the storage intention matching score : ; Storage intention matching score The higher the value, the clearer the vehicle's intention to enter the warehouse; Vehicle attribute matching score The calculation formula is as follows: Where, Represents the normalized vehicle attribute feature vector The elements; Represents the normalized value of the authorized vehicle attribute.

8. The intelligent ground lock control method based on multimodal perception and authority game according to claim 5 is characterized in that: Classify vehicles into long-term rental users, temporary visitors, and emergency vehicles, calculate the second permission score based on identity matching and real-time permission validity, and output the normalized result; Among them, if the license plate recognition feature vector and normalized vehicle attribute feature vector If the information matches the reserved information of the parking space, the current vehicle is considered to be a long-term rental user and the second authority score is directly assigned. ; If no long-term rental user is matched, the user is further judged to be a temporary visitor. In this case: If the license plate recognition feature vector of the current vehicle is in the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1, otherwise the authorization validity is assigned a value of 0; if the current time is within the authorized period, the time matching degree is assigned a value of 1, otherwise the time exceeding the period is reduced proportionally; Use the camera to identify the emergency license plate and / or emergency lights. If the current vehicle is identified as an emergency vehicle, the second authority score is directly assigned. .

9. The intelligent ground lock control method based on multimodal perception and authority game according to claim 1 is characterized in that: Fusion of the first authority score and the second authority score , calculate the final authority score ;in is the fusion weight; Determine whether to unlock based on the final authority score S: Where T is the preset threshold. The preset threshold for long-term rental user parking spaces is the highest, the preset threshold for community public parking spaces is medium, and the preset threshold for temporary parking lots is the lowest.

10. An intelligent ground lock control system based on multimodal perception and authority game, characterized in that: include: At least one camera, several millimeter-wave radars, several pressure sensors arranged in an array, and an execution module; the cameras, millimeter-wave radars, and pressure sensors are all arranged in the parking area where the ground lock is located; The execution module establishes communication with the camera, millimeter wave radar, and pressure sensor; The execution module can automatically execute the intelligent ground lock control method based on multimodal perception and authority game described in any one of claims 1 to 9.

Images