A Smart Ground Lock Control Method and System Based on Multimodal Perception and Authority Game Theory

Abstract

This invention provides a smart parking lock control method and system based on multimodal perception and permission game theory, relating to the field of smart parking management. The method uses millimeter-wave radar to scan the parking intention trajectory to determine if there is an intention to enter; if so, it collects license plate recognition features. It then scans the vehicle's width and height using millimeter-wave radar and measures the tire contact area using a pressure sensor array to obtain vehicle attribute features. The parking intention trajectory, license plate recognition features, and vehicle attribute features are combined to form a fused feature. A first permission score is calculated based on the fused feature. Vehicles are categorized as long-term renters, temporary visitors, and emergency vehicles, and a second permission score is calculated based on identity matching and real-time permission validity. The first and second permission scores are combined to obtain a final permission score, which is used to determine whether to unlock the vehicle. This invention optimizes the control logic of smart parking locks and improves the parking experience in public parking areas.

Description

Technical Field

[0001] This invention relates to the field of intelligent parking management, and in particular to an intelligent parking lock control method and system based on multimodal perception and permission game theory. Background Technology

[0002] A parking space lock is a mechanical device installed on the ground. Traditional parking locks are mechanical locks, which require manual unlocking and release before parking, which is very inconvenient. Therefore, parking space locks with automatic release mechanisms have been developed. Currently, some public parking areas have installed such locks to achieve intelligent management of vehicle parking.

[0003] However, currently, these types of parking space locks are usually linked to license plate recognition. The essence of license plate recognition is OCR recognition technology, which is easy to bypass (such as through images or even handwritten license plates).

[0004] Therefore, there is an urgent need for a reliable control method for smart parking locks to improve the control logic of existing smart parking locks in public parking areas, so as to enhance security and optimize the parking experience. Summary of the Invention

[0005] Purpose of the invention: To propose an intelligent ground lock control method and system based on multimodal perception and permission game theory, so as to solve the above-mentioned problems existing in the prior art.

[0006] In a first aspect, this invention proposes an intelligent ground lock control method based on multimodal perception and permission game theory, comprising the following steps:

[0007] Cameras, millimeter-wave radar, and pressure sensor arrays are installed in the parking area where the ground lock is located;

[0008] Before a vehicle enters the parking space, the ground lock is in a pre-judgment state. When the vehicle approaches the parking area, the millimeter-wave radar is triggered to scan the vehicle's intended trajectory to determine whether the vehicle intends to enter the parking space. If so, the camera is triggered to capture the vehicle's license plate and obtain the license plate recognition features.

[0009] After obtaining the license plate recognition features, the width and height of the vehicle are scanned by the millimeter-wave radar, and the tire contact area is measured by the pressure sensor array to obtain the vehicle attribute features;

[0010] The data entry intent trajectory, license plate recognition features, and vehicle attribute features are combined to form a fused feature.

[0011] Based on the fusion features, calculate the first permission score;

[0012] Vehicles are categorized into long-term rental users, temporary visitors, and emergency vehicles. Second permission scores are calculated based on identity matching and real-time permission validity.

[0013] By combining the first permission score and the second permission score, a final permission score is calculated, and a decision is made on whether to unlock the device based on the final permission score.

[0014] In a further embodiment of the first aspect, the millimeter-wave radar scans the vehicle's intended trajectory for entering the parking space and constructs an intended trajectory feature vector; the intended trajectory for entering the parking space is the movement trajectory of the vehicle 3 to 8 meters away from the parking lock.

[0015] Constructing the data entry intent trajectory feature vector specifically includes:

[0016] The sequence of coordinates of the vehicle's movement trajectory at a distance of 3 to 8 meters from the ground lock is recorded as follows: ;

[0017] The trajectory directional is calculated based on the cosine of the angle between the vehicle's direction of movement and the centerline of the parking space. : ;

[0018] In the formula, The direction vector of the centerline of the parking space; Let be the displacement vector of the vehicle's motion;

[0019] Determine whether a vehicle intends to slow down when entering a parking space based on the average rate of change of speed at three consecutive sampling points, using the deceleration trend. express: ;

[0020] In the formula, This represents the instantaneous speed of the vehicle corresponding to the k-th sampling point; Indicates the first The instantaneous speed of the vehicle corresponding to each sampling point; This represents the timestamp of the k-th sampling point; Indicates the first Timestamp of each sampling point;

[0021] Calculate the rate of change of the distance between the vehicle and the ground lock over time, and normalize it to... With distance convergence rate express: ;

[0022] In the formula, This is the initial distance, which is 8 meters from the vehicle to the ground lock; Current distance;

[0023] Constructing the feature vector of the intention trajectory for data entry : ;

[0024] In the formula, To counter the deceleration trend The function that performs nonnegation, when hour, ,when hour, ; This is the preset maximum deceleration rate threshold.

[0025] In a further embodiment of the first aspect, the camera captures the vehicle license plate, and OCR is used to recognize the characters to obtain a license plate recognition feature vector. ,in This represents the nth character.

[0026] In a further embodiment of the first aspect, a license plate recognition feature vector is obtained. Then, the width W and height H of the vehicle are scanned by the millimeter-wave radar, and the tire contact area is measured by the pressure sensor array. Construct normalized vehicle attribute feature vectors : ;

[0027] In the formula, , , These are the preset maximum width, maximum height, and maximum tire contact area for family car models.

[0028] In a further embodiment of the first aspect, the feature vector of the intention to be stored in the database is aggregated. License plate recognition feature vector Vehicle attribute feature vector , forming a fused feature vector .

[0029] In a further embodiment of the first aspect, based on the fused feature vector Calculate the first-level permission score : ;

[0030] In the formula, , , These are the dynamically adjusted weighting coefficients; The license plate matching score, i.e., the license plate recognition feature vector. The ratio of the number of successfully matched characters in the license plate to the total number of characters in the license plate; A score is assigned based on the intent to enter the database; Scoring based on vehicle attributes;

[0031] Based on the inbound intention trajectory feature vector The mean of the three elements is used to obtain the input intent matching score. : Intention matching score The higher the value, the clearer the vehicle's intention to enter the parking space;

[0032] Vehicle attribute matching score The calculation formula is as follows: ;

[0033] In the formula, Represents the normalized vehicle attribute feature vector The first in One element; This represents the normalized value of the authorized vehicle attributes.

[0034] In a further embodiment of the first aspect, vehicles are classified into long-term rental users, temporary visitors, and emergency vehicles. A second permission score is calculated based on identity matching degree and real-time permission validity, and a normalized result is output.

[0035] Among them, if the license plate recognition feature vector and normalized vehicle attribute feature vector If the information matches the reserved information for the available parking space, the current vehicle is identified as a long-term rental user, and the second permission score is directly assigned. ;

[0036] If no long-term rental user is matched, the system will further determine whether the user is a temporary visitor. In this case: If the license plate recognition feature vector of the current vehicle is within the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1; otherwise, the authorization validity is assigned a value of 0. If the current time is within the authorized time period, the time matching degree is assigned a value of 1; otherwise, it is reduced proportionally if the time exceeds the limit.

[0037] The system uses a camera to identify emergency vehicle license plates and / or emergency lights. If the current vehicle is identified as an emergency vehicle, a second-level permission score is directly assigned. .

[0038] In a further embodiment of the first aspect, the first permission score is integrated. Second-level permission score The final permission score is calculated. ;in For weighting;

[0039] Determine whether to unlock based on the final permission score S: ;

[0040] In the formula, T is the preset threshold, with the highest preset threshold for long-term rental parking spaces, the medium preset threshold for community public parking spaces, and the lowest preset threshold for temporary parking lots.

[0041] A second aspect of the present invention discloses an intelligent parking lock control system based on multimodal perception and permission game theory. The system includes: at least one camera, several millimeter-wave radars, several pressure sensors arranged in an array, and an execution module; the camera, millimeter-wave radars, and pressure sensors are all arranged in the parking area where the parking lock is located.

[0042] The execution module establishes communication with the camera, millimeter-wave radar, and pressure sensor;

[0043] The execution module can automatically execute the intelligent ground lock control method based on multimodal perception and permission game as described in the first aspect.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This technical solution employs multimodal perception fusion (camera recognition of license plates, millimeter-wave radar detection of vehicle attributes, and pressure sensor collection of tire features). It calculates the first permission score by fusing feature vectors and verifies legality from multiple dimensions such as vehicle identity, size, and intention to enter the parking space, avoiding misjudgment based on a single feature.

[0046] Based on multimodal perception fusion, vehicles are classified into long-term rental users, temporary visitors, and emergency vehicles. Dynamic permission game is introduced to take into account scenarios such as daily use, temporary access, and emergency situations, resulting in stronger dynamic management capabilities.

[0047] In summary, this invention, through "multi-dimensional perception and categorized permission game," takes into account the convenience and scenario adaptability of various users, and solves the problems of "low anti-scratching accuracy and cumbersome authorization management" of traditional ground locks. Attached Figure Description

[0048] Figure 1 This is a flowchart of the intelligent ground lock control method in the embodiment. Detailed Implementation

[0049] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention. Example 1:

[0050] This embodiment discloses an intelligent parking lock control method based on multimodal perception and permission game theory. A prerequisite is the deployment of cameras, millimeter-wave radar, and pressure sensor arrays in the parking area where the parking lock is located. Before the vehicle enters the parking space, the parking lock is in a pre-emptive lifting state. See [link / details]. Figure 1 The specific execution steps are as follows:

[0051] S1. When a vehicle approaches the parking area, the millimeter-wave radar is triggered to scan the vehicle's intended trajectory for parking and determine whether the vehicle intends to park. If so, the camera is triggered to capture the vehicle's license plate and obtain the license plate recognition features.

[0052] S2. After obtaining the license plate recognition features, the width and height of the vehicle are scanned by millimeter-wave radar, and the tire contact area is measured by a pressure sensor array to obtain the vehicle attribute features.

[0053] S3. Summarize the intent trajectory of the data entering the database, license plate recognition features, and vehicle attribute features to form a fused feature.

[0054] S4. Calculate the first permission score based on the fusion features.

[0055] S5. Classify vehicles into long-term rental users, temporary visitors, and emergency vehicles, and calculate the second permission score based on identity matching degree and real-time permission validity.

[0056] S6. Combine the first permission score and the second permission score to calculate the final permission score, and determine whether to unlock based on the final permission score. Example 2:

[0057] Based on the solution disclosed in Embodiment 1 above, the parking intention trajectory can be the movement trajectory of the vehicle 3 to 8 meters away from the parking lock. A feature vector of the parking intention trajectory is constructed using this trajectory. The specific implementation steps are as follows:

[0058] The sequence of coordinates of the vehicle's movement trajectory at a distance of 3 to 8 meters from the ground lock is recorded as follows: ;

[0059] The trajectory directional is calculated based on the cosine of the angle between the vehicle's direction of movement and the centerline of the parking space. See the following formula (1): ;

[0060] In the formula, The direction vector of the centerline of the parking space; Let be the displacement vector of the vehicle's motion;

[0061] Determine whether a vehicle intends to slow down when entering a parking space based on the average rate of change of speed at three consecutive sampling points, using the deceleration trend. This is expressed as shown in equation (2): ;

[0062] In the formula, This represents the instantaneous speed of the vehicle corresponding to the k-th sampling point; Indicates the first The instantaneous speed of the vehicle corresponding to each sampling point; This represents the timestamp of the k-th sampling point; Indicates the first Timestamp of each sampling point;

[0063] Calculate the rate of change of the distance between the vehicle and the ground lock over time, and normalize it to... With distance convergence rate This is expressed in equation (3): ;

[0064] In the formula, This is the initial distance, which is 8 meters from the vehicle to the ground lock; Current distance;

[0065] Constructing the feature vector of the intention trajectory for data entry See equation (4). ;

[0066] In the formula, To counter the deceleration trend The function that performs nonnegation, when hour, ,when hour, ; This is the preset maximum deceleration rate threshold. Example 3:

[0067] Based on the solution disclosed in Embodiment 1 above, this embodiment discloses in detail a feasible solution for constructing vehicle attribute features.

[0068] The camera captures vehicle license plates, and OCR is used to recognize the characters, resulting in a license plate recognition feature vector. ,in This represents the nth character. The resulting license plate recognition feature vector is obtained. Then, the width W and height H of the vehicle are scanned by millimeter-wave radar, and the tire contact area is measured by a pressure sensor array. Construct normalized vehicle attribute feature vectors See equation (5): ;

[0069] In the formula, , , These are the preset maximum width, maximum height, and maximum tire contact area for family car models. Example 4:

[0070] Based on the solutions disclosed in the aforementioned embodiments 1 to 3, this embodiment further provides a feasible method for calculating the first permission score, the second permission score, and the final permission score.

[0071] Summarize the feature vector of the intention to enter the database License plate recognition feature vector Vehicle attribute feature vector , forming a fused feature vector .

[0072] Based on fusion feature vectors Calculate the first authority score according to formula (6). : ;

[0073] In the formula, , , These are the dynamically adjusted weighting coefficients; The license plate matching score, i.e., the license plate recognition feature vector. The ratio of the number of successfully matched characters in the license plate to the total number of characters in the license plate; A score is assigned based on the intent to enter the database; Scoring based on vehicle attributes;

[0074] According to the following formula (7), based on the feature vector of the intention to enter the database, The mean of the three elements is used to obtain the input intent matching score. : ;

[0075] Inbound Intent Matching Score The higher the value, the clearer the vehicle's intention to enter the parking space;

[0076] Vehicle attribute matching score The calculation is as follows (8): ;

[0077] In the formula, Represents the normalized vehicle attribute feature vector The first in One element; This represents the normalized value of the authorized vehicle attributes.

[0078] Vehicles are categorized into long-term rental users, temporary visitors, and emergency vehicles. Second permission scores are calculated based on identity matching degree and real-time permission validity, and normalized results are output.

[0079] Among them, if the license plate recognition feature vector and normalized vehicle attribute feature vector If the information matches the reserved information for the available parking space, the current vehicle is identified as a long-term rental user, and the second permission score is directly assigned. ;

[0080] If no long-term rental user is matched, the system will further determine whether the user is a temporary visitor. .

[0081] If the license plate recognition feature vector of the current vehicle is within the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1; otherwise, the authorization validity is assigned a value of 0. If the current time is within the authorized time period, the time matching degree is assigned a value of 1; otherwise, it is reduced proportionally if the time exceeds the limit.

[0082] The system uses a camera to identify emergency vehicle license plates and / or emergency lights. If the current vehicle is identified as an emergency vehicle, a second-level permission score is directly assigned. .

[0083] Merge first-level authority score Second-level permission score The final permission score is calculated. ;in For weighting;

[0084] The decision to unlock a parking space is based on the final access score S. If the final access score S exceeds a preset threshold T, unlocking is performed; otherwise, unlocking is refused. The preset threshold T can be adjusted; for example, the preset threshold is highest for long-term rental parking spaces, medium for community public parking spaces, and lowest for temporary parking lots. Example 5:

[0085] This embodiment discloses an intelligent parking lock control system based on multimodal perception and permission game theory. The system includes at least one camera, several millimeter-wave radars, several arrayed pressure sensors, and an execution module. The camera, millimeter-wave radar, and pressure sensors are all located within the parking area where the parking lock is located. The execution module communicates with the camera, millimeter-wave radar, and pressure sensors. The execution module can automatically execute the intelligent parking lock control methods based on multimodal perception and permission game theory described in embodiments 1 to 4 above. Specifically, the execution module consists of a trajectory prediction unit, a license plate recognition unit, a vehicle attribute recognition unit, a first calculation unit, a second calculation unit, and an unlocking execution unit. Before a vehicle enters the parking space, the parking lock is in a raised prediction state. When the vehicle approaches the parking area, the trajectory prediction unit triggers the millimeter-wave radar to scan the vehicle's intended parking trajectory, determining whether the vehicle intends to enter. If so, the system feeds back to the license plate recognition unit, triggering the camera to capture the vehicle's license plate and obtain license plate recognition features. After obtaining the license plate recognition features, the vehicle attribute recognition unit triggers the millimeter-wave radar to scan the vehicle's width and height, and the pressure sensor array measures the tire contact area to obtain vehicle attribute features. The first calculation unit aggregates the input intent trajectory, license plate recognition features, and vehicle attribute features to form a fused feature, and calculates a first permission score based on the fused feature. The second calculation unit classifies vehicles into long-term rental users, temporary visitors, and emergency vehicles, and calculates a second permission score based on identity matching degree and real-time permission validity. The unlocking execution unit merges the first and second permission scores to calculate the final permission score, and determines whether to execute unlocking based on the final permission score.

[0086] The logical ideas behind the methods disclosed in the above embodiments can be implemented, in whole or in part, through software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs.

[0087] When computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions according to the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives (SSDs).

[0088] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0089] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A smart lock control method based on multimodal perception and permission game theory, characterized in that, Includes the following steps: Cameras, millimeter-wave radar, and pressure sensor arrays are installed in the parking area where the ground lock is located; Before a vehicle enters the parking space, the ground lock is in a pre-judgment state. When the vehicle approaches the parking area, the millimeter-wave radar is triggered to scan the vehicle's intention trajectory to enter the parking space, construct an intention trajectory feature vector, and determine whether the current vehicle has the intention to enter the parking space. If so, the camera is triggered to collect the vehicle's license plate and obtain the license plate recognition feature. The intended entry trajectory is the movement trajectory of the vehicle 3 to 8 meters away from the ground lock; After obtaining the license plate recognition features, the width and height of the vehicle are scanned by the millimeter-wave radar, and the tire contact area is measured by the pressure sensor array to obtain the vehicle attribute features; The data entry intent trajectory, license plate recognition features, and vehicle attribute features are combined to form a fused feature. Based on the fusion features, calculate the first permission score. : In the formula, , , These are the dynamically adjusted weighting coefficients; The license plate matching score, i.e., the license plate recognition feature vector. The ratio of the number of successfully matched characters in the license plate to the total number of characters in the license plate; A score is assigned based on the intent to enter the database; Scoring based on vehicle attributes; Vehicles are categorized into long-term rental users, temporary visitors, and emergency vehicles. A second permission score is calculated based on identity matching and real-time permission validity. If the license plate recognition feature vector and the normalized vehicle attribute feature vector match the reserved information for the opened parking space, the current vehicle is identified as a long-term rental user, and a second permission score is directly assigned. ; If no long-term rental user is matched, the system will further determine whether the user is a temporary visitor. In this case: If the license plate recognition feature vector of the current vehicle is within the whitelist preset by the long-term rental user, the authorization validity is assigned a value of 1; otherwise, the authorization validity is assigned a value of 0. If the current time is within the authorized time period, the time matching degree is assigned a value of 1; otherwise, it is reduced proportionally if the time exceeds the limit. The system uses a camera to identify emergency vehicle license plates and / or emergency lights. If the current vehicle is identified as an emergency vehicle, a second-level permission score is directly assigned. ; Integrate the first permission score Second-level permission score The final permission score is calculated. ;in For weighting; Determine whether to unlock based on the final permission score S: In the formula, T is the preset threshold, with the highest preset threshold for long-term rental parking spaces, the medium preset threshold for community public parking spaces, and the lowest preset threshold for temporary parking lots.

2. The intelligent ground lock control method based on multimodal perception and permission game theory according to claim 1, characterized in that, Constructing the data entry intent trajectory feature vector specifically includes: The sequence of coordinates of the vehicle's movement trajectory at a distance of 3 to 8 meters from the ground lock is recorded as follows: ; The trajectory directional is calculated based on the cosine of the angle between the vehicle's direction of movement and the centerline of the parking space. : In the formula, The direction vector of the centerline of the parking space; Let be the displacement vector of the vehicle's motion; Determine whether a vehicle intends to slow down when entering a parking space based on the average rate of change of speed at three consecutive sampling points, using the deceleration trend. express: In the formula, This represents the instantaneous speed of the vehicle corresponding to the k-th sampling point; This represents the instantaneous speed of the vehicle corresponding to the (k+1)th sampling point; This represents the timestamp of the k-th sampling point; This represents the timestamp of the (k+1)th sampling point; Calculate the rate of change of the distance between the vehicle and the ground lock over time, and normalize it to... With distance convergence rate express: In the formula, This is the initial distance, which is 8 meters from the vehicle to the ground lock; Current distance; Constructing the feature vector of the intention trajectory for data entry : In the formula, To counter the slowdown trend The function that performs nonnegation, when hour, ,when hour, ; This is the preset maximum deceleration rate threshold.

3. The intelligent ground lock control method based on multimodal perception and permission game theory according to claim 2, characterized in that, The camera captures vehicle license plates, and OCR is used to recognize the characters, resulting in a license plate recognition feature vector. ,in This represents the nth character.

4. The intelligent ground lock control method based on multimodal perception and permission game theory according to claim 3, characterized in that, Obtain the license plate recognition feature vector Then, the width W and height H of the vehicle are scanned by the millimeter-wave radar, and the tire contact area is measured by the pressure sensor array. Construct normalized vehicle attribute feature vectors : In the formula, , , These are the preset maximum width, maximum height, and maximum tire contact area for family car models.

5. The intelligent ground lock control method based on multimodal perception and permission game theory according to claim 4, characterized in that, Summarize the feature vector of the intention to enter the database License plate recognition feature vector Vehicle attribute feature vector , forming a fused feature vector .

6. The intelligent ground lock control method based on multimodal perception and permission game theory according to claim 5, characterized in that, Based on the inbound intention trajectory feature vector The mean of the three elements is used to obtain the input intent matching score. : Inbound Intent Matching Score The higher the value, the clearer the vehicle's intention to enter the parking space; Vehicle attribute matching score The calculation formula is as follows: In the formula, Represents the normalized vehicle attribute feature vector The i-th element in; This represents the normalized value of the authorized vehicle attributes.

7. An intelligent ground lock control system, characterized in that, include: The system includes at least one camera, several millimeter-wave radars, several pressure sensors arranged in an array, and an execution module; the camera, millimeter-wave radars, and pressure sensors are all located within the parking area where the parking lock is located. The execution module establishes communication with the camera, millimeter-wave radar, and pressure sensor; The execution module can automatically execute the intelligent ground lock control method based on multimodal perception and permission game as described in any one of claims 1 to 6.

Images