Pickup method of a vehicle pass and vehicle

Abstract

This application provides a method for picking up a vehicle access card and a vehicle. The method is applied in the field of vehicle control technology. The method includes: in response to the vehicle being in a access card picking scenario, obtaining the landing position of the picking device, generating a picking trajectory of the picking device based on the landing position and the current position of the vehicle, generating a control command for the picking device based on the picking trajectory, and controlling the picking device to execute the control command to pick up the access card. By setting up a picking device on the vehicle to achieve intelligent and convenient card picking, the accuracy of access card picking can be effectively improved, and the safety and passage efficiency of the vehicle can be improved.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and more specifically, to a method for picking up vehicle access cards and a vehicle. Background Technology

[0002] In scenarios such as manual toll lanes, semi-automatic card dispensing lanes, and toll stations not covered by ETC on highways, drivers typically need to stop and manually pick up and exchange highway toll cards. Related technologies address this by modifying toll station equipment, such as adding auxiliary card-collecting robotic arms and optimizing parking guidance devices, to increase the convenience of stopping and collecting cards.

[0003] However, the relevant technologies have problems such as high transformation costs, long construction periods, and inability to adapt to all old stations and temporary lanes, making it difficult to popularize on a large scale. They have low versatility and rely too much on the transformation of roadside facilities. When the parking position deviates significantly, the driver still needs to manually take the card, which fails to fundamentally eliminate the safety hazards of taking the card and reduces the safety and convenience of vehicle passage. Summary of the Invention

[0004] This application provides a method for picking up vehicle toll cards and a vehicle. The method can determine the landing position of the picking device based on the landing point of the toll card, adaptively plan the picking trajectory based on the vehicle position, and precisely control the execution, effectively improving the accuracy of toll card picking and enhancing vehicle safety and traffic efficiency.

[0005] In a first aspect, a method for picking up a vehicle toll card is provided, wherein the vehicle is equipped with a toll card picking device, the method comprising: in response to the vehicle being in a toll card picking scenario, obtaining the landing position of the picking device; generating a picking trajectory of the picking device based on the landing position and the current position of the vehicle; generating a control command for the picking device based on the picking trajectory, and controlling the picking device to execute the control command to pick up the toll card.

[0006] By utilizing the above technical solution and the pickup device installed on the vehicle, when the vehicle is in a toll card pickup scenario, the landing position of the pickup device can be obtained first. The pickup trajectory of the pickup device can be generated based on the landing position of the toll card and combined with the current position of the vehicle. Finally, the pickup device is driven by the control command to complete the toll card pickup, realizing fully automatic toll card pickup without the need for manual pickup by the driver, effectively improving the efficiency of toll station passage and driving safety.

[0007] In conjunction with the first aspect, in some possible implementations, obtaining the landing point location of the picking device includes: identifying the location of the access card and determining the landing point location based on the location of the access card.

[0008] By using the above technical solution, the actual spatial location of the access card can be identified in real time and the landing point of the pickup device can be accurately calibrated based on this. This can significantly improve the accuracy of the pickup device's landing point positioning, environmental adaptability, and the stability and success rate of the device's automatic card retrieval.

[0009] In conjunction with the first aspect, in some possible implementations, identifying the position of the access card and determining the landing point position based on the position of the access card includes: identifying the target card issuance method of the access card, wherein the target card issuance method is a manual card issuance method or a card issuing machine card issuance method; in response to the card issuance method of the access card being a manual card issuance method, identifying the hand gesture of the card issuer, determining the target card delivery type based on the hand gesture, and predicting the first position of the access card based on the target card delivery type, so as to determine the first landing point coordinates of the pickup device based on the first position of the access card, thereby determining the landing point position; if the card issuance method of the access card is a card issuing machine card issuance method, determining the coordinate information of the card delivery point of the card issuing machine, and using the coordinate information to determine the second position of the access card, so as to determine the second landing point coordinates of the pickup device based on the second position of the access card, thereby determining the landing point position.

[0010] The above technical solution can identify the card issuance method and, for both manual and machine issuance, determine the card delivery type by recognizing hand gestures to predict the card's position. Based on the card's position, the first landing point coordinates of the pickup device are determined, and the second position of the card is determined by directly acquiring the coordinates of the card dispenser's landing point. The second landing point coordinates of the pickup device are then determined based on the card's second position, achieving differentiated and accurate detection of the card's landing point. By adapting to different card issuance methods and detecting landing points, the landing point coordinates of the pickup device are determined, effectively improving the accuracy and robustness of landing point prediction and providing a reliable basis for subsequent trajectory planning and pickup device control.

[0011] In conjunction with the first aspect, in some possible implementations, identifying the hand gesture of the card issuer and determining the target card delivery type based on the hand gesture includes: acquiring multiple key hand points of the card issuer, and determining the wrist coordinates, fingertip coordinates, and palm coordinates of the card issuer based on the multiple key hand points; determining the hand gesture of the card issuer using the wrist coordinates, fingertip coordinates, and palm coordinates; and determining the target card delivery type as either a throwing card delivery type or a flat pushing card delivery type based on the hand gesture.

[0012] The above technical solution can determine the hand posture by acquiring and analyzing multiple key points of the card issuer's wrist, fingertips, and palm, and then identify two types of target card delivery: throwing and pushing. This allows for precise identification of the card delivery type through detailed analysis of the key hand points, providing an adaptability basis for predicting the landing point and trajectory planning of different card delivery actions, and effectively improving the success rate of card retrieval.

[0013] In conjunction with the first aspect, in some possible implementations, determining whether the target card delivery type is a throw-type or a push-type card delivery based on the hand posture includes: acquiring the vertical coordinates of key points on the wrist of the card issuer in multiple frames of images to determine the vertical velocity of the wrist; determining the distance change curve between the fingertip and the wrist based on the fingertip coordinates and the palm coordinates; in response to the wrist vertical velocity being greater than a first preset velocity threshold and the distance change curve satisfying a preset throw trend, determining that the target card delivery type is a throw-type card delivery; if the wrist vertical velocity is less than a second preset velocity threshold and the distance change curve satisfies a preset push trend, then determining that the target card delivery type is a push-type card delivery; wherein, the first preset velocity threshold is greater than the second preset velocity threshold.

[0014] The above technical solution can determine the vertical velocity of the wrist by obtaining the vertical coordinates of the key points of the wrist of the card issuer in multiple frames of images. Combined with the distance change curve between the fingertip and the wrist, and based on certain velocity thresholds and trend rules, the card delivery type can be determined as either throwing or pushing. This allows for accurate differentiation of card delivery types through quantified velocity and distance change features, improving the objectivity and anti-interference of the determination, and providing a reliable action classification basis for subsequent trajectory prediction.

[0015] In conjunction with the first aspect, in some possible implementations, generating the pickup trajectory of the pickup device based on the landing point location and the current location of the vehicle includes: responding to the card issuance method of the access card being a manual issuance method, identifying the actual hand gesture of the card issuer, correcting the first landing point coordinates based on the actual hand gesture, and updating the landing point location using the corrected first landing point coordinates to determine the third landing point coordinates of the pickup device; determining the motion path corresponding to the pickup device based on the third landing point coordinates and the current location of the vehicle to generate the pickup trajectory.

[0016] The above technical solution can correct the previously predicted first landing point coordinates by recognizing the actual hand posture of the card issuer in the manual card issuance method, thereby updating the landing point position and obtaining the final third landing point coordinates. Then, combined with the current position of the vehicle, the movement path is determined, thereby generating the picking trajectory of the picking device. By dynamically correcting the landing point coordinates based on the actual hand posture, the accuracy and adaptability of trajectory planning are effectively improved, ensuring the success rate of toll card picking in manual card issuance scenarios.

[0017] In conjunction with the first aspect, in some possible implementations, generating the pickup trajectory of the pickup device based on the landing point position and the current position of the vehicle includes: responding to the card issuing method of the toll card being a card issuing machine issuing method, obtaining the card issuing speed and card issuing direction of the card issuing machine, correcting the second landing point coordinates based on the card issuing speed and card issuing direction, and updating the landing point position using the corrected second landing point coordinates to determine the fourth landing point coordinates of the pickup device; determining the movement path of the pickup device based on the fourth landing point coordinates and the current position of the vehicle to generate the pickup trajectory.

[0018] The above technical solution enables the correction of the previously predicted second landing point coordinates by acquiring the card dispensing speed and direction of the card dispenser in card dispensing scenarios. This updates the landing point position and yields the final fourth landing point coordinates. Combined with the vehicle's current position, the movement path is determined, generating the pickup trajectory of the pickup device. By dynamically correcting the landing point coordinates in conjunction with the card dispensing parameters of the card dispenser, the accuracy and adaptability of trajectory planning in card dispensing scenarios are improved, ensuring the success rate of toll card pickup.

[0019] In conjunction with the first aspect, in some possible implementations, after picking up the access card, the method further includes: in response to the picking device picking up the access card, controlling the picking device to deliver the access card to a target accessible area of ​​the user in the vehicle cabin; or, based on the placement command set by the user, controlling the picking device to place the picked-up access card in a preset storage location.

[0020] The above technical solution enables the card to be delivered to a user-accessible area inside the vehicle or placed in a storage location according to user instructions after the card is successfully picked up by the picking device. This allows for flexible adaptation to different usage habits and improves ease of use and driving safety.

[0021] In conjunction with the first aspect, in some possible implementations, before obtaining the landing position of the picking device, the method further includes: in response to detecting that the actual distance between the front position of the vehicle and the card issuer or the card issuer is greater than a preset safe distance, controlling the vehicle to automatically enter the toll card picking scenario; or, in response to a picking scenario command input by the user, controlling the vehicle to enter the toll card picking scenario.

[0022] The above technical solution enables automatic triggering or response to user commands before the detection point is reached, based on the distance between the vehicle's front position and the card issuing end (e.g., the card issuer or card issuer) exceeding a threshold. This allows the vehicle to enter the card picking scenario, achieving both automatic judgment and manual triggering of the picking scenario, adapting to different scenarios and improving the vehicle's intelligence and operational flexibility.

[0023] In conjunction with the first aspect, in some possible implementations, the method of this application embodiment further includes: in response to the vehicle being in a toll card return scenario, determining the target return area of ​​the toll card; generating the return trajectory of the pickup device according to the target return area, and controlling the pickup device to extend along the return trajectory to deliver the toll card to the target return area.

[0024] The above technical solution enables the determination of the target return area when a vehicle is in a toll card return scenario. Based on this, a return trajectory is generated and the picking device is extended to deliver the card to the designated return area. This achieves full automation of the card retrieval and return process, fully covering all toll station passage scenarios and further improving vehicle passage efficiency and ease of use.

[0025] Secondly, a vehicle toll card picking device is provided, wherein a vehicle is equipped with a toll card picking device, the vehicle toll card picking device comprising: an acquisition module, configured to acquire the landing position of the picking device in response to the vehicle being in a toll card picking scenario; a generation module, configured to generate a picking trajectory of the picking device based on the landing position and the current position of the vehicle; and a picking module, configured to generate a control command for the picking device based on the picking trajectory, and control the picking device to execute the control command to pick up the toll card.

[0026] In conjunction with the second aspect, in some possible implementations, the acquisition module includes: a first identification unit, used to identify the location of the access card and determine the landing point location based on the location of the access card.

[0027] In conjunction with the second aspect, in some possible implementations, the first identification unit includes: a first identification subunit, configured to identify the target card issuance method of the access card, wherein the target card issuance method is a manual card issuance method or a card issuing machine card issuance method; a second identification subunit, configured to, in response to the card issuance method of the access card being a manual card issuance method, identify the hand gesture of the card issuer and determine the target card delivery type based on the hand gesture, predict the first position of the access card based on the target card delivery type, determine the first landing point coordinates of the pickup device based on the first position of the access card, and determine the landing point position; and a determining subunit, configured to, if the card issuance method of the access card is a card issuing machine card issuance method, determine the coordinate information of the card drop point of the card issuing machine, and use the coordinate information to determine the second position of the access card, determine the second landing point coordinates of the pickup device based on the second position of the access card, and determine the landing point position.

[0028] In conjunction with the second aspect, in some possible implementations, the second identification subunit includes: an acquisition component for acquiring multiple hand key points of the card issuer and determining the wrist coordinates, fingertip coordinates, and palm coordinates of the card issuer based on the multiple hand key points; a first determination component for determining the hand posture of the card issuer using the wrist coordinates, fingertip coordinates, and palm coordinates; and a second determination component for determining whether the target card delivery type is a throwing card type or a flat pushing card type based on the hand posture.

[0029] In conjunction with the second aspect, in some possible implementations, the second determining component includes: a coordinate acquisition subcomponent, used to acquire the vertical coordinates of key points on the wrist of the card issuer in multiple frames of images to determine the vertical speed of the wrist; a curve determination subcomponent, used to determine the distance change curve between the fingertip and the wrist based on the fingertip coordinates and the palm coordinates; a first determination subcomponent, used to determine that the target card delivery type is a throwing card delivery type in response to the wrist vertical speed being greater than a first preset speed threshold and the distance change curve satisfying a preset throwing trend; and a second determination subcomponent, used to determine that the target card delivery type is a pushing card delivery type if the wrist vertical speed is less than a second preset speed threshold and the distance change curve satisfies a preset pushing trend; wherein the first preset speed threshold is greater than the second preset speed threshold.

[0030] In conjunction with the second aspect, in some possible implementations, the generation module includes: a second identification unit, configured to, in response to the card issuance method being manual issuance, identify the actual hand gesture of the card issuer, correct the first landing point coordinates based on the actual hand gesture, and update the landing point position using the corrected first landing point coordinates to determine the third landing point coordinates of the pickup device; and a first generation unit, configured to, based on the third landing point coordinates and the current position of the vehicle, determine the motion path corresponding to the pickup device to generate the pickup trajectory.

[0031] In conjunction with the second aspect, in some possible implementations, the generation module includes: an acquisition unit, configured to, in response to the card issuance method of the pass card being a card issuing machine issuance method, acquire the card issuing speed and card issuing direction of the card issuing machine, correct the second landing point coordinates based on the card issuing speed and card issuing direction, and update the landing point position using the corrected second landing point coordinates to determine the fourth landing point coordinates of the pickup device; and a second generation unit, configured to, based on the fourth landing point coordinates and the current position of the vehicle, determine the movement path of the pickup device to generate the pickup trajectory.

[0032] In conjunction with the second aspect, in some possible implementations, the apparatus of this application embodiment further includes: a delivery module, configured to, after picking up the access card, in response to the picking device picking up the access card, control the picking device to deliver the access card to a target accessible area of ​​the user in the vehicle cabin; or, configured to, after picking up the access card, control the picking device to place the picked-up access card in a preset storage location based on a placement instruction set by the user.

[0033] In conjunction with the second aspect, in some possible implementations, the apparatus of this application embodiment further includes: a control module, configured to, before acquiring the landing position of the picking device, control the vehicle to automatically enter the toll card picking scenario in response to detecting that the actual distance between the front position of the vehicle and the card issuer or the card issuer is greater than a preset safe distance; or, before acquiring the landing position of the picking device, control the vehicle to enter the toll card picking scenario in response to a picking scenario command input by the user.

[0034] In conjunction with the second aspect, in some possible implementations, the apparatus of this application embodiment further includes: a determining module, configured to determine a target return area for the toll card in response to the vehicle being in a toll card return scenario; and a return module, configured to generate a return trajectory for the picking device based on the target return area, and control the picking device to extend along the return trajectory to deliver the toll card to the target return area.

[0035] Thirdly, a vehicle is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to perform the method of the first aspect or any possible implementation thereof.

[0036] Fourthly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0037] Fifthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof. Attached Figure Description

[0038] Figure 1 This is a design location and structural diagram of a vehicle pass card pickup device according to an embodiment of this application; Figure 2 This is a flowchart illustrating a method for picking up a vehicle pass card according to an embodiment of this application; Figure 3 This is a schematic diagram of a vehicle pass card pickup device according to an embodiment of this application; Figure 4 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0039] The technical solutions in this application will be clearly and thoroughly described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in the text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0040] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0041] In scenarios such as manual toll lanes, semi-automatic card issuing lanes, and toll stations not covered by ETC on highways, drivers usually need to stop and manually pick up and return highway toll cards when passing through. Due to limited driver visibility, inaccurate judgment of parking position, unclear lane guidance signs, and adverse environmental conditions such as rain, snow, fog, and night, many vehicles cannot stop at the ideal card-collecting position, and the distance between the front of the vehicle and the card issuing window is generally too far.

[0042] At this time, drivers often need to unbuckle their seat belts, open the car door, lean out or even get out of the car to complete the card taking and returning operation. This is not only cumbersome and inefficient, but also very likely to cause safety accidents such as door collisions, lane scraping, and rear-end collisions. There are also safety hazards such as people falling or being affected by traffic.

[0043] Among related technologies, modifying toll station equipment by adding auxiliary card-retrieving robotic arms and optimizing parking guidance devices can reduce the difficulty for drivers in retrieving cards. However, this approach suffers from high modification costs, long construction periods, and incompatibility with all older toll stations and temporary lanes, hindering large-scale adoption. Other vehicle-mounted card-retrieving devices have complex structures and fixed extension strokes, making them prone to issues such as card reception failure, card drop, and mechanical collisions, resulting in insufficient stability and versatility.

[0044] To address the aforementioned issues, this application embodiment can generate a pickup trajectory for the pickup device based on the landing point of the pickup device and the current position of the vehicle when the vehicle is in a toll card pickup scenario. This allows the pickup device to be controlled to accurately pick up the toll card based on the pickup trajectory. In other words, intelligent and convenient card retrieval is achieved through a pickup device installed on the vehicle, effectively improving vehicle safety and traffic efficiency.

[0045] As one possible way to achieve this, such as Figure 1 As shown in (a), this embodiment of the application designs a pickup device for picking up a toll card below the air conditioning vent on the left side of the driver's side of the vehicle. The specific structural composition of the pickup device is as follows: Figure 1 As shown in (b), specifically, the pickup device is a retractable magnetic card delivery device, which consists of a telescopic rod recovery base, a micro motor, a motor drive mechanism, a retractable robotic arm, an infrared / laser radar, a magnetic card slot + magnet, and a control unit.

[0046] Specifically, the retractable magnetic card delivery device is located below the left-side air conditioning vent on the driver's side. For aesthetic purposes, it can be designed to be hidden, without affecting the vehicle's exterior and interior. When needed, it extends through a hidden track beneath the left-side air conditioning vent on the driver's side, and immediately retracts back into its completely hidden state after the action is complete, conforming to automotive aesthetics and safety regulations. Of course, the retractable magnetic card delivery device can also be designed without being hidden. The micro motor in the device can be a micro linear motor. When energized, the micro linear motor outputs linear thrust, directly driving the retractable robotic arm to extend smoothly outward from below the left-side air conditioning vent on the driver's side along a straight trajectory. The extension length is 0.6–1.2 meters. Furthermore, the magnetic card slot at the end of the device is a flexible silicone card slot, and the magnetic card slot contains an N52-grade micro magnet, which can attract the metal anti-counterfeiting mark or magnetic strip on the surface of the card. The device uses infrared / LiDAR, along with vehicle-mounted millimeter-wave radar and a vehicle-mounted vision camera, to identify the location of the card issuer / card dispenser and the trajectory of the pass card in real time. When the vehicle stops and is more than a safe threshold away from the card dispenser, the device automatically pops out a flexible silicone card slot and absorbs the pass card that falls or is delivered in the air, and smoothly delivers it to a location inside the vehicle that the driver can reach or a pre-set storage location.

[0047] The mechanism control unit is electrically connected to the motor drive mechanism and can be connected to the vehicle ECU controller via a CAN bus. The physical switch or voice module of the vehicle's central control screen issues bus control commands and transmits them to the vehicle ECU in the form of bus signals. The vehicle ECU then forwards the commands to the mechanism control unit via the CAN bus. After receiving the commands, the mechanism control unit outputs drive control signals to the motor drive mechanism. Based on these signals, the motor drive mechanism drives its core component, the micro linear motor, to output power, thereby driving the telescopic robotic arm to complete the extension or retraction operation.

[0048] In summary, the embodiments of this application can place the retractable magnetic card delivery device below the air conditioning vent on the left side of the driver's side of the vehicle. The driver can automatically obtain the access card in the original position (fastening the seat belt and closing the door) through the vehicle's own system, thus ensuring driving safety.

[0049] It should be noted that the retractable magnetic card delivery device can also be installed in other locations such as inside the door below the window on the driver's side of the vehicle. The specific installation can be carried out by relevant technical personnel and is not specifically limited here. In this embodiment, the retractable magnetic card delivery device will be described in detail as being installed below the air conditioning vent on the left side of the driver's side of the vehicle.

[0050] Figure 2 This is a schematic flowchart illustrating a method for picking up a vehicle access card provided in an embodiment of this application.

[0051] For example, such as Figure 2 As shown, the method for collecting vehicle access cards includes the following steps: In step S201, in response to the vehicle being in a toll card picking scenario, the landing position of the picking device is obtained.

[0052] It is understood that the embodiments of this application can respond to a vehicle being in a toll card collection scenario. For example, when a vehicle enters a highway toll station, if the vehicle's onboard millimeter-wave radar sensor identifies a card issuing booth or machine structure ahead and the vehicle is stationary, it can automatically enter the toll card collection scenario based on the vehicle's location information. Alternatively, it can enter the toll card collection scenario manually triggered by a physical button. After the vehicle is in the toll card collection scenario, the onboard millimeter-wave radar sensor installed on the vehicle's front grille can detect in real time the location of the card issuer or machine, the card issuing action, and the toll card delivery trajectory. This allows the determination of the landing point of the extended pickup device. It should be noted that the landing point of the extended pickup device is also the landing point of the toll card; the pickup device can attract the toll card at this landing point. This enables real-time and accurate perception and positioning of the pickup landing point, effectively improving the accuracy and reliability of the pickup trajectory planning.

[0053] Optionally, in one embodiment of this application, obtaining the landing position of the picking device includes: identifying the position of the access card and determining the landing position based on the position of the access card.

[0054] In this embodiment of the application, the position of the access card is the final landing position of the access card, which is also the landing position of the picking device.

[0055] In some embodiments, this application embodiment can use an on-board millimeter-wave radar sensor installed on the front grille of the vehicle to detect in real time the location of the card issuer or card issuer, the card issuing action, and the trajectory of the pass card delivery, thereby determining the real-time location of the pass card and predicting the final landing point of the pass card. Then, based on the final landing point of the pass card, the landing point of the pickup device is determined. The pickup device can then attract the pass card at this landing point, thereby achieving real-time and accurate perception and positioning of the landing point, effectively improving the positioning accuracy of the pickup device and the reliability and stability of automatic card retrieval.

[0056] Optionally, in one embodiment of this application, before obtaining the landing position of the picking device, the method further includes: in response to detecting that the actual distance between the front position of the vehicle and the card issuer or card issuer is greater than a preset safe distance, controlling the vehicle to automatically enter the toll card picking scenario; or, in response to a picking scenario command input by the user, controlling the vehicle to enter the toll card picking scenario.

[0057] In actual implementation, the vehicle in this embodiment can use an onboard millimeter-wave radar sensor and an onboard vision sensor installed above the rearview mirror or windshield, such as a vision camera, to detect that the actual distance between the front of the vehicle and the position of the card issuer in the toll window, or the card dispensing slot of the automatic card dispenser, is greater than a certain safe distance, for example, greater than 1.8 meters. The onboard control unit automatically determines that the current scenario is not suitable for manual card retrieval and controls the vehicle to lock the doors and automatically enter the toll card retrieval scenario; or, in response to a manual trigger command input by the driver through the onboard touch screen, voice assistant, or physical button, the vehicle can be controlled to actively enter the toll card retrieval scenario. By combining automatic determination with manual triggering, the retrieval scenario can be flexibly and on-demand activated, greatly improving the intelligence level of the retrieval scenario triggering and the convenience of user operation.

[0058] Furthermore, in this embodiment of the application, the safe distance between the front of the vehicle and the card issuer or the card issuer exit can be set to 1.8 meters. Taking an SUV as an example, the distance between the driver's seat and the front of the vehicle is generally 1.5 meters. Using the Pythagorean theorem, this is equivalent to a distance of 0.99 meters between the driver and the window, which exceeds the comfortable distance for a normal person to reach out and take the card. In this case, the card can be taken by the access card picking device.

[0059] It should be noted that the preset safety distance can be modified and adjusted according to the vehicle type or actual size, such as sedan models, commercial vehicle models, etc. The specific settings can be made by relevant technical personnel, and no specific limitations are made here.

[0060] Alternatively, millimeter-wave radar sensors can be installed on the driver's side door or exterior rearview mirror of the vehicle to detect the safe distance between the driver and the card issuer or the card issuer exit. The specific settings can be configured by relevant technical personnel according to actual needs.

[0061] Optionally, in one embodiment of this application, identifying the location of the access card and determining the landing point location based on the location of the access card includes: identifying the target card issuance method of the access card, wherein the target card issuance method is a manual card issuance method or a card issuing machine card issuance method; in response to the card issuance method of the access card being a manual card issuance method, identifying the hand gesture of the card issuer, determining the target card delivery type based on the hand gesture, and predicting the first position of the access card based on the target card delivery type, so as to determine the first landing point coordinates of the picking device based on the first position of the access card, thereby determining the landing point location; if the card issuance method of the access card is a card issuing machine card issuance method, determining the coordinate information of the card delivery point of the card issuing machine, and using the coordinate information to determine the second position of the access card, so as to determine the second landing point coordinates of the picking device based on the second position of the access card, thereby determining the landing point location.

[0062] In this embodiment, the first position of the access card is the position predicted by manual card issuance; the second position of the access card is the position predicted by card issuing machine issuance; the first landing point coordinates are the predicted landing point position of the access card picking device when the card is issued manually; and the second landing point coordinates are the predicted landing point position of the access card picking device when the card is issued by card issuing machine.

[0063] As one possible implementation, this application embodiment can use an onboard vision sensor to acquire images and detect features of the toll window area in front to identify whether the toll card is issued manually or by a card issuing machine. If at least one feature of the card issuer's upper body, face, hand movements, or work clothing is detected, or scene features such as a manual toll booth window or manual operating table are detected, then the current toll card issuance method is determined to be manual. Then, the onboard vision sensor collects the three-dimensional pose data of the card issuer's hand in real time, and combines it with ergonomic hand shape parameters and card delivery trajectory to identify the target card delivery type, such as a flat push card delivery type with one hand, a two-hand lifting type, or a throwing type such as a side delivery. Based on the kinematic characteristics of the target card delivery type and the toll card release trajectory, the coordinates of the first landing point of the pickup device in space are predicted to determine the landing point position of the pickup device.

[0064] In addition, for the automatic card dispensing scenario, this application realizes automatic card picking in three stages: First, stable positioning of card dispensing ports from different manufacturers is achieved through template matching and edge gradient verification, and three types of mainstream card dispensing port templates are constructed, such as rectangles, round holes, and oblique slots; Second, the initial velocity of the card is obtained using virtual detection zones and optical flow methods; Finally, the landing point is predicted through robust parabolic fitting, and the results are synchronized to the vehicle's HMI (Human Machine Interface), adapting the computing power and communication architecture of the vehicle's domain control to achieve automatic card identification and trajectory prediction with low CPU usage and high anti-interference.

[0065] For example, when the vehicle-mounted vision sensor identifies at least one feature among the automatic card dispenser housing, card dispensing slot, card dispensing indicator light, and card pushing mechanism, or detects a standardized card dispensing equipment table without manual operation, it is determined that the current card dispensing method is automatic card dispensing. By scanning the structural features and mechanical state of the card dispensing area of ​​the card dispenser with the vision sensor, the release position of the card tray inside the card dispenser and the coordinates of the card drop point are accurately located. Based on this coordinate information and the vehicle's parking posture fine-tuning, the coordinates of the second drop point of the pickup device are determined, thus determining the drop point position of the pickup device.

[0066] Thus, by differentiating the landing point determination for different card issuance scenarios, accurate prediction and adaptive determination of landing point coordinates are achieved, effectively improving the success rate and reliability of toll card retrieval in complex toll scenarios.

[0067] Optionally, in one embodiment of this application, identifying the hand posture of the card issuer and determining the target card delivery type based on the hand posture includes: acquiring multiple key points of the card issuer's hand, and determining the wrist coordinates, fingertip coordinates, and palm coordinates of the card issuer based on the multiple key points of the hand; determining the hand posture of the card issuer using the wrist coordinates, fingertip coordinates, and palm coordinates; and determining the target card delivery type as either a throwing card delivery type or a flat pushing card delivery type based on the hand posture.

[0068] In some embodiments, this application achieves accurate card recognition and action determination through a three-stage technology: First, YOLOv5s + a lightweight hand detection head is used to quickly and coarsely locate the hand area; second, the coordinates of key points such as the wrist and fingertips are extracted through an improved Lite-HRNet to provide accurate pose information for trajectory calculation; finally, a state machine is constructed based on the velocity vectors of key points to distinguish between "flat card delivery" and "upward card throwing" actions, and the relevant data is adapted to the vehicle domain control architecture to achieve high real-time card issuance action determination with low computing power overhead.

[0069] For example, in this embodiment of the application, an in-vehicle vision sensor can be used to collect images of the card issuer's hand and obtain the coordinates of multiple key points of the hand, such as the wrist, fingertips, and palm. For example, the wrist key point coordinates are W (x1, y1), the index fingertip key point coordinates are T (x2, y2), and the palm key point coordinates are P (x3, y3). Based on the direction of the line connecting the wrist coordinates W and the fingertip coordinates T, the direction of the fingertip extension is determined to be horizontal forward or upward. Based on the relative position of the palm coordinates P and the wrist coordinates W, the palm orientation is determined to be facing the inside of the vehicle or facing diagonally upward. Combining the above directions and orientations, the current hand posture is finally determined to be either a card throwing posture or a flat card passing posture.

[0070] Specifically, when the palm faces diagonally upwards based on the line connecting the palm key point and the wrist key point, and the fingertip key point shows an upward movement trend, the current target card delivery type is determined to be a throwing card delivery type. When the palm faces horizontally forward based on the relative position of the palm key point and the wrist key point, and the fingertip key point maintains a steady-state characteristic of straight extension, the current target card delivery type is determined to be a pushing card delivery type. Through refined recognition of hand spatial posture, intelligent classification and accurate prediction of different card delivery actions are achieved, improving the accuracy of landing point prediction and the matching degree of pickup action, as well as improving the adaptive ability of trajectory planning and the pickup success rate.

[0071] Optionally, in one embodiment of this application, determining whether the target card delivery type is a throw-type or a push-type card delivery based on hand posture includes: acquiring the vertical coordinates of key points on the wrist of the card issuer in multiple frames of images to determine the vertical speed of the wrist; determining the distance change curve between the fingertip and the wrist based on the fingertip coordinates and the palm coordinates; in response to the wrist vertical speed being greater than a first preset speed threshold and the distance change curve satisfying a preset throw trend, determining that the target card delivery type is a throw-type card delivery; if the wrist vertical speed is less than a second preset speed threshold and the distance change curve satisfies a preset push trend, then determining that the target card delivery type is a push-type card delivery; wherein, the first preset speed threshold is greater than the second preset speed threshold.

[0072] In this embodiment, the preset throwing trend is that the distance change curve between the fingertip and the wrist shows an upward change characteristic of first increasing rapidly and then stabilizing; the preset pushing trend is that the distance change curve between the fingertip and the wrist maintains a small and stable fluctuation with a gentle change characteristic without obvious sudden changes.

[0073] In some embodiments, the present application embodiments can obtain the vertical coordinates of key points of the wrist of the card issuer in multiple consecutive frames of images, calculate the vertical movement speed of the wrist based on the difference in coordinates between adjacent frames and the time interval; at the same time, based on the relative positional relationship between the fingertip coordinates and the palm coordinates, determine the curve of the distance between the fingertip and the wrist changing with time. For example, when the vertical speed of the wrist exceeds a first preset speed threshold, such as 0.25 m / s, and the distance change curve between the fingertip and wrist shows a rapid upward trend (e.g., wrist Z-axis speed > 0.8 m / s, fingertip-wrist distance Δd > 12 cm entering an "upward state"), and Δd continues to increase while the distance suddenly drops by > 8 cm within 200 ms after the peak fingertip speed, then "release" is determined, corresponding to throwing card types such as two-handed lifting and side delivery, and the target card delivery type is determined to be a throwing card type. When the vertical speed of the wrist is less than a second preset speed threshold, such as 0.10 m / s, and the distance change curve remains stable with small changes, corresponding to a flat-push card type of single-handed flat delivery, the target card delivery type is determined to be a flat-push card type. The first preset speed threshold is greater than the second preset speed threshold. By combining the movement speed and distance change curves to distinguish different card delivery types, the accuracy and scene adaptability of card delivery type recognition are improved.

[0074] It should be noted that the first preset speed threshold and the second preset speed threshold are set by those skilled in the art, and are not specifically limited here.

[0075] In step S202, the pickup trajectory of the pickup device is generated based on the landing point location and the current position of the vehicle.

[0076] It is understood that, in this embodiment of the application, the landing point of the toll card pickup device can be determined based on the first landing point coordinates of the manual card issuance method or the second landing point coordinates of the card issuance machine method as determined in the above steps. Then, based on the vehicle's current position information, the lateral offset and longitudinal distance of the vehicle body relative to the card issuance end, the actual parking posture of the vehicle in the toll lane is determined. Based on the vehicle's structural parameters and current position, the driver's cabin area, the window opening status, and the effective working space of the toll card pickup device are further determined to avoid interference between the pickup trajectory and the vehicle's interior, windows, or driver area. For example, if the vehicle is laterally offset to the left by 15cm and longitudinally by 0.8m relative to the card issuance end, and the window opening width is 12cm, then it is determined that the driver is located in a fixed area on the left side of the cabin. Based on this, the trajectory of the toll card pickup device is planned to extend smoothly outward from the window opening and approach the landing point diagonally upward, so that the pickup trajectory avoids the steering wheel, center console, and driver's limb area. This improves the rationality of automatic card retrieval triggering and the safety of trajectory planning.

[0077] Optionally, in one embodiment of this application, generating the pickup trajectory of the pickup device based on the landing point location and the current location of the vehicle includes: responding to the card issuance method being manual card issuance, identifying the actual hand gesture of the card issuer, correcting the first landing point coordinates based on the actual hand gesture, and updating the landing point location using the corrected first landing point coordinates to determine the third landing point coordinates of the pickup device; and determining the motion path corresponding to the pickup device based on the third landing point coordinates and the current location of the vehicle to generate the pickup trajectory.

[0078] In this embodiment of the application, the first landing point coordinates are the landing point position of the card picking device predicted by the manual card issuing method; the third landing point coordinates are the final landing point position of the card picking device determined by the manual card issuing method after correcting the first landing point coordinates.

[0079] In actual implementation, this embodiment can identify the card issuance method as manual when the card is detected. It uses an onboard vision sensor to identify the actual hand position and dynamic changes of the card issuer in real time. For example, if the card issuer's hand is detected to be 3cm inward from the predicted position, the first landing point coordinates are corrected based on this actual hand position. The landing point position of the pickup device is then updated based on the corrected first landing point coordinates to determine the third landing point coordinates of the pickup device. This eliminates errors caused by posture deviations or changes in card delivery actions, resulting in a third landing point coordinate that more closely matches the actual card delivery position. Then, based on the third landing point coordinates, the vehicle's current position, the window opening range, the driver's cabin area, and the effective working space, the movement path of the pickup device from the base to the target landing point is planned, ultimately generating a smooth, interference-free pickup trajectory. This improves the accuracy of the pickup trajectory matching and the success rate of card retrieval in complex manual card issuance scenarios.

[0080] Optionally, in one embodiment of this application, generating the pickup device based on the landing point location and the current location of the vehicle includes: responding to the card issuing method of the toll card being a card issuing machine, obtaining the card issuing speed and direction of the card issuing machine, correcting the second landing point coordinates based on the card issuing speed and direction, and updating the landing point location using the corrected second landing point coordinates to determine the fourth landing point coordinates of the pickup device; determining the movement path of the pickup device based on the fourth landing point coordinates and the current location of the vehicle to generate a pickup trajectory.

[0081] In this embodiment of the application, the second landing point coordinates are the landing point position of the access card picking device predicted by the card issuing method of the card issuing machine; the fourth landing point coordinates are the final landing point position of the access card picking device determined by the card issuing method of the card issuing machine after correcting the second landing point coordinates.

[0082] As one possible implementation, this application embodiment can, when the card issuance method is identified as card issuing by a card issuer, obtain the card issuing speed and direction of the card issuer through an onboard vision sensor or communication interaction between the vehicle and the card issuer. For example, if it is detected that the card issuer is pushing the card at a horizontal card issuing speed of 0.2 m / s and in a downward diagonal direction, the original second landing point coordinates are slightly adjusted forward based on the card issuing speed and direction. The landing point position of the pickup device is then updated according to the corrected second landing point coordinates to obtain a more accurate fourth landing point coordinate of the pickup device. Based on the fourth landing point coordinates, the vehicle's current position, the window opening range, the driver's cabin area, and the effective working space, a smooth and interference-free movement path for the pickup device is planned, thereby generating the final pickup trajectory. This improves the accuracy of pickup trajectory generation and the card retrieval success rate in automatic card issuance scenarios.

[0083] In step S203, a control command for the picking device is generated based on the picking trajectory, and the picking device is controlled to execute the control command to pick up the access card.

[0084] It is understood that, according to the generated pickup trajectory, the embodiments of this application can decompose the pickup device's extension length, rotation angle, movement speed, and adsorption timing into control parameters, and generate corresponding motor drive commands, angle control commands, and magnetic attraction energization commands; control the pickup device to smoothly extend along the planned trajectory according to the commands, for example, generating control commands to extend horizontally by 5cm, raise the elevation angle by 15°, move at a speed of 0.1m / s, and activate the magnetic card suction after a 50ms delay after reaching the fourth or third landing point coordinates, so as to control the pickup device to complete the precise card retrieval action and pick up the card. The device retracts slowly at a speed of 0.2 m / s, delivering the toll card to a position within easy reach of the driver's naturally hanging hand (approximately 15 cm from the seat). Then, a green light illuminates on the dashboard, and a voice prompt states, "Toll card delivered, please retrieve it." After 3 seconds, the device uses a combination of camera and magnetic card sensing to determine if the toll card has been retrieved. If not, the toll card is automatically placed in a preset storage location; alternatively, the retrieved toll card can be placed in a preset storage location such as the vehicle's card slot, door storage compartment, or center console storage box, and the retrieval device automatically retracts and returns to its concealed state. This embodiment converts the retrieval trajectory into control commands adapted to the magnetic retrieval device, improving the control accuracy, stability, and reliability of the card retrieval action.

[0085] Optionally, in one embodiment of this application, after picking up the access card, the method further includes: in response to the picking device picking up the access card, controlling the picking device to deliver the access card to a target accessible area of ​​the user in the vehicle cabin; or, based on a placement instruction set by the user, controlling the picking device to place the picked-up access card in a preset storage location.

[0086] In some embodiments, after the card-collecting device picks up the access card, it can control the device to deliver the access card along a safe path to a target accessible area of ​​the user inside the vehicle cabin, such as in front of the center console, to the right of the steering wheel, or near the armrest box, so that the user can easily retrieve the access card; or, based on a pre-set placement command by the user, the device can control the device to place the picked-up access card in a preset storage location such as the vehicle card slot, door storage compartment, or center console storage box. Thus, the access card collecting device can automatically complete the delivery or storage of the card after retrieval, improving ease of use and safety during operation inside the cabin.

[0087] Optionally, in one embodiment of this application, the method further includes: in response to a vehicle being in a toll card return scenario, determining a target return area for the toll card; generating a return trajectory for the pickup device based on the target return area, and controlling the pickup device to extend along the return trajectory to deliver the toll card to the target return area.

[0088] In this embodiment of the application, when a vehicle travels to the highway exit toll area and detects a lane indication signal, a card recycling prompt, or a user-triggered return operation, it is determined that the vehicle is currently in a toll card return scenario.

[0089] In some embodiments, this application can determine the target return area for toll cards, such as the toll station's card return area or card reader area, through visual recognition or positioning information when the vehicle is in a toll card return scenario. Examples include the card handing counter at a manual toll booth or the card return platform next to an automatic card reader at a highway exit. Based on the location and height of the target return area and the vehicle's current parking posture, a return trajectory for the pickup device is generated. The pickup device is then controlled to smoothly extend along this trajectory, accurately delivering the magnetically picked-up toll card to the target return area. After the control device delivers the toll card to the return area, it is powered off and released, automatically retracting the card after the return is complete. This automatically achieves accurate toll card return, improves traffic efficiency, reduces the need for drivers to manually retrieve cards, and enhances driving safety.

[0090] In this embodiment, the pickup device can extend from below the left air vent on the driver's side to the final corrected landing position within 0.8 seconds, and initiate magnetic attraction to complete the card reception the instant the card falls. After the pickup device is powered on, it confirms the initial position through a motor encoder or Hall sensor, and determines the specific extension path according to the vehicle status, such as the degree of window opening and closing, and the user's preset mode. For example, the degree of window opening and closing can be a fully open state, a half open state, and a closed state. When the window is closed, it can remind the user to open the window, or automatically open the window to a suitable size within a certain period of time, and send the path parameters to the vehicle domain controller through the CAN (Controller Area Network) bus to realize a precise and safe automatic card delivery and reception process.

[0091] Secondly, the picking device in this embodiment is equipped with an obstacle detection and avoidance mechanism, ensuring operational safety through multi-sensor configuration and hierarchical control logic: In terms of sensor configuration, the front end of the picking device is equipped with infrared / LiDAR, which can scan obstacles within 0.5 meters ahead of the path in real time with an accuracy of ±1cm; simultaneously, ultrasonic sensors are configured to detect side and oblique obstacles, such as vehicle B-pillars and canopy supports; optional cameras combined with AI algorithms can identify dynamic obstacles such as pedestrians, and visual SLAM (Simultaneous Localization and Interference) is used. Mapping (synchronous localization and mapping) constructs a local environment map; in terms of dynamic obstacle avoidance logic, a hierarchical response mechanism is adopted: when encountering immovable first-level obstacles such as vehicle structures, the mechanism immediately stops moving and triggers an alarm; when encountering movable second-level obstacles such as pedestrians, it decelerates to a safe speed, such as 5cm / s, and dynamically generates S-shaped avoidance trajectories based on the obstacle's position, adjusting the motor torque through PID closed-loop control to achieve detour; at the same time, a safety redundancy mechanism is set up. If obstacle avoidance fails 3 times in a row, the mechanism will automatically retract and record a fault code, such as (E03:OBSTACLE_AVOIDANCE_FAILURE), to ensure the safe and reliable operation of the pickup device.

[0092] Furthermore, the pickup device in this embodiment is equipped with a fault detection and emergency handling mechanism, ensuring operational safety through two dimensions: motion timing monitoring and hardware status detection. Regarding motion timing monitoring, the pickup device sets a timeout threshold for the extension / retraction action. If the preset travel distance is not completed within 0.8 seconds during the extension phase (e.g., extending 5cm), or if it fails to return to its original position within 1.2 seconds during the retraction phase, it is determined to be stalled or mechanically jammed, triggering a safety lock. The motor power is immediately cut off, an alarm message is sent to the vehicle control module via the CAN bus, and an abnormality is displayed in a pop-up window on the user's APP. Regarding hardware status detection, if the infrared / ultrasonic sensor data is abnormal, it will switch to a safety mode that only allows manual card retrieval, prohibiting automatic card delivery. Simultaneously, the Hall sensor monitors the current of the magnetic attraction mechanism. When the attraction force is insufficient, such as when the current is below 80% of the threshold, the backup attraction module is automatically activated, forming a multi-dimensional fault protection system.

[0093] In addition, the dynamic adsorption control of the pickup device in this embodiment includes two parts: card trajectory prediction and adsorption synchronization, and feedback closed-loop optimization. When the predicted distance of the card from the card delivery mechanism is ≤3cm, the magnetic attraction is initiated 0.2 seconds in advance to avoid collision impact, and the adsorption force is dynamically adjusted as the distance decreases through the adsorption coefficient calibrated by experiments. At the same time, a vibration suppression mechanism is set. If the acceleration of the pickup device after adsorption is >2g, a buffer time of 0.5 seconds will be added and the magnetic attraction force will be gradually released. This application also has a self-learning function. With user authorization, it records parameters such as the weight of the card and the falling angle of each operation, continuously optimizes the adsorption force model, and achieves accurate and stable card adsorption control.

[0094] Once this application enters the pickup scenario, it can provide multi-dimensional information feedback through user interaction design to ensure a good user experience. For example, in the fault prompting stage, this application will display fault codes and corresponding solutions on the dashboard or central control screen, such as prompting users to check for obstacles in the window gaps; in the operation guidance stage, when automatic card delivery fails, it will prompt users to manually push the toll card or retry in conjunction with voice broadcast; at the same time, the system will store the data of the last 10 operations, including timestamps, fault codes and other information, and support OTA remote diagnosis, so that users can clearly understand the status of the device and facilitate troubleshooting and maintenance.

[0095] In this embodiment, a "two-stage coupled prediction" architecture is adopted, which achieves high-precision prediction of the card landing point error of <15cm through the feedforward-feedback collaboration of vision and radar: (1) the initial state of the card is calculated based on multiple frames of visual observation within a 200ms window, and the visual extrapolation landing point is generated; (2) the airborne card is tracked by micro-motion point cloud using a 77GHz millimeter-wave radar, and the visual deviation is corrected in real time by Kalman filtering; the final landing point is obtained by weighted fusion of vision and radar, and the fusion weight is dynamically adjusted by the confidence of both. This architecture effectively makes up for the defect of vision in strong backlight and occlusion scenarios where it is easy to lose the target, and at the same time, it greatly improves the robustness and accuracy of prediction by taking advantage of the radar's advantages of not being affected by light and its high speed measurement accuracy.

[0096] Specifically, this application constructs a three-stage technical process for visual initial value calculation (keyframes: release time t0 to t1 = t0 + 200 ms, where t0 is the release time and t1 is the sampling termination time 200 ms later): First, based on the hand keypoint sequence (wrist W, fingertip T) and the access card bounding box (BBox), the release time t0 is defined as: when ||T(t)| ... After W(t) reaches its peak, it drops by ≥8cm within 200ms, and the distance between the center of the BBox and the fingertip is <3cm. The release point P0 = the pixel coordinates of the center of the BBox, converted to world coordinates (Z-axis aligned with the lane plane) by a monocular depth estimation algorithm (Lite-MiDaS quantized version). That is, using the Lite-MiDaS quantized monocular depth estimation method, combined with lane plane prior and real-time camera parameter calibration, the initial three-dimensional coordinates P0 of the toll card at release time t0 are accurately determined, P0 = (x0, y0, z0) ∈R. 3 In this embodiment, depth estimation can be performed by introducing lane plane prior: assuming the toll card lands on a horizontal road surface (z=0), the depth network output is constrained to z∈[0.8m, 1.5m] (typical toll booth card issuance height). Calibration compensation can also be performed: camera intrinsic parameters (f...) x ,f y ,c x ,cy f and extrinsic parameters (R,T), where f x ,f y c represents the focal length in the x and y directions, respectively. x ,c y The coordinates are: R is the rotation matrix, T is the translation vector, and the system is automatically calibrated every 24 hours (based on the static calibration plate image). This allows the initial error of P0 to be controlled within a very small range, providing a reliable starting point for subsequent initial velocity calculation, parabola modeling, and landing point prediction. It is the most basic error control step in the entire card throwing prediction process.

[0097] Secondly, using the center pixel coordinates (p1, p2, p3) of three consecutive frames of the access card, corresponding to the timestamps (t1, t2, t3), the calculation logic is as follows: pixel displacement Δp i = p i+1 -p i , where p i Let p be the pixel coordinates of the center of the card in the i-th frame of the image. i+1 The pixel coordinates of the center of the card in the (i+1)th frame are solved by the camera projection model to obtain the three-dimensional displacement ΔP. i Solve for the initial velocity vector: v0 = (ΔP1 + ΔP2) / (t3) t1), perform weighted averaging, suppress single-frame noise, and finally output v0=(v x0 , v y0 , v z0 ), where v0 is the initial velocity vector at the moment the pass card is released, v x0 v y0 v z0 All are three-dimensional velocity components; furthermore, error control can use RANSAC to remove outlier frames: for 3 frames fitted with a straight line, frames with a reprojection error > 5px are rejected; velocity limiting can also be performed: v0∈ [1.2, 3.5] m / s (the physical limit of manual card throwing, if the limit is exceeded, the radar will be forcibly taken over).

[0098] Finally, P0 and v0, where P0 is the initial three-dimensional coordinate at the moment the pass card is released, are used to establish the equation of motion based on the gravitational acceleration g = 9.78033 m / s² (localized correction), and to establish a world coordinate system: origin O = P0, Z-axis vertically upward, and the equation of motion for the pass card is as follows: x(t) = x0 + v x0 ·t, y(t) = y0 + v y0 ·t, z(t) = z0 + v z0 ·t ½ g·t², Where x(t), y(t), and z(t) are the position coordinates of the pass card at time t in the three-dimensional coordinate system, and t is the flight time of the pass card calculated from the time of release; Landing condition: z(t) = 0 (at the moment of contact with the ground), solving for the theoretical flight time of the pass from release to contact with the ground: t_f = [v z0 + √(v z0 ² + 2·g·z0)] / g, The theoretical landing point of the final pass: P_f = (x(t_f), y(t_f), 0), Finally, output the visual landing point prediction value: P_f^vision = (x_f, y_f, 0).

[0099] Among them, error compensation can introduce an air resistance correction term to compensate for the high-speed card throwing method, that is, to introduce an air resistance correction term (activated only when v0>2.5m / s): Δz_drag = 0.02·v0²·t_f (empirical coefficient, calibrated by wind tunnel experiment). Finally, after incorporating air resistance correction, the recalculated actual flight time of the toll card is as follows: t_f' = [v z0 + √(v_z0² + 2g(z0+Δz_drag))] / g, The visual landing point prediction value Pf_vision is obtained after 200ms, which provides a high-precision feedforward initial value for subsequent landing point fusion and trajectory planning.

[0100] Furthermore, the radar closed-loop correction phase (time window t1-t2, lasting until 50ms before impact) uses radar data as the core to complete the closed-loop correction of the impact point: First, through CFAR (Constant False Alarm Rate) detection and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering, the access card point cloud is extracted from the original radar data. That is, the original radar ADC (Analog-to-Digital Converter) data (Chirp signal) is input, and after CFAR detection + DBSCAN clustering, candidate point cloud clusters of access cards (number of points ≥ 5, RCS > 0.5%) are extracted. 25dBsm), and the radar point cloud is mapped to the visual coordinate system through the joint calibration matrix H_radar2cam (including rotation R, translation T, and time offset Δt=3ms), resulting in P_radar(t) ={p1,p2,...,p X}∈R 3 The error control method is dynamic time synchronization: the PPS pulse (Pulse Per Second) (1Hz) broadcast by the CAN bus is used to calibrate the clock offset between the radar and the camera, with a residual error of <1ms.

[0101] Next, a Kalman filter embedding the gravity model is constructed: state vector X = [x, y, z, v] x , v y , v z ] Where x, y, z are the three-dimensional position coordinates of the access card, and v x , v y , v z The three-dimensional velocity components are used; the observation matrix H = [1,0,0,0,0,0; 0,1,0,0,0,0; 0,0,1,0,0,0]; the process noise Q is dynamically adjusted according to v0 (Q increases at high speeds to enhance tracking robustness) to track the card's motion state in real time.

[0102] Finally, based on the deviation between the radar extrapolated landing point and the visually predicted landing point, and combined with the radar point cloud confidence level, the compensation weight is dynamically adjusted. That is, the estimated card state value input to the Kalman filter output is... (t) (t∈ [t1, t_f) [0.05s]), the coordinates of the toll card landing point predicted by the visual algorithm P_f^vision, and the radar extrapolated landing point: Let (t)=0, the solution is based on radar data extrapolation to obtain the toll card's ground contact time t_f^radar and the toll card's landing point coordinates P_f^radar obtained from radar extrapolation. (t_f^radar), (t_f^radar), 0); Calculate the vision-radar landing point deviation: δP = P_f^radar P_f^vision, where δP is the deviation vector between the radar and visual prediction landing points; Dynamic compensation: P_f = P_f^vision + γ·δP, where the dynamic compensation weight coefficient γ = min(0.7 + 0.3·C_radar, 1.0), and C_radar is the radar point cloud confidence (based on cluster density and RCS stability). The radar point confidence C_radar is calculated as follows: C_radar = exp( ‖δv‖² / δv ²) × (N_points / 10) × (RCS_mean / 20), Where δv is the variance of the radar measured velocity or the reference velocity threshold, ‖δv‖² is the square of the radar measured velocity deviation, N_points is the number of effective point clouds detected by the radar, and RCS_mean is the average value of the radar cross section (RCS).

[0103] This outputs the final fused landing point P_f = (x_f, y_f, 0), achieving closed-loop correction of the visual initial value calculation error and improving the overall accuracy and robustness of the landing point prediction.

[0104] Figure 3 This is a schematic diagram of the structure of a vehicle access card picking device provided in an embodiment of this application.

[0105] For example, such as Figure 3 As shown, the vehicle pass card picking device 10 may include: an acquisition module 100, a generation module 200, and a picking module 300.

[0106] Specifically, the acquisition module 100 is used to acquire the landing position of the pickup device in response to the vehicle being in a toll card pickup scenario.

[0107] The generation module 200 is used to generate the pickup trajectory of the pickup device based on the landing point location and the current position of the vehicle.

[0108] The pickup module 300 is used to generate control commands for the pickup device based on the pickup trajectory, and to control the pickup device to execute the control commands in order to pick up the access card.

[0109] Optionally, in one embodiment of this application, the acquisition module 100 includes: a first identification unit.

[0110] The first identification unit is used to identify the location of the access card and determine the landing point based on the location of the access card.

[0111] Optionally, in one embodiment of this application, the first identification unit includes: a first identification subunit, a second identification subunit, and a determination subunit.

[0112] The first identification subunit is used to identify the target card issuance method of the pass card, wherein the target card issuance method is manual card issuance or card issuance machine issuance.

[0113] The second identification subunit is used to identify the hand gesture of the card issuer in response to the card issuance method being manual card issuance, and to determine the target card type based on the hand gesture, so as to predict the first position of the card based on the target card type, and to determine the first landing point coordinates of the picking device based on the first position of the card, so as to determine the landing point position.

[0114] The determination subunit is used to determine the coordinate information of the card issuing machine's card drop point if the card is issued by a card issuing machine, and to determine the second position of the card using the coordinate information, so as to determine the second drop point coordinates of the picking device based on the second position of the card, and thus determine the drop point position.

[0115] Optionally, in one embodiment of this application, the second identification subunit includes: an acquisition component, a first determination component, and a second determination component.

[0116] The acquisition component is used to acquire multiple key points of the card issuer's hands and determine the wrist coordinates, fingertip coordinates, and palm coordinates of the card issuer based on these key points.

[0117] The first determining component is used to determine the hand position of the card issuer using wrist coordinates, fingertip coordinates, and palm coordinates.

[0118] The second determining component is used to determine whether the target card delivery type is a throwing card delivery type or a flat pushing card delivery type based on the hand position.

[0119] Optionally, in one embodiment of this application, the second determining component includes: a coordinate acquisition subcomponent, a curve determining subcomponent, a first determination subcomponent, and a second determination subcomponent.

[0120] The coordinate acquisition sub-component is used to obtain the vertical coordinates of key points on the wrist of the person issuing the card in multiple frames of images in order to determine the vertical speed of the wrist.

[0121] The curve determination subcomponent is used to determine the distance change curve between the fingertip and the wrist based on the fingertip coordinates and palm coordinates.

[0122] The first determination sub-component is used to determine the target card delivery type as a throwable card type in response to the wrist vertical speed being greater than a first preset speed threshold and the distance change curve satisfying a preset throwing trend.

[0123] The second determination sub-component is used to determine the target card delivery type as a flat-push card issuing type if the vertical speed of the wrist is less than the second preset speed threshold and the distance change curve satisfies the preset flat-push trend; wherein, the first preset speed threshold is greater than the second preset speed threshold.

[0124] Optionally, in one embodiment of this application, the generation module 200 includes a second identification unit and a first generation unit.

[0125] The second identification unit is used to identify the actual hand position of the person issuing the card in response to the card issuance method being manual issuance, correct the first landing point coordinates based on the actual hand position, and update the landing point position using the corrected first landing point coordinates to determine the third landing point coordinates of the picking device.

[0126] The first generation unit is used to determine the motion path corresponding to the pickup device based on the coordinates of the third landing point and the current position of the vehicle, so as to generate the pickup trajectory.

[0127] Optionally, in one embodiment of this application, the generation module 200 includes: an acquisition unit and a second generation unit.

[0128] The acquisition unit is used to respond to the card issuance method of the pass card being the card issuing machine issuance method, acquire the card issuing speed and card issuing direction of the card issuing machine, correct the second landing point coordinates based on the card issuing speed and card issuing direction, and update the landing point position using the corrected second landing point coordinates, so as to determine the fourth landing point coordinates of the picking device.

[0129] The second generation unit is used to determine the movement path of the pickup device based on the coordinates of the fourth landing point and the current position of the vehicle, so as to generate a pickup trajectory.

[0130] Optionally, in one embodiment of this application, the apparatus 10 of this application embodiment further includes: a delivery module.

[0131] The delivery module is used to, after picking up the access card, control the picking device to deliver the access card to the target accessible area of ​​the user in the vehicle cabin in response to the picking device picking up the access card; or, after picking up the access card, control the picking device to place the picked-up access card in a preset storage location based on the placement command set by the user.

[0132] Optionally, in one embodiment of this application, the apparatus 10 of this application embodiment further includes a control module.

[0133] The control module is used to control the vehicle to automatically enter the toll card collection scenario before obtaining the landing position of the pickup device, in response to detecting that the actual distance between the front position of the vehicle and the card issuer or card issuer is greater than a preset safe distance; or, before obtaining the landing position of the pickup device, it is used to control the vehicle to enter the toll card collection scenario in response to the pickup scenario command input by the user.

[0134] Optionally, in one embodiment of this application, the apparatus of this application embodiment further includes: a determining module and a returning module.

[0135] The determination module is used to determine the target return area of ​​the toll card in response to a vehicle being in a toll card return scenario.

[0136] The handover module is used to generate a handover trajectory for the pickup device based on the target handover area, and control the pickup device to extend along the handover trajectory to deliver the access card to the target handover area.

[0137] Figure 4 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0138] When the processor 402 executes the program, it implements the vehicle pass card picking method provided in the above embodiments.

[0139] Furthermore, the vehicle also includes: Communication interface 403 is used for communication between memory 401 and processor 402.

[0140] The memory 401 is used to store computer programs that can run on the processor 402.

[0141] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0142] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0143] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0144] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0145] Furthermore, embodiments of this application also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform the vehicle access card picking method provided in embodiments of this application.

[0146] This embodiment can divide the device into functional modules based on the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0147] When the functional modules are divided according to their respective functions, the device may further include a first acquisition module, a generation module, a first control module, a first determination module, and a second control module. It should be noted that all relevant content of each step involved in the above method embodiments can be referenced to the functional description of the corresponding functional module, and will not be repeated here.

[0148] It should be understood that the device provided in this embodiment is used to execute the above-described method for picking up a vehicle pass card, and therefore can achieve the same effect as the above-described implementation method.

[0149] When using integrated units, the device may include a processing module and a storage module. When applied to an automobile, the processing module can be used to control and manage the vehicle's movements. The storage module can be used to support the vehicle in executing program code, etc.

[0150] The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as disclosed in this application. The processor may also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0151] In addition, the device provided in the embodiments of this application may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the vehicle pass card picking method provided in the above embodiments.

[0152] This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described method steps to implement the vehicle pass card picking method provided in the above embodiment.

[0153] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the vehicle pass card retrieval method provided in the above embodiment.

[0154] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0155] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0156] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0157] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for picking up a vehicle pass, characterized in that, The vehicle is equipped with a card-collecting device, wherein the method includes the following steps: In response to a vehicle being in a toll card collection scenario, the landing position of the collection device is obtained; The pickup trajectory of the pickup device is generated based on the landing point location and the current position of the vehicle; Based on the picking trajectory, a control command is generated for the picking device, and the picking device is controlled to execute the control command to pick up the access card.

2. The method according to claim 1, characterized in that, The step of obtaining the landing position of the pickup device includes: Identify the location of the access card and determine the landing point based on the location of the access card.

3. The method according to claim 2, characterized in that, The process of identifying the location of the access card and determining the landing point location based on the location of the access card includes: Identify the target card issuance method of the access card, wherein the target card issuance method is a manual card issuance method or a card issuance machine issuance method; In response to the card issuance method being manual, the hand gesture of the card issuer is identified, the target card type is determined based on the hand gesture, and the first position of the card is predicted based on the target card type. The first landing point coordinates of the picking device are then determined based on the first position of the card to determine the landing point position. If the card is issued by a card issuer, the coordinates of the card drop point of the card issuer are determined, and the second position of the card is determined using the coordinates. The second drop point coordinates of the pickup device are then determined based on the second position of the card to determine the drop point position.

4. The method according to claim 3, characterized in that, The process of identifying the hand gesture of the card issuer and determining the target card type based on the hand gesture includes: Multiple key hand points of the card issuer are obtained, and the wrist coordinates, fingertip coordinates, and palm coordinates of the card issuer are determined based on the multiple key hand points; The hand position of the person issuing the card is determined using the wrist coordinates, fingertip coordinates, and palm coordinates. Based on the hand position, the target card delivery type is determined to be either a throwing card delivery type or a flat pushing card delivery type.

5. The method according to claim 4, characterized in that, The step of determining whether the target card delivery type is a throwing card delivery type or a flat-pushing card delivery type based on the hand position includes: The vertical coordinates of key points on the wrist of the person issuing the card are obtained in multiple frames of images to determine the vertical velocity of the wrist. The distance variation curve between the fingertip and the wrist is determined based on the fingertip coordinates and the palm coordinates; In response to the wrist vertical speed being greater than a first preset speed threshold and the distance change curve satisfying a preset throwing trend, the target card delivery type is determined to be a throwing type. If the vertical speed of the wrist is less than the second preset speed threshold, and the distance change curve satisfies the preset horizontal pushing trend, then the target card delivery type is determined to be a horizontal pushing card issuing type; wherein, the first preset speed threshold is greater than the second preset speed threshold.

6. The method according to claim 3, characterized in that, The step of generating the pickup trajectory of the pickup device based on the landing point location and the current position of the vehicle includes: In response to the card issuance method being manual, the actual hand position of the card issuer is identified, the first landing point coordinates are corrected based on the actual hand position, and the landing point position is updated using the corrected first landing point coordinates to determine the third landing point coordinates of the picking device. The motion path corresponding to the pickup device is determined based on the coordinates of the third landing point and the current position of the vehicle, so as to generate the pickup trajectory.

7. The method according to claim 3, characterized in that, The step of generating the pickup trajectory of the pickup device based on the landing point location and the current position of the vehicle includes: In response to the card issuing method of the access card being a card issuing machine, the card issuing speed and direction of the card issuing machine are obtained, the second landing point coordinates are corrected based on the card issuing speed and direction, and the landing point position is updated using the corrected second landing point coordinates to determine the fourth landing point coordinates of the picking device. The movement path of the pickup device is determined based on the coordinates of the fourth landing point and the current position of the vehicle to generate the pickup trajectory.

8. The method according to claim 1, characterized in that, After picking up the access card, the process also includes: In response to the pickup device picking up the access card, the pickup device is controlled to deliver the access card to the target accessible area of ​​the user inside the vehicle cabin; Alternatively, based on the placement instructions set by the user, the picking device can be controlled to place the picked-up access card in a preset storage location.

9. The method according to claim 3, characterized in that, Before obtaining the landing position of the pickup device, the process also includes: In response to the detection that the actual distance between the front of the vehicle and the card issuer or the card issuer is greater than a preset safe distance, the vehicle is controlled to automatically enter the toll card picking scene; Alternatively, in response to a user-inputted pickup scenario command, the vehicle can be controlled to enter the toll card pickup scenario.

10. A vehicle, characterized in that, The vehicle includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method as described in any one of claims 1 to 9.

Images