Bicycle unlocking method, device and equipment based on AR system and storage medium

Abstract

The application belongs to the technical field of wearable devices, and discloses a bicycle unlocking method, device and equipment based on an AR system and a storage medium. The method comprises the following steps: when a bicycle use instruction sent by a user is received, the current position of the user is acquired; the user is guided according to the current position, and the distance between the user and a bicycle parking area is detected in real time; when the distance is less than a first distance threshold, the gaze information of the user is acquired according to a target bicycle area; the relative position of an initial bicycle is determined according to the gaze information and the target bicycle area; the target bicycle is determined through bicycle matching according to the relative position and a preset bicycle map, and the target bicycle is unlocked. In the above manner, the target bicycle can be quickly positioned and unlocked, the problems of unlocking difficulty and car searching difficulty when a two-dimensional code is scanned are avoided, the user experience when the bicycle is used is improved, and the accuracy when the bicycle is positioned is improved.

Description

Technical Field

[0001] This invention relates to the field of wearable device technology, and in particular to a bicycle unlocking method, apparatus, device, and storage medium based on an AR system. Background Technology

[0002] The current mainstream method for unlocking shared bikes is by scanning the QR code on the bike with a mobile phone. However, QR codes are prone to damage or smudges, affecting normal use by users. Furthermore, QR codes are easily forged by criminals for fraud and to obtain user information, posing a risk to user data security. To address the issue of QR codes failing to scan properly, bike companies currently offer a solution: users need to select "Find a Bike" on their mobile app or mini-program, locate nearby bikes on a map, tap the bell to locate the bike, and after the bike sounds, return to the bike-finding interface, tap navigation, and record the bike's code. Finally, they select the code to unlock. This process is cumbersome and prone to discrepancies between the bike displayed on the map and its actual location, making it difficult to find a bike and impacting user experience. Summary of the Invention

[0003] The main objective of this invention is to provide a bicycle unlocking method, device, equipment, and storage medium based on an AR system, aiming to solve the technical problem of how to quickly and accurately locate and unlock bicycles.

[0004] To achieve the above objectives, the present invention provides a single-vehicle unlocking method based on an AR system, the single-vehicle unlocking method based on an AR system comprising:

[0005] Upon receiving a user's instruction to use a bicycle, the system obtains the user's current location.

[0006] The user is guided to their current location, and the distance between the user and the bicycle parking area is detected in real time.

[0007] When the distance is less than a first distance threshold, the user's gaze information is obtained based on the target bicycle area;

[0008] The initial relative position of the bicycle is determined based on the gaze information and the target bicycle area;

[0009] The target bicycle is determined by matching the relative position and the preset bicycle map, and then the target bicycle is unlocked.

[0010] Optionally, obtaining the user's current location upon receiving a bicycle usage instruction from the user includes:

[0011] When a user sends a bicycle usage instruction, the bicycle usage program is started.

[0012] The current operating mode is adjusted to the bicycle usage mode according to the bicycle usage procedure.

[0013] The user's current location is detected based on the bicycle usage mode.

[0014] Optionally, the step of guiding the user to a location based on the current location and detecting the distance between the user and the bicycle parking area in real time includes:

[0015] The bicycle parking area is determined based on the current location and the preset bicycle map;

[0016] Based on the preset map construction method, the current location, and the bicycle parking area, a path planning is performed to obtain the movement planning path;

[0017] The user is guided to their location based on the planned movement path, and the distance between the user and the bicycle parking area is detected in real time.

[0018] Optionally, when the distance is less than a first distance threshold, obtaining the user's gaze information based on the target bicycle area includes...

[0019] When the distance is less than a first distance threshold, multiple parked bicycles within the user's field of vision are identified;

[0020] Determine the relative position of each parked bicycle to the user;

[0021] Based on the relative position between each parked bicycle and the user and the user's current position, multiple parked bicycles are locked, and multiple alternative bicycles are determined from among the multiple parked bicycles;

[0022] Determine the target bicycle area based on the bicycle area corresponding to each candidate bicycle;

[0023] When the distance is less than the second distance threshold, the user's gaze information is obtained based on the target bicycle area.

[0024] Optionally, determining the initial relative position of the bicycle based on the gaze information and the target bicycle area includes:

[0025] Based on the gaze information, determine the user's multiple gaze points within the target bicycle area and the cumulative number of gazes at each gaze point;

[0026] The cumulative fixation counts for each fixation point are sorted to obtain the sorting results;

[0027] Based on the sorting results, an initial bicycle is determined within the target bicycle area;

[0028] Calculate the relative position between the user and the initial bicycle.

[0029] Optionally, calculating the relative position between the user and the initial bicycle includes:

[0030] Obtain the user's eye-locking position;

[0031] The initial bicycle area is determined based on the eyeball locking position and the preset radius;

[0032] Extract multiple bicycle feature points within the initial bicycle area, and determine the feature point position of each bicycle feature point according to a preset visual positioning method;

[0033] The relative position between the initial bicycle and the user is determined based on the location of each bicycle's feature points.

[0034] Optionally, the step of matching the target bicycle based on the relative position and a preset bicycle map, and then unlocking the target bicycle, includes:

[0035] Determine the position rotation matrix and position translation matrix based on the relative positions;

[0036] The actual absolute position of the initial bicycle is determined by calculating the position based on the user's current position, the position rotation matrix, and the position translation matrix.

[0037] The target bicycle is determined by matching the actual absolute location with the preset bicycle map, and then the target bicycle is unlocked.

[0038] Optionally, the step of matching the target bicycle based on the actual absolute location and a preset bicycle map, and then unlocking the target bicycle, includes:

[0039] Error calculations are performed based on the actual absolute position and the virtual absolute position of each preset bicycle in the preset bicycle map to determine multiple error calculation results;

[0040] Based on the error calculation results, the target bicycle is determined from each preset bicycle, and the basic information of the target bicycle is obtained;

[0041] The basic information is broadcast to prompt the user to respond with a bicycle unlock command.

[0042] Upon receiving the user's bicycle unlocking command, the target bicycle is unlocked.

[0043] Optionally, after determining the target bicycle by matching it with the relative position and the preset bicycle map, and unlocking the target bicycle, the method further includes:

[0044] Obtain the stationary time of the target bicycle;

[0045] When the stillness time is not less than a first time threshold, the user's current location is obtained;

[0046] The current distance is determined based on the current location and the current location of the target bicycle;

[0047] When the current distance is not less than the lock distance threshold, a lock reminder command is broadcast so that the user can respond with a lock command according to the lock reminder command;

[0048] Upon receiving the user's lock command, the target bicycle is locked according to the lock command.

[0049] Furthermore, to achieve the above objectives, the present invention also proposes a bicycle unlocking device based on an AR system, the bicycle unlocking device based on an AR system comprising:

[0050] The acquisition module is used to acquire the user's current location when it receives a bicycle usage instruction sent by the user;

[0051] The detection module is used to guide the user's location based on the current location and to detect the distance between the user and the bicycle parking area in real time.

[0052] The acquisition module is further configured to acquire the user's gaze information based on the target bicycle area when the distance is less than the first distance threshold;

[0053] The determination module is used to determine the initial relative position of the bicycle based on the gaze information and the target bicycle area;

[0054] The unlocking module is used to match and determine the target bicycle based on the relative position and the preset bicycle map, and to unlock the target bicycle.

[0055] Furthermore, to achieve the above objectives, the present invention also proposes a bicycle unlocking device based on an AR system. The bicycle unlocking device based on an AR system includes: a memory, a processor, and a bicycle unlocking program based on an AR system stored in the memory and executable on the processor. The bicycle unlocking program based on an AR system is configured to implement the bicycle unlocking method based on an AR system as described above.

[0056] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a bicycle unlocking program based on an AR system, wherein the bicycle unlocking program based on an AR system is executed by a processor to implement the bicycle unlocking method based on an AR system as described above.

[0057] This invention obtains the user's current location upon receiving a bicycle usage command; guides the user based on the current location and monitors the distance between the user and the bicycle parking area in real time; when the distance is less than a first distance threshold, obtains the user's gaze information based on the target bicycle area; determines the initial relative position of the bicycle based on the gaze information and the target bicycle area; matches the relative position with a preset bicycle map to determine the target bicycle and unlocks it. Through this method, the user is guided based on their current location, and when the distance between the user and the bicycle parking area is less than a first distance threshold, the initial relative position of the bicycle is determined based on the user's gaze information and the target bicycle area. The target bicycle is then finally determined based on the relative position and the preset bicycle map, and unlocked using an AR system. This allows for quick and accurate location and unlocking of the target bicycle, avoiding the difficulties of unlocking when using QR codes and finding bicycles when using software. This improves the user experience and enhances the accuracy of bicycle location. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the structure of a single-vehicle unlocking device based on an AR system in the hardware operating environment of the embodiment of the present invention;

[0059] Figure 2 This is a flowchart illustrating the first embodiment of the single-vehicle unlocking method based on an AR system according to the present invention.

[0060] Figure 3 This is a flowchart illustrating the second embodiment of the single-vehicle unlocking method based on an AR system according to the present invention.

[0061] Figure 4 This is a flowchart illustrating the third embodiment of the single-vehicle unlocking method based on an AR system of the present invention.

[0062] Figure 5 This is a schematic diagram of the overall process of an embodiment of the single-vehicle unlocking method based on an AR system of the present invention;

[0063] Figure 6 This is a structural block diagram of the first embodiment of the single-vehicle unlocking device based on the AR system of the present invention.

[0064] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0065] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0066] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a single-vehicle unlocking device based on an AR system, which is part of the hardware operating environment of the embodiment of the present invention.

[0067] like Figure 1 As shown, the AR-based bicycle unlocking device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0068] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on AR-based bicycle unlocking devices and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0069] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a bicycle unlocking program based on an AR system.

[0070] exist Figure 1 In the AR-based bicycle unlocking device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the AR-based bicycle unlocking device of the present invention can be set in the AR-based bicycle unlocking device, and the AR-based bicycle unlocking device calls the AR-based bicycle unlocking program stored in the memory 1005 through the processor 1001 and executes the AR-based bicycle unlocking method provided in the embodiment of the present invention.

[0071] This invention provides a single-vehicle unlocking method based on an AR system, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of a single-vehicle unlocking method based on an AR system according to the present invention.

[0072] The AR-based single-vehicle unlocking method includes the following steps:

[0073] Step S10: Upon receiving a bicycle usage instruction from a user, obtain the user's current location.

[0074] It should be noted that the execution subject of this embodiment is the control unit in the AR-based bicycle unlocking system, which can also be a unit or module that can achieve the same function. This embodiment does not limit this. The AR-based bicycle unlocking system is installed on an AR wearable device.

[0075] It is understandable that when AR users need to find and unlock shared bicycles, they can interact with AR wearable devices to enable the AR-based bicycle unlocking system to find and unlock shared bicycles when it receives the user's bicycle usage command, thus realizing the application value of AR wearable devices in real-world scenarios and improving the travel experience of AR users.

[0076] In practice, when users need to use a bicycle, they can interact with the AR wearable device through any of the following methods: voice, gesture, or virtual touch, thereby generating instructions for using the bicycle. Voice interaction allows users to issue voice commands, such as "Hey A, Hey A, I want to ride a shared bike." The control unit in the AR system's bike unlocking system recognizes the voice command and receives the user's bike usage instruction. Gesture interaction allows users to issue gestures. The control unit performs a similarity match between the user's gesture and pre-stored gestures for bike finding. When the match reaches a preset value, the user's gesture is considered a bike finding gesture, indicating that the user's bike usage instruction has been received. Virtual touch interaction allows users to issue gestures. The control unit performs a similarity match between the user's gesture and pre-stored gestures for virtual touch input. When the match reaches a preset value, the user's gesture is considered a virtual touch input gesture. The control unit then obtains the fingertip position used for clicking and maps it to a virtual interface position. When the user clicks any of the following on the virtual interface—opening the bike-sharing app, the bike-sharing mini-program, or the start-up button—the user's bike usage instruction has been received.

[0077] It should be noted that when the system receives a user's command to use the bicycle, it detects the user's current location in real time. The user's current location is the current location. Since the user is wearing an AR wearable device, the control unit can determine the user's current location by directly obtaining the location of the AR wearable device (i.e., its own location).

[0078] Understandably, to ensure accuracy when locating bicycles later, the bicycle usage mode needs to be activated upon receiving a bicycle usage instruction. Furthermore, obtaining the user's current location upon receiving the user's bicycle usage instruction includes: starting the bicycle usage program upon receiving the user's instruction; adjusting the current operating mode to bicycle usage mode according to the bicycle usage program; and detecting the user's current location according to the bicycle usage mode.

[0079] In its implementation, upon receiving a bicycle usage command, the control unit activates the bicycle usage program within the AR-based bicycle unlocking system. This bicycle usage program refers to a mini-program for using the bicycle; alternatively, a bicycle-using app can also be launched. After activating either the app or the bicycle usage program, the current operating mode is switched to bicycle usage mode. In bicycle usage mode, the control unit of the AR-based bicycle unlocking system continuously monitors the user's current location.

[0080] Step S20: Guide the user to the correct location based on the current location, and detect the distance between the user and the bicycle parking area in real time.

[0081] It should be noted that after determining the user's current location, the control unit searches for the nearest bicycle parking area based on the bicycle user program or app. The bicycle user program or app will have a preset bicycle map, which will have multiple areas specifically designated for bicycle parking. These multiple areas specifically designated for user bicycle parking are the preset parking areas, which refer to the area within the preset parking areas that is closest to the user's current location.

[0082] Understandably, in order to accurately guide the user's location, the step of guiding the user's location based on the current location and detecting the distance between the user and the bicycle parking area in real time includes: determining the bicycle parking area based on the current location and a preset bicycle map; performing path planning based on the preset map construction method, the current location, and the bicycle parking area to obtain a movement planning path; guiding the user's location based on the movement planning path and detecting the distance between the user and the bicycle parking area in real time.

[0083] In the specific implementation, the distance between the current location and multiple preset parking areas in the preset bicycle map is calculated, and the area closest to the user's current location in the preset parking area is determined and used as the bicycle parking area.

[0084] It should be noted that the preset map building method refers to SLAM (Simultaneous Localization and Mapping). After determining the current location and the bicycle parking area, the preset map building method is used to plan the path for the user to move to the bicycle parking area. The user is then guided to move to the bicycle parking area according to the path, and the distance between the user and the bicycle parking area is detected in real time during the movement.

[0085] Step S30: When the distance is less than the first distance threshold, obtain the user's gaze information based on the target bicycle area.

[0086] It should be noted that the first distance threshold is a set distance threshold used to determine the target bicycle area. The target bicycle area refers to the area where the nearest stationary bicycle to the user is located within the bicycle parking area. The gaze information is obtained by activating the eye tracking module on the AR wearable device to track the user's gaze and gaze points. The gaze information includes, but is not limited to, multiple gaze points of the user, gaze time information corresponding to each gaze point, and cumulative gaze count of each gaze point.

[0087] It is understandable that when the distance is less than the first distance threshold, it indicates that the user is approaching the bicycle parking area. The target bicycle parking area within the bicycle parking area is determined, and the user's gaze information is obtained when the user gets closer.

[0088] In a specific implementation, to accurately determine the target bicycle area, the step of obtaining the user's gaze information based on the target bicycle area when the distance is less than a first distance threshold includes: identifying multiple parked bicycles within the user's field of vision when the distance is less than the first distance threshold; determining the relative position between each parked bicycle and the user; locking the multiple parked bicycles based on the relative position between each parked bicycle and the user and the user's current position, and determining multiple candidate bicycles among the multiple parked bicycles; determining the target bicycle area based on the bicycle area corresponding to each candidate bicycle; and obtaining the user's gaze information based on the target bicycle area when the distance is less than a second distance threshold.

[0089] It should be noted that the second distance threshold is a set distance threshold used to obtain gaze information. The second distance threshold is less than the first distance threshold. Parked bicycles refer to bicycles that are stationary within the user's field of vision in the bicycle parking area. Alternate bicycles refer to a preset number of bicycles that are closest to the user among the parked bicycles.

[0090] Understandably, when the distance is less than the first distance threshold, the camera image target recognition module is activated. Multiple parked bicycles within the field of view are identified through image template matching, and the relative positions of each parked bicycle and the user's parked bicycle are determined through dual-camera binocular visual positioning. As the user gradually approaches the parked bicycles, the image recognition and binocular visual positioning functions combine to gradually and accurately lock onto and track a preset number of candidate bicycles closest to the user's relative position, determining the bicycle area corresponding to each candidate bicycle and its boundary position. Based on the bicycle area corresponding to each candidate bicycle and its boundary position, the target bicycle area is determined. In this embodiment, the first distance threshold can be 10 meters, the second distance threshold can be 5 meters, or other set values; this embodiment does not impose any limitations on these.

[0091] In the specific implementation, when the distance between the user and the bicycle parking area is less than the second distance threshold, it means that the user has moved closer to the bicycle parking area. At this time, the distance to the target bicycle that the user needs to select is closer, and the user can clearly see the bicycle parked in the target bicycle area (that is, in front of the user). At this time, the eye tracking module is activated to track the user's line of sight and gaze point in real time and obtain gaze information in the target bicycle area.

[0092] Step S40: Determine the initial relative position of the bicycle based on the gaze information and the target bicycle area.

[0093] It should be noted that the initial bicycle refers to the bicycle selected from multiple candidate bicycles in the target bicycle area that has received the most cumulative glances from the user. The initial bicycle is also the user's intended bicycle that has not been matched with the preset bicycle map. The relative position between the initial bicycle and the user is calculated based on the gaze information and the target bicycle area. In this embodiment, the relative position is always a three-dimensional coordinate.

[0094] Step S50: Based on the relative position and the preset bicycle map, perform bicycle matching to determine the target bicycle, and unlock the target bicycle.

[0095] It should be noted that by performing coordinate transformation based on relative position, the absolute position of the initial bicycle in the actual scene can be determined. Then, the absolute position of the initial bicycle is matched with the absolute positions of each preset bicycle in the preset bicycle map. The preset bicycle that matches the initial bicycle in the preset bicycle map is identified as the target bicycle. The control unit unlocks the target bicycle in the bicycle user program or bike-sharing APP and sends a command to the target bicycle to make the target bicycle ring in the actual scene, helping the user quickly confirm the bicycle recognized by the AR-based bicycle unlocking system.

[0096] Understandably, to ensure the rational use of bicycle resources, target bicycles need to be locked in a timely manner. Furthermore, after determining the target bicycle by matching it with the relative position and a preset bicycle map, and unlocking the target bicycle, the process further includes: obtaining the stationary time of the target bicycle; when the stationary time is not less than a first time threshold, obtaining the user's current position; determining the current distance based on the current position and the current position of the target bicycle; when the current distance is not less than a locking distance threshold, broadcasting a locking reminder command so that the user can respond with a locking command; and upon receiving the user's locking command, locking the target bicycle according to the locking command.

[0097] In the specific implementation, the first time threshold refers to the set critical time value for locking. When the target bicycle remains stationary for no less than the first time threshold, the user's current location and the target bicycle's current location are obtained to determine the current distance between the user and the target bicycle. If the current distance is no less than the locking distance threshold, it indicates that the user wearing the AR wearable device has moved away from the target bicycle. At this time, the control unit broadcasts a locking reminder command, such as "The vehicle is not locked, please lock it," so that the user can respond with a locking command. Upon receiving the locking command, the target bicycle is locked according to the command, and the user exits the bicycle usage mode after locking. For example, when the target bicycle is detected to be stationary for more than 20 seconds and the user's distance from the target bicycle exceeds 5 meters, an automatic voice reminder is given to the user, "The vehicle is not locked, please lock it." The user can respond with a locking command via voice / gesture / virtual touch to complete the locking process.

[0098] It should be noted that users can interact with the AR wearable device through any of the following methods: voice, gesture, or virtual touch, to generate locking commands. Voice interaction allows users to issue voice commands, such as "Hey A, please lock." The control unit in the AR system's bike unlocking system recognizes the voice command and receives the user's locking command. Gesture interaction involves the control unit matching the user's gesture with pre-stored bike locking gestures. If the matching score reaches a preset value, the user's gesture is considered the correct one for locking the bike, and the locking command is received. Virtual touch interaction involves the control unit matching the user's gesture with pre-stored virtual touch input gestures. If the matching score reaches a preset value, the user's gesture is considered the correct one for virtual touch input. The control unit then obtains the fingertip position used for the click, maps the click position to a location on the virtual interface, and receives the user's lock command when the user clicks on the virtual interface.

[0099] Understandably, when interacting via virtual touch, if the target bicycle remains stationary for at least a first time threshold, a lock control automatically pops up. Upon receiving a lock command from the user, the lock is completed. If the user waits at a traffic light or makes a brief stop, and does not click the lock button within a second time threshold, the lock control disappears, and the stationary time of the target bicycle is recalculated. The second time threshold is greater than the first time threshold.

[0100] In practice, users can also manually lock the target bicycle. When the system detects that the user has manually locked the target bicycle, it will automatically exit the bicycle usage mode.

[0101] This embodiment obtains the user's current location upon receiving a bicycle usage command from the user; guides the user based on the current location and monitors the distance between the user and the bicycle parking area in real time; when the distance is less than a first distance threshold, obtains the user's gaze information based on the target bicycle area; determines the initial relative position of the bicycle based on the gaze information and the target bicycle area; matches the relative position with a preset bicycle map to determine the target bicycle and unlocks it. Through this method, the user is guided based on their current location, and when the distance between the user and the bicycle parking area is less than a first distance threshold, the initial relative position of the bicycle is determined based on the user's gaze information and the target bicycle area. The target bicycle is then finally determined based on the relative position and the preset bicycle map, and unlocked using an AR system. This allows for quick and accurate location and unlocking of the target bicycle, avoiding the difficulties of unlocking when using QR codes and finding bicycles when using software. This improves the user experience and enhances the accuracy of bicycle location.

[0102] refer to Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of a single-vehicle unlocking method based on an AR system according to the present invention.

[0103] Based on the first embodiment described above, step S40 in the single-vehicle unlocking method based on the AR system in this embodiment includes:

[0104] Step S41: Determine multiple gaze points of the user within the target bicycle area and the cumulative number of gazes at each gaze point based on the gaze information.

[0105] It should be noted that, since users may be looking at other information around them while searching for bicycles in open outdoor spaces, it is necessary to determine the user's multiple fixation points within the target bicycle area and the cumulative number of fixations for each fixation point.

[0106] Step S42: Sort the cumulative number of fixations for each fixation point to obtain the sorting results.

[0107] Step S43: Determine the initial bicycle within the target bicycle area based on the sorting results.

[0108] It should be noted that since users will look at the bicycle they want to ride more often than other content during the search process, the system determines the gaze point with the highest cumulative gaze count within the target bicycle area based on the sorting results, and then determines whether the content corresponding to the gaze point with the highest cumulative gaze count is a bicycle. If so, it means that the bicycle is the initial bicycle.

[0109] Step S44: Calculate the relative position between the user and the initial bicycle.

[0110] It should be noted that, in order to accurately calculate the relative position between the user and the initial bicycle, the calculation of the relative position between the user and the initial bicycle further includes: obtaining the user's eye-locking position; determining the initial bicycle area based on the eye-locking position and a preset radius; extracting multiple bicycle feature points within the initial bicycle area, and determining the feature point position of each bicycle feature point according to a preset visual positioning method; and determining the relative position between the initial bicycle and the user based on the feature point position of each bicycle feature point.

[0111] In the specific implementation, the initial bicycle position locked by eye tracking is the eye-lock position. An initial bicycle area is defined with the eye-lock position as the center and a preset radius as the radius. A binocular camera identifies multiple feature points of the initial bicycle within this area. Based on a preset visual positioning method, the position of each feature point within the initial bicycle area is determined. The average position of all feature points is calculated, and this average position is the initial value of the relative position between the initial bicycle and the user. A Kalman filter algorithm is used to track and optimize the initial value, ultimately obtaining the accurate relative position between the initial bicycle and the user. The preset visual positioning method refers to the positioning method corresponding to the binocular visual positioning principle.

[0112] In this embodiment, multiple gaze points of the user within the target bicycle area and the cumulative number of gazes at each gaze point are determined based on the gaze information; the cumulative number of gazes at each gaze point is sorted to obtain a sorting result; an initial bicycle is determined within the target bicycle area based on the sorting result; and the relative position between the user and the initial bicycle is calculated. Through this method, the initial bicycle can be quickly and accurately located within the target bicycle area, and the accuracy of subsequent bicycle positioning is ensured by calculating the relative position.

[0113] refer to Figure 4 , Figure 4 This is a flowchart illustrating a second embodiment of a single-vehicle unlocking method based on an AR system according to the present invention.

[0114] Based on the above embodiments, step S50 in the single-vehicle unlocking method based on the AR system in this embodiment includes:

[0115] Step S51: Determine the position rotation matrix and position translation matrix based on the relative positions.

[0116] It should be noted that the initial relative position of the bicycle is represented by the position rotation matrix R and the position translation matrix T.

[0117] Step S52: Calculate the position based on the user's current position, the position rotation matrix, and the position translation matrix to determine the actual absolute position of the initial bicycle.

[0118] It should be noted that the user's current location is determined through initial navigation positioning and real-time precise SLAM positioning. Based on the current position, the position rotation matrix R, and the position translation matrix T, the position calculation determines the initial absolute position of the bicycle in the actual scene. This initial absolute position is the actual absolute position. .

[0119] Step S53: Based on the actual absolute position and the preset bicycle map, perform bicycle matching to determine the target bicycle, and unlock the target bicycle.

[0120] It should be noted that, in order to accurately locate and unlock the target bicycle, the step of matching the target bicycle with the actual absolute position and the preset bicycle map to determine the target bicycle and then unlocking the target bicycle includes: calculating the error between the actual absolute position and the virtual absolute position of each preset bicycle in the preset bicycle map, and determining multiple error calculation results; determining the target bicycle among the preset bicycles based on each error calculation result, and obtaining the basic information of the target bicycle; broadcasting the basic information to prompt the user to provide a bicycle unlocking command; and unlocking the target bicycle upon receiving the user's bicycle unlocking command.

[0121] It is understandable that the virtual absolute position refers to the absolute position of each preset bicycle in the actual scene. The position error is calculated by comparing the actual absolute position of the initial bicycle with the virtual absolute position of each preset bicycle. Multiple error calculation results are determined. Among the multiple error calculation results, the preset bicycle with the smallest position error with the initial bicycle is determined. The preset bicycle with the smallest position error with the initial bicycle is the target bicycle. The basic information of the target bicycle is obtained, including but not limited to the virtual absolute position and current usage information of the target bicycle.

[0122] In the specific implementation, after obtaining the basic information of the target bicycle, it means that the bicycle that the user is about to ride has been identified. At this time, a preset bicycle map marked with the target bicycle is displayed on the virtual interface, and the basic information is displayed in text near the target bicycle. The system also broadcasts information such as "Bicycle found for you, please confirm whether to unlock" to prompt the user to provide a bicycle unlocking command. When the user provides a bicycle unlocking command, the target bicycle is unlocked.

[0123] It should be noted that when users need to unlock a bicycle, they can interact with the AR wearable device through any of the following methods: voice, gesture, or virtual touch, to generate a bicycle unlocking command. Voice interaction allows users to issue voice commands, such as "unlock." The control unit in the AR system's bike unlocking system recognizes the voice command and thus receives the user's unlocking instruction. Gesture interaction allows users to issue gestures. The control unit performs a similarity match between the user's gesture and pre-stored gestures for unlocking bikes. If the similarity reaches a preset value, the user's gesture is considered the unlocking gesture, and the system receives the unlocking instruction. Virtual touch interaction allows users to issue gestures. The control unit performs a similarity match between the user's gesture and pre-stored gestures for virtual touch input. If the similarity reaches a preset value, the user's gesture is considered the virtual touch input gesture. The system then obtains the fingertip position used for the click, maps it to a virtual interface position, and receives the user's confirmation of unlocking by clicking the corresponding option control on the virtual interface.

[0124] It is understandable that, such as Figure 5As shown, when a user needs to use a bicycle, the bicycle usage mode is activated. When the user interacts with the bicycle, various interaction methods such as voice, gesture, and virtual touch can be used. When the user approaches the bicycle, the system accurately identifies and locates the target bicycle's position through technologies such as image recognition, SLAM positioning, binocular visual positioning, and eye tracking. After automatically matching the bicycle with the preset bicycle map and obtaining the bicycle information, the system quickly unlocks the bicycle. When the trip is over, the user can actively lock the bicycle through various interaction methods. If the user forgets to lock the bicycle, the system will remind the user to lock it via voice to ensure that the user locks the bicycle in time and exits the bicycle usage mode after locking it.

[0125] This embodiment determines the position rotation matrix and position translation matrix based on the relative position; it then calculates the actual absolute position of the initial bicycle based on the user's current position, the position rotation matrix, and the position translation matrix; finally, it matches the actual absolute position with a preset bicycle map to determine the target bicycle and unlocks it. This method ensures accurate bicycle positioning, avoids incorrect bicycle selection, and improves the user experience.

[0126] In addition, refer to Figure 6 This invention also proposes a bicycle unlocking device based on an AR system, the bicycle unlocking device based on an AR system comprising:

[0127] The acquisition module 10 is used to acquire the current location of the user when it receives a bicycle usage instruction sent by the user.

[0128] The detection module 20 is used to guide the user's location based on the current location and to detect the distance between the user and the bicycle parking area in real time.

[0129] The acquisition module 10 is further configured to acquire the user's gaze information based on the target bicycle area when the distance is less than the first distance threshold.

[0130] The determination module 30 is used to determine the initial relative position of the bicycle based on the gaze information and the target bicycle area.

[0131] The unlocking module 40 is used to match and determine the target bicycle based on the relative position and the preset bicycle map, and to unlock the target bicycle.

[0132] This embodiment obtains the user's current location upon receiving a bicycle usage command from the user; guides the user based on the current location and monitors the distance between the user and the bicycle parking area in real time; when the distance is less than a first distance threshold, obtains the user's gaze information based on the target bicycle area; determines the initial relative position of the bicycle based on the gaze information and the target bicycle area; matches the relative position with a preset bicycle map to determine the target bicycle and unlocks it. Through this method, the user is guided based on their current location, and when the distance between the user and the bicycle parking area is less than a first distance threshold, the initial relative position of the bicycle is determined based on the user's gaze information and the target bicycle area. The target bicycle is then finally determined based on the relative position and the preset bicycle map, and unlocked using an AR system. This allows for quick and accurate location and unlocking of the target bicycle, avoiding the difficulties of unlocking when using QR codes and finding bicycles when using software. This improves the user experience and enhances the accuracy of bicycle location.

[0133] In one embodiment, the acquisition module 10 is further configured to start the bicycle usage program when it receives a bicycle usage instruction sent by the user;

[0134] The current operating mode is adjusted to the bicycle usage mode according to the bicycle usage procedure.

[0135] The user's current location is detected based on the bicycle usage mode.

[0136] In one embodiment, the detection module 20 is further configured to determine a bicycle parking area based on the current location and a preset bicycle map;

[0137] Based on the preset map construction method, the current location, and the bicycle parking area, a path planning is performed to obtain the movement planning path;

[0138] The user is guided to their location based on the planned movement path, and the distance between the user and the bicycle parking area is detected in real time.

[0139] In one embodiment, the acquisition module 10 is further configured to identify multiple parked bicycles within the user's field of vision when the spacing is less than a first distance threshold;

[0140] Determine the relative position of each parked bicycle to the user;

[0141] Based on the relative position between each parked bicycle and the user and the user's current position, multiple parked bicycles are locked, and multiple alternative bicycles are determined from among the multiple parked bicycles;

[0142] Determine the target bicycle area based on the bicycle area corresponding to each candidate bicycle;

[0143] When the distance is less than the second distance threshold, the user's gaze information is obtained based on the target bicycle area.

[0144] In one embodiment, the determining module 30 is further configured to determine, based on the gaze information, multiple gaze points of the user within the target bicycle area and the cumulative number of gazes at each gaze point;

[0145] The cumulative fixation counts for each fixation point are sorted to obtain the sorting results;

[0146] Based on the sorting results, an initial bicycle is determined within the target bicycle area;

[0147] Calculate the relative position between the user and the initial bicycle.

[0148] In one embodiment, the determining module 30 is further configured to obtain the user's eye-locking position;

[0149] The initial bicycle area is determined based on the eyeball locking position and the preset radius;

[0150] Extract multiple bicycle feature points within the initial bicycle area, and determine the feature point position of each bicycle feature point according to a preset visual positioning method;

[0151] The relative position between the initial bicycle and the user is determined based on the location of each bicycle's feature points.

[0152] In one embodiment, the unlocking module 40 is further configured to determine a position rotation matrix and a position translation matrix based on the relative position;

[0153] The actual absolute position of the initial bicycle is determined by calculating the position based on the user's current position, the position rotation matrix, and the position translation matrix.

[0154] The target bicycle is determined by matching the actual absolute location with the preset bicycle map, and then the target bicycle is unlocked.

[0155] In one embodiment, the unlocking module 40 is further configured to perform error calculation based on the actual absolute position and the virtual absolute position of each preset bicycle in the preset bicycle map, and determine multiple error calculation results;

[0156] Based on the error calculation results, the target bicycle is determined from each preset bicycle, and the basic information of the target bicycle is obtained;

[0157] The basic information is broadcast to prompt the user to respond with a bicycle unlock command.

[0158] Upon receiving the user's bicycle unlocking command, the target bicycle is unlocked.

[0159] In one embodiment, the unlocking module 40 is further configured to obtain the stationary time of the target bicycle;

[0160] When the stillness time is not less than a first time threshold, the user's current location is obtained;

[0161] The current distance is determined based on the current location and the current location of the target bicycle;

[0162] When the current distance is not less than the lock distance threshold, a lock reminder command is broadcast so that the user can respond with a lock command according to the lock reminder command;

[0163] Upon receiving the user's lock command, the target bicycle is locked according to the lock command.

[0164] Since this device adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0165] Furthermore, this embodiment of the invention also proposes a storage medium storing a bicycle unlocking program based on an AR system. When the bicycle unlocking program based on an AR system is executed by a processor, it implements the steps of the bicycle unlocking method based on an AR system as described above.

[0166] Since this storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0167] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0168] In addition, for technical details not described in detail in this embodiment, please refer to the single-vehicle unlocking method based on the AR system provided in any embodiment of the present invention, which will not be repeated here.

[0169] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0170] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0171] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0172] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for unlocking a bicycle based on an AR system, characterized in that, The AR-based single-vehicle unlocking method includes: Upon receiving a user's instruction to use a bicycle, the system obtains the user's current location. The user is guided to their current location, and the distance between the user and the bicycle parking area is detected in real time. When the distance is less than a first distance threshold, the user's gaze information is obtained based on the target bicycle area; The initial relative position of the bicycle is determined based on the gaze information and the target bicycle area; The target bicycle is determined by matching the relative position and the preset bicycle map, and the target bicycle is then unlocked. When the distance is less than a first distance threshold, obtaining the user's gaze information based on the target bicycle area includes: When the distance is less than a first distance threshold, multiple parked bicycles within the user's field of vision are identified; Determine the relative position of each parked bicycle to the user; Based on the relative position between each parked bicycle and the user and the user's current position, multiple parked bicycles are locked, and multiple alternative bicycles are determined from the multiple parked bicycles; Determine the target bicycle area based on the bicycle area corresponding to each candidate bicycle; When the distance is less than the second distance threshold, the user's gaze information is obtained based on the target bicycle area; Determining the initial relative position of the bicycle based on the gaze information and the target bicycle area includes: Based on the gaze information, determine the user's multiple gaze points within the target bicycle area and the cumulative number of gazes at each gaze point; The cumulative fixation counts for each fixation point are sorted to obtain the sorting results; Based on the sorting results, an initial bicycle is determined within the target bicycle area; Calculate the relative position between the user and the initial bicycle.

2. The single-vehicle unlocking method based on an AR system as described in claim 1, characterized in that, Upon receiving a bicycle usage instruction from a user, obtaining the user's current location includes: When a user sends a bicycle usage instruction, the bicycle usage program is started. The current operating mode is adjusted to the bicycle usage mode according to the bicycle usage procedure. The user's current location is detected based on the bicycle usage mode.

3. The single-vehicle unlocking method based on an AR system as described in claim 1, characterized in that, The step of guiding the user to a location based on the current location and detecting the distance between the user and the bicycle parking area in real time includes: The bicycle parking area is determined based on the current location and the preset bicycle map; Based on the preset map construction method, the current location, and the bicycle parking area, a path planning is performed to obtain the movement planning path; The user is guided to their location based on the planned movement path, and the distance between the user and the bicycle parking area is detected in real time.

4. The single-vehicle unlocking method based on an AR system as described in claim 1, characterized in that, The calculation of the relative position between the user and the initial bicycle includes: Obtain the user's eye-locking position; The initial bicycle area is determined based on the eyeball locking position and the preset radius; Extract multiple bicycle feature points within the initial bicycle area, and determine the feature point position of each bicycle feature point according to a preset visual positioning method; The relative position between the initial bicycle and the user is determined based on the location of each bicycle's feature points.

5. The single-vehicle unlocking method based on an AR system as described in claim 1, characterized in that, The step of matching and determining the target bicycle based on the relative position and a preset bicycle map, and then unlocking the target bicycle, includes: Determine the position rotation matrix and position translation matrix based on the relative positions; The actual absolute position of the initial bicycle is determined by calculating the position based on the user's current position, the position rotation matrix, and the position translation matrix. The target bicycle is determined by matching the actual absolute location with the preset bicycle map, and then the target bicycle is unlocked.

6. The single-vehicle unlocking method based on an AR system as described in claim 5, characterized in that, The step of matching and determining the target bicycle based on the actual absolute location and the preset bicycle map, and unlocking the target bicycle, includes: Error calculations are performed based on the actual absolute position and the virtual absolute position of each preset bicycle in the preset bicycle map to determine multiple error calculation results; Based on the error calculation results, the target bicycle is determined from each preset bicycle, and the basic information of the target bicycle is obtained; The basic information is broadcast to prompt the user to respond with a bicycle unlock command. Upon receiving the user's bicycle unlocking command, the target bicycle is unlocked.

7. The single-vehicle unlocking method based on an AR system as described in any one of claims 1 to 6, characterized in that, After determining the target bicycle by matching it with the relative position and the preset bicycle map, and unlocking the target bicycle, the method further includes: Obtain the stationary time of the target bicycle; When the stillness time is not less than a first time threshold, the user's current location is obtained; The current distance is determined based on the current location and the current location of the target bicycle; When the current distance is not less than the lock distance threshold, a lock reminder command is broadcast so that the user can respond with a lock command according to the lock reminder command; Upon receiving the user's lock command, the target bicycle is locked according to the lock command.

8. A bicycle unlocking device based on an AR system, characterized in that, The AR-based single-vehicle unlocking device includes: The acquisition module is used to acquire the user's current location when it receives a bicycle usage instruction sent by the user; The detection module is used to guide the user's location based on the current location and to detect the distance between the user and the bicycle parking area in real time. The acquisition module is further configured to acquire the user's gaze information based on the target bicycle area when the distance is less than the first distance threshold; The determination module is used to determine the initial relative position of the bicycle based on the gaze information and the target bicycle area; The unlocking module is used to match bicycles to determine the target bicycle based on the relative position and the preset bicycle map, and to unlock the target bicycle. The acquisition module is also used for: When the distance is less than a first distance threshold, multiple parked bicycles within the user's field of vision are identified; Determine the relative position of each parked bicycle to the user; Based on the relative position between each parked bicycle and the user and the user's current position, multiple parked bicycles are locked, and multiple alternative bicycles are determined from the multiple parked bicycles; Determine the target bicycle area based on the bicycle area corresponding to each candidate bicycle; When the distance is less than the second distance threshold, the user's gaze information is obtained based on the target bicycle area; The determining module is further configured to: Based on the gaze information, determine the user's multiple gaze points within the target bicycle area and the cumulative number of gazes at each gaze point; The cumulative fixation counts for each fixation point are sorted to obtain the sorting results; Based on the sorting results, an initial bicycle is determined within the target bicycle area; Calculate the relative position between the user and the initial bicycle.

9. A bicycle unlocking device based on an AR system, characterized in that, The device includes: a memory, a processor, and an AR-based vehicle unlocking program stored in the memory and executable on the processor, the AR-based vehicle unlocking program being configured to implement the AR-based vehicle unlocking method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a bicycle unlocking program based on an AR system, which, when executed by a processor, implements the bicycle unlocking method based on an AR system as described in any one of claims 1 to 7.

Images