A system for charting an obstacle avoidance navigation path and a method thereof

Abstract

Method comprising receiving, by a computing device communicably coupled with a vehicle, a first set of data packets from a first device which pertain to a target location. The computing device determi

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an improved, accurate, and efficient solution to enable environment awareness to a self-driving vehicle on an area of interest and an obstacle avoidance navigation and guidance system that is enabled. BACKGROUND

[0002] The background description includes information that may be useful in understanding the present invention. It is not an admission that any of the information provided herein is prior art or relevant to the presently claimed invention, or that any publication specifically or implicitly referenced is prior art.

[0003] Self-driving vehicles, or fully / semi- autonomous vehicles are being rapidly introduced to service. Such vehicles may be any, such as drones, cars, boats, trucks, etc. advances in path planning and tracking have encouraged rapid deployment of such vehicles. With the advancement of technology, motorized self-driven vehicles were introduced, which can be controlled by a remote connected through a network so that the users don't need to push / pull the self-driven vehicle around. With just a few presses of a button, the user is able to bring the self-driven vehicle to the required location. Recently, many self-driven vehicles were introduced, which can follow the user without any user interaction.

[0004] The existing remote connected through a network-controlled self-driven vehicles involve a straightforward way of finding the global position of the user using a global positioning system (GPS) so that the self-driven vehicle can automatically navigate to the position of the user. However, one cannot rely only on the existing methods of reinforcement learning as for example, if you were to deploy a robot that was reliant on reinforcement learning to navigate a complex physical environment, it would seek new states and take different actions as it moves. It is difficult to consistently take the best actions in a real-world environment, however, because of how frequently the environment changes.

[0005] The existing navigation method accurately tracks and follows a user who actuates the self-driven vehicle. However, the existing method does not take into account the immediate surroundings and obstacles and the immediate ambience changes around the self-driven vehicle.

[0006] There is, therefore, a need to overcome the drawbacks, shortcomings, and limitations associated with existing navigation and path plotting engines for self-driving vehicle, by providing an improved, accurate, and efficient solution to enable environment awareness and path planning for the self-driving vehicle that helps to navigate the best possible route during automated tracking of user and navigation of self-driving vehicle to the user in difficult real-world conditions and also when the connection between the self-driving vehicle and remote connected through a network of the user is interrupted. SUMMARY

[0007] The present disclosure relates to an improved, accurate, and efficient solution to enable environment awareness to a self-driving vehicle on an area of interest and an obstacle avoidance navigation and guidance system that is enabled.

[0008] In a first aspect, the present disclosure provides a method for determining a path from current location to a target location for a vehicle. The method includes receiving, by a computing device communicably coupled with the vehicle, a first set of data packets from a first device, wherein the first set of data packets pertain to the target location. The method further includes determining, by the computing device, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) including the current location and target location of the vehicle. The second set of data packets pertain to geographical features of the AOI. The method further includes determining, by the computing device, a reward parameter corresponding to the AOI including the current location and the target location of the vehicle, the reward parameter based, at least in part, on an execution of a pre-determined reward function. The pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters. The method further includes determining, by the computing device, an optimized path from the plurality of potential paths based on the determined reward parameter. The optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter.

[0009] In some embodiments, the method further includes receiving, by the computing device, a real time geo-location of the vehicle. The real time geo-location corresponds to the current location of the vehicle.

[0010] In some embodiments, the second set of data packets is received, by the computing device, responsive to the receipt of the current location of the vehicle.

[0011] In some embodiments, the computing device is configured to determine the map of the AOI based on stored map data of the current location of the vehicle and data obtained from one or more sensors provided on the vehicle. The one or more sensors are configured to detect presence of objects in the AOI.

[0012] In some embodiments, the first set of data packets further include an instruction to the vehicle to begin movement from the current location to the target location.

[0013] In some embodiments, the first device is associated with a user of the vehicle.

[0014] In some embodiments, the first device transmits the first set of data packets to the computing device. Responsive to receipt of the first set of data packets and the second set of data packets, the computing device is configured to generate a grid map associated with the AOI based on the extracted set of images and the received current location from the vehicle. The grid map has a plurality of sub-grids, said grid map being transmitted to the vehicle and said hyperparameters being tuned for each of the plurality of sub-grids within the grid map by the vehicle.

[0015] In some embodiments, the set of hyperparameters includes a learning rate and an exploration rate. The hyperparameters are dynamically assigned optimal values by trial and error such that, on execution of the predefined reward function, the reward parameters are within a pre-defmed range.

[0016] In some embodiments, determining, by the computing device, the reward parameter includes defining, by the computing device, a state space ‘s’. The state space being represented as a n*n-dimensional matrix corresponding to the received grid map such that the state space is defined for each of the received plurality of sub-grids. Determining, by the computing device, the reward parameter further includes initializing, computing device, the learning rate for each of the plurality of sub-grids in the state space. Determining, by the computing device, the reward parameter further includes initializing, computing device, the exploration-rate for each of the plurality of blocks in the state space. Determining, by the computing device, the reward parameter further includes defining, by the computing device, an action space ‘a’ for each of the plurality of blocks as one of eight orthogonal directions. Determining, by the computing device, the reward parameter further includes determining, by the computing device, a reward function based on an obstacle avoidance rule Rav (s, a) with ‘s’ representing the defined space state and ‘a’ representing the defined action space. Determining, by the computing device, the reward parameter further includes calculating, by the computing device, a reward function for each of the plurality of sub-grids to generate a corresponding reward matrix for the n*n dimensional matrix.

[0017] In some embodiments, magnitude of n for the n*n dimensional matrix is determined based on size of the received grid and the plurality of sub-grids within the received grid.

[0018] In some embodiments, the orthogonal directions for the action space ‘a’ includes one or more of an up direction, a down direction, a left direction, a right direction, an up-right direction, an up-left direction, a down-right direction, and a down-left direction.

[0019] In some embodiments, the optimized path is determined by identifying and acquiring, by the computing device, a geo-location of a hazard or an obstacle; comparing, by the computing device, the user location with the geo-location of the hazard or the obstacle and on the user geo-location being determined outside the geo-location of the hazard or the obstacle, executing the follow command; generating, by the computing device, the n*n-dimensional state matrix based at least on the current location, the target location of the vehicle and the geo-location of the hazard or the obstacle; obtaining, by the computing device, the reward matrix by calculating the corresponding reward for each of the plurality of blocks in the n*n-dimensional state matrix; traversing, by the computing device, the action space by sequentially selecting actions to execute, and performing a one-step prediction from a current state to a predicted next state Q (s, a); initializing, by the computing device, the predicted next state Q (s, a) to obtain the reward of the next state, and calculating a total reward value for the predicted state by selecting the action with maximum magnitude of the reward value; and repeating, by the computing device, the one-step prediction from a current state to a predicted next state Q (s, a) for n iterations to obtain the path, wherein the reward values are updated dynamically in the generated reward matrix until the vehicle traverses from the current location to the target location.

[0020] In some embodiments, the one-step prediction is based on pre-determined criteria, said pre-determined criteria including any or a combination of avoiding the hazard and travelling to the target location by traversing shortest distance, plotting the shortest obstacle avoidance and navigation path with minimal run time, and plotting the shortest path with lowest necessary runtime.

[0021] In some embodiments, the shortest obstacle avoidance and navigation path is updated dynamically in real-time during the motion of the vehicle from the current location to the dynamic location.

[0022] In some embodiments, traversing the action space and performing the one step prediction includes: selecting the best action for the vehicle based on the pre-defined hyperparameters, the action being selected such that, upon the learning rate value being closer to 0, physical distance to be traversed by the vehicle is longer and upon the learning rate value closer to 1, the physical distance to be traversed by the vehicle is shorter; and upon the exploration rate value being closer to 1, the space to be traversed by the vehicle is wider.

[0023] In some embodiments, the on-site execution stage further includes calculating an objective function, said objective function being a sum of an inverse of the reward, 0.8 times an average computation time and 0.2 times an average computational cost.

[0024] In some embodiments, the average computation time is total time taken in predicting next state Q (s, a) and inputting predicted next state Q (s, a) and, the average computational cost is a quantification of a resource utilization in predicting next state Q (s, a) and inputting predicted next state Q (s, a) into a pre-defined Q-policy.

[0025] In some embodiments, the method further includes sorting, by an avoidance and path navigation module, the received values of n*n dimensional state and selecting the values of n*n dimensional state with the lowest values. The method further includes updating circumstances of the learning rate and the exploration rate based on the selected lowest values to determine the sub-grid with greater reward parameter. The method further includes avoiding the identified geo-location of the hazard by the obstacle avoidance rule, wherein the obstacle avoidance rule enables rewarding the vehicle based on whether the vehicle arrives at the target location, or arrives within a pre-defined threshold distance from the location of the hazard.

[0026] In some embodiments, selecting the state space ‘s’ further includes receiving environmental information using one or more GPS sensors or pre-stored images that are stored on the second device.

[0027] In a second aspect, the present disclosure provides a system for determining a path from current location to a target location for a vehicle. The system includes a computing device operatively coupled to the vehicle, the computing device including a processor and a memory communicably coupled with the processor, the memory storing instructions executable by the processor. The computing device is configured to receive a first set of data packets from a first device, wherein the first set of data packets pertain to the target location. The computing device is further configured to determine, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) comprising the current location and target location of the vehicle, wherein the second set of data packets pertain to geographical features of the AOI. The computing device is further configured to determine a reward parameter corresponding to the AOI comprising the current location and the target location of the vehicle, the reward parameter based ,at least in part, on an execution of a pre-determined reward function, wherein the pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters. The computing device is further configured to determine an optimized path from the plurality of potential paths based on the determined reward parameter. The optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0029] In the drawings, similar components and / or features may have the same reference label. Further, various components of the same type may be distinguished by following the reference label with a second label that distinguishes among the similar components. If only the first reference label is used in the specification, the description is applicable to any one of the similar components having the same first reference label irrespective of the second reference label.

[0030] FIG. 1 illustrates an exemplary block diagram of the proposed system comprising the network (remote connected through a network carried by the user) in communication with the self-driven device (self-driven vehicle) connected to a cloud server through a network according to an embodiment of the invention.

[0031] FIG. 2 illustrates an exemplary overview of the processing engine of the proposed system according to an embodiment of the invention.

[0032] FIG. 3 illustrates an exemplary overview of the various engines being configured on the self-driven vehicle of the proposed system according to an embodiment of the invention.

[0033] FIGs. 4A and 4B are graphs illustrating the variance of the cost function relative to the resource function as assigned by the self-driven vehicle according to an embodiment of the invention.

[0034] FIG. 5 illustrates an exemplary computer system to implement the proposed system in accordance with embodiments of the present disclosure.

[0035] FIG. 6 is a flowchart illustrating the sequence of steps of the proposed method for path planning by grid-based navigation and avoidance in an embodiment of the present invention.

[0036] FIG. 7A is an image that is transmitted to the self-driven vehicle by the server in an embodiment of the present disclosure.

[0037] FIG. 7B illustrates the image after a gridding operation by the gridding engine of the proposed system in an embodiment of the present disclosure.

[0038] FIG. 7C illustrates a sub-gridding of the gridded image by the self-driven vehicle of the proposed system in an embodiment of the present disclosure.

[0039] FIG. 8 A illustrates a position of the self-driven vehicle computed positions for the self-driven vehicle to move to as computed by the method of the proposed system.

[0040] FIG. 8B illustrates the number of steps in the navigation path for the self-driven vehicle as computed by the method of the proposed system.

[0041] FIG. 8C illustrates the number of steps for higher learning rate during hyperparameter tuning to compute positions for the self-driven vehicle to move to by the method of the proposed system.

[0042] FIG. 8D illustrates the number of steps for a lower learning rate during hyperparameter tuning to compute positions for the self-driven vehicle to move to by the method of the proposed system.

[0043] FIG. 9A illustrates the number of positions available for the self-driven vehicle to move exploration rate close to zero

[0044] FIG. 9B illustrates the number of positions available for the exploration rate close to 1 during hyperarameter tuning to compute positions for the self-driven vehicle to move in an embodiment of the proposed method.

[0045] FIG. 9C illustrates the number of steps available for the exploration rate close to 0 during hyperarameter tuning in an embodiment of the proposed method.

[0046] FIG. 9D illustrates the number of steps available for the exploration rate close to 1 during hyperarameter tuning in an embodiment of the proposed method.

[0047] FIG. 10A is a satellite image of the steps for the self-driven vehicle in conformance with the reward matrix.

[0048] FIG. 10B is an exemplary reward matrix corresponding to the most preferred step and the least preferred step as computed by the proposed method.

[0049] FIG. 10C is an exemplary reward function matrix illustrating the path calculation based on highest reward parameter values. DETAILED DESCRIPTION

[0050] The following is a detailed description of embodiments of the disclosure depicted in the accompanying drawings. The embodiments are in such detail as to clearly communicate the disclosure. However, the amount of detail offered is not intended to limit the anticipated variations of embodiments; on the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.

[0051] In the specification, reference may be made to the spatial relationships between various components and to the spatial orientation of various aspects of components as the devices are depicted in the attached drawings. However, as will be recognized by those skilled in the art after a complete reading of the present application, the devices, members, devices, etc. described herein may be positioned in any desired orientation. Thus, the use of terms such as “above,” “below,” “upper,” “lower,” “first”, “second” or other like terms to describe a spatial relationship between various components or to describe the spatial orientation of aspects of such components should be understood to describe a relative relationship between the components or a spatial orientation of aspects of such components, respectively, as the device described herein may be oriented in any desired direction.

[0052] In a first aspect, the present disclosure provides a method for determining a path from current location to a target location for a vehicle. The method includes receiving, by a computing device communicably coupled with the vehicle, a first set of data packets from a first device, wherein the first set of data packets pertain to the target location. The method further includes determining, by the computing device, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) including the current location and target location of the vehicle. The second set of data packets pertain to geographical features of the AOI. The method further includes determining, by the computing device, a reward parameter corresponding to the AOI including the current location and the target location of the vehicle, the reward parameter based, at least in part, on an execution of a pre-determined reward function. The pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters. The method further includes determining, by the computing device, an optimized path from the plurality of potential paths based on the determined reward parameter. The optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter.

[0053] In a second aspect, the present disclosure provides a system for determining a path from current location to a target location for a vehicle. The system includes a computing device operatively coupled to the vehicle, the computing device including a processor and a memory communicably coupled with the processor, the memory storing instructions executable by the processor. The computing device is configured to receive a first set of data packets from a first device, wherein the first set of data packets pertain to the target location. The computing device is further configured to determine, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) comprising the current location and target location of the vehicle, wherein the second set of data packets pertain to geographical features of the AOI. The computing device is further configured to determine a reward parameter corresponding to the AOI comprising the current location and the target location of the vehicle, the reward parameter based ,at least in part, on an execution of a pre-determined reward function, wherein the pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters. The computing device is further configured to determine an optimized path from the plurality of potential paths based on the determined reward parameter. The optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter.

[0054] FIG. 1 illustrates an exemplary block diagram 100 of the proposed system 102 for tracking a user 108 and navigating a self-driving self-driven vehicle 104 (or vehicle 104) to the user 108. The system 100 includes a remote connected through a network 106 that can be carried by the user 108. The remote connected through a network 106 is configured to communicatively interact with the self-driven vehicle 104 (also referred to as self-driven vehicle 104, herein) to determine a distance between them, which enables the self-driven vehicle 104 of the system to determine the exact positions of the remote connected through a network 106 / user 108 and self-driven vehicle 104 in an area of interest (AOI) and further enables the self-driven vehicle 104 to track and navigate to the remote connected through a network 106 / user 108. In an exemplary embodiment, the AOI may be a golf course, but not limited to the like, and the self-driven vehicle 104 may be a self-driving golf caddy (also referred to as the self-driven vehicle 104, herein) that is configured to carry golfing tools and accessories for the user 108.

[0055] In another exemplary embodiment, the AOI may be an airfield and the self-driven vehicle 104 may be an unmanned aerial vehicle (UAV). In yet another embodiment, the AOI may be a water body and the self-driven vehicle 104 may be an unmanned water vehicle (UWV). Those skilled in the art would appreciate that while various embodiments and figures of the present application herein have been elaborated in terms of the golf caddy / self-driven vehicle as the self-driven vehicle, and a golfer as the user 108 in a golf course as the AOI for sake of simplicity and easier explanation, however, the teachings of the present application is also equally implementable for UAVs, UWVs, self-driving vehicles, and the likes, and all such embodiments are well within the scope of the present application without any limitation.

[0056] FIG. 1 illustrates an exemplary block diagram of the proposed system comprising the self-driven vehicle 104 (remotely connected with a user 108 through a network 106 by a user device carried by the user) in communication with the self-driven device (self-driven vehicle) connected to a cloud server 112 through a network 106 according to an embodiment of the invention.

[0057] The self-driven vehicle 104 (self-driven vehicle) includes a global positioning system (GPS) module 104-1, an inertial measurement unit (IMU) 104-2, an altitude sensor 104-3, one or more communication units (CUs) or receiving means (for example, but not limited to CU-1, CL / 2, or CU-3, or a combination thereof), a mini-processor, and a display 104-C comprising a monitor 104-5. The CUs can be selected from any or a combination of ultra-wideband (UWB) modules, radio frequency (RF) modules, Bluetooth (BLE) modules, transceivers, image sensors / cameras, infrared (IR) sensors, ultrasonic sensors, time of flight (TOF) sensors, and other known wireless communication modules available in the art. Further, a data encoder 104-4 is configured on the self-driven vehicle 104 to decrypt the data captured by the sensors before transmitting the data to the server 102 or other self-driven vehicles 104, and a decoder is configured to decrypt the data received from the server 102 or multiple selfdriven vehicles. The self-driven vehicle 104 includes an inertial measurement unit 106-1, an altitude sensor 106-2, and one communication unit (CU-4). In an exemplary embodiment, one CU can be configured on the self-driven vehicle 104, which can communicate with the CU of the user remote connected through a network 106. In another exemplary embodiment, two CUs can be configured on the self-driven vehicle 104, wherein both the CUs of the self-driven vehicle 104 can communicate with the CU of the remote connected through a network 106 While various embodiments and figures of the present disclosure elaborate upon the use of three CUs (CU-1 to CU-3) on the self-driven vehicle just for the sake of exemplary explanation purpose, however, the number of CUs on the self-driven vehicle 104 can be kept one or two or more than three also, based on the type of the CU being employed in the system and as per the requirement, and all such embodiments are well within the scope of the present disclosure without any limitation.

[0058] System 100 includes a server 102 in communication with the self-driven vehicle or self-driven vehicle 104 through a network. Further, the self-driven vehicle 104 remains in communication with the remote connected through a network 106 using the CUs of the selfdriven vehicle 104 and remote connected through a network 106. The IMU 104-2 and GPS 104-1 of the self-driven vehicle 104 enable server 102 to determine and monitor the exact 2D position of the self-driven vehicle 104 in the AOI. The altitude sensor 104-3 further helps determine the altitude of the self-driven vehicle 104, which helps convert the determined 2D position of the self-driven vehicle 104 into a 3D position. Further, the interaction between the CU of the remote connected through a network 106 and the CUs of the self-driven vehicle 104, and the IMU 104-1 of the remote connected through a network 106 enables the self-driven vehicle 104 to determine a distance between the self-driven vehicle 104 and the remote connected through a network 106. The self-driven vehicle 104 then determines the exact 2D position of the remote connected through a network 106 / user 108 in the AOI based on the position of the self-driven vehicle 104 and the distance between the self-driven vehicle 104 and the remote connected through a network 106. Later, the altitude sensor 106-2 of the remote connected through a network 106 helps determine the altitude of the remote connected through a network 106 / user 108 and enables the self-driven vehicle 104 to convert the 2D position of the remote connected through a network 106 / user 108 into a 3D position. Accordingly, the selfdriven vehicle 104 is actuated to move forward and navigate to the identified 3D position of the remote connected through a network 106 / user 108 in the AOI, efficiently and accurately. In yet another embodiment, the self-driven vehicle 104 is also configured to follow the remote connected through a network 106 / user 108 in front of the remote connected through a network 106 / user 108, when the self-driven vehicle 104 is detected to be in front of the remote connected through a network 106 / user 108 in the AOI.

[0059] In an embodiment, the self-driven vehicle or vehicle 104 can include a controller configured to identify the global location of the self-driven vehicle or vehicle 104 based on the obtained satellite location data, and further determine a relative position of the network 106 to the self-driven vehicle or vehicle 104, based on at least one radio signal communicated between the CUs of the self-driven vehicle 104 and the network 106. The controller can determine the global location of the user 108 based on the global location of the self-driven vehicle or vehicle 104 and the position of the user 108 and can correspondingly provide a control signal to direct the self-driven vehicle or vehicle 104 towards the user 108 based on the determined global position of the self-driven vehicle 104.

[0060] It would be obvious for a person skilled in the art that one cannot rely only on GPS for determining the accurate position of the self-driven vehicle 104 because GPS is not accurate all the time specified in stationary points as in the case of the existing technologies. So, to attain an accurate global position, the IMU sensor is integrated into the proposed system 100 / self-driven vehicle 104 along with the GPS module 104. The IMU sensor is configured on a stable platform in the remote connected through a network 106 or the self-driven vehicle 104. The IMU sensor measures the acceleration, using a gyroscope, and magnetometer. The coupled GPS and IMU data enable sensor fusion, which processes the GPS and IMU data to stabilize the location, velocity, and acceleration for both the remote connected through a network 106 and self-driven vehicle 104. As the position of both the self-driven vehicle 104 and GPS is needed in the proposed system 100 for efficient and accurate tracking and navigation, however, a GPS cannot be configured within the remote connected through a network 106 because of various technical reasons and hardware restrictions associated with GPS. For instance, as GPS provided on the remote connected through a network will remain ON in searching mode, the GPS will consume more electrical power, as a result, frequent charging of the remote connected through a network 106 will be required or a higher capacity battery will be required in the remote connected through a network which will make the remote connected through a network 106 bulkier and heavy, making it unpleasant for the user to carry the remote connected through a network 106. In addition, as GPS generally has an accuracy of up to 5 meters or so, as a result, GPS implemented in the remote connected through a network 106 may not be able to provide an accurate location of the remote connected through a network 106 to the self-driven vehicle 104, which will make the overall navigation process inaccurate. Further, GPS also fails to work properly in cloudy / rainy conditions and when the line of sight is disturbed due to vegetation coverage, due to the presence of unaware obstacles, or when the remote connected through a network remains in the pocket of the user, thereby making it unreliable. To overcome this, the proposed system 100 determines the local position of the remote connected through a network 106 with respect to the self-driven vehicle 104 based on the interaction between the communication units of the remote connected through a network 106 and the self-driven vehicle 104. The remote may be communicably coupled to the network 106 via being communicably coupled to the vehicle 104. Further, by adding to the global position data of the self-driven vehicle 104, the global position of the remote connected through a network 106 is also determined.

[0061] In some embodiments, the vehicle 104 may further includes an encoder coupled to one or more wheels of the vehicle 104. The encoder may be configured to determine a speed of rotation of the wheels, as well and an angular motion of the wheels to provide a trajectory of the wheels of the vehicle. In combination with the GPS and IMU data, the encoder may enhance the accuracy of real-time location of the vehicle 104.

[0062] FIG. 2 illustrates an exemplary overview of the processing engine of the proposed system according to an embodiment of the invention. Specifically, FIG. 2 illustrates an exemplary overview of the processing engine at the cloud server of the proposed system according to an embodiment of the invention. The cloud server 112 may include a processor 202 operatively coupled with a memory 204, the memory 204 storing instructions which enable the operation of an IMU module 212 that receives real-time data from an IMU on the selfdriven vehicle. Further, the cloud server 112 may further comprise a GPS module 214 and a gridding engine 218. The GPS module 214 may be configured to track in real-time positions of the one or more self-driven vehicles and the user / user devices. The gridding engine 218 may be configured to generate a tile-based grid of a satellite image of an AOI. The satellite image may be captured in real-time or may be a pre-stored image that is divided into multiple grids and transmitted to the self-driven vehicle. A database 210 may be configured to store satellite images of a particular area to enable offline functionality of the system.

[0063] FIG. 3 illustrates an exemplary overview of the various engines being configured on the self-driven vehicle of the proposed system according to an embodiment of the invention. Specifically, FIG. 3 illustrates an exemplary overview of the various engines being configured on the self-driven vehicle (self-driven mobile device 104) of the proposed system according to an embodiment of the invention. The self-driven vehicle 104 may be comprised of an edge computing module / AI engine 308 that receives data from the cloud server 112 in real time. The AI engine 308 of the self-driven vehicle 104 may be comprised of an obstacle avoidance engine 314, a path plotting engine 312, a navigation engine 316 and other engines 318 to receive and process data obtained from the cloud server 112.

[0064] IMU includes gyroscope and accelerometer sensors. Gyroscopes and accelerometers are motion-sensing devices that measure the rate of rotation (angular velocity) and linear acceleration, respectively. The velocity, position, and orientation of an object are tracked by integrating acceleration and angular velocity over time. IMU suffers error propagation in measurements. The accumulated error, which is known as drift, grows rapidly and makes the IMU output unreliable for navigation purposes. Thus, usually, IMU is fused with the GPS for improved results. Generally, the IMU needs calibration, which is done by rotating the sensor in all directions for accurate measurements. However, in the present implementation of the vehicle 104, the IMU may not require additional calibration. The IMU includes gyroscope and accelerometer sensors. Gyroscopes and accelerometers are motionsensing devices that measure the rate of rotation (angular velocity) and linear acceleration, respectively. The velocity, position, and orientation of an object are tracked by integrating acceleration and angular velocity over time. IMU suffers error propagation in measurements. The accumulated error, which is known as drift, grows rapidly and makes the IMU output unreliable for navigation purposes. Thus, usually, IMU is fused with the GPS for improved results. But, the IMU needs calibration, which is done by rotating the sensor in all directions for accurate measurements. Further, typically, the calibration needs to be repeated once the IMU is turned on again, which is time-consuming and hectic.

[0065] The proposed system may include a means to implement a self-calibration of the IMU. The steps include receiving raw data monitored by the IMUs of the vehicle 104 and the remote and correspondingly creating a rotation matrix. Further, Euler angles are extracted, which include roll, pitch, and yaw associated with the vehicle 104 and the remote, based on the rotation matrix. Furthermore, the extracted Euler angles are stabilized using known filtration techniques to provide stabilized values of roll, pitch, and yaw associated with the vehicle 104 and the remote, which facilitates the trolley and the remote in determining true north and yaw angle without manual calibration.

[0066] In an embodiment, the user device may transmit the first set of data packets to the cloud server 112, and wherein responsive to receipt of the first set of data packets and the second set of data packets, the cloud server 112 may be configured to generate a grid map associated with the AOI based on the extracted set of images and the received current location from the self-driven vehicle, the grid map having a plurality of sub-grids, said grid map being transmitted to the self-driven vehicle and said hyperparameters being tuned for each of the plurality of sub-grids within the grid map by the self-driven vehicle 104.

[0067] The method includes determining, by the self-driven vehicle 104, in runtime, a reward parameter corresponding to the second set of data packets, based at least in part on an execution of a pre-determined reward function, wherein the pre-defined reward function is executed based on a hyperparameter tuning of a set of hyperparameters. The set of hyperparameters may comprise a learning rate and an exploration rate dynamically assigned optimal values by trial and error such that the learning rate and exploration rate values are within a pre-defined range of 0 and 1.

[0068] The method then includes dynamically plotting, by the self-driven vehicle, the obstacle avoidance navigation path based on the hyper-parameter tuning for the self-driven vehicle 104 based on the determined reward parameter.

[0069] In an embodiment, the reward parameter may be calculated at the cloud server 112 or at an edge computing at the self-driven vehicle 104. The calculation of the reward parameter may include defining, by the self-driven vehicle, a state space ‘s’. The state space can be represented as a n*n-dimensional matrix corresponding to the received grid map such that the state space is defined for each of the received plurality of sub-grids. The self-driven vehicle is configured to initializing a value of learning rate for each of the plurality of sub-grids that are generated within the received image of the AOI in the state space. Further step involves initializing, by the self-driven vehicle, the exploration-rate for each of the plurality of blocks in the state space and defining an action space ‘a’, by the self-driven vehicle, for each of the plurality of blocks as one of eight orthogonal directions. The reward function is determined based on an obstacle avoidance rule Rav (s, a) with ‘s’ representing the defined space state and ‘a’ representing the defined action space. The reward parameter value is calculated for each of the plurality of sub-grids to generate a corresponding reward matrix for the n*n dimensional matrix.

[0070] In an embodiment, the magnitude of n for the n*n dimensional matrix is determined based on size of the received grid and the plurality of sub-grids within the received grid.

[0071] On calculation of the reward function, the obstacle avoidance navigation path is plotted in an on-site execution stage at the self-driven vehicle 104. The calculation of the reward function includes, identifying and acquiring, by the self-driven vehicle, a geo-location of a hazard or an obstacle. The location of the identified hazard or obstacle is then compared with the user location. On the user geo-location being determined outside the geo-location of the hazard or the obstacle, the self-driven vehicle is configured to execute a follow command. The follow command may include tracking the location of the user / golfer and reaching the user / golfer.

[0072] The self-driven vehicle then generates an n*n-dimensional state matrix based at least on the current location, the target location of the self-driven vehicle and the geo-location of the hazard or the obstacle. The reward matrix is obtained by calculating the corresponding reward for each of the plurality of blocks in the n*n-dimensional state matrix.

[0073] In the path planning case, the model tries to learn the next best directions from possible sets of {Up, Down, Left, Right, Up-Right, Up-left, Down-Right, Down-Left}. These actions must first avoid the hazard in their path and second choose the shortest path for reaching to the end. Reinforcement learning in Q learning needs to observe the environment which in an exemplary embodiment may be the golf course and with determination of start and end point it can choose the best path with testing different structure. In summary Q learning tries to predict expected future rewards to choose the best action which leads to the shortest path to the goal. To choose such an action, the proposed model has two hyper parameters which is defined as ‘learning rate’ and ‘exploration rate’. Learning rate is a hyper parameter that determines the action of the model to choose the next action. Learning rate range must be between 0 and 1. As much as the learning rate will get closer to the 0 it means that the steps toward reaching to the end will be much higher and the model tries to test more actions to choose best from them. If the learning rate gets closer to the 1 the model will try to reach the goal with lowest testing actions. Another hyperparameter which must be tuned here is the exploration rate. Exploration rate specifies the space that the agent of the model must explore to reach the end goal. This value must be between 0 and 1 too. As the value of this model gets closer to 1 the space which model can choose for its action will increase dramatically.

[0074] The result of more exploration will be a more zigzagging pattern from start point and end point. This process will make the patterns more drivers and strange to the observer. A sample of this work has been shown in FIGs. 8A-8D.

[0075] Tuning these hyperparameters is very important in Q-Learning. In an exemplary embodiment the tuning criteria may be defined as follows: 1 .Reaching to the end with the shortest path while avoiding the hazard. 2.Tacking the shortest path with run time of below 5 seconds. 3.Tacking the shortest path with lowest necessary runtime.

[0076] In an embodiment, to tackle these three criteria, a reward system may be presented to preserve the location of the hazard at lowest value possible, the goal will be the highest possible value and ordinary path will be rewarded as zero. Many values have been tested for target and hazard locations.

[0077] The reward matrix is calculated by the reward function. Changing this reward function will be based on the policy function. Q-Leaming works by watching an agent play and gradually improving its estimates of the Q-Values. Once it has accurate Q-Value estimates (or close enough), then the optimal policy is choosing the action that has the highest Q-Value. Q-Value is defined as a pair of state -action which tries. The Q-Value will be determined by estimated future value of reward times, a discount factor. Q-Leaming had considered the effect of rewards which were received earlier higher than those received later.

[0078] On calculation of the reward matrix, the self-driven vehicle is configured to traverse the action space by sequentially selecting actions to execute and performing a one-step prediction from a current state to a predicted next state Q (s, a). temporal difference Qnew(st,at) *- Q(st,at) + a • ( n + y : - Q(s,,dj) old value learning rate reward discount factor —-—p————' old value estimate of optimal future value new value {temporal different target) where rt is the reward received when moving from the state st to the state st+1, and a is the learning rate (0 <a <1). An episode of the algorithm ends when state st+1 is a final or terminal state. However, Q-learning can also learn in non-episodic tasks (as a result of the property of convergent infinite series). If the discount factor is lower than 1, the action values are finite even if the problem can contain infinite loops. For indicating better effect of the action, we banded the agent to get closer to the hazard by mentioning the opposite action when the agent wants to choose an action toward the hazard.

[0079] The method then comprises initializing the predicted next state Q (s, a) to obtain the reward of the next state, and calculating a total reward value for the predicted state by selecting the action with maximum magnitude of the reward value and repeating the one-step prediction from a current state to a predicted next state Q (s, a) for n iterations to obtain the path, wherein the reward values are updated dynamically in the generated reward matrix until the self-driven vehicle traverses from the current location to the target location.

[0080] The orthogonal directions for the action space ‘a’ comprises one or more of an Up direction, a Down direction, a Left direction, a Right direction, an Up-Right direction, an Up-left direction, a Down-Right direction, and a Down-Left direction.

[0081] After the path have been found by the RL algorithms, then because information about each step in square grid coordination is clear commands which direction and how much each step must be will send to IMU sensors. So IMU sensors without any prior information can follow proposed path until reach to the goal position.

[0082] The one-step prediction is based on pre-determined criteria, said pre-determined criteria including any or a combination of avoiding the hazard and travelling to the target location by traversing shortest distance, plotting the shortest obstacle avoidance and navigation path with minimal run time, and plotting the shortest path with lowest necessary runtime. Step (^(6371000)2 * |sm( n 180 sin lat2) * |Zoni - Ion2\ Where latl and lonl are latitude and longitude information of top left and top left location of the grid on real life. Lat2 and lon2 are information of lower right of the grid on real life and is total number of squares in the grid. Information about choosing best directions come from surveying 8 direction reward around position of the trolley. A simple of using such a structure has shown below that best direction for next step is Southeast. An exemplary list of positions for the next step are provided below. 0.82 0.9 0.9 0.84 Trolley 0.85 0.8 0.86 0.93

[0083] In the above scenario, the trolley shall move to the bottom right square as the highest next step prediction value is 0.93.

[0084] In an embodiment, the shortest obstacle avoidance and navigation path is updated dynamically in real-time during the motion of the self-driven vehicle from the current location to the dynamic location.

[0085] The step of traversing the action space and performing the one step prediction includes selecting the best action for the self-driven vehicle based on the pre-defined hyperparameters. Upon the learning rate value being closer to 0, physical distance to be traversed by the self-driven vehicle is longer and upon the learning rate value closer to 1, the physical distance to be traversed by the self-driven vehicle is shorter. Upon the exploration rate value being closer to 1, the space to be traversed by the self-driven vehicle is wider.

[0086] In an embodiment, the on-site execution stage may further comprise calculating an objective function. The objective function may be a sum of an inverse of the reward, 0.8 times an average computation time and 0.2 times an average computational cost. The average computation time is total time taken in predicting next state Q (s, a) and inputting predicted next state Q (s, a) and, the average computational cost is a quantification of a resource utilization in predicting next state Q (s, a) and inputting predicted next state Q (s, a) into a predefined Q-policy.

[0087] The method may further comprise sorting, by an avoidance and path navigation module, the received values of n*n dimensional state and selecting the values of n*n dimensional state with the lowest values and updating circumstances of the learning rate and the exploration rate based on the selected lowest values to determine the sub-grid with greater reward parameter. The values of the matrix with the greater reward parameter value are selected as the next step. The method includes avoiding the identified geo-location of the hazard by the obstacle avoidance rule, wherein the obstacle avoidance rule enables providing a positive reward to the self-driven vehicle on arriving at the target location and providing a negative reward to the self-driven vehicle on arriving within a pre-defined threshold distance from the location of the hazard.

[0088] In some embodiments, the vehicle may be configured to determine a shortest path between the origin and the destination. The vehicle may determine a way around existing obstacles by resuming the shortest path at an earliest point after deviating from the shortest path due to presence of the obstacle.

[0089] In some embodiments, the vehicle is further configured to determine one or more obstacles in its path in real time. Typically, maps available to the vehicle may be from available databases, and may not be complete. For instance, the map may not include later updates and location / position of obstacles. In another instance, the map may not include topographical data of the AOI. As a result, the vehicle may confront obstacles in its path.

[0090] In some embodiments, the vehicle is configured to detect one or more obstacles using one or more sensors provided on the vehicle. For example, the vehicle may include sensors, such as LiDAR, RADAR, stereo-multipurpose cameras (SMPC), etc. that may be configured to monitor the surroundings of the vehicle, as the vehicle is moving. In another example, the vehicle may utilize the IMU, and / or gyroscope to determine a gradient of the path that the vehicle is traversing. If the obstacle, or the gradient is prohibitive, the vehicle may be configured to determine the selected path containing the obstacle or the gradient as prohibited.

[0091] In some embodiments, the vehicle may assign a negative reward to the first path determined on which it detected the obstacles or the gradient. In some embodiments, the vehicle may determine that the first path may be a forbidden path for a foreseeable future.

[0092] The vehicle may utilize the existing map of the AOI to determine a next optimal path.

[0093] In an aspect, a system for identifying topography of an area of interest (AOI) and charting an obstacle avoidance navigation path is disclosed. The system comprises a self-driven vehicle in operative communication with a cloud server and a geo-network carried by a user, said cloud server comprising a processor in operative coupling with a memory storing instructions which when executed enables the processor to receive a real-time geo-location of the self-driven vehicle and correspondingly extract an image of the area of interest (AOI) surrounding the received real-time geo-location of the self-driven vehicle and generate a grid map of the AOI based on location metadata received from the self-driven vehicle. The processor is in operative coupling with an avoidance and path navigation module, said avoidance and path navigation module being configured to dynamically receive the grid map from the AI engine and, chart an obstacle avoidance navigation path for the self-driven vehicle based on a reinforcement learning framework pre-stored in the self-driven vehicle.

[0094] The reinforcement learning framework generates an obstacle avoidance navigation path for the self-driven vehicle based on a reward determined by a pre-stored Q-Learning policy.

[0095] In an embodiment, a first self-driven vehicle being communicatively coupled with one or more other self-driven vehicles such that the obstacle avoidance navigation path is stored at the cloud server for a given set of a current location and a target location of the first selfdriven vehicle, other self-driven vehicle is transmitted to the cloud server that is extracted for path planning for a subsequent second self-driven vehicle traversing on the given set of current location and the target location.

[0096] In an embodiment, the self-driven vehicle may further comprise an auto calibrated inertial measurement unit (IMU) comprising a gyroscope, and an accelerometer, the IMU being configured to monitor velocity, angular velocity, orientation, and linear acceleration of the selfdriven vehicle, a geo-network (GPS), wherein the IMU and GPS are configured to enable the cloud server to monitor a real-time 2D geo-location of the self-driven vehicle in the AOI, an altitude sensor configured to monitor altitude of the self-driven vehicle in the AOI, wherein the altitude sensor enables conversion of the 2D geo-location of the self-driven vehicle into a 3D geo-location based on the monitored altitude, and correspondingly facilitate the cloud server to determine and monitor the real-time 3D geo-location of the self-driven vehicle in the AOI and at least one communication unit configured at predetermined geo-locations on the self-driven vehicle. The data collected by the communication unit, and the altitude sensor of the selfdriving vehicle and the communication unit, and the altitude sensor of the geo-network are denoised and stabilized.

[0097] In an embodiment, the self-driven vehicle may be a self-driven caddy, with the user being a golfer, the AOI being a golf course such that a plurality of self-driven vehicles on the golf course are communicatively coupled to one another by the cloud server and to the users through one or more user devices or remote-controlled devices.

[0098] Referring to FIGs. 2, and 3, the path plotting engine 312 of the proposed system 100 involves any or a combination of time of flight (TOF) data, time of arrival (TOA) data, time difference of arrival (TODA) data or angle of arrival (AOA) data coming from the communication units of the self-driven vehicle 104 and remote connected through a network 106 at block 308. In another implementation, one CU can be configured on the remote connected through a network 106, and two communication units can be configured on the self- driven vehicle 104, wherein one BLE (first CU) is used for AOA data, and one UWB (second CU) is used for TOF data, which correspondingly enables the path plotting engine to determine the distance between the self-driven vehicle 104 and the remote connected through a network 106. In yet another implementation, a directional antenna (for example a patch antenna) having multiple Cus can be configured around the self-driven vehicle 104 such that a complete 360o coverage around the self-driven vehicle is achieved, and one CU can be configured on the remote connected through a network 106, which helps determine the AOA data, and correspondingly enables the path plotting engine to determine the distance between the selfdriven vehicle 104 and the remote connected through a network 106.

[0099] These communication unit outputs (AOA data and / or TOA data and / or TOF data) are generally noisy. The data (AOA)is further processed at block 204, which involves denoising the data using denoising techniques available in the art to clean the data. After this, the clean data is further processed to estimate the 2D position of the self-driven vehicle 104 in the AOI and determine the distance between the 104 self-driven vehicle and remote connected through a network 106. The resulted 2D position may again not be clean and may be noisy, so filtration is performed using known filters to stabilize the estimated position of the self-driven vehicle 104 and the remote connected through a network 106, which helps the self-driven vehicle 104 to determine the 2D location of the remote connected through a network 106 with respect to the self-driven vehicle 104. Accordingly, the wheels of the self-driven vehicle 104 can be controlled to track and navigate to the remote connected through a network 106, which is played in real-time. Thus, by adding the GPS and altitude sensor data of the remote connected through a network 106 and the self-driven vehicle 104, the 3D position of both the remote connected through a network 106 and self-driven vehicle 106 can be found.

[00100] Thus, system 100 uses communication units, altitude sensors, and IMUs to make a triangulation with the remote connected through a network 106 and the self-driven vehicle 104. The system 100 measures the AOA of the transmitting signal from the communication unit to determine the angle, local position, and global position of the remote connected through a network 106 / user 108. For instance, the resulting 2D position means the location in (x, y) which can be local or global. For local positioning, denoising, filtration, and stabilization of sensor data can be used, but for the global position, GPS 104 and IMU 104 are used. In the path plotting engine, one of the Cus on self-driven vehicle 104 is considered the origin of the local position system. And other Cus have a relative (x, y) which is constant and known.

[00101] The motion of the self-driven vehicle and remote connected through a network has rotation angles with respect to the Cartesian system. The angles include Roll, Pitch, and Yaw. The roll is around the X-axis, the pitch is around the Y-axis, and the yaw is around the Z-axis. These angles are named Euler angles herein. The yaw is interpreted as the rotation in left and right which is very important. The yaw helps find the heading of the user and self-driven vehicle for better tracking. The pitch is interpreted as the rotation of the user and self-driven vehicle up and down. Both roll and pitch help to understand the slope of the ground in the AOI for better wheel slip management and tracking.

[00102] The path plotting engine of the proposed system 100 is implemented by denoising techniques and filters, and communication units like UWB, RF, BLE, image sensors / cameras, IR sensors, ultrasonic sensors, time of flight (TOF) sensors, and the like, to determine the user position in 2D space, and an altitude sensor to determine the location in the 3D space building on the determined position on 2D space. Even though this communication unit-based tracking gives very accurate tracking of the user 108, its accuracy is limited to a range of less than 10-12 meters.

[00103] If and when the distance between remote connected through a network 106 and selfdriven vehicle 104 is greater than a predefined distance (for example, but not limited to 10 meters) or when communication between the CU of the remote connected through a network 106 and the CU(s) of the self-driven vehicle 104 is interrupted, the path plotting engine starts fluctuating, which leads to zig-zag movement of the self-driven vehicle 104 while following forward and navigating to the remote connected through a network 106 / user 108, which is not ideal and highly undesirable.

[00104] The self-driven vehicle 104 always tries to track the remote connected through a network 106 carried by the user 108. However, there are some positions where the distance between the self-driven vehicle and remote connected through a network is more than the predefined distance for example, but not limited to 10m, and because of hardware restrictions the path plotting engine can’t find the direction of the remote connected through a network 106 correctly to follow. Additionally, this may happen when the distance between the remote connected through a network 106 and self-driven vehicle 104 is long and the remote connected through a network turns ON, then the self-driven vehicle 104 cannot find the direction of the remote connected through a network 106 or may happen in other chances where the self-driven vehicle 104 starts zigzag movement irrespective of distance. To overcome this zigzag movement, the proposed system 100 involves a grid-based tracking technique involving a gridding engine configured with server 102 and self-driven vehicle 104. Depending on the distance between the self-driven vehicle 104 and the remote connected through a network 106 / user 108, the grid-based tracking repeats itself every predetermined distance for example but is not limited to 2-meters, which means the self-driven vehicle 104 course corrects itself every 2 meters till the distance between the self-driven vehicle 104 and the user 108 is less than the predefined distance for example, but not limited to 10m. In that case, the self-driven vehicle 104 stops following the user 108 using the grid-based tracking and starts following the user 108 with the path plotting engine which solely uses the communication units till the user 108 stops driven or the distance between the user 108 and self-driven vehicle 104 exceeds the predefined distance (example 10 meters) again.

[00105] In an exemplary embodiment, when the distance between the self-driven vehicle 104 and user 108 is greater than the predefined distance for example, but not limited to 10 meters, the approximate position of the remote connected through a network 106 / user 108 and the self-driven vehicle 104 is sent to the server. The server 102 can also include important information about hazards and paths and other important information in these grids and provides it back in the scale of a first predefined resolution, in order to reduce the data transfer limitations and reduce latency. Then this first predefined resolution grid information is sent back to the self-driven vehicle 104. Further, during the navigation of the self-driven vehicle 104 towards the user 108, as the distance between the self-driven vehicle 104 and remote connected through a network 106 / user 108 changes, the self-driven vehicle 104 then subdivides the grid information further to increase to a second predefined resolution of the grids, which can be dynamic, to achieve very precise tight travel paths.

[00106] FIGs. 4A and 4B are graphs illustrating the variance of the cost function relative to the resource function as assigned by the self-driven vehicle according to an embodiment of the invention. Referring to FIG. 4A, selection of values for the hyperparameters are disclosed wherein the values for each hyperparameter (learning rate and exploration rate) are selected dynamically as a dependent of the cost function. FIG. 4C is a 3-D representation of the selection of the values for the hyperparameter values for each grid obtained within the satellite image of the AOI.

[00107] FIG. 5 illustrates an exemplary computer system to implement the proposed system in accordance with embodiments of the present disclosure.

[00108] As shown in FIG. 5, computer system can include an external storage device 510, a bus 520, a main memory 530, a read only memory 540, a mass storage device 550, communication port 550, and a processor 570. A person skilled in the art will appreciate that computer system may include more than one processor and communication ports. Examples of processor 570 include, but are not limited to, an Intel® Itanium® or Itanium 2 processor!s), or AMD® Opteron® or Athlon MP® processor(s), Motorola® lines of processors, FortiSOC™ system on a chip processors or other future processors. Processor 570 may include various modules associated with embodiments of the present invention. Communication port 550 can be any of an RS-232 port for use with a modem-based dialup connection, a 10 / 100 Ethernet port, a Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or other existing or future ports. Communication port 550 may be chosen depending on a network, such a Local Area Network (LAN), Wide Area Network (WAN), or any network to which computer system connects.

[00109] Memory 530 can be Random Access Memory (RAM), or any other dynamic storage device commonly known in the art. Read only memory 540 can be any static storage device(s) e.g., but not limited to, a Programmable Read Only Memory (PROM) chips for storing static information e.g., start-up or BIOS instructions for processor 570. Mass storage 550 may be any current or future mass storage solution, which can be used to store information and / or instructions. Exemplary mass storage solutions include, but are not limited to, Parallel Advanced Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and / or Firewire interfaces), e.g. those available from Seagate (e.g., the Seagate Barracuda 7102 family) or Hitachi (e.g., the Hitachi Deskstar 7K1000), one or more optical discs, Redundant Array of Independent Disks (RAID) storage, e.g. an array of disks (e.g., SATA arrays), available from various vendors including Dot Hill Systems Corp., LaCie, Nexsan Technologies, Inc. and Enhance Technology, Inc.

[00110] Bus 520 communicatively couples processor(s) 570 with the other memory, storage, and communication blocks. Bus 520 can be, e.g., a Peripheral Component Interconnect (PCI) / PCI Extended (PCI-X) bus, Small Computer System Interface (SCSI), USB or the like, for connecting expansion cards, drives and other subsystems as well as other buses, such a front side bus (FSB), which connects processor 570 to software system.

[00111] Optionally, operator and administrative interfaces, e.g., a display, keyboard, and a cursor control device, may also be coupled to bus 520 to support direct operator interaction with computer system. Other operator and administrative interfaces can be provided through network connections connected through communication port 550. External storage device 510 can be any kind of external hard-drives, floppy drives, IOMEGA® Zip Drives, Compact Disc - Read Only Memory (CD-ROM), Compact Disc - Re-Writable (CD-RW), Digital Video Disk - Read Only Memory (DVD-ROM). Components described above are meant only to exemplify various possibilities. In no way should the aforementioned exemplary computer system limit the scope of the present disclosure.

[00112] FIG. 6 is a flowchart illustrating the sequence of steps of the proposed method for path planning by grid-based navigation and avoidance in an embodiment of the present invention.

[00113] FIG. 7A is an image that is transmitted to the self-driven vehicle by the server in an embodiment of the present disclosure. As can be seen, a satellite image of the golf-course has been obtained. The gridding engine at the cloud server then generates a grid for the obtained image and identifies the presence of users / golfers on the golf-course as well as the self-driven vehicles corresponding to each golfer. FIG. 7B illustrates the image after a gridding operation by the gridding engine of the proposed system in an embodiment of the present disclosure. FIG. 7C illustrates a sub-gridding of the gridded image by the self-driven vehicle of the proposed system in an embodiment of the present disclosure.

[00114] FIG. 8A illustrates a position of the self-driven vehicle computed positions for the self-driven vehicle to move to as computed by the method of the proposed system.

[00115] FIG. 8B illustrates the number of steps in the navigation path for the self-driven vehicle as computed by the method of the proposed system.

[00116] FIG. 8C illustrates the number of steps for higher learning rate during hyperparameter tuning to compute positions for the self-driven vehicle to move to by the method of the proposed system.

[00117] FIG. 8D illustrates the number of steps for a lower learning rate during hyperparameter tuning to compute positions for the self-driven vehicle to move to by the method of the proposed system.

[00118] FIG. 9A illustrates the number of positions available for the self-driven vehicle to move exploration rate close to zero

[00119] FIG. 9B illustrates the number of positions available for the exploration rate close to 1 during hyperarameter tuning to compute positions for the self-driven vehicle to move in an embodiment of the proposed method.

[00120] FIG. 9C illustrates the number of steps available for the exploration rate close to 0 during hyperarameter tuning in an embodiment of the proposed method.

[00121] FIG. 9D illustrates the number of steps available for the exploration rate close to 1 during hyperarameter tuning in an embodiment of the proposed method.

[00122] FIG. 10A is a satellite image of the steps for the self-driven vehicle in conformance with the reward matrix.

[00123] FIG. 10B is an exemplary reward matrix corresponding to the most preferred step and the least preferred step as computed by the proposed method.

[00124] FIG. 10C is an exemplary reward function matrix illustrating the path calculation based on highest reward parameter values.

[00125] Thus, the present invention (proposed system and method) overcomes the drawbacks, shortcomings, and limitations associated with existing navigation and path plotting engines for self-driving self-driven vehicles, by providing an improved, accurate, and efficient solution to enable automated tracking of golfer (user) and navigation of self-driving caddy (similar unmanned self-driving devices) to the golfer (user) in difficult real-world conditions while ensuring surrounding awareness for the self-driven vehicle.

[00126] In one implementation, a network can be a wireless network, a wired network, or a combination thereof. Network can be implemented as one of the different types of networks, such as an intranet, local area network (LAN), wide area network (WAN), the internet, and the like. Further, the network may either be a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol / Internet Protocol (TCP / IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further, network can include a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like. In another implementation the network can be a cellular network or mobile communication network based on various technologies, including but not limited to, Global System for Mobile (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Long Term Evolution (LTE), WiMAX, and the like.

[00127] Various terms are used herein. To the extent a term used in a claim is not defined below, it should be given the broadest definition persons in the pertinent art have given that term as reflected in printed publications and issued patents at the time of filing.

[00128] As used in the description herein and throughout the claims that follow, the meaning of “a,” “an,” and “the” includes plural reference unless the context clearly dictates otherwise. Also, as used in the description herein, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise. The recitation of ranges of values herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within the range. Unless otherwise indicated herein, each individual value is incorporated into the specification as if it were individually recited herein.

[00129] All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided with respect to certain embodiments herein is intended merely to better illuminate the invention and does not pose a limitation on the scope of the invention otherwise claimed. No language in the specification should be construed as indicating any non-claimed element essential to the practice of the invention.

[00130] Groupings of alternative elements or embodiments of the invention disclosed herein are not to be construed as limitations. Each group member can be referred to and claimed individually or in any combination with other members of the group or other elements found herein. One or more members of a group can be included in, or deleted from, a group for reasons of convenience and / or patentability. When any such inclusion or deletion occurs, the specification is herein deemed to contain the group as modified thus fulfilling the written description of all groups used in the appended claims.

[00131] Embodiments of the present invention include various steps, which will be described below. The steps may be performed by hardware components or may be embodied in machine-executable instructions, which may be used to cause a general-purpose or specialpurpose processor programmed with the instructions to perform the steps. Alternatively, steps may be performed by a combination of hardware, software, firmware, and / or by human operators.

[00132] Embodiments of the present invention may be provided as a computer program product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process. The machine-readable medium may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media / machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).

[00133] Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to the present invention with appropriate standard computer hardware to execute the code contained therein. An apparatus for practicing various embodiments of the present invention may involve one or more computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the invention could be accomplished by modules, routines, subroutines, or subparts of a computer program product.

[00134] In interpreting the specification, all terms should be interpreted in the broadest possible manner consistent with the context. In particular, the terms “comprise” and “comprising” should be interpreted as referring to elements, components, or steps in a nonexclusive manner, indicating that the referenced elements, components, or steps may be present, or utilized, or combined with other elements, components, or steps that are not expressly referenced. Where the specification claims refer to at least one of something selected from the group consisting of A, B, C ... .and N, the text should be interpreted as requiring only one element from the group, not A plus N, or B plus N, etc.

[00135] While the foregoing describes various embodiments of the invention, other and further embodiments of the invention may be devised without departing from the basic scope thereof. The scope of the invention is determined by the claims that follow. The invention is not limited to the described embodiments, versions, or examples, which are included to enable a person having ordinary skill in the art to make and use the invention when combined with information and knowledge available to the person having ordinary skill in the art.

Claims

1. A method for determining a path from current location to a target location for a vehicle, the method comprising:receiving, by a computing device communicably coupled with the vehicle, a first set of data packets from a first device, wherein the first set of data packets pertain to the target location;determining, by the computing device, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) comprising the current location and target location of the vehicle, wherein the second set of data packets pertain to geographical features of the AOI;determining, by the computing device, a reward parameter corresponding to the AOI comprising the current location and the target location of the vehicle, the reward parameter based, at least in part, on an execution of a pre-determined reward function, wherein the pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters; anddetermining, by the computing device, an optimized path from the plurality of potential paths based on the determined reward parameter,wherein the optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter.

2. The method as set forth in claim 1 further comprising: receiving, by the computing device, a real time geo-location of the vehicle, and wherein the real time geo-location corresponds to the current location of the vehicle.

3. The method as set forth in claim 2, wherein the second set of data packets is received, by the computing device, responsive to the receipt of the current location of the vehicle.

4. The method as set forth in claim 1, wherein the computing device is configured to determine the map of the AOI based on stored map data of the current location of the vehicle and data obtained from one or more sensors provided on the vehicle, and wherein the one or more sensors are configured to detect presence of objects in the AOI.

5. The method as put forth in claim 1, wherein the first set of data packets further comprise an instruction to the vehicle to begin movement from the current location to the target location.

6. The method as put forth in claim 5, wherein the first device is associated with a user of the vehicle.

7. The method as set forth in claim 4, wherein the first device transmits the first set of data packets to the computing device, and wherein responsive to receipt of the first set of data packets and the second set of data packets, the computing device is configured to generate a grid map associated with the AOI based on the extracted set of images and the received current location from the vehicle, the grid map having a plurality of subgrids, said grid map being transmitted to the vehicle and said hyperparameters being tuned for each of the plurality of sub-grids within the grid map by the vehicle.

8. The method as set forth in claim 7, wherein the set of hyperparameters comprises a learning rate and an exploration rate, wherein the hyperparameters are dynamically assigned optimal values by trial and error such that, on execution of the predefined reward function, the reward parameters are within a pre-defined range.

9. The method as set forth in claim 1, wherein determining, by the computing device, the reward parameter comprises:defining, by the computing device, a state space ‘s’, said state space being represented as a n*n-dimensional matrix corresponding to the received grid map such that the state space is defined for each of the received plurality of sub-grids;initializing, computing device, the learning rate for each of the plurality of subgrids in the state space;initializing, computing device, the exploration-rate for each of the plurality of blocks in the state space;defining, by the computing device, an action space ‘a’ for each of the plurality of blocks as one of eight orthogonal directions;determining, by the computing device, a reward function based on an obstacle avoidance rule RaV (s, a) with ‘s’ representing the defined space state and ‘a’ representing the defined action space; andcalculating, by the computing device, a reward function for each of the plurality of sub-grids to generate a corresponding reward matrix for the n*n dimensional matrix.

10. The method as set forth in claim 9, wherein magnitude of n for the n*n dimensional matrix is determined based on size of the received grid and the plurality of sub-grids within the received grid.

11. The method as set forth in claim 10, wherein the orthogonal directions for the action space ‘a’ comprises one or more of an up direction, a down direction, a left direction, a right direction, an up-right direction, an up-left direction, a down-right direction, and a down-left direction.

12. The method as set forth in claim 1, wherein the optimized path is determined by: identifying and acquiring, by the computing device, a geo-location of a hazard or an obstacle;comparing, by the computing device, the user location with the geo-location of the hazard or the obstacle and on the user geo-location being determined outside the geo-location of the hazard or the obstacle, executing the follow command;generating, by the computing device, the n*n-dimensional state matrix based at least on the current location, the target location of the vehicle and the geo-location of the hazard or the obstacle;obtaining, by the computing device, the reward matrix by calculating the corresponding reward for each of the plurality of blocks in the n*n-dimensional state matrix;traversing, by the computing device, the action space by sequentially selecting actions to execute, and performing a one-step prediction from a current state to a predicted next state Q (s, a);initializing, by the computing device, the predicted next state Q (s, a) to obtain the reward of the next state, and calculating a total reward value for the predicted state by selecting the action with maximum magnitude of the reward value; andrepeating, by the computing device, the one-step prediction from a current state to a predicted next state Q (s, a) for n iterations to obtain the path, wherein the reward values are updated dynamically in the generated reward matrix until the vehicle traverses from the current location to the target location.

13. The method as set forth in claim 12, wherein the one-step prediction is based on predetermined criteria, said pre-determined criteria comprising any or a combination of avoiding the hazard and travelling to the target location by traversing shortest distance, plotting the shortest obstacle avoidance and navigation path with minimal run time, and plotting the shortest path with lowest necessary runtime.

14. The method as set forth in claim 13, wherein the shortest obstacle avoidance and navigation path is updated dynamically in real-time during the motion of the vehicle from the current location to the dynamic location.

15. The method as set forth in claim 13, wherein traversing the action space and performing the one step prediction includes:selecting the best action for the vehicle based on the pre-defined hyperparameters, the action being selected such that,upon the learning rate value being closer to 0, physical distance to be traversed by the vehicle is longer and upon the learning rate value closer to 1, the physical distance to be traversed by the vehicle is shorter; and,upon the exploration rate value being closer to 1, the space to be traversed by the vehicle is wider.

16. The method as set forth in claim 11, wherein the on-site execution stage further comprises calculating an objective function, said objective function being a sum of an inverse of the reward, 0.8 times an average computation time and 0.2 times an average computational cost.

17. The method as set forth in claim 16, wherein the average computation time is total time taken in predicting next state Q (s, a) and inputting predicted next state Q (s, a) and, the average computational cost is a quantification of a resource utilization in predicting next state Q (s, a) and inputting predicted next state Q (s, a) into a pre-defined Q-policy.

18. The method as set forth in claim 15, further comprising:sorting, by an avoidance and path navigation module, the received values of n*n dimensional state and selecting the values of n*n dimensional state with the lowest values;updating circumstances of the learning rate and the exploration rate based on the selected lowest values to determine the sub-grid with greater reward parameter;avoiding the identified geo-location of the hazard by the obstacle avoidance rule, wherein the obstacle avoidance rule enables providing a positive reward to the vehicle on arriving at the target location and providing a negative reward to the vehicle on arriving within a pre-defined threshold distance from the location of the hazard.

19. The method as set forth in claim 10, wherein selecting the state space ‘s’ further includes receiving environmental information using one or more GPS sensors or prestored images that are stored on the second device.

20. A system for determining a path from current location to a target location for a vehicle, the system comprising:a computing device operatively coupled to the vehicle, the computing device comprising a processor and a memory communicably coupled with the processor, the memory storing instructions executable by the processor, the computing device configured to:receive a first set of data packets from a first device, wherein the first set of data packets pertain to the target location;determine, based on the current location of the vehicle, the first set of data packets, and a second set of data packets, a map of an area of interest (AOI) comprising the current location and target location of the vehicle, wherein the second set of data packets pertain to geographical features of the AOI;determine a reward parameter corresponding to the AOI comprising the current location and the target location of the vehicle, the reward parameter based, at least in part, on an execution of a pre-determined reward function, wherein the pre-defined reward function is indicative of a plurality of potential paths between the current location and the target location, and wherein the predefined reward function is based on a set of tunable hyperparameters; anddetermine an optimized path from the plurality of potential paths based on the determined reward parameter,wherein the optimized path of the plurality of potential paths between the current location and the target location corresponds to the highest value of the reward parameter.

Images