Computer-implemented method and server for determining trajectory of vehicle

Abstract

The present disclosure provides a computer-implemented method and server for determining a trajectory of a vehicle. The method comprises: acquiring motion data representing movement of the vehicle in a given road section; generating a ground live simulation environment for modeling motion of the vehicle based on the motion data; during a given modeling iteration of a plurality of modeling iterations: generating, based on the motion data, a respective simulated trajectory of the vehicle in the ground live simulation environment during the given modeling iteration; determining a simulated behavior of a given surrounding object based on the respective simulated trajectory; in response to the respective ground-live behavior of the given surrounding object differing from the simulated behavior of the given surrounding object during the given modeling iteration: replacing its respective ground-live behavior with the simulated behavior of the given surrounding object, thereby generating a modified ground-live simulated environment.

Description

[0001] Cross-reference

[0002] This application claims priority to Russian Patent Application No. 2024101314, filed on January 19, 2024, titled "Method and a System for Generating a Trajectory for a Vehicle", which is incorporated herein by reference in its entirety. Technical Field

[0003] The present technology generally relates to self-driving cars (SDCs); and in particular, to methods and systems for using motion planning algorithms to generate trajectories of SDCs. Background Art

[0004] Fully or highly automated driving systems can be designed to operate a vehicle on a road without driver interaction (e.g., in a driverless mode) or other external control. For example, autonomous vehicles and / or self-driving vehicles are designed to operate a vehicle in a computer-aided manner.

[0005] Self-driving vehicles (such as, but not limited to, self-driving cars (abbreviated as SDCs), delivery robots, warehouse robots, etc.) are configured to travel along a planned path between their current location and a target future location without (or with minimal) input from a driver. For this purpose, an SDC can access multiple sensors to "sense" its surrounding area. For example, a given implementation of an SDC can include one or more cameras, one or more LIDARs, and one or more radars. The SDC can further access a 3D map to locate its own position in space.

[0006] One of the technical challenges associated with SDCs is its predictive ability or otherwise determining the trajectories of other road users (such as other vehicles) traveling in the surrounding area of the SDC (e.g., in adjacent lanes). When a given vehicle traveling in front of the SDC in an adjacent lane, for example, is about to perform a maneuver (e.g., turn left or right), its trajectory may (at least partially) overlap and / or intersect with the trajectory of the SDC, which can pose a high risk of collision between the SDC and one of the other vehicles in the surrounding area (including the given vehicle). Therefore, this may require the SDC to take corrective measures, whether braking or otherwise actively accelerating, resulting in the construction of an SDC trajectory that ensures the minimum risk of an accident.

[0007] Typically, the SDC can be configured to determine a trajectory based on objects located in the current surrounding environment of the SDC. For this purpose, for example, the processor of the SDC can be configured to: (i) receive sensed data of the surrounding environment of the SDC (e.g., LIDAR, camera, and other sensor data); (ii) use an object detection machine learning algorithm (MLA) to determine the positions and object classes (e.g., vehicles, passengers, streetlights, etc.) of the objects in the surrounding environment of the SDC based on the sensed data; and (iii) determine the behaviors of the surrounding environment objects based on the corresponding positions and object classes. In addition, based on the behaviors of the surrounding objects thus determined, the processor can be configured to generate a trajectory of the SDC.

[0008] The process of generating a trajectory can be iteratively performed (i.e., during each given iteration (lasting, for example, 1, 5, or 10 milliseconds)) by the processor of the SDC, which can be configured to determine the trajectory of the SDC considering its surrounding objects. For this, the processor of the SDC can be configured to execute a motion planning algorithm. For example, the processor can be configured to generate a corresponding trajectory during each iteration such that the SDC will avoid collisions with its surrounding objects.

[0009] One of the technical challenges that the SDC may encounter when traversing a given trajectory is objects of unknown classes, i.e., objects that the object detection MLA has not been trained to identify. Another challenge for the SDC may be suddenly appearing objects, i.e., objects for which the object detection algorithm will require more processing time to identify. Either of these situations can lead to navigation errors or accidents associated with the SDC. For example, since a woman pushing a collapsible stroller is relatively less encountered on the road compared to other pedestrians, the object detection MLA of the SDC may identify this object as an inanimate object. In another example, unusual animals in an area (e.g., sheep or cows in the city, as an instance), for which the SDC has been trained to operate, may also be identified as static objects.

[0010] It can be appreciated that it may be infeasible to train the object detection MLA to detect all object classes regardless of the operating conditions (e.g., geographical location) of the SDC, as this would require a significant amount of computational resources, time, and effort for labeling the training dataset and further training and using such an object detection MLA.

[0011] Specific background art approaches have been proposed to address the above-identified technical problems.

[0012] U.S. Patent No. 11,551,414-B2, issued on January 10, 2023, and assigned to Magna Automated Systems LLC and Woven by Toyota USA, Inc., and titled "SIMULATION ARCHITECTURE FOR ON-VEHICLE TESTING AND VALIDATION," discloses a computing system for a vehicle that generates perception data based on sensor data captured by one or more sensors of the vehicle. The perception data includes one or more representations of physical objects in an environment associated with the vehicle. The computing system further determines simulated perception data that includes one or more representations of virtual objects within the environment and generates modified perception data based on the perception data and the simulated perception data. The modified perception data includes at least one of the one or more representations of the physical objects and the one or more representations of the virtual objects. The computing system further determines a travel path for the vehicle based on the modified perception data that includes the one or more representations of the virtual objects.

[0013] U.S. Patent No. 11,494,533-B2, issued on November 8, 2022, and assigned to Waymo LLC and titled "SIMULATIONS WITH MODIFIED AGENTS FOR TESTING AUTONOMOUS VEHICLE SOFTWARE," discloses simulation software that can be run using log data collected during operation of a vehicle in an autonomous driving mode. The simulation can be run using software for controlling a simulated vehicle and modified characteristics of agents identified in the log data. During the simulation, it can be determined that a first type of interaction will occur between a first simulated vehicle and a modified agent. In response to determining that a particular type of interaction will occur, the modified agent can be replaced by an interactive agent that simulates a road user corresponding to the modified agent that can respond to actions performed by the simulated vehicle. It can be determined that a particular type of interaction has occurred in the simulation between the simulated vehicle and the interactive agent.

[0014] U.S. Patent No. 11,338,825-B2, issued on May 24, 2022, and assigned to Zoox, Inc., and titled "AGENT BEHAVIOR MODEL FOR SIMULATION CONTROL," discloses the simulated reality movement of objects (e.g., vehicles or pedestrians) and explains unusual behaviors. The simulation may include generating an agent behavior model at least in part based on the output of a perception component of a self-driving vehicle and determining a difference between the output and log data indicative of an actual maneuver including the position of the object. The simulated movement of the object may include using the perception component to determine a predicted movement of the object and modifying the predicted movement at least in part based on the agent behavior model. SUMMARY OF THE INVENTION

[0015] Accordingly, there is a need for systems and methods that avoid, reduce, or overcome the limitations of the background art.

[0016] Developers of the present technology have recognized that the representation of the SDC and the sensed data of suddenly appearing objects can be used to generate a simulation environment in which new versions of motion planning algorithms configured to generate SDC trajectories can be tested.

[0017] More specifically, developers of the present technology have recognized that a simulation environment for modeling the movement of an SDC can be generated based on past sensed data indicative of the SDC moving in a given road segment, the past sensed data including the past behavior of each surrounding object relative to the SDC, e.g., a woman pushing a collapsible stroller towards a crosswalk that intersects the trajectory of a past SDC; or a sheep frantically fleeing a van parked along the road on which the SDC is traveling.

[0018] In addition, similar to the way in which the trajectory of the SDC is generated during runtime, the movement along a given simulated trajectory in such a generated simulation environment can also be modeled iteratively. However, to avoid spatial conflicts between the SDC and surrounding objects in the simulation environment and to model more realistically, developers have recognized that the simulation environment can be dynamically updated by adjusting the behavior of the surrounding objects to the current simulated trajectory of the SDC during a given modeling iteration.

[0019] In other words, specific non-limiting embodiments of the present technology are directed to: (i) determining a corresponding simulated behavior of a given surrounding object in response to the current simulated trajectory of the SDC during a given modeling iteration; (ii) determining whether the corresponding simulated behavior is different from the corresponding past behavior of the given surrounding object; and if so: (iii) replacing the corresponding past behavior of the given surrounding object with the corresponding simulated behavior in at least one subsequent modeling iteration in the simulation environment.

[0020] Therefore, the simulated environment thus generated can allow for a relatively realistic modeling of the movement of the SDC. Considering the simulated behaviors of various objects (including unanticipated and / or less frequently occurring objects) in the surrounding environment of the SDC, it can further allow for a more qualitative selection among several versions of the motion planning algorithm for further use in generating the trajectory of the SDC during runtime.

[0021] By doing so, the methods and systems described herein can provide increased safety and comfort for the SDC.

[0022] More specifically, according to a first general aspect of the present technology, there is provided a computer-implemented method for determining a trajectory of a vehicle using a motion planning algorithm. The method includes: obtaining motion data representing the movement of the vehicle in a given road segment, the motion data including data of surrounding objects of the vehicle within the given road segment; generating, based on the motion data, a ground truth simulation environment for modeling the movement of the vehicle, the ground truth simulation environment representing the respective ground truth behaviors of each surrounding object of the vehicle in the given road segment during multiple modeling iterations. Further, during a given modeling iteration among the multiple modeling iterations, the method includes performing: generating, based on the motion data, a respective simulated trajectory of the vehicle in the ground truth simulation environment during the given modeling iteration using the current version of the motion planning algorithm; determining the simulated behavior of a given surrounding object based on the respective simulated trajectory; in response to the respective ground truth behavior of the given surrounding object being different from the simulated behavior of the given surrounding object during the given modeling iteration: in the ground truth simulation environment, for at least one subsequent modeling iteration among the multiple modeling iterations, replacing the respective ground truth behavior thereof with the simulated behavior of the given surrounding object, thereby generating a modified ground truth simulation environment; using a corresponding instance of the modified ground truth simulation environment from one of the multiple modeling iterations to determine the trajectory of the vehicle using a subsequent version of the motion planning algorithm.

[0023] In some embodiments of the method, the generating the ground truth simulation environment includes determining a respective object category of each surrounding object in the given road segment.

[0024] In some embodiments of the method, the determining the respective object category of each surrounding object includes soliciting a respective label for the surrounding object from a human assessor.

[0025] In some embodiments of the method, the motion data includes a bounding box representing the surrounding object; and determining the respective object class of each surrounding object includes applying a machine learning algorithm (MLA) trained to determine the respective object class of the given surrounding object based on the respective bounding box representing the given surrounding object.

[0026] In some embodiments of the method, determining the simulated behavior of the given surrounding object includes applying an MLA trained to determine the actual behavior of the surrounding object based on the current trajectory of the vehicle.

[0027] In some embodiments of the method, the replacement includes making replacements until, at a given subsequent modeling iteration among the multiple modeling iterations, the respective simulated behavior of the given surrounding object corresponds to its respective ground truth behavior for the given modeling iteration.

[0028] In some embodiments of the method, the method further includes: in response to a stop event during the given modeling iteration: aborting the modeling of the motion of the vehicle without performing subsequent modeling iterations; and removing the current version of the motion planning algorithm from further consideration for determining the trajectory of the vehicle.

[0029] In some embodiments of the method, the stop event includes an accident associated with the vehicle occurring during the given modeling iteration.

[0030] According to a second broad aspect of the present technology, there is provided a server for determining a trajectory of a vehicle using a motion planning algorithm. The server includes at least one processor and at least one non-transitory computer-readable memory storing executable instructions which, when executed by the at least one processor, cause the server to: obtain motion data representing the movement of the vehicle in a given road segment, the motion data including data of surrounding objects of the vehicle within the given road segment; generate a ground truth simulation environment for modeling the motion of the vehicle based on the motion data, the ground truth simulation environment representing the respective ground truth behaviors of each surrounding object of the vehicle in the given road segment during multiple modeling iterations; during a given modeling iteration among the multiple modeling iterations, perform: generating a respective simulated trajectory of the vehicle in the ground truth simulation environment during the given modeling iteration using a current version of the motion planning algorithm based on the motion data; determining a simulated behavior of a given surrounding object based on the respective simulated trajectory; in response to the respective ground truth behavior of the given surrounding object being different from the simulated behavior of the given surrounding object during the given modeling iteration: in the ground truth simulation environment, for at least one subsequent modeling iteration among the multiple modeling iterations, replacing the respective ground truth behavior thereof with the simulated behavior of the given surrounding object, thereby generating a modified ground truth simulation environment; using a respective instance of the modified ground truth simulation environment from one of the multiple modeling iterations to determine the trajectory of the vehicle using a subsequent version of the motion planning algorithm.

[0031] In an implementation of the server, to generate the ground truth simulation environment, the at least one processor causes the server to determine a respective object category of each surrounding object in the given road segment.

[0032] In an implementation of the server, to determine the respective object category of each surrounding object, the at least one processor causes the server to solicit a respective label for the surrounding object from a human assessor.

[0033] In an implementation of the server, the motion data includes a bounding box representing the surrounding object; and to determine the respective object category of each surrounding object, the at least one processor causes the server to apply a machine learning algorithm (MLA) trained to determine the respective object category of the given surrounding object based on the respective bounding box representing the given surrounding object.

[0034] In an implementation of the server, to determine the simulated behavior of the given surrounding object, the at least one processor causes the server to apply an MLA trained to determine the actual behavior of surrounding objects based on the current trajectory of the vehicle.

[0035] In an implementation of the server, the replacement includes making replacements until, at a given subsequent modeling iteration among the multiple modeling iterations, the corresponding simulated behavior of the given surrounding object corresponds to its corresponding ground truth behavior for the given modeling iteration.

[0036] In an implementation of the server, in response to a stop event during the given modeling iteration, the at least one processor further causes the server to: abort modeling of the movement of the vehicle without performing subsequent modeling iterations; and remove the current version of the motion planning algorithm from further consideration for determining the trajectory of the vehicle.

[0037] In an implementation of the server, the stop event includes an accident associated with the vehicle occurring during the given modeling iteration.

[0038] In the context of this specification, the term "light source" broadly refers to a device configured to emit radiation, which is a radiation signal in the form of, for example, a light beam (e.g., but not limited to a light beam containing radiation of one or more corresponding wavelengths within the electromagnetic spectrum). In one example, the light source can be a "laser source". Thus, a light source can include a laser (e.g., a solid-state laser, a laser diode, a high-power laser) or an alternative light source (e.g., a light-emitting diode (LED)-based light source). Some (non-limiting) examples of laser sources include: Fabry-Perot laser diodes, quantum well lasers, distributed Bragg reflector (DBR) lasers, distributed feedback (DFB) lasers, fiber lasers, or vertical cavity surface emitting lasers (VCSELs). Additionally, a laser source can emit light beams in different formats (e.g., light pulses, continuous wave (CW), quasi-CW, etc.). In some non-limiting examples, a laser source can include a laser diode configured to emit light at a wavelength between approximately 650 nm and 1150 nm. Alternatively, a light source can include a laser diode configured to emit a light beam at a wavelength between approximately 800 nm and approximately 1000 nm, between approximately 850 nm and approximately 950 nm, between approximately 1300 nm and approximately 1600 nm, or between any other suitable ranges. Unless otherwise indicated, the term "about" with respect to a numerical value is defined as a variance of up to 10% relative to the stated value.

[0039] In the context of this specification, the term "surroundings" of a given vehicle refers to the area or volume around the given vehicle, including a part of its current environment, which can be scanned using one or more sensors mounted on the given vehicle, for example, to generate a 3D map of such surroundings or to detect objects therein.

[0040] In the context of this specification, a "server" is a computer program that runs on suitable hardware and is capable of receiving requests via a network (e.g., from an electronic device) and executing those requests or causing those requests to be executed. The hardware can be implemented as a single physical computer or a physical computer system, but for the purposes of this technology, neither is necessarily so. In the current context, the use of the expression "server" is not intended to mean that every task (e.g., the received instructions or requests) or any particular task will be received, executed, or caused to be executed by the same server (i.e., the same software and / or hardware); it is intended to mean that any number of software components or hardware devices can be involved in receiving / sending, executing any instructions or requests, or causing them to be executed; and all of this software and hardware can be a single server or multiple servers, both of which are included within the expression "at least one server".

[0041] In the context of this specification, an "electronic device" is any computer hardware capable of running software suitable for the relevant task at hand. In the context of this specification, the term "electronic device" implies that the device can be used as a server for other electronic devices, but this is not necessarily so for the purposes of this technology. Thus, some (non-limiting) examples of electronic devices include autonomous driving units, personal computers (desktop computers, laptop computers, notebook computers, etc.), smartphones and tablets, and network equipment (e.g., routers, switches, and gateways). It should be understood that in the current context, a device being used as an electronic device does not mean that it cannot be used as a server for other electronic devices.

[0042] In the context of this specification, the expression "information" includes any nature or type of information that can be stored in a database. Thus, information includes, but is not limited to, visual works (e.g., maps), audiovisual works (e.g., images, movies, recordings, presentations, etc.), data (e.g., location data, weather data, traffic data, numerical data, etc.), text (e.g., opinions, comments, questions, messages, etc.), documents, spreadsheets, etc.

[0043] In the context of this specification, a "database" is any structured collection of data, regardless of its specific structure, on which database management software or computer hardware for storing, implementing, or otherwise presenting data for use is stored. The database may reside on the same hardware as the process that stores or utilizes the information stored in the database or the database may reside on separate hardware (e.g., a dedicated server or multiple servers).

[0044] In the context of this specification, the words "first", "second", "third", etc. are used as adjectives solely for the purpose of allowing a distinction to be made between several nouns that they modify and not for the purpose of describing any particular relationship between those nouns. Additionally, as described in other contexts herein, the reference to a "first" element and a "second" element does not exclude the actual real-world possibility that the two elements are the same.

[0045] Embodiments of the present technology each have at least one of the objects and / or aspects mentioned above, but do not necessarily have all of the said objects and / or aspects. It should be understood that some aspects of the present technology resulting from an attempt to achieve the above-mentioned objectives may not meet this objective and / or may meet other objectives not specifically recited herein.

[0046] Additional and / or alternative features, aspects, and advantages of embodiments of the present technology will become apparent from the following specification, drawings, and appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] These and other features, aspects, and advantages of the present technology will be better understood with respect to the following description, appended claims, and drawings, in which:

[0048] Figure 1 A schematic diagram depicting an example computer system for implementing a particular embodiment of a system and / or method of the present technology;

[0049] Figure 2 A networked computing environment suitable for use with some embodiments of the present technology;

[0050] Figure 3 Depicting, according to a particular non-limiting embodiment of the present technology, Figure 2 the steps of a LIDAR data acquisition program executed by a processor of an electronic device in a networked computing environment for receiving 3D point cloud data captured by a LiDAR sensor of a vehicle present in the Figure 2 networked computing environment;

[0051] Figure 4 Depicting, according to a particular non-limiting embodiment of the present technology, Figure 2A schematic diagram of a vehicle driving within a given road segment in a networked computing environment, where the given road segment is represented by 3D point cloud data, and the 3D point cloud data is received by a processor of the networked computing environment that executes Figure 3 the LIDAR data acquisition program step of Figure 2 the networked computing environment;

[0052] Figure 5 A schematic diagram depicting a ground truth simulation environment according to a particular non - limiting embodiment of the present technology, where the ground truth simulation environment is generated by a server existing in the networked computing environment based on past sensed data captured by sensors of a vehicle existing in the networked computing environment by Figure 2 ; Figure 2 the networked computing environment;

[0053] Figure 6 A schematic diagram depicting a modified simulation environment according to a particular non - limiting embodiment of the present technology, where the modified simulation environment is generated by a server existing in the networked computing environment using the Figure 2 simulation trajectory of a vehicle existing in the networked computing environment and the Figure 5 ground truth simulation environment; Figure 2 the networked computing environment;

[0054] Figure 7 A flowchart depicting a computer - implemented method for determining the Figure 2 trajectory of a vehicle existing in the networked computing environment. Detailed Description

[0055] The examples and conditional language described herein are, in principle, intended to assist the reader in understanding the principles of the present technology and not to limit its scope to these specifically described examples and conditions. It will be understood that those skilled in the art may envision various arrangements, which, although not explicitly described or shown herein, embody the principles of the present technology and are included within the spirit and scope of the present technology.

[0056] In addition, for the sake of understanding, the following description may depict relatively simplified implementations of the present technology. As those skilled in the art will understand, various implementations of the present technology may have greater complexity.

[0057] In some cases, the content of useful examples of what are considered to be modifications to the present technology may also be stated. This is done solely for the purpose of assisting understanding and is not intended to define the scope or state the boundaries of the present technology. These modifications are not an exhaustive list, and those skilled in the art may make other modifications while still remaining within the scope of the present technology. Additionally, in cases where examples of modifications have not been stated, it should not be construed that no modifications can exist and / or that the disclosed content is the only way to implement the elements of the present technology.

[0058] In addition, all statements herein reciting the principles, aspects, and embodiments of the technology, as well as specific examples thereof, are intended to cover both structural and functional equivalents thereof, whether currently known or developed in the future. Thus, for example, those skilled in the art will understand that any block diagrams herein represent conceptual diagrams of illustrative circuit systems embodying the principles of the technology. Similarly, it will be understood that any flowcharts, flow diagram forms, state transition diagram forms, pseudocode, etc., represent various processes that may be substantially represented in a computer-readable medium and thus executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0059] The functionality of the various elements shown in the figures, including any functional blocks labeled "processor," can be provided by using dedicated hardware and hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functionality can be provided by a single dedicated processor, by a single shared processor, or by multiple individual processors, some of which may be shared. Moreover, the explicit use of the term "processor" or "controller" should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, but is not limited to, digital signal processor (DSP) hardware, network processors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage devices. Conventional and / or custom other hardware may also be included.

[0060] A software module, or simply a module implied to be software, can be represented herein as any combination of flowchart elements or other elements indicating performance of process steps and / or text descriptions. These modules can be executed by hardware that is explicitly or implicitly shown.

[0061] With these basics in mind, some non-limiting examples will now be considered to illustrate various embodiments of aspects of the technology.

[0062] Computer System

[0063] Reference Figure 1 , depicts a schematic diagram of a computer system 100 suitable for use with some embodiments of the technology. Computer system 100 includes various hardware components, including one or more single or multi-core processors represented jointly by processor 110, solid state drive 120, and memory 130, which can be random access memory or any other type of memory.

[0064] Communication between the various components of computer system 100 may be enabled by one or more internal and / or external buses (not shown) (e.g., PCI bus, Universal Serial Bus, IEEE 1394 “FireWire” bus, SCSI bus, Serial ATA bus, etc.), to which the various hardware components are electronically coupled. According to an embodiment of the present technology, solid state drive 120 stores program instructions suitable for being loaded into memory 130 and executed by processor 110 to determine the presence of an object. For example, the program instructions may be part of a vehicle control application executable by processor 110. It should be noted that computer system 100 may have additional and / or optional components (not depicted), such as a network communication module, a positioning module, etc.

[0065] Networked computing environment

[0066] Reference Figure 2 , depicts a networked computing environment 200 suitable for use with some non-limiting embodiments of the present technology. Networked computing environment 200 includes an electronic device 210 that is associated with vehicle 220 and / or with a user (not depicted) associated with vehicle 220 (e.g., an operator of vehicle 220). Environment 200 also includes a server 235 that communicates with electronic device 210 via a communication network 240 (e.g., the Internet, etc., as will be described in more detail below).

[0067] In at least some non-limiting embodiments of the present technology, electronic device 210 is communicatively coupled to the control system of vehicle 220. Electronic device 210 may be arranged and configured to control different operating systems of vehicle 220, which include but are not limited to: ECU (engine control unit), steering system, braking system, and signaling and lighting systems (i.e., headlights, brake lights, and / or turn signals). In such an embodiment, vehicle 220 may be an autonomous vehicle.

[0068] In some non-limiting embodiments of the present technology, networked computing environment 200 may include GPS satellites (not depicted) that transmit GPS signals to and / or receive GPS signals from electronic device 210. It will be understood that the present technology is not limited to GPS and may employ positioning technologies other than GPS. It should be noted that GPS satellites may be completely omitted.

[0069] The vehicle 220 associated with the electronic device 210 can be any transportation vehicle for leisure or other purposes, such as, for example, a private or commercial vehicle, a truck, a motorcycle, and so on. Although the vehicle 220 is depicted as a land vehicle, this may not be the case in every non-limiting embodiment of the present technology. For example, in certain non-limiting embodiments of the present technology, the vehicle 220 can be a watercraft (e.g., a boat) or an aircraft (e.g., a drone).

[0070] The vehicle 220 can be a user-operated or self-driving (driverless) vehicle. In some non-limiting embodiments of the present technology, the vehicle 220 can be implemented as a self-driving car (SDC). It should be noted that the specific parameters of the vehicle 220 are non-limiting. For example, these specific parameters include: vehicle manufacturer, vehicle model, vehicle manufacturing year, vehicle weight, vehicle size, vehicle weight distribution, vehicle surface area, vehicle height, driveline type (e.g., 2x or 4x), tire type, braking system, fuel system, mileage, vehicle identification number, and engine size.

[0071] In other non-limiting embodiments of the present technology, the vehicle 220 can be implemented as a delivery robot vehicle and used to transport various items to users. In this regard, in some non-limiting embodiments of the present technology, the items can include items ordered by the user, such as, for example, (e.g., from an online catalog platform (e.g., Yandex TM Market TM online catalog platform, Avito TM online catalog platform, etc.) consumer goods. In this example, the vehicle 220 can be owned by the online catalog platform or by another entity associated with the online catalog platform. In still other non-limiting embodiments of the present technology, the vehicle 220 can be implemented as a warehouse robot vehicle and used to perform at least one of moving, unloading, and loading various inventory items in a warehouse.

[0072] According to the present technology, the implementation of the electronic device 210 is not limited. For example, the electronic device 210 can be implemented as a vehicle engine control unit, a vehicle CPU, a vehicle navigation device (e.g., TomTom TM , Garmin TM ) built into the vehicle 220, a tablet computer, a personal computer, and so on. Therefore, it should be noted that the electronic device 210 may or may not be permanently associated with the vehicle 220. Additionally or alternatively, the electronic device 210 can be implemented in a wireless communication device (e.g., a mobile phone (e.g., a smartphone or a wireless phone)). In a specific embodiment, the electronic device 210 has a display 270.

[0073] Depending on the particular embodiment, the electronic device 210 may include Figure 1 some or all of the components of the computer system 100 depicted in. In a particular embodiment, the electronic device 210 is an on-board computer device and includes a processor 110, a solid state drive 120, and a memory 130. In other words, the electronic device 210 includes hardware and / or software and / or firmware or a combination thereof for processing data as will be described in more detail below.

[0074] In some non-limiting embodiments of the present technology, the communication network 240 is the Internet. In alternative non-limiting embodiments of the present technology, the communication network 240 may be implemented as any suitable local area network (LAN), wide area network (WAN), personal communication network, and the like. It should be clearly understood that the implementation of the communication network 240 is for illustrative purposes only. A communication link (not separately numbered) is provided between the electronic device 210 and the communication network 240, and its implementation will depend in particular on how the electronic device 210 is implemented. By way of example only and not limitation, in those non-limiting embodiments of the present technology in which the electronic device 210 is implemented as a wireless communication device (e.g., a smart phone or a navigation device), the communication link may be implemented as a wireless communication link. Examples of wireless communication links may include but are not limited to 3G communication network links, 4G communication network links, and the like. The communication network 240 may also use a wireless connection with the server 235.

[0075] In some embodiments of the present technology, the server 235 is implemented as a computer server and may thus include Figure 1 some or all of the components of the computer system 100 of. In one non-limiting example, the server 235 is implemented as a Dell TM Windows server TM operating system running on a TM PowerEdge TM server, but may also be implemented in any other suitable hardware, software, and / or firmware or a combination thereof. In the depicted non-limiting embodiment of the present technology, the server 235 is a single server. In alternative non-limiting embodiments of the present technology, the functionality of the server 235 may be distributed and may be implemented via multiple servers (not depicted).

[0076] In some non - limiting embodiments of the present technology, the processor 110 of the electronic device 210 may communicate with the server 235 to receive one or more updates. These updates may include, but are not limited to, software updates, map updates, route updates, weather updates, and so on. In some non - limiting embodiments of the present technology, the processor 110 may also be configured to transmit specific operation data (e.g., travel route, traffic data, performance data, etc.) to the server 235. Some or all of such data transmitted between the vehicle 220 and the server 235 may be encrypted and / or anonymized.

[0077] It should be noted that the electronic device 210 may use a variety of sensors and systems to collect information about the surrounding environment 250 of the vehicle 220. As Figure 2 seen in, the vehicle 220 may be equipped with multiple sensor systems 280. It should be noted that different sensor systems from the multiple sensor systems 280 may be used to collect different types of data about the surrounding environment 250 of the vehicle 220.

[0078] In one example, the multiple sensor systems 280 may include various optical systems, which particularly include one or more camera - type sensor systems (e.g., camera sensor 290), and the camera - type sensor systems are mounted to the vehicle 220 and communicatively coupled to the processor 110 of the electronic device 210. Broadly speaking, the camera sensor 290 may be configured to collect image data of various parts of the surrounding environment 250 of the vehicle 220, e.g., an image or a series thereof.

[0079] For example, in certain non - limiting embodiments of the present technology, the camera sensor 290 may be implemented as a monocular camera having a resolution sufficient to detect surrounding objects at a predetermined distance up to about 80 m in the surrounding environment 250 of the vehicle 220 (however, camera systems with other resolutions and ranges are within the scope of the present disclosure). The camera sensor 290 may be mounted on the upper inner part of the windshield of the vehicle 220, but other locations of the vehicle 220 are also within the scope of the present disclosure, including on the rear window, side window, front hood, roof box, front grille, or front bumper. In some non - limiting embodiments of the present technology, the camera sensor 290 may be mounted in a dedicated housing (not depicted) mounted on the top of the vehicle 220.

[0080] In some non - limiting embodiments of the present technology, the camera sensor 290 is configured to capture a predetermined portion of the surrounding environment 250 around the vehicle 220. In some embodiments of the present technology, the camera sensor 290 is configured to capture image data representing the surrounding environment 250 around the vehicle 220 at approximately 90 degrees along the movement path of the vehicle 220.

[0081] In other non-limiting embodiments of the present technology, the camera sensor 290 is configured to capture an image (or a series of images) representing the surrounding environment 250 around the vehicle 220 along the movement path of the vehicle 220 for approximately 180 degrees. In still other non-limiting embodiments of the present technology, the camera sensor 290 is configured to capture image data representing the surrounding environment 250 around the vehicle 220 along the movement path of the vehicle 220 for approximately 360 degrees (in other words, the entire surrounding area around the vehicle 220).

[0082] In a particular non-limiting example, the camera sensor 290 may be implemented as a camera of the type available from FLIR INTEGRATED IMAGING SOLUTIONS INC., 12051 Riverside Way, Richmond, BC V6W 1K7, Canada. It should be expressly understood that the camera sensor 290 may be implemented in any other suitable equipment.

[0083] In some cases, the electronic device 210 may use the image data provided by the camera sensor 290 to perform object detection program steps as will be described in detail below.

[0084] In another example, the plurality of sensor systems 280 may include one or more radar-type sensor systems (not individually labeled) that are mounted to the vehicle 220 and communicatively coupled to the processor 110 of the electronic device 210. Broadly speaking, one or more radar-type sensor systems may be configured to use radio waves to collect data on various parts of the surrounding environment 250 of the vehicle 220. For example, one or more radar-type sensor systems may be configured to collect radar data on potential surrounding objects around the vehicle 220, such data potentially representing the distance of the surrounding object from the radar-type sensor system, the orientation of the surrounding object, the rate and / or speed of the surrounding object, and so on.

[0085] It should be noted that the plurality of sensor systems 280 may include additional types of sensor systems of the sensor systems described above non-exhaustively without departing from the scope of the present technology.

[0086] For example, according to a particular non-limiting embodiment of the present technology and as Figure 2As illustrated, vehicle 220 may be equipped with at least one light detection and ranging (LiDAR) system (e.g., LiDAR sensor 300) for collecting information about the surrounding environment 250 of vehicle 220. Although described herein only in the context of being attached to vehicle 220, it is also contemplated that LiDAR sensor 300 may be an independently operating device or connected to another system.

[0087] According to non-limiting embodiments of the present technology, LiDAR sensor 300 of vehicle 220 is communicatively coupled to electronic device 210. In some non-limiting embodiments, information received by electronic device 210 from LiDAR sensor 300 may be used at least in part to control vehicle 220. For example, in an embodiment where vehicle 220 is an autonomous vehicle, a 3D map formed based on information determined by LiDAR sensor 300 may be used by electronic device 210 to at least partially control vehicle 220. In another example, processor 110 of electronic device 210 may be configured to use information received by LiDAR sensor 300 to detect surrounding objects present in the surrounding environment 250 of vehicle 220 in real time for planning the movement of vehicle 220.

[0088] In some non-limiting embodiments of the present technology, for planning the movement of vehicle 220, based on information of the detected surrounding objects, processor 110 of electronic device 210 may be configured to generate and / or modify the movement trajectory of vehicle 220. Additionally, although most of the examples provided in the following description detect movement planning of vehicle 220 in which it is generating its movement trajectory, in broader non-limiting embodiments of the present technology, the movement planning of vehicle 220 may include determining, by processor 110 of electronic device 210, specific movement parameters of vehicle 220 at a given future moment, such as, at least one of the following: displacement, speed, and acceleration of vehicle 220 at a given future moment.

[0089] According to specific non-limiting embodiments of the present technology, a given surrounding object may include at least one of a moving surrounding object and a stationary surrounding object. For example, a moving surrounding object may include, but is not limited to, another vehicle, a train, a tram, a cyclist, or a pedestrian. As an example, a stationary surrounding object may include, but is not limited to, a traffic light, a road sign, a street lamp, a curb, a tree, a fire hydrant, a stopped or parked vehicle, and a trash can.

[0090] It is expected that those skilled in the art will understand the functionality of the LiDAR sensor 300. Briefly, the light source (e.g., a laser, not depicted) of the LiDAR sensor 300 is configured to emit a light beam, which, after being reflected from one or more surrounding objects in the surrounding environment 250 of the vehicle 220, scatters back to the receiver (not depicted) of the LiDAR sensor 300. A telescope is used to collect the photons returning to the receiver and count them as a function of time. Using the speed of light (~3x10 8 m / s), the processor 110 of the electronic device 210 can then calculate how far the photons have traveled (round trip). The photons can scatter back from many different entities around the vehicle 220, such as other particles (aerosols or molecules) of water, dust, or smoke in the atmosphere, other vehicles, stationary surrounding objects, or potential obstacles in front of the vehicle 220.

[0091] Depending on the embodiment, the vehicle 220 may include more or fewer LiDAR sensors 300 than illustrated. Depending on the particular embodiment, the selection of a particular sensor system among the multiple sensor systems 280 may depend on the particular embodiment of the LiDAR sensor 300. The LiDAR sensor 300 can be mounted or retrofitted to the vehicle 220 in a variety of locations and / or in a variety of configurations.

[0092] For example, depending on the implementation of the vehicle 220 and the LiDAR sensor 300, the LiDAR sensor 300 can be mounted on the upper inner part of the windshield of the vehicle 220. Nevertheless, as Figure 2 illustrated, other locations for mounting the LiDAR sensor 300 are within the scope of the present disclosure, including on the rear window, side window, front hood, roof box, front grille, front bumper, or sides of the vehicle 220. In some cases, the LiDAR sensor 300 can even be mounted in a dedicated housing mounted on the top of the vehicle 220.

[0093] In some non-limiting embodiments of the present technology (e.g., Figure 2 embodiments), the LiDAR sensor 300 is mounted to the roof box of the vehicle 220 in a rotatable configuration. For example, the LiDAR sensor 300 mounted to the vehicle 220 in a rotatable configuration may include at least some components that can rotate 360 degrees about the axis of rotation of a given LiDAR sensor 300. When mounted in a rotatable configuration, a given LiDAR sensor 300 can collect most of the data about the surrounding environment 250 of the vehicle 220.

[0094] In some non-limiting embodiments of the present technology (e.g., Figure 2In an embodiment, the LiDAR sensor 300 is mounted in a non-rotatable configuration, such as on a side or a front grille. For example, the LiDAR sensor 300 mounted in a non-rotatable configuration to the vehicle 220 may include at least some components that are not rotatable 360 degrees and are configured to collect data about a predetermined portion of the surroundings 250 of the vehicle 220.

[0095] Regardless of the specific location and / or specific configuration of the LiDAR sensor 300, the LiDAR sensor is configured to capture data about the surroundings 250 of the vehicle 220 used, for example, to construct a multi-dimensional map of the surrounding objects in the surroundings 250 of the vehicle 220. Details regarding the configuration of the LiDAR sensor 300 for capturing data about the surroundings 250 of the vehicle 220 will now be described.

[0096] In a particular non-limiting example, the LiDAR sensor 300 may be implemented as a LiDAR-based sensor, which may be of a type available from VELODYNE LiDAR, INC., 5521 Hellyer Avenue, San Jose, CA 95138, USA. It should be expressly understood that the LiDAR sensor 300 may be implemented in any other suitable equipment.

[0097] It should be noted that although in the description provided herein the LiDAR sensor 300 is implemented as a time-of-flight LiDAR system—and as such, includes the corresponding components suitable for such an implementation—other implementations of the LiDAR sensor 300 are also possible without departing from the scope of the present technology. For example, in a particular non-limiting embodiment of the present technology, as disclosed in the co-owned U.S. Patent Application Publication No. 2021 / 373,172-A1, titled "LiDAR Detection Methods and Systems", and published on December 2, 2021, the LiDAR sensor 300 may also be implemented as a frequency-modulated continuous wave (FMCW) LiDAR system, according to one or more implementation variants and based on their corresponding components; the content of the application publication is hereby incorporated by reference in its entirety.

[0098] Reference Figure 3 , depicts a schematic diagram of the LiDAR data acquisition program step 302 executed by the processor 110 of the electronic device 210 according to a particular non-limiting embodiment of the present technology, the step being for generating 3D point cloud data 310 representing the surrounding objects present in the surroundings 250 of the vehicle 220.

[0099] In some non - limiting embodiments of the present technology, the LiDAR data acquisition program step 302 of receiving 3D point cloud data 310 can be executed in a continuous manner. In other embodiments of the present technology, the LiDAR data acquisition program step 302 of receiving 3D point cloud data 310 can be implemented at a predetermined interval (e.g., every 2 milliseconds or any other suitable time interval).

[0100] To execute the LiDAR data acquisition program step 302, when the vehicle 220 is traveling on the road 304, the processor 110 of the electronic device 210 is configured to use the LiDAR sensor 300 to obtain sensor data 306 representing objects in the surrounding area 250 of the vehicle 220. According to a particular non - limiting embodiment of the present technology, the processor 110 can be configured to receive the sensor data 306, which represents objects in the surrounding area 250 of the vehicle 220 at different positions on the road 304 in the form of one or more 3D point clouds (e.g., 3D point cloud 312).

[0101] Generally speaking, the 3D point cloud 312 is a set of LiDAR points in the form of a 3D point cloud, where a given LiDAR point 314 is a point that indicates at least a part of the surface of a given surrounding object on or around the road 304 in 3D space. In some non - limiting embodiments of the present technology, the 3D point cloud 312 can be organized in layers, where the points in each layer are also organized in an elliptical manner and the starting points of all the elliptical layers are considered to share a similar orientation.

[0102] A given LiDAR point 314 in the 3D point cloud 312 is associated with LiDAR parameters 316 ( Figure 3 depicted as L1, L2, and LN in the figure). As a non - limiting example, the LiDAR parameters 316 can include: distance, intensity, and angle, as well as other parameters related to the information that can be obtained by the LiDAR sensor 300. When the vehicle 220 is traveling, the LiDAR sensor 300 can obtain a 3D point cloud at each time step t, thereby obtaining a set of 3D point clouds similar to the 3D point cloud data 310 regarding the surrounding environment 250 of the vehicle 220.

[0103] Consider that, in some non-limiting embodiments of the present technology, the processor 110 of the electronic device 210 may be configured to enrich the 3D point cloud 312 by using the image data obtained from the camera sensor 290. For this purpose, the processor 110 may be configured to apply one or more ways described in the co-owned U.S. Patent No. 11,551,365-B2, titled "METHODS AND SYSTEMS FOR COMPUTER-BASED DETERMINING OF PRESENCE OF OBJECTS", published on January 23, 2023; the content of the said patent is hereby incorporated by reference in its entirety.

[0104] In addition, referring back to Figure 2 , using the 3D point cloud 312, the processor 110 may be configured to: (i) detect objects in the surrounding environment 250 of the vehicle 220; and (ii) determine the movement trajectory of the vehicle 220 based on the detected objects. Referring to Figure 4 , a schematic diagram depicting the vehicle 220 driving within a given road section 402 according to a specific non-limiting embodiment of the present technology is shown.

[0105] It can be understood from Figure 4 that when the vehicle 220 approaches an intersection in a given road section 402, the vehicle may be configured to make a right turn maneuver 404 to the intersection according to its predetermined (previous) movement trajectory. However, to avoid a collision with the oncoming vehicle 420 driving along the intersection in the straight direction 406, the vehicle 220 must be able to: (i) detect the oncoming vehicle 420; and (ii) take specific corrective measures relative to the previously predetermined movement trajectory, for example, one of deceleration, acceleration, or braking, thereby re-determining the movement trajectory of the vehicle 220.

[0106] According to a specific non-limiting embodiment of the present technology, to detect the oncoming vehicle 420, the processor 110 may be configured to determine: (i) the position of the oncoming vehicle 420 in the coordinate system of the vehicle 220; and (ii) the object category of the oncoming vehicle 420.

[0107] According to certain non - limiting embodiments of the present technology, to adjust the movement trajectory of vehicle 220 to its current environment and respond to surrounding objects in a timely manner, according to certain non - limiting embodiments of the present technology, the processor 110 of vehicle 220 may be configured to iteratively determine the movement trajectory. More specifically, during a given iteration (for example, which may include 0.1, 1, or 5 milliseconds or 1, 10, or 20 seconds), the processor 110 may be configured to: (1) determine the position and object category of an object in the surrounding environment 250, for example, the position and object category of an upcoming vehicle 402; (ii) based on the respective position and object category, determine the corresponding behavior of the upcoming vehicle 420 for the given iteration, for example, moving in a straight direction 406; and (iii) based on the respective position, object category, and behavior of the upcoming vehicle 420, generate the corresponding movement trajectory of vehicle 220 for the given iteration.

[0108] To determine the corresponding object category of surrounding objects, in some non - limiting embodiments of the present technology, the processor 110 may be configured to use a first machine learning algorithm (MLA) 260 hosted by server 235 and configured to detect objects in the surrounding environment 250 of vehicle 220. In non - limiting embodiments of the present technology, the first MLA 260 may be based on a neural network (NN), for example, a convolutional NN (CNN), a transformer - based NN, etc., which will be described in more detail below.

[0109] According to certain non - limiting embodiments of the present technology, to determine the corresponding object category of surrounding objects, the first MLA 260 may be trained to determine a plurality of object features that describe a given surrounding object. Merely as an example, and in no way as a limitation, in a case where the given surrounding object is an upcoming vehicle 420, the object features may include, but are not limited to: (i) the type of the given surrounding object, for example, a movable object; (ii) the type of the movable object, for example, an inanimate object; (iii) the type of the movable inanimate object, for example, a vehicle; (iv) the make and model of the upcoming vehicle 420; (v) the release year of the upcoming vehicle 420; (vi) the body type of the upcoming vehicle 420, for example, a sedan, a hatchback, a van - type vehicle, a minivan, etc.; (vii) the control type of the upcoming vehicle 420, for example, traditional (operated by a driver) or driverless (i.e., other SDC); (viii) the current speed of the upcoming vehicle 420 in the straight direction 406; (ix) the distance 408 from vehicle 220 to the upcoming vehicle 420; etc.

[0110] In another example, where a given surrounding object is a pedestrian (not depicted), the object features may include, but are not limited to: (i) the type of the object, e.g., a movable object; (ii) the type of the movable object, e.g., a living object; (iii) the type of the movable living object, e.g., a human; (iv) the gender of the human, e.g., a woman; (v) the height of the pedestrian; (vi) the weight of the pedestrian; (vii) the age group of the pedestrian, e.g., an adult or a child; (viii) the current moving direction of the pedestrian; (ix) the current speed of the pedestrian in the current moving direction of the pedestrian; (x) the distance to the pedestrian; and so on.

[0111] In yet another example, in a case where a given surrounding object is the traffic light 410, the object features may include, but are not limited to: (i) the type of the object, e.g., a stationary object; (ii) the type of the stationary object, e.g., a traffic rule object; (iii) the type of the traffic rule object, e.g., a traffic light; (iv) the height of the traffic light 410; (v) the distance to the traffic light 410; and so on.

[0112] In some non - limiting embodiments of the present technology, the server 235 may be configured to use a first training data set including a first plurality of training digital objects to train the first MLA 260. A given one of the first plurality of training digital objects includes: (i) a given training bounding box (e.g., the bounding box 421 defined around the upcoming vehicle 420) defined around at least one training surrounding object in a given portion of the surrounding environment 250 in a training road segment (not depicted); and (ii) a corresponding feature vector including features representing at least one training object; and (iii) a corresponding label indicating the position and object category of at least one training object in the given portion of the surrounding environment 250. According to a particular non - limiting embodiment of the present technology, the features representing at least one training object may include, but are not limited to: (i) the surface area of the given training bounding box; (ii) the number of LiDAR points falling within the given training bounding box; (iii) the density of LiDAR points within the given training bounding box; (iv) the light intensity value if each of the LiDAR points falls within the given training bounding box; and so on.

[0113] In some non - limiting embodiments of the present technology, the server 235 may be configured to train the first MLA 260, which may be implemented as described in co - owned Russian Patent Application No. 2023119351, filed on July 21, 2023, and titled "METHOD AND ASYSTEM OF DETERMINING A TRAJECTORY FOR AN AUTONOMOUS VEHICLE", the content of which is incorporated herein by reference in its entirety. However, in other non - limiting embodiments of the present technology, the first MLA 235 may be trained by a third - party server (not depicted), and the server 235 may be configured to access the first MLA 260 via the communication network 240 or locally.

[0114] In addition, to determine the corresponding behavior of an upcoming vehicle 402, during a given iteration, according to certain non - limiting embodiments of the present technology, the processor 110 of the electronic device 210 associated with the vehicle 220 may be configured to use a second MLA 360, which may also be hosted by the server 235 and may be trained to determine its behavior based on various object features of movable surrounding objects (e.g., another vehicle or a pedestrian). In some specific non - limiting embodiments of the present technology, similar to the first MLA 260, the second MLA 360 may also be implemented based on a neural network.

[0115] In some non - limiting embodiments of the present technology, the corresponding behavior of a given movable surrounding object may include, but is not limited to, starting or continuing to move, stopping, accelerating, decelerating, maneuvering, etc. Thus, to train the second MLA 360 to determine the corresponding behavior of a given movable surrounding object, in some non - limiting embodiments of the present technology, the server 235 may be configured to use a second training data set that includes a second plurality of training digital objects, where a given one of the second plurality of training digital objects includes: (i) training sensed data received from a plurality of sensor systems 280 representing a given portion of the surrounding environment 250 in a training road segment (not depicted) during a given training iteration; (ii) a training feature vector containing a plurality of training object features of at least one training movable surrounding object present in the given portion of the surrounding environment 250; and (iii) a corresponding label representing the corresponding training behavior of at least one training movable surrounding object during the given and subsequent training iterations.

[0116] In some non-limiting embodiments of the present technology, the training sensed data can be the past sensed data generated by the electronic device 210 using multiple sensor systems 280 during a given training (past) iteration. Thus, the training road segment can represent the surrounding environment 250 of the vehicle 220 during a given training iteration (e.g., at a given moment during a given training iteration), including the positions and object classes of the corresponding past objects. For this purpose, for example, the training sensed data generated at different training iterations can indicate the same past object (e.g., a traffic light), but are captured by the multiple sensor systems 280 from different corresponding perspectives, at different lighting values, or in different relationships with other past objects in the training road segment (e.g., a given past object is blocked by other past objects or is in front of (or above) other past objects). Additionally, during a given training iteration, the training sensed data can include, but is not limited to, data representing road signs, pedestrian areas (e.g., crosswalks), snowdrifts, fences, guardrails, and so on.

[0117] Furthermore, according to a particular non-limiting embodiment of the present technology, the training feature vectors can be generated using a suitable encoding algorithm configured to represent the training sensed data of the training road segment during a given training iteration in a format that can be received by a given implementation of the MLA 360 (e.g., a neural network as mentioned above).

[0118] In some non-limiting embodiments of the present technology, the second MLA 360 can be trained to determine the corresponding behaviors of surrounding objects in response to the current movement trajectory of the vehicle 220 from one of a given iteration and at least one previous iteration among multiple iterations. In this regard, in these embodiments, a given training digital object of the second plurality of training digital objects can further include data representing the corresponding training movement trajectory of the vehicle 220 during one of a given training iteration and at least one previous training iteration. According to a particular non-limiting embodiment of the present technology, the data representing the corresponding training movement trajectory can include, but is not limited to, the training motion parameters of the vehicle 220 (e.g., speed, acceleration, jerk, etc.), data representing maneuvers (e.g., lane change, U-turn, highway exit), and so on.

[0119] Similar to the case of the first MLA 260, in some non-limiting embodiments of the present technology, the server 235 can be configured to access the second MLA 360 that has been trained by a third-party server (not depicted) via the communication network 240.

[0120] In addition, to generate a corresponding movement trajectory of vehicle 220 based on the corresponding behavior of the upcoming vehicle 402, for a given iteration, according to a particular non-limiting embodiment of the present technology, the processor 110 may be configured to execute a motion planning algorithm 460. Broadly speaking, the motion planning algorithm 460 may be configured to generate a corresponding trajectory of vehicle 220 for each iteration based on at least one of the corresponding positions, object categories, and behaviors of the objects in the surrounding environment 250. More specifically, continuing Figure 4 with the example of Figure 4 , using the motion planning algorithm 460, the processor 110 may be configured to determine current object kinematic data, which includes current object motion parameters (e.g., current speed, acceleration, jerk, braking curve, etc.) of the upcoming vehicle 402 in the straight direction 406. In addition, based on the current object kinematic data, the processor 110 may be configured to: determine the current vehicle kinematic data of vehicle 220 for a given iteration, and use the current vehicle kinematic data to cause vehicle 220 to move along the direction of the right turn maneuver 404 in order to avoid a collision with the upcoming vehicle 402 and / or with any other object in the surrounding environment 250.

[0121] In some non-limiting embodiments of the present technology, the motion planning algorithm 460 may include a kinematic model. Broadly speaking, the kinematic model 402 includes a combination of mathematical models that are configured to calculate the current object kinematic data of the surrounding objects of vehicle 220. In some non-limiting embodiments of the present technology, the kinematic model may be implemented as described in co-owned U.S. Patent No. 11,753,037-B2, issued on September 12, 2023, and titled "METHOD AND PROCESSOR FOR CONTROLLING IN-LANE MOVEMENT OF AUTONOMOUS VEHICLE", the content of which is incorporated herein by reference in its entirety. Looking ahead to other implementations of the motion planning algorithm 460 (e.g., implementations that include MLA), they do not depart from the scope of the present technology. As will be apparent from the description provided hereinafter in this document, there may be multiple versions of the motion planning algorithm 460 that can be uploaded to the electronic device 210 for execution by the processor 110 to generate the motion trajectory of vehicle 220. Each of several versions of the motion planning algorithm 460 may be configured to generate a movement trajectory in a different manner for a given motion direction (e.g., the right turn maneuver 404 illustrated as Figure 4 in the example), which may include different curvatures and vehicle kinematic data.

[0122] However, one of the challenges that may be encountered while the vehicle 220 is traversing a given route is a sudden or unknown object for the first MLA 260. For example, unknown objects can include, but are not limited to, objects of relatively rarely encountered object classes, such as a person pushing a collapsible stroller, a person in a wheelchair, livestock (such as sheep, pigs, or cows, especially in urban areas). On the other hand, a suddenly appearing object can include an object that is not visible to the vehicle's multiple sensor systems 280 during a portion of a given iteration (e.g., due to being blocked by other objects). Such objects can include objects moving at relatively high speeds, such as, but not limited to, a fleeing animal, a cyclist, or another vehicle. Thus, when these objects become visible to the multiple sensor systems 280 only at some points during a given iteration, the processor 110 may not have enough time to determine the object class, behavior, and current object kinematic data of these objects, which can lead to an inaccurate determination of the corresponding trajectory of the vehicle 220.

[0123] One of the direct solutions to the above-identified technical problem would be: (i) training the first MLA 260 and the second MLA 360 to detect objects of all possible object classes and further determine the behavior of these objects respectively; and (ii) testing a new version of the motion planning algorithm 460 based on the predictions of the first MLA 260 and the second MLA 360 for these specific objects. However, it can be appreciated that training the first MLA 260 and the second MLA 360 to detect and determine the behavior of more objects may require: (i) a significant amount of time and effort for labeling the corresponding data sets; and (ii) computational resources of the processor 110 of the server 235 and / or the electronic device 210 for training and using the first MLA 260 and the second MLA 360.

[0124] Therefore, the developers of the present technology have recognized that the sensed data representing these specific objects generated by the multiple sensor systems 280 in the past can be used to generate a simulated environment in which different versions of the motion planning algorithm 460 can be tested. The versions of the motion planning algorithm 460 can be used to generate different simulated trajectories, and the different simulated trajectories can be further analyzed for safety. In addition, the version of the motion planning algorithm 460 that has determined a safe generated modeled trajectory can be further used to generate the motion trajectory of the vehicle 220 during runtime.

[0125] Now, it will be described how a simulated environment can be generated according to specific non-limiting embodiments of the present technology.

[0126] Simulated Environment

[0127] Reference Figure 5, schematically depicts a ground truth simulation environment 500 according to a particular non - limiting embodiment of the present technology, the ground truth simulation environment being generated by server 235 and representing a vehicle 220 traveling within a past road segment 502 during a first past iteration 501 (T1), a second past iteration 503 (T2), and a third past iteration 505 (T3). It can be understood that in the illustrated example, the vehicle 220 travels along the past road segment 502 in a first direction 510 (i.e., in the forward direction of the vehicle 220, i.e., from left to right in the orientation of Figure 5 ).

[0128] According to a particular non - limiting embodiment of the present technology, to generate the ground truth simulation environment 500, server 235 may be configured to use past sensed data generated by a plurality of sensor systems 280 of vehicle 220 and received by processor 110 of electronic device 210. As mentioned above herein, the past sensed data may include: data indicating the positions of past surrounding objects, the object categories of past surrounding objects, and the behaviors of past surrounding objects during each of a plurality of past iterations (e.g., the first past iteration 501, the second past iteration 503, and the third past iteration 505).

[0129] In some non - limiting embodiments of the present technology, server 235 may be configured to use the first MLA 260 trained as mentioned above to determine the object categories of past surrounding objects. In other non - limiting embodiments of the present technology, server 235 may be configured to solicit human - generated labels from human assessors representing the object categories of past surrounding objects. For example, in these embodiments, server 235 may be configured to submit past sensed data in a computer - readable format via communication network 240 to a crowdsourcing platform (e.g., Yandex TM Toloka TM crowdsourcing platform or Amazon TM Mechanical Turk TM crowdsourcing platform) having corresponding labeling authority. For example, if using the first MLA 260, server 235 may be unable to identify a given object 504 as a person in a wheelchair, then server 235 may be configured to obtain a label for the given object 504 from a human assessor.

[0130] Therefore, as Figure 5As illustrated, before vehicle 220, a given object 504 (a person in a wheelchair) is traversing the oncoming road segment 502 in a second direction 512 that is perpendicular to the first direction 510. Thus, for each of a first pass iteration 501, a second pass iteration 503, and a third pass iteration 505, the processor 110 of the electronic device 210 may be configured to: (i) use a first MLA 260 to detect the given object 504; (ii) use a second MLA 360 to determine the corresponding passing behavior of the given object 504, the corresponding passing behavior being an intention to traverse the oncoming road segment 502 in the second direction 512 at a crosswalk (not individually numbered); and (iii) based on the corresponding passing behavior, use a previous version of a motion planning algorithm 460 to determine the passing kinematic data of the vehicle 220, the passing kinematic data defining its corresponding passing trajectory during each of the first pass iteration 501, the second pass iteration 503, and the third pass iteration 505. As Figure 5 illustrated in the example of, each of the passing trajectories of the vehicle 220 includes making way for the given object 504.

[0131] Thus, by using the sensed past data to generate a ground truth simulation environment 500, the server 235 may be configured to model the actual past movement of the vehicle 220 and the corresponding past behavior of the past surrounding objects in the past road segment 502. In some non-limiting embodiments of the present technology, the server 235 may be configured to generate the ground truth simulation environment 500 as a 2D simulation environment that represents the movement of the vehicle 220 and the past surrounding objects in the past road segment 502 in a given projection (e.g., in a top view), as Figure 5 schematically depicted. In these embodiments, the ground truth simulation environment 500 may be represented as a sequence of images that represent the positions of the vehicle 220 and the past surrounding objects in the past road segment 502 at each pass iteration. In other non-limiting embodiments of the present technology, the server 235 may be configured to generate the ground truth simulation environment 500 as a 3D simulation environment, where the vehicle 220 and the past surrounding objects are represented as 3D models, for example, the 3D models including mesh elements.

[0132] Furthermore, according to certain non-limiting embodiments of the present technology, the server 235 may be configured to use the ground truth simulation environment 500 to test a new version of the motion planning algorithm 460 for further use in generating new trajectories for the vehicle 220. In some non-limiting embodiments of the present technology, the server 235 may be configured to cause the new trajectories to be simulated in the ground truth simulation environment 500 before using these new trajectories for the actual movement of the vehicle 220.

[0133] A given simulated trajectory of the vehicle 220 in a past road segment 502 can differ from a past trajectory in at least one of (i) the kinematic data of the vehicle 220 and (ii) the geometry of the path defined by the given simulated trajectory. More specifically, by modifying the past kinematic data of the vehicle 220, the server 235 can be configured to cause the vehicle 220 to move in the first direction 510 in at least one of the following ways: (1) more slowly or more quickly; (2) with acceleration or deceleration; and (3) with higher or lower jerk. On the other hand, by modifying the geometry of the past trajectory, the server 235 can be configured to cause the vehicle 220 to deviate from the first direction 510. For example, using one of the new versions of the motion planning algorithm 460, the server 235 can be configured to cause the vehicle 220 to change lanes during a given one of the modeling iterations.

[0134] Once the server 235 has generated a given simulated trajectory of the vehicle 220, the server 235 can further be configured to cause the vehicle 220 to perform a corresponding modeled motion in the ground truth simulation environment 500. However, the corresponding modeled motion of the vehicle 220 can cause a conflict with the movement of past surrounding objects in the ground truth simulation environment 500. For example, if the given simulated trajectory includes moving the vehicle 220 at a higher speed during at least one of the first past iteration 501, the second past iteration 503, and the third past iteration 505, then the representation of the vehicle 220 and the representation of the given object 504 can overlap in the ground truth simulation environment 500. Such modeling of the motion of the vehicle 220 can have a lower value for testing new versions of the motion planning algorithm 460 because this simulation may not allow determining how surrounding objects will respond to the changed trajectory of the vehicle 220 and whether the modeled trajectory is safe and / or comfortable for the passengers of the vehicle 220 and other road users.

[0135] Therefore, the developers of the present technology have recognized that to more realistically model the motion of the vehicle 220 and estimate the comfort and safety of the simulated trajectory, the ground truth simulation environment 500 can be dynamically adjusted in response to a given simulated trajectory of the vehicle 220. In other words, according to a non-limiting embodiment of the present technology, in response to each new simulated trajectory of the vehicle 220, the server 235 can be configured to re-determine the behavior of surrounding objects in the ground truth simulation environment 500 at each modeling iteration, thereby generating a modified simulation environment, such as the modified simulation environment 600.

[0136] Reference Figure 6 shows schematically a first modeling iteration 601 (T1) by the server 235 according to a particular non-limiting embodiment of the present technology ’) The modified simulation environment 600 generated using a given new version of the motion planning algorithm 460 during the second modeling iteration 603 (T2 ’ ) and the third modeling iteration 605 (T3 ’ ). According to certain non-limiting embodiments of the present technology, a given modeling iteration among multiple modeling iterations (e.g., one of the first modeling iteration 601, the second modeling iteration 603, and the third modeling iteration 605) may correspond in duration to a corresponding previous iteration. In some non-limiting embodiments of the present technology, a given modeling iteration may be shorter or longer in duration than the corresponding previous iteration.

[0137] In Figure 6 the example illustrated, the initial positions of the vehicle 220 and the given object 504 in the previous road segment 502 (i.e., their positions at the start of the first modeling iteration 601) correspond to the initial positions of the vehicle 220 and the given object 504 in the ground truth simulation environment 500 at the start of the first previous iteration 501.

[0138] Furthermore, during the first modeling iteration 601, using a given new version of the motion planning algorithm 460 and based on previously sensed data (which the server 235 uses to generate the ground truth simulation environment 500), the server 235 may be configured to generate a first simulated trajectory of the vehicle 220, which is defined by the first modeling kinematic data and the first direction 510 of the vehicle 220. Continuing to refer to Figure 6 and referring back to Figure 5 , it can be appreciated that the vehicle 220 moves faster in the first direction 510 during the first modeling iteration 601 using the first modeling kinematic data as compared to using the previous kinematic data during the first previous iteration 501.

[0139] Furthermore, using the second MLA 360 and based on the first simulated trajectory, the server 235 may be configured to determine a first simulated behavior of the given object 504 for the first modeling iteration 601. For example, the server 235 may be configured to determine that during the first modeling iteration 601, when the vehicle 220 is moving relatively fast along the first simulated trajectory, to avoid a collision with the vehicle 220, the given object 504 must not move in the second direction 512 but must rest at the initial position.

[0140] In addition, according to certain non-limiting embodiments of the present technology, the server 235 may be configured to determine whether a first simulated behavior of a given object 504 in the ground truth simulation environment 500 is different from its corresponding past behavior. It can be appreciated that in the current instance, the corresponding past behavior of the given object 504 during the first past iteration 501 (corresponding to the first modeling iteration 601) is to cross the past road segment 502 in the second direction 512; while in the modified simulation environment 600, the given object 504 rests during the first modeling iteration 601.

[0141] Accordingly, in response to determining that the first simulated behavior of the given object 504 is different from its corresponding past behavior during the first modeling iteration 601, in some non-limiting embodiments of the present technology, the server 235 may be configured to replace the corresponding past behavior of the given object 504 with the first simulated behavior of at least one subsequent modeling iteration among multiple modeling iterations. In some non-limiting embodiments of the present technology, the server 235 may be configured to replace the corresponding past behavior of the given object 504 for a predetermined number (e.g., 1, 5, or 10) of subsequent modeling iterations. Once the predetermined number of subsequent modeling iterations has elapsed, during subsequent modeling iterations, the server 235 may be configured to: (i) re-determine the corresponding simulated behavior of the given object 504; and (ii) compare the corresponding simulated behavior of the given object 504 during subsequent modeling iterations with its corresponding past behavior during the first past iteration 501.

[0142] For example, the server 235 may be configured to replace the corresponding past behavior of the given object 504 with the first simulated behavior during the second modeling iteration 603, thereby causing the given object 504 to rest during the second modeling iteration 603.

[0143] Meanwhile, during the second modeling iteration 603, using a given new version of the motion planning algorithm 460, the server 235 may be configured to determine a second simulated trajectory of the vehicle 220, the second simulated trajectory including second modeling kinematic data of the vehicle 220, in which case the vehicle 220 continues to move in the first direction 510 while the given object 504 rests. It can be appreciated that by assigning the first simulated behavior to the given object 504, the visual representations of the vehicle 220 and the given object 504 will not overlap as the vehicle 220 moves along the modeled trajectory.

[0144] Finally, during the third modeling iteration 605, the server 235 may be configured to generate a third simulated trajectory of the vehicle 220, the third simulated trajectory including third modeled kinematic data, in which case the vehicle 220 continues to move forward in the first direction 510 in the past road section 502. Additionally, as mentioned above, during the third modeling iteration 605, the server 235 may be configured to re-determine whether the current simulated behavior of a given object 504 corresponds to its corresponding past behavior during the first past iteration 501. More specifically, during the third modeling iteration 605, the server 235 may be configured to: (i) use the second MLA 360 to determine a third simulated behavior of the given object 504 based on the third simulated trajectory of the vehicle 220; and (ii) determine whether the third simulated behavior of the given object 504 during the third modeling iteration 605 is different from the corresponding past behavior of the given object 504 during the first past iteration 501.

[0145] For example, when there is no vehicle 220 in the second direction 512 during the third modeling iteration 605, using the second MLA 360, the server 235 may be configured to determine a third simulated behavior of the given object 504, the third simulated behavior corresponding to its corresponding past behavior during the first past iteration 501, i.e., moving along the second direction 512 to cross the past road section 502 at the crosswalk.

[0146] By doing so, the server 235 may be configured to continue modeling the movement of the vehicle 220 for further modeling iterations, adjusting the behavior of the past surrounding objects to the current simulated trajectory of the vehicle 220, thereby generating a corresponding instance of the modified simulated environment 600 based on the ground truth simulated environment 500. According to a particular non-limiting embodiment of the present technology, the server 235 may be configured to continue modeling the movement of the vehicle 220 for further modeling iterations among multiple modeling iterations until a stop event occurs.

[0147] In some non-limiting embodiments of the present technology, the stop event may be associated with the safety of a given simulated trajectory of the vehicle 220. For example, the stop event may include an accident between the vehicle 220 and other objects (not depicted) in the past road section 502 in the modified simulated environment 600. Suppose that at the first modeling iteration 601, other objects also intend to move in the second direction 512; however, the other objects are not visible to the vehicle 220, e.g., blocked by the given object 504. In such a case, the server 235 may not be able to determine the corresponding simulated behavior of the other objects; and during one of the first modeling iteration 601, the second 603, or the third modeling iteration 605, the other objects may start to move in the second direction 512 and collide with the vehicle 220.

[0148] Other examples of stop events may include, but are not limited to: reaching a predetermined jerk value of vehicle 220 along the simulated trajectory, reaching a predetermined number of lane changes, occurrence of a traffic rule violation, vehicle 220 crossing a predetermined safety zone defined around another road user in the oncoming road segment 502, and so on.

[0149] According to certain non-limiting embodiments of the present technology, in response to the occurrence of a stop event, server 235 may be configured to: (i) use a given new version of motion planning algorithm 460 to abort the modeling of the motion of vehicle 220; (ii) remove the given new version of motion planning algorithm 460 from further consideration for generating the actual trajectory of vehicle 220; and (iii) continue to test other new versions of motion planning algorithm 460 in the ground truth simulation environment 500. In other words, in these embodiments, server 235 may be configured to: (i) undo all modifications to the ground truth simulation environment 500 made during the testing of a given new version of motion planning algorithm 460; and (ii) start testing its other new version from the very beginning as described above.

[0150] However, if after reaching a threshold predetermined number of modeling iterations (as examples, for instance, 100, 5000, or 1000000) using a given new version of motion planning algorithm 460, a stop event has not occurred, then in some non-limiting embodiments of the present technology, server 235 may be configured to save the given new version of motion planning algorithm 460 in the memory 130 of server 235 for further use in generating the actual trajectory of the navigating vehicle 220 during runtime. In other words, if a given new version of motion planning algorithm 460 has passed the tests in the ground truth simulation environment 500, then server 235 may be configured to transmit this version of motion planning algorithm 460 to electronic device 210, whereby the processor 110 of electronic device 210 can use the given new version of motion planning algorithm 460 to generate the actual trajectory of vehicle 220, as referenced above Figure 4 described.

[0151] In some non-limiting embodiments of the present technology, server 235 may further be configured to test other new versions of motion planning algorithm 460 in the modified simulation environment 600 generated during the testing of a given new version of motion planning algorithm 460.

[0152] Accordingly, by doing so, the server 235 can be configured to generate a modified simulation environment 600 that represents real conditions for modeling the movement of the vehicle 220 along a simulated trajectory generated by a new version of the motion planning algorithm 460. Based on the modeled movement of the vehicle 220, the server 235 can be configured to reject further use of a given new version of the motion planning algorithm 460 or save it for further determination of the trajectory in the use of the vehicle 220 at runtime.

[0153] Method

[0154] In view of the architecture and examples provided hereinabove, a method for determining the trajectory of the vehicle 220 can be performed. Now refer to Figure 7 , which depicts a flowchart of a method 700 according to a particular non-limiting embodiment of the present technology. The method 700 can be executed by the server 235.

[0155] Step 702: Obtain motion data representing the movement of the vehicle in a given road segment, the motion data including data of surrounding objects of the vehicle within the given road segment

[0156] The method 700 begins at step 702, where the server 235 is configured to obtain past sensed data representing the past movement of the vehicle 220 in the past road segment 502. As mentioned hereinabove, the past sensed data can be generated by multiple sensor systems 280 of the vehicle 220, received by the processor 110 of the electronic device 210, and transmitted to the server 235.

[0157] As further mentioned above, the past sensed data can include data that represents: the position of a given past surrounding object (e.g., a given object 504) during each past iteration (e.g., one of the first past iteration 501, the second past iteration 503, and the third past iteration 505) of multiple past iterations of generating the past trajectory of the vehicle 220; the corresponding object category of the given object 504; and the corresponding past behavior of the given object 504, as described in detail above with reference to Figure 5 Described in detail.

[0158] The method 700 then proceeds to step 704.

[0159] Step 704: Generate a ground truth simulation environment for modeling the movement of the vehicle based on the motion data, the ground truth simulation environment representing the corresponding ground truth behavior of each surrounding object of the vehicle in the given road segment during multiple modeling iterations

[0160] At step 704, according to a particular non-limiting embodiment of the present technology, the server 235 may be configured to generate a ground truth simulation environment 500 based on past sensed data, as described above with reference to Figure 5 Description. The ground truth environment 500 represents the actual past movement of the vehicle 220 and the corresponding past behavior of its past surrounding objects (e.g., the corresponding past behavior of a given object 504) during each of the first past iteration 501, the second past iteration 503, and the third past iteration 505.

[0161] Method 700 thus proceeds to step 706.

[0162] Step 706: Generate a corresponding simulated trajectory of the vehicle in the ground truth simulation environment during a given modeling iteration using a current version of a motion planning algorithm based on motion data

[0163] At step 706, according to a particular non-limiting embodiment of the present technology, the server 235 may be configured to begin testing a given new version of the motion planning algorithm 460, which may be used by the processor 110 of the electronic device 210 to generate a trajectory of the vehicle 220. More specifically, at step 706, the server 235 may be configured to use a given new version (or otherwise the current version) of the motion planning algorithm 460 to generate a first simulated trajectory of the vehicle 220 in the ground truth simulation environment 500 for the first modeling iteration 601. As mentioned above with reference to Figure 6 The first modeling iteration 601 may be the same as or different from the first past iteration 501.

[0164] In Figure 6 the instance, the server 235 has generated a first simulated trajectory that includes first kinematic data of the vehicle 220, in which case the vehicle 220 will move faster in the first direction 510 during the first modeling iteration 501 than during the first past iteration 501.

[0165] Accordingly, method 700 continues to step 708.

[0166] Step 708: Determine the simulated behavior of a given surrounding object based on the corresponding simulated trajectory

[0167] At step 708, using the second MLA 360 and based on the first simulated trajectory of the vehicle 220, the server 235 may be configured to determine a first simulated behavior of a given object 504 for the first modeling iteration. As referenced Figure 6As mentioned above, since the chance of a collision between a given object 504 and its first simulated trajectory with a given vehicle 220 is relatively high, the server 235 may be configured to determine a first simulated behavior of the given object 504, including resting in an initial site at the start of a first modeling iteration 601.

[0168] Method 700 thus proceeds to step 710.

[0169] Step 710: In response to the corresponding ground truth behavior of a given surrounding object being different from its simulated behavior during a given modeling iteration: In a ground truth simulation environment, for at least one subsequent modeling iteration among multiple modeling iterations, replace the corresponding ground truth behavior of the given surrounding object with its simulated behavior, thereby generating a modified ground truth simulation environment

[0170] At step 710, according to a particular non - limiting embodiment of the present technology, the server 235 may be configured to determine whether the first simulated behavior of a given object 504 is different from its corresponding past behavior during a first past iteration 501.

[0171] In the example described above Figure 5 and 6 the first simulated behavior (resting) of the given object 504 during the first modeling iteration 601 is different from the corresponding past behavior (moving in a second direction 512) of the given object 504 during the first past iteration 501. In response, according to a particular non - limiting embodiment of the present technology, the server 235 may be configured to replace the corresponding past behavior of the given object 504 with its first simulated behavior for at least one subsequent modeling iteration among multiple modeling iterations. For example, the server 235 may be configured to assign the first simulated behavior to the given object 504 for a second modeling iteration 603.

[0172] Thus, by modifying the behavior of the given object 504 in the ground truth simulation environment 500, the server 235 is configured to generate a corresponding instance of the modified simulation environment 600.

[0173] In addition, in some non - limiting embodiments of the present technology, the server 235 may be configured to replace the corresponding past behavior of the given object 504 for a predetermined number (e.g., 1, 5, or 10) of subsequent modeling iterations. Once the predetermined number of subsequent modeling iterations has passed, during subsequent modeling iterations, the server 235 may be configured to: (i) re - determine the corresponding simulated behavior of the given object 504; and (ii) compare the corresponding simulated behavior of the given object 504 during subsequent modeling iterations with its corresponding past behavior during the first past iteration 501.

[0174] For example, as referred to aboveFigure 6 Further, during the third modeling iteration 605, the server 235 may be configured to generate a third simulated trajectory of the vehicle 220, the third simulated trajectory including third modeled kinematic data, in which case the vehicle 220 continues to move forward in the first direction 510 in the past road section 502. Subsequently, the server 235 may be configured to: use the second MLA 360 to determine a third simulated behavior of a given object 504 based on the third simulated trajectory of the vehicle 220; and (ii) determine whether the third simulated behavior of the given object 504 during the third modeling iteration 605 is different from the corresponding past behavior of the given object 504 during the first past iteration 501.

[0175] For example, when there is no vehicle 220 in the second direction 512 during the third modeling iteration, using the second MLA 360, the server 235 may be configured to determine a third simulated behavior of the given object 504, the third simulated behavior corresponding to its corresponding past behavior during the first past iteration 501, that is, moving along the second direction 512 to cross the past road section 502 at the crosswalk.

[0176] By doing so, the server 235 may be configured to continue modeling the movement of the vehicle 220 using a given new version of the motion planning algorithm 460 for further modeling iterations among multiple modeling iterations until a stop event (e.g., an accident associated with the vehicle 220) occurs.

[0177] Method 700 then proceeds to step 712.

[0178] Step 712: Use the corresponding instance of the modified ground truth simulation environment from one of the multiple modeling iterations to determine the trajectory of the vehicle using a subsequent version of the motion planning algorithm

[0179] At step 712, in the absence of a stop event occurring, according to a particular non-limiting embodiment of the present technology, the server 235 may be configured to save the corresponding instance of the modified simulation environment 600 to test a subsequent new version of the motion planning algorithm 460.

[0180] In addition, if a stop event has not occurred and the server 235 has tested a given new version of the motion planning algorithm 460, then the server 235 may be further configured to save the given new version of the motion planning algorithm 460 for further use in generating the in-use trajectory of the vehicle 220 during runtime.

[0181] However, if a stop event has occurred, then server 235 may be configured to abort modeling the movement of vehicle 220 using a given new version of motion planning algorithm 460 and remove it from further consideration for determining the future trajectory of vehicle 220.

[0182] Method 700 thus terminates.

[0183] Accordingly, a particular non-limiting embodiment of method 700 allows the movement of vehicle 220 to be safely modeled along a newly generated trajectory in a realistic simulation environment.

[0184] Modifications and improvements to the above-described embodiments of the technology will be apparent to those skilled in the art. The foregoing description includes example embodiments of the technology and is in no way intended to be limiting. Accordingly, the scope of the technology is intended to be limited only by the scope of the appended claims.

[0185] Although the embodiments described above are described and shown with reference to particular steps performed in a particular order, it will be understood that some of these steps may be combined, subdivided, or reordered without departing from the teachings of the technology. Accordingly, the order and grouping of the steps are not limitations of the technology.

Claims

1. A computer-implemented method for determining a trajectory of a vehicle using a motion planning algorithm, the method comprising: Obtaining motion data representing the movement of the vehicle in a given road segment, The motion data including data of surrounding objects of the vehicle within the given road segment; Generating a ground truth simulation environment for modeling the motion of the vehicle based on the motion data, The ground truth simulation environment representing the corresponding ground truth behavior of each surrounding object of the vehicle in the given road segment during multiple modeling iterations; During a given modeling iteration among the multiple modeling iterations, performing: Generating a corresponding simulated trajectory of the vehicle in the ground truth simulation environment during the given modeling iteration using the current version of the motion planning algorithm based on the motion data; Determining the simulated behavior of a given surrounding object based on the corresponding simulated trajectory; In response to the corresponding ground truth behavior of the given surrounding object being different from the simulated behavior of the given surrounding object during the given modeling iteration: In the ground truth simulation environment, for at least one subsequent modeling iteration among the multiple modeling iterations, replacing the corresponding ground truth behavior of the given surrounding object with the simulated behavior of the given surrounding object, thereby generating a modified ground truth simulation environment; Using a corresponding instance of the modified ground truth simulation environment from one of the multiple modeling iterations to determine the trajectory of the vehicle using a subsequent version of the motion planning algorithm.

2. The computer-implemented method according to claim 1, wherein generating the ground truth simulation environment includes determining a corresponding object category of each surrounding object in the given road segment.

3. The computer-implemented method according to claim 2, wherein determining the corresponding object category of each surrounding object includes soliciting corresponding labels for the surrounding objects from a human assessor.

4. The computer-implemented method according to claim 2, wherein: The motion data includes bounding boxes representing the surrounding objects; and Determining the corresponding object category of each surrounding object includes applying a machine learning algorithm MLA trained to determine the corresponding object category of the given surrounding object based on the corresponding bounding box representing the given surrounding object.

5. The computer-implemented method according to claim 1, wherein determining the simulated behavior of the given surrounding object includes applying an MLA trained to determine the actual behavior of surrounding objects based on the current trajectory of the vehicle.

6. The computer-implemented method according to claim 1, wherein the replacement includes performing the replacement until, at a given subsequent modeling iteration among the multiple modeling iterations, the corresponding simulated behavior of the given surrounding object corresponds to its corresponding ground truth behavior for the given modeling iteration.

7. The computer-implemented method according to claim 1, further comprising, in response to a stop event during the given modeling iteration: Abort modeling the movement of the vehicle without performing subsequent modeling iterations; and Remove the current version of the motion planning algorithm from further consideration for determining the trajectory of the vehicle.

8. The computer-implemented method according to claim 7, wherein the stop event includes an accident associated with the vehicle occurring during the given modeling iteration.

9. A server for determining a trajectory of a vehicle using a motion planning algorithm, the server including at least one processor and at least one non-transitory computer-readable memory storing executable instructions that, when executed by the at least one processor, cause the server to: Obtain motion data representing the vehicle moving in a given road segment, wherein the motion data includes data of surrounding objects of the vehicle within the given road segment; Generate a ground truth simulation environment for modeling the movement of the vehicle based on the motion data, wherein the ground truth simulation environment represents the corresponding ground truth behavior of each surrounding object of the vehicle in the given road segment during multiple modeling iterations; During a given modeling iteration among the multiple modeling iterations, perform: Generate a corresponding simulated trajectory of the vehicle in the ground truth simulation environment during the given modeling iteration using the current version of the motion planning algorithm based on the motion data; Determine the simulated behavior of a given surrounding object based on the corresponding simulated trajectory; In response to the corresponding ground truth behavior of the given surrounding object being different from the simulated behavior of the given surrounding object during the given modeling iteration: In the ground truth simulation environment, for at least one subsequent modeling iteration among the multiple modeling iterations, replace the corresponding ground truth behavior thereof with the simulated behavior of the given surrounding object, thereby generating a modified ground truth simulation environment; Use a corresponding instance of the modified ground truth simulation environment from one of the multiple modeling iterations to determine the trajectory of the vehicle using a subsequent version of the motion planning algorithm.

10. The server according to claim 9, wherein to generate the ground truth simulation environment, the at least one processor causes the server to determine the corresponding object category of each surrounding object in the given road segment.

11. The server according to claim 10, wherein to determine the corresponding object category of each surrounding object, the at least one processor causes the server to solicit corresponding labels for the surrounding objects from a human assessor.

12. The server according to claim 10, wherein: The motion data includes bounding boxes representing the surrounding objects; and To determine the corresponding object category of each surrounding object, the at least one processor causes the server to apply a machine learning algorithm MLA trained to determine the corresponding object category of the given surrounding object based on the corresponding bounding box representing the given surrounding object.

13. The server according to claim 9, wherein, to determine the simulated behavior of the given surrounding object, the at least one processor causes the server to apply an MLA trained to determine the actual behavior of surrounding objects based on the current trajectory of the vehicle.

14. The server according to claim 9, wherein the replacement includes making replacements until, at a given subsequent modeling iteration among the multiple modeling iterations, the corresponding simulated behavior of the given surrounding object corresponds to its corresponding ground truth behavior for the given modeling iteration.

15. The server according to claim 9, wherein, During the given modeling iteration in response to a stop event, the at least one processor further causes the server to: abort modeling the movement of the vehicle without performing subsequent modeling iterations; and remove the current version of the motion planning algorithm from further consideration for determining the trajectory of the vehicle.

16. The server according to claim 15, wherein the stop event includes an accident associated with the vehicle occurring during the given modeling iteration.

Images