Memory-based system and method for vehicle environment map updating

Abstract

A method of adjusting operation of a vehicle based on a traffic control device can include receiving information about a traffic control device in an environment surrounding the vehicle. The method can also include determining a relevance state of the traffic control device based at least in part on the information about the traffic control device. The method can also include determining a road configuration indicated by the information about the traffic control device in response to determining that the relevance state is a relevant state. The method can also include generating an updated environmental map based at least in part on the road configuration indicated by the information about the traffic control device and an existing environmental map. The method can also include adjusting operation of an autonomous driving system of the vehicle based at least in part on the updated environmental map.

Description

Technical Field

[0001] This disclosure relates to systems and methods for real-time environmental mapping of vehicles. Background Technology

[0002] To support functions such as navigation, obstacle detection, and autonomous operation, real-time environment mapping systems can be utilized. These systems are configured to generate a representation of the vehicle's surrounding environment using data collected from one or more sensors. In some examples, sensors include cameras, light detection and ranging (LIDAR), radar, etc. Data from these sensors can be processed by an onboard computing system to construct a dynamic map of the vehicle's environment. These maps can represent static features (such as roads, curbs, and traffic signs) and dynamic objects (such as pedestrians, vehicles, and animals). In some implementations, environment mapping can be combined with localization techniques to achieve situational awareness and localization relative to known features. The environment map can be continuously updated to reflect changes in the vehicle's environment. In some examples, the environment mapping system can operate in conjunction with autonomous driving functions to support navigation and control decisions.

[0003] While systems and methods for environmental mapping have achieved their intended purpose, there is still a need for new and improved systems and methods for environmental mapping, including those for adjusting vehicle operation based on environment-based traffic control devices. Summary of the Invention

[0004] According to several aspects, a method for adjusting vehicle operation based on traffic control devices is provided. The method may include receiving information about traffic control devices in the environment surrounding the vehicle. The method may further include determining a relevance state of the traffic control devices based at least in part on the information about the traffic control devices. The relevance state includes one of a relevant state and an irrelevant state. The method may further include determining a road configuration indicated by the information about the traffic control devices in response to determining that the relevance state is a relevant state. The method may further include generating an updated environment map based at least in part on the road configuration indicated by the information about the traffic control devices and an existing environment map. The updated environment map includes information about road configurations in the environment surrounding the vehicle. The method may further include adjusting the operation of the vehicle's automated driving system based at least in part on the updated environment map.

[0005] In another aspect of this disclosure, determining the relevance status of a traffic control device may further include determining the novelty status of the traffic control device. A novelty status includes either a novel state or a non-novel state. Determining the relevance status of a traffic control device may further include, in response to determining that the novelty status of the traffic control device is a novel state, determining the relevance status of the traffic control device as a relevant state. Determining the relevance status of a traffic control device may further include, in response to determining that the novelty status of the traffic control device is a non-novel state, determining the relevance status of the traffic control device based at least in part on the position of the vehicle relative to the traffic control device.

[0006] In another aspect of this disclosure, determining the novel state of the traffic control device may further include querying the non-transitory memory of the vehicle's controller to obtain a matching record corresponding to the traffic control device, based at least in part on information about the traffic control device and its location. Determining the novel state of the traffic control device may further include, in response to determining that the non-transitory memory does indeed contain a matching record corresponding to the traffic control device, defining the novel state of the traffic control device as a non-novel state. Determining the novel state of the traffic control device may further include, in response to determining that the non-transitory memory does not contain a matching record corresponding to the traffic control device, defining the novel state of the traffic control device as a novel state.

[0007] In another aspect of this disclosure, determining the relevance state of the traffic control device may further include determining the location of the road configuration indicated by the information about the traffic control device, at least in part, based on road rules. Determining the relevance state of the traffic control device may further include determining the position of the vehicle relative to the location of the road configuration indicated by the information about the traffic control device. Determining the relevance state of the traffic control device may further include determining the relevance state of the traffic control device as irrelevant in response to determining that the distance between the vehicle's position and the location of the road configuration indicated by the information about the traffic control device increases over time. Determining the relevance state of the traffic control device may further include determining the relevance state of the traffic control device as relevant in response to determining that the distance between the vehicle's position and the location of the road configuration indicated by the information about the traffic control device decreases over time.

[0008] In another aspect of this disclosure, determining the road configuration indicated by information about the traffic control device may further include querying the non-transitory memory of the vehicle controller to obtain similar records, at least in part based on the information about the traffic control device. Similar records include information similar to the information about the traffic control device. Determining the road configuration indicated by the information about the traffic control device may further include retrieving the road configuration indicated by the information about the traffic control device from the non-transitory memory in response to determining that the non-transitory memory does indeed contain similar records. Determining the road configuration indicated by the information about the traffic control device may further include using a machine learning model to determine the road configuration indicated by the information about the traffic control device, at least in part based on the information about the traffic control device, in response to determining that the non-transitory memory does not contain similar records.

[0009] In another aspect of this disclosure, determining the road configuration indicated by information about the traffic control device using a machine learning model may further include: determining the road configuration indicated by information about the traffic control device, at least in part, using a machine learning model. The machine learning model is a large language model (LLM).

[0010] In another aspect of this disclosure, querying a non-transitory memory to obtain similar records may further include: determining a similarity score between information about the traffic control device and information in each of a plurality of records in the non-transitory memory. Querying a non-transitory memory to obtain similar records may further include: identifying similar records among the plurality of records in response to determining that the similarity score between information about the traffic control device and information in the similar records is greater than or equal to a predetermined similarity threshold.

[0011] In another aspect of this disclosure, generating an updated environmental map may further include identifying one or more differences between the existing environmental map and the road configuration indicated by information about the traffic control device. Generating an updated environmental map may further include determining one or more modifications to the existing environmental map, at least in part based on the one or more differences and the road configuration indicated by the information about the traffic control device. Generating an updated environmental map may further include modifying the existing environmental map, at least in part based on the one or more modifications, to generate the updated environmental map.

[0012] In another aspect of this disclosure, determining one or more modifications to an existing environmental map may also include determining the one or more modifications to the existing environmental map using a rule-based algorithm, based at least in part on the one or more differences and road configurations indicated by information about traffic control devices.

[0013] In another aspect of this disclosure, receiving information about traffic control devices in the environment surrounding the vehicle may further include capturing images of the traffic control devices. The traffic control devices include traffic signs.

[0014] According to several aspects, a system for adjusting vehicle operation based on traffic signs is provided. The system may include a camera, an autonomous driving system, and a controller in electrical communication with the camera and the autonomous driving system. The controller is programmed to capture images of traffic signs in the environment surrounding the vehicle using the camera. The controller is also programmed to determine the relevance state of the traffic signs, at least in part, based on the images. The relevance state includes one of a relevant state and an irrelevant state. The controller is further programmed to determine the road configuration indicated by the images of the traffic signs in response to determining the relevant state as relevant. The controller is also programmed to generate an updated environmental map, at least in part, based on the road configuration indicated by the images of the traffic signs and an existing environmental map. The updated environmental map includes information about the road configuration in the environment surrounding the vehicle. The controller is also programmed to adjust the operation of the autonomous driving system, at least in part, based on the updated environmental map.

[0015] In another aspect of this disclosure, to determine the relevance state of a traffic sign, the controller is also programmed to determine the novelty state of the traffic sign. The novelty state includes either a novel state or a non-novel state. To determine the relevance state of a traffic sign, the controller is also programmed to determine the relevance state of the traffic sign as a relevant state in response to determining that the novelty state of the traffic sign is a novel state. To determine the relevance state of a traffic sign, the controller is also programmed to determine the relevance state of the traffic sign at least in part based on the position of the vehicle relative to the road configuration indicated by the image of the traffic sign.

[0016] In another aspect of this disclosure, to determine the novelty state of a traffic sign, the controller is further programmed to query the controller's non-transitory memory to obtain a matching record corresponding to the traffic sign, based at least in part on the image and location of the traffic sign. To determine the novelty state of a traffic sign, the controller is also programmed to determine the novelty state of the traffic sign as non-novel in response to determining that the non-transitory memory does indeed contain a matching record corresponding to the traffic sign. To determine the novelty state of a traffic sign, the controller is further programmed to determine the novelty state of the traffic sign as novel in response to determining that the non-transitory memory does not contain a matching record corresponding to the traffic sign. To determine the novelty state of a traffic sign, the controller is also programmed to store a new record for the traffic sign in the non-transitory memory in response to determining the novelty state of the traffic sign as novel.

[0017] In another aspect of this disclosure, to determine the relevance state of a traffic sign, the controller is further programmed to determine the location of a road configuration indicated by the image of the traffic sign, at least in part, based on road rules. To determine the relevance state of a traffic sign, the controller is also programmed to determine the position of a vehicle relative to the location of the road configuration indicated by the image of the traffic sign. To determine the relevance state of a traffic sign, the controller is further programmed to determine the relevance state of the traffic sign as irrelevant in response to an increase in the distance between the vehicle's position and the location of the road configuration indicated by the image of the traffic sign over time. To determine the relevance state of a traffic sign, the controller is further programmed to determine the relevance state of the traffic sign as relevant in response to a decrease in the distance between the vehicle's position and the location of the road configuration indicated by the image of the traffic sign over time. To determine the relevance state of a traffic sign, the controller is further programmed to query the controller's non-transitory memory to obtain a matching record corresponding to the traffic sign, at least in part, based on the image of the traffic sign and the location of the traffic sign. To determine the relevance state of a traffic sign, the controller is further programmed to delete the matching record corresponding to the traffic sign from the non-transitory memory in response to determining the relevance state of the traffic sign as irrelevant.

[0018] In another aspect of this disclosure, to determine the road configuration indicated by an image of a traffic sign, the controller is also programmed to query the controller's non-transitory memory at least in part based on the image of the traffic sign to obtain a similarity record. The similarity record includes information similar to the image of the traffic sign. To determine the road configuration indicated by the image of the traffic sign, the controller is also programmed to retrieve the road configuration indicated by the image of the traffic sign from the non-transitory memory in response to determining that the non-transitory memory does indeed contain a similar record. To determine the road configuration indicated by the image of the traffic sign, the controller is also programmed to determine the road configuration indicated by the image of the traffic sign using a Large Language Model (LLM) machine learning model, at least in part based on the image of the traffic sign, in response to determining that the non-transitory memory does not contain a similar record. To determine the road configuration indicated by the image of the traffic sign, the controller is also programmed to save a new record for the traffic sign in the non-transitory memory in response to determining the road configuration indicated by the image of the traffic sign using the machine learning model.

[0019] In another aspect of this disclosure, to generate an updated environmental map, the controller is also programmed to determine one or more differences between the existing environmental map and the road configuration indicated by images of traffic signs. To generate the updated environmental map, the controller is also programmed to determine one or more modifications to the existing environmental map using a rule-based algorithm, based at least in part on the one or more differences and the road configuration indicated by images of traffic signs. To generate the updated environmental map, the controller is also programmed to modify the existing environmental map, based at least in part on the one or more modifications, to generate the updated environmental map.

[0020] In another aspect of this disclosure, to adjust the operation of the autonomous driving system, the controller is also programmed to recalculate the vehicle path using the autonomous driving system's path planning algorithm, at least in part, based on modifications to the existing environment map. To further adjust the operation of the autonomous driving system, the controller is also programmed to use the autonomous driving system to control the vehicle, thereby driving the vehicle along the driving path.

[0021] According to several aspects, a method for adjusting vehicle operation based on a traffic control device is provided. The method may include receiving information about traffic signs in the environment surrounding the vehicle. The method may further include determining a novelty state of the traffic signs. A novelty state includes either a novel state or a non-novel state. The method may further include determining a relevant state of the traffic signs as a relevant state in response to determining that the novelty state of the traffic signs is a novel state. The method may further include determining the relevant state of the traffic signs at least partially based on the vehicle's position relative to the traffic signs in response to determining that the novelty state of the traffic signs is a non-novel state. The method may further include determining a road configuration indicated by the information about the traffic signs in response to determining that the relevant state is a relevant state. The method may further include generating an updated environment map based at least partially on the road configuration indicated by the information about the traffic signs and an existing environment map. The updated environment map includes information about the road configuration in the environment surrounding the vehicle. The method may further include adjusting the operation of the vehicle's automated driving system based at least partially on the updated environment map.

[0022] In another aspect of this disclosure, determining the road configuration indicated by information about traffic signs may further include: querying the non-transitory memory of the vehicle's controller to obtain similar records, at least in part based on the information about traffic signs. Similar records include information similar to the information about traffic signs. Determining the road configuration indicated by the information about traffic signs may further include: retrieving the road configuration indicated by the information about traffic signs from the transient memory in response to determining that the non-transitory memory does indeed include similar records. Determining the road configuration indicated by the information about traffic signs may further include: in response to determining that the non-transitory memory does not include similar records, using a Large Language Model (LLM) machine learning model, at least in part based on the information about traffic signs, to determine the road configuration indicated by the information about traffic signs.

[0023] In another aspect of this disclosure, querying a non-transitory memory to obtain similar records may further include: determining a similarity score between information about a traffic sign and information in each of a plurality of records in the non-transitory memory. Querying a non-transitory memory to obtain similar records may further include: identifying similar records among the plurality of records in response to determining that the similarity score between information about a traffic sign and information in similar records is greater than or equal to a predetermined similarity threshold.

[0024] Other applicable areas will be apparent from the description provided herein. It should be understood that the specification and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure. Attached Figure Description

[0025] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0026] Figure 1 This is a schematic diagram of a system for adjusting vehicle operation based on a traffic control device, according to an exemplary embodiment.

[0027] Figure 2 This is a flowchart of a method for adjusting vehicle operation based on a traffic control device, according to an exemplary embodiment; and

[0028] Figure 3 This is a schematic diagram of an exemplary road used to illustrate the functionality of an exemplary memory controller module according to an exemplary embodiment. Detailed Implementation

[0029] The following description is merely exemplary and is not intended to limit this disclosure, its application, or its uses.

[0030] In various aspects of this disclosure, updating environmental maps based on traffic control devices (e.g., traffic signs) in the environment surrounding the vehicle is advantageous. This improves the performance of driver assistance and autonomous driving systems for the vehicle by providing more accurate and up-to-date information. However, utilizing information from traffic control devices requires understanding the context of the traffic control devices within the environment. Therefore, this disclosure provides a novel and improved system and method for updating environmental maps and adjusting vehicle operation, which utilizes memory to accurately interpret traffic control devices.

[0031] Reference Figure 1 The diagram illustrates a system for adjusting the operation of a vehicle based on traffic control devices, and the system is generally indicated by reference numeral 10. System 10 is illustrated with an exemplary vehicle 12. Although a passenger vehicle is shown, it should be understood that vehicle 12 can be any type of vehicle without departing from the scope of this disclosure. System 10 typically includes a controller 14, one or more sensors 16, and an autonomous driving system 18.

[0032] The controller 14 is used to implement a method 100 for adjusting vehicle operation based on traffic control devices, as will be described below. The controller 14 includes at least one processor 20 and a non-transitory computer-readable storage device, memory, or medium 22. The processor 20 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 14, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or generally a means for executing instructions.

[0033] Computer-readable storage device or medium 22 may include volatile and non-volatile storage devices in the form of read-only memory (ROM), random access memory (RAM), and retention memory (KAM). KAM is a persistent or non-volatile memory that can be used to store various operational variables when the processor 20 is powered off. Computer-readable storage device or medium 22 may be implemented using various memory devices, such as PROM (programmable read-only memory), EPROM (electrically programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), flash memory, or another electrical, magnetic, optical, or combined memory device capable of storing data, some of which represents executable instructions used by the controller 14 to control various systems of the vehicle 12. It should be understood that, without departing from the scope of this disclosure, medium 22 may be implemented as a single unified storage device, multiple independent storage devices, and / or multiple virtual storage containers (e.g., partitions) on one or more memory devices.

[0034] Controller 14 may also include multiple controllers that are electrically communicating with each other. Controller 14 may interconnect with additional systems and / or controllers of vehicle 12, thereby allowing controller 14 to acquire data such as vehicle speed, acceleration, braking, and steering angle.

[0035] The controller 14 communicates electrically with the one or more sensors 16 and the autonomous driving system 18. In an exemplary embodiment, the electrical communication is established using, for example, a CAN network, a FLEXRAY network, a local area network (such as WiFi, Ethernet, etc.), a Serial Peripheral Interface (SPI) network, etc. It should be understood that various additional wired and wireless technologies and communication protocols used for communicating with the controller 14 are within the scope of this disclosure. It should also be understood that, within the scope of this disclosure, electrical communication also includes power and / or energy transfer between electrical devices (e.g., through the use of conductive wires and / or wireless power transfer technologies).

[0036] The one or more sensors 16 are used to acquire information related to the vehicle 12. In an exemplary embodiment, the one or more sensors 16 include at least a camera system 26.

[0037] Camera system 26 is a sensing sensor for capturing images and / or video of the environment 28 surrounding vehicle 12. In an exemplary embodiment, camera system 26 includes a camera and / or video camera positioned to observe the environment 28 surrounding vehicle 12. In a non-limiting example, camera system 26 includes a camera attached to the interior of vehicle 12 (e.g., attached to the headliner of vehicle 12) and having a field of view through the windshield of vehicle 12. In another non-limiting example, camera system 26 includes a camera attached to the exterior of vehicle 12 (e.g., attached to the top of vehicle 12) and having a field of view of the environment 28 in front of vehicle 12.

[0038] In another exemplary embodiment, camera system 26 is a panoramic camera system comprising a plurality of cameras (also referred to as satellite cameras) arranged to provide a field of view of environment 28 adjacent to the entire side of vehicle 12. In a non-limiting example, camera system 26 includes a forward-facing camera (e.g., mounted in the front grille of vehicle 12), a rear-facing camera (e.g., mounted on the tailgate of vehicle 12), and two side cameras (e.g., mounted below each of the two side mirrors of vehicle 12). In another non-limiting example, camera system 26 further includes an additional rear-view camera mounted near the high-mounted brake light of vehicle 12.

[0039] It should be understood that camera systems with additional cameras and / or additional mounting locations are within the scope of this disclosure. As discussed above, camera system 26 is in electrical communication with controller 14.

[0040] In another exemplary embodiment, the one or more sensors 16 further include sensors for determining performance data regarding the vehicle 12. In a non-limiting example, the one or more sensors 16 further include at least one of a motor speed sensor, a motor torque sensor, an electric drive motor voltage and / or current sensor, an accelerator pedal position sensor, a brake position sensor, a coolant temperature sensor, a cooling fan speed sensor, and a transmission oil temperature sensor.

[0041] In another exemplary embodiment, the one or more sensors 16 further include sensors for determining information about the environment inside the vehicle 12. In a non-limiting example, the one or more sensors 16 further include at least one of a seat occupancy sensor, a cabin air temperature sensor, a cabin motion detection sensor, a cabin camera, and a cabin microphone.

[0042] In another exemplary embodiment, the one or more sensors 16 further include sensors for determining information about the environment 28 surrounding the vehicle 12. In a non-limiting example, the one or more sensors 16 also include at least one of an ambient temperature sensor, an atmospheric pressure sensor, a global navigation satellite system (GNSS), a vehicle communication system 30, etc.

[0043] The controller 14 uses the vehicle communication system 30 to communicate with other systems outside the vehicle 12. For example, the vehicle communication system 30 has the capability to communicate with vehicles (“vehicle-to-vehicle (V2V)” communication), infrastructure (“vehicle-to-infrastructure (V2I)” communication), remote systems at remote call centers (such as on-star systems of general motors), and / or personal devices. Generally, the term “vehicle-to-everything communication” (“V2X” communication) refers to communication between the vehicle 12 and any remote system (e.g., vehicles, infrastructure, and / or remote systems).

[0044] In some embodiments, the vehicle communication system 30 is a wireless communication system configured to communicate via a wireless local area network (WLAN) using the IEEE 802.11 standard or by using cellular data communication (e.g., using GSMA standards such as SGP.02, SGP.22, SGP.32, etc.). Therefore, the vehicle communication system 30 may also include an embedded universal integrated circuit card (eUICC) configured to store a profile of at least one cellular connectivity configuration (e.g., an embedded user identity module (eSIM) profile).

[0045] The vehicle communication system 30 is also configured to communicate via a personal area network (e.g., Bluetooth), near field communication (NFC), and / or any additional type of radio frequency communication. However, additional or alternative communication methods, such as Dedicated Short Range Communication (DSRC) channels and / or mobile communication protocols based on 3GPP standards, are also considered within the scope of this disclosure. A DSRC channel refers to a unidirectional or bidirectional short- to medium-range wireless communication channel designed specifically for automotive use, along with a corresponding set of protocols and standards. 3GPP refers to a partnership among several standards organizations that develop protocols and standards for mobile communications. 3GPP standards are architected in “releases”. Therefore, communication methods based on 3GPP Releases 14, 15, 16, and / or future 3GPP releases are considered within the scope of this disclosure.

[0046] Therefore, the vehicle communication system 30 may include one or more antennas and / or a communication transceiver for receiving and / or transmitting signals (e.g., Cooperative Sensing Messages (CSM)). The vehicle communication system 30 is configured to wirelessly transmit information between vehicle 12 and another vehicle. Further, the vehicle communication system 30 is configured to wirelessly transmit information between vehicle 12 and infrastructure or other vehicles. It should be understood that, without departing from the scope of this disclosure, the vehicle communication system 30 may be integrated with the controller 14 (e.g., integrated on the same circuit board as the controller 14, or otherwise as part of the controller 14).

[0047] In another exemplary embodiment, the one or more sensors 16 include additional sensing sensors capable of sensing objects in the environment 28 surrounding the vehicle 12 and / or measuring distances. In a non-limiting example, the one or more sensors 16 also include at least one of an ultrasonic ranging sensor, a radar sensor, a time-of-flight sensor, a lidar (LIDAR) sensor 32, etc.

[0048] LIDAR sensor 32 is used for remote sensing and environmental mapping by emitting laser pulses and measuring the time required for the laser pulses to return to the LIDAR sensor after impacting an object. In an exemplary embodiment, LIDAR sensor 32 includes a LIDAR laser source, a LIDAR scanner or mirror, a LIDAR photodetector, and a LIDAR time-of-flight measurement system. In a non-limiting example, the LIDAR laser source emits laser pulses that travel to a target area, and the LIDAR scanner guides these pulses in different directions. The emitted laser pulses interact with objects in the environment, and the reflected light from the laser pulses is captured by the LIDAR photodetector. The LIDAR time-of-flight measurement system calculates the distance to the object based on the time between the emission of the laser pulse by the LIDAR laser source and the reception of the reflected laser pulse by the LIDAR photodetector. LIDAR sensor 32 is in electrical communication with controller 14, as discussed above.

[0049] In one example, at least one of the one or more sensors 16 is attached to the interior of the vehicle 12 (e.g., to the headliner of the vehicle 12), thereby providing a view through the windshield of the vehicle 12. In another example, at least one of the one or more sensors 16 is attached to the exterior of the vehicle 12 (e.g., to the roof of the vehicle 12), thereby providing a view of the environment 28 surrounding the vehicle 12. It should be understood that the various types of sensors discussed above are merely exemplary in nature, and the one or more sensors 16 may include various additional and / or alternative sensors without departing from the scope of this disclosure. The one or more sensors 16 are in electrical communication with the controller 14, as discussed above.

[0050] The automated driving system 18 is used to assist the occupants of the vehicle 12 to enhance their perception and / or control of the vehicle 12. Within the scope of this disclosure, the automated driving system 18 encompasses systems that provide any level of assistance to the occupants (e.g., blind spot warning, lane departure warning, etc.) and systems capable of autonomously driving the vehicle 12 under some or all conditions (e.g., automatic lane keeping, adaptive cruise control, fully automated driving, etc.). It should be understood that all levels of driving automation (i.e., SAE Level 0, SAE Level 1, SAE Level 2, SAE Level 3, SAE Level 4, and SAE Level 5) as defined, for example, by SAE J3016 are within the scope of this disclosure.

[0051] In an exemplary embodiment, the autonomous driving system 18 is configured to detect and / or receive information about the environment 28 surrounding the vehicle 12 and process that information to provide assistance to the occupants. In some embodiments, the autonomous driving system 18 is a software module running on the controller 14. In other embodiments, the autonomous driving system 18 includes a separate autonomous driving system controller, similar to the controller 14, capable of processing information about the environment 28 surrounding the vehicle 12. In an exemplary embodiment, the autonomous driving system 18 can operate in a manual operation mode, a partially automated operation mode, and a fully automated operation mode.

[0052] Within the scope of this disclosure, manual operation mode means that the autonomous driving system 18 provides warnings or prompts to the occupants but does not directly intervene in or control the vehicle 12. In a non-limiting example, the autonomous driving system 18 receives information from the one or more sensors 16. For example, by using techniques such as computer vision, the autonomous driving system 18 understands the environment 28 surrounding the vehicle 12 and provides assistance to the occupants. For example, if the autonomous driving system 18 identifies, based on data from the one or more sensors 16, that the vehicle 12 may collide with a remote vehicle, the autonomous driving system 18 may use a display to provide a warning to the occupants.

[0053] Within the scope of this disclosure, a partially automated operating mode means that the autonomous driving system 18 provides warnings or notifications to the occupants and may, in certain circumstances, directly intervene in or control the vehicle 12. In a non-limiting example, the autonomous driving system 18 additionally communicates electrically with components of the vehicle 12, such as its braking system, propulsion system, and / or steering system, enabling the autonomous driving system 18 to control the behavior of the vehicle 12. In a non-limiting example, the autonomous driving system 18 may control the behavior of the vehicle 12 by applying the brakes to avoid an impending collision. In another non-limiting example, the autonomous driving system 18 may control the steering system of the vehicle 12 to achieve an automatic lane-keeping function. In yet another non-limiting example, the autonomous driving system 18 may control the braking system, propulsion system, and steering system of the vehicle 12 to temporarily drive the vehicle 12 toward a predetermined destination. However, occupant intervention may be required at any time. In an exemplary embodiment, the autonomous driving system 18 may include additional components, such as an eye-tracking device configured to monitor the occupant's level of attention and ensure the occupant is ready to take over control of the vehicle 12.

[0054] Within the scope of this disclosure, the fully automatic operating mode means that the autonomous driving system 18 uses data from the one or more sensors 16 to understand the environment 28 and control the vehicle 12, thereby driving the vehicle 12 to a predetermined destination without the need for control or intervention by the occupants.

[0055] The autonomous driving system 18 operates using a path planning algorithm configured to generate a safe and efficient trajectory for the vehicle to navigate in the environment surrounding the vehicle 12. In an exemplary embodiment, the path planning algorithm is a machine learning algorithm trained to output control signals for the vehicle 12 based on input data collected from the one or more sensors 16. In another exemplary embodiment, the path planning algorithm is a deterministic algorithm programmed to output control signals for the vehicle 12 based on data collected from the one or more sensors 16.

[0056] In a non-limiting example, the path planning algorithm generates a series of waypoints or continuous paths that vehicle 12 should follow to reach its destination while complying with rules, regulations, and safety constraints. This series of waypoints or continuous paths is generated at least in part based on an environmental map and the current state of vehicle 12 (i.e., the position, speed, and orientation of vehicle 12). The environmental map contains information, for example, about lane boundaries, road geometry, and / or other relevant features. In an exemplary embodiment, the environmental map is stored in medium 22 of controller 14 and / or in a remote database or server. In another exemplary embodiment, data collected from the one or more sensors 16 is parsed to create, update, and / or expand the environmental map, as discussed in more detail below.

[0057] It should be understood that the autonomous driving system 18 may include any software and / or hardware modules configured to operate in the manual operation mode, partial autonomous driving mode, or fully autonomous driving mode as described above. The autonomous driving system 18 maintains electrical communication with the controller 14, as discussed above.

[0058] Reference Figure 2A flowchart of a method 100 for adjusting vehicle operation based on traffic control devices is shown. Method 100 begins at block 102 and proceeds to block 104. At block 104, controller 14 uses one or more sensors 16 to receive information about traffic control devices in the environment 28 surrounding vehicle 12. Within the scope of this disclosure, traffic control devices include signs, signals, markings, or other features (e.g., traffic signs, traffic lights, road markings, speed bumps, etc.) for regulating traffic flow. Within the scope of this disclosure, information about traffic control devices may include images of the traffic control devices, physical characteristics (e.g., shape, size, color, etc.), the content of text / images / pictographs printed or displayed on the traffic control devices, the status of lights / lights or other dynamic visual indicators of the traffic control devices, and / or scene information about the traffic control devices (e.g., position, height, or angle, etc.). In an exemplary embodiment, controller 14 uses a camera system 26 of one or more sensors 16 to capture images of traffic control devices (e.g., traffic signs) in the environment 28. In another exemplary embodiment, controller 14 uses one or more sensors 16 to determine the position of the traffic control devices relative to vehicle 12. Following box 104, method 100 continues to box 106.

[0059] In block 106, controller 14 determines the novelty state of the traffic control device in block 104. Within the scope of this disclosure, the novelty state indicates whether controller 14 has received information about this particular traffic control device. For example, while crossing a road, one or more sensors 16 may receive information about the same traffic control device multiple times (e.g., as vehicle 12 approaches a traffic sign, camera system 26 may capture multiple images at different times, where two or more of the multiple images include the same traffic sign). A novelty state includes either a novelty state (i.e., the information received at block 104 is the first information received about a particular traffic control device) or a non-novelty state (i.e., the information received at block 104 is not the first information received about a particular traffic control device).

[0060] In an exemplary embodiment, to determine a novelty state, controller 14 queries medium 22 to obtain a matching record corresponding to the traffic control device. In a non-limiting example, controller 14 queries medium 22 based on information about the traffic control device determined at block 104. For example, if the information includes an image, controller 14 queries medium 22 to obtain a record with a matching shape, size, text, color, etc. For example, controller 14 can extract text from information about the traffic control device (e.g., text written on a traffic sign) and search medium 22 based on the extracted text. In another non-limiting example, controller 14 queries medium 22 based on the location of the traffic control device. For example, controller 14 uses dead reckoning based on the speed and heading of vehicle 12 to track the position of the traffic control device relative to vehicle 12. Controller 14 then queries medium 22 to obtain records of locations within a predetermined radius (e.g., 1 meter) of the location of the traffic control device. If medium 22 does indeed include a matching record corresponding to the traffic control device at block 104, the novelty state is determined to be a non-novelty state, and method 100 continues to block 108. If medium 22 does not include a matching record corresponding to the traffic control device in box 104, the novel state is identified as a novel state, and method 100 continues to box 110, as will be discussed in detail below.

[0061] In block 108, controller 14 determines the relevance state of the traffic control device in block 104. Within the scope of this disclosure, the relevance state indicates whether the traffic control device in block 104 is still relevant to the future path of vehicle 12. The relevance state includes one of the following: a relevant state and an irrelevant state. In an exemplary embodiment, the relevance state is determined at least in part based on the position of vehicle 12 relative to the traffic control device in response to determining that the novelty state is a non-novel state. In a non-limiting example, controller 14 first determines the location of a road configuration indicated by information about the traffic control device. Within the scope of this disclosure, a road configuration indicated by information about the traffic control device refers to a road characteristic, condition, or configuration indicated or involved by the traffic control device. For example, if the traffic control device is a traffic sign marked "Lane End," the location of the road configuration indicated by the traffic sign is at the lane end ahead of the sign.

[0062] In an exemplary embodiment, controller 14 determines the location of a road configuration indicated by information about a traffic control device based on one or more road rules or regulations. In a non-limiting example, controller 14 uses road rules or regulations from the Uniform Traffic Control Device Manual (MUTCD) to determine the location of the road configuration indicated by information about the traffic control device based on the shape, size, color, content, etc. of the traffic control device. For example, MUTCD specifies that a "lane termination" sign should be placed to provide vehicles with sufficient time (e.g., 14.0 to 14.5 seconds) before the lane terminates. Therefore, the location of the road configuration indicated by information about the traffic control device is determined based on the location of the traffic control device, road regulations, and / or additional information (e.g., the speed limit or average driving speed of the road).

[0063] In another exemplary embodiment, the controller 14 determines the location of a road configuration indicated by information about a traffic control device based on data received using the one or more sensors 16. In a non-limiting example, the one or more sensors 16 are used to collect perception data and infer the location of the road configuration indicated by information about a traffic control device from the perception data.

[0064] After determining the location of the road configuration indicated by the information from the traffic control device, the controller 14 determines the position of the vehicle 12 relative to the road configuration. In an exemplary embodiment, the controller 14 uses a dead reckoning algorithm to determine the position of the vehicle 12 relative to the road configuration, tracking the position of the vehicle 12 relative to the traffic control device based on the vehicle 12's speed and heading, and the time elapsed since receiving information about the traffic control device from block 104. In another exemplary embodiment, the controller 14 uses one or more sensors 16 to determine the position of the vehicle 12 relative to the road configuration based on, for example, sensing data and / or GNSS data.

[0065] After determining the position of vehicle 12 relative to the road configuration, controller 14 repeatedly recalculates the position of vehicle 12 relative to the road configuration to determine whether the distance between the position of vehicle 12 and the position of the road configuration increases or decreases over time. If the distance between the position of vehicle 12 and the position of the road configuration increases, vehicle 12 is deviating from the position of the road configuration indicated by the traffic control device. Therefore, the relevant state of the traffic control device is determined to be irrelevant, and method 100 continues to block 112. If the distance between the position of vehicle 12 and the position of the road configuration decreases, vehicle 12 is approaching the position of the road configuration indicated by the traffic control device. Therefore, the relevant state of the traffic control device is determined to be relevant, and method 100 continues to block 114, as will be discussed in more detail below.

[0066] In block 112, in response to determining that the relevance state of the traffic control device is irrelevant, controller 14 deletes the matching record corresponding to the traffic control device found in block 106 from medium 22. In another exemplary embodiment, instead of deleting the matching record, controller 14 marks the matching record as irrelevant but retains it. After block 112, method 100 proceeds to block 116 and enters a standby state.

[0067] In block 110, in response to determining that the novel state of the traffic control device is a novel state, controller 14 saves the new record for the traffic control device from block 104 in medium 22. After block 110, method 100 continues to block 118.

[0068] In block 118, controller 14 queries medium 22 to obtain records similar to the traffic control device in block 104. Within the scope of this disclosure, similar records correspond to traffic control devices having a meaning similar to the specific traffic control device in block 104. In an exemplary embodiment, to query medium 22 to obtain records similar to the traffic control device in block 104, controller 14 determines a similarity score between the information about the traffic control device determined at block 104 and the information in each of a plurality of records stored in medium 22. In a non-limiting example, to determine the similarity score, controller 14 generates an embedding vector for the information about the traffic control device determined at block 104, and generates an embedding vector for each of the plurality of records stored in medium 22. By comparing the embedding vectors (e.g., using a dot product operation), controller 14 determines a similarity score for each of the plurality of records.

[0069] In a non-limiting example, controller 14 identifies a similar record as the record with the highest similarity score among the plurality of records. In another non-limiting example, controller 14 identifies a similar record as the record with a similarity score greater than or equal to a predetermined similarity threshold among the plurality of records. In yet another non-limiting example, controller 14 identifies a similar record as the record with the highest similarity score among the plurality of records with a similarity score greater than or equal to a predetermined similarity threshold. If no similar record is found in medium 22, method 100 proceeds to box 120. If a similar record is found in medium 22, method 100 proceeds to box 114, as will be discussed in more detail below.

[0070] In block 120, in response to determining that medium 22 does not contain similar records, controller 14 uses a machine learning model to determine the road configuration indicated by the traffic control device based on the information received at block 104 regarding the traffic control device. In an exemplary embodiment, controller 14 passes the information received at block 104 regarding the traffic control device (e.g., an image of the traffic control device) to the machine learning model, and the machine learning model provides a machine-readable response (e.g., in the form of a data structure) describing the road configuration indicated by the traffic control device.

[0071] For example, if the information about the traffic control device is an image of an intersection lane control traffic sign (i.e., a traffic sign indicating which lanes are permitted to turn and / or go straight at an intersection), the machine learning model provides a data structure (e.g., an array) representing the permitted passage for each lane indicated by that traffic sign. In another example, if the information about the traffic control device is an image of a warning traffic sign (e.g., a "Lane End" traffic sign), the machine learning model provides a data structure (e.g., a tuple) including alphanumeric sign symbols (e.g., "W4-2" for a "Lane End" sign) and the corresponding lane to which the traffic sign applies (e.g., "1" for the first lane).

[0072] It should be understood that the examples of machine learning model inputs and outputs discussed above are illustrative in nature, and machine learning models can be configured to provide outputs in any type of machine-readable format. It should be understood that various types and configurations of machine learning models, including, for example, large language models (LLMs), can be utilized within the scope of this disclosure. It should also be understood that machine learning models can be executed using controller 14, integrated or external graphics processing units (GPUs), other dedicated modules / controllers / computers of vehicle 12, and / or external devices or servers. Following box 120, method 100 continues to box 122.

[0073] In block 122, in response to determining the road configuration indicated by information about the traffic control device at block 120, controller 14 stores the new record for the traffic control device from block 104 in medium 22. In an exemplary embodiment, this record includes the information about the traffic control device received at block 104, and the corresponding machine-readable road configuration determined at block 120 using a machine learning model. Therefore, if a similar traffic control device is encountered in the future, medium 22 can be queried to determine the meaning of the traffic control device (i.e., the road configuration indicated by the information about the traffic control device) without executing a machine learning model, thus allowing for faster processing. Additionally, medium 22 will accumulate records corresponding to common traffic control devices to reduce the frequency of executing machine learning models, thereby saving computational resources. After block 122, method 100 continues to block 114.

[0074] In block 114, in response to determining at block 106 that medium 22 does indeed include a matching record and at block 108 that the relevance status is relevant, or in response to determining at block 118 that medium 22 does indeed include a similar record, or in response to saving a new record to medium 22 at block 122, controller 14 retrieves a road configuration indicated by traffic control device information from medium 22. In an exemplary embodiment, controller 14 queries medium 22 to retrieve a road configuration indicated by information about the traffic control device. In a non-limiting example, controller 14 retrieves a road configuration from matching records in medium 22. In another non-limiting example, controller 14 retrieves a road configuration from similar records in medium 22. In yet another non-limiting example, controller 14 retrieves a road configuration from a new record in medium 22. After block 114, method 100 continues to 124.

[0075] In block 124, controller 14 generates an updated environmental map based at least in part on road configurations indicated by information about the traffic control device and an existing environmental map. Within the scope of this disclosure, the existing environmental map is an environmental map generated based on perception data (e.g., camera and / or LiDAR data) and / or navigation map data (e.g., GNSS data) received using one or more sensors 16. The existing environmental map is modified based on the road configurations indicated by information about the traffic control device to generate the updated environmental map. Therefore, the updated environmental map includes information about road configurations in the environment 28 surrounding vehicle 12, and more specifically, it includes information about road configurations indicated by information about the traffic control device. Thus, using system 10 and method 100, the existing environmental map is updated to incorporate information provided by the traffic control device in environment 28.

[0076] In an exemplary embodiment, to generate an updated environment map, controller 14 determines one or more differences between the existing environment map and the road configuration indicated by information about the traffic control device. Then, controller 14 uses a rule-based algorithm to determine one or more modifications to the existing environment map, based at least in part on the one or more differences and the road configuration indicated by the information about the traffic control device. In a non-limiting example, the rule-based algorithm performs pre-programmed modifications to the existing environment map based on machine-readable road configurations provided by a machine learning model. Controller 14 then modifies the existing environment map, at least in part, based on the one or more modifications, to generate the updated environment map.

[0077] For example, if information from a traffic control device indicates that the road configuration specifies that the third lane terminates at 400 meters, but the existing environment map includes a third lane extending beyond 400 meters, the existing environment map is modified to remove the third lane beyond 400 meters. This modified existing environment map is referred to as the updated environment map. In an exemplary embodiment, the updated environment map is stored in medium 22 for later use. In another exemplary embodiment, the updated environment map is also transmitted to one or more remote vehicle and / or server systems for storage, aggregation, and / or distribution using vehicle communication system 30. Following box 124, method 100 continues to box 126.

[0078] In block 126, controller 14 adjusts the operation of autonomous driving system 18 based at least in part on the updated environment map generated at block 124. In an exemplary embodiment, to adjust the operation of autonomous driving system 18, controller 14 recalculates the vehicle path using the path planning algorithm of autonomous driving system 18, based at least in part on the modification of the existing environment map performed at block 124. In a non-limiting example, controller 14 transmits the updated environment map generated at block 124 to autonomous driving system 18, and autonomous driving system 18 recalculates the vehicle path using the updated environment map. In an exemplary embodiment, controller 14 uses autonomous driving system 18 to control vehicle 12 to drive the vehicle along the vehicle path. After block 126, method 100 proceeds to block 116 and enters a standby state.

[0079] In an exemplary embodiment, controller 14 repeatedly exits standby state 116 and restarts method 100 at block 102. In a non-limiting example, controller 14 exits standby state 116 and restarts method 100 based on a timer, for example, once every 300 milliseconds.

[0080] Continue to refer to Figure 2 In a non-limiting example, controller 14 is configured with a set of computer-executable instructions (i.e., a computer program) to execute method 100. This set of computer-executable instructions can be organized into a software architecture. This software architecture comprises multiple software modules. Within the scope of this disclosure, a software module represents an independent software component capable of receiving input data, manipulating the input data according to computer-executable instructions, storing intermediate values ​​(i.e., variables) and / or persistent data (i.e., utilizing non-transitory memory), and providing output data.

[0081] In a non-limiting example, the software modules are implemented programmatically as functions, methods, subroutines, subroutines, etc. In an exemplary embodiment, one or more of the software modules are configured to operate in a non-blocking manner, meaning that a failure or pause in one software module will not cause a failure or pause in any other software module. In a non-limiting example, one or more of the software modules execute concurrently on independent threads of controller 14. In another non-limiting example, one or more of the software modules execute on independent controllers distributed throughout vehicle 12.

[0082] In an exemplary embodiment, the software architecture includes a memory controller module 202, a machine learning model controller module 204, a machine learning model execution module 206, and an environment map updater module 208. In a non-limiting example, the memory controller module 202 is responsible for determining the novelty and relevance of traffic control devices and tracking previously observed traffic control devices to prevent duplicate identification / interpretation. The memory controller module 202 executes blocks 106, 108, 110, and 112 of method 100.

[0083] Reference Figure 3 A schematic diagram of an exemplary road 302 is shown to illustrate the function of the memory controller module 202. The exemplary road 302 includes a termination lane 304 and an example traffic sign 306. In a non-limiting example, the exemplary traffic sign 306 includes text and / or an image indicating that the termination lane 304 is about to end. While the vehicle 12 is in the first area 308, the one or more sensors 16 can repeatedly detect the exemplary traffic sign 306, as discussed with reference to block 106 of method 100. Therefore, the memory controller module 202 stores information about the exemplary traffic sign 306 in the medium 22 and prevents redundant processing of the exemplary traffic sign 306.

[0084] While vehicle 12 is in the second region 310, memory controller module 202 tracks the position of vehicle 12 relative to the road configuration indicated by exemplary traffic sign 306 and determines the relevance state of exemplary traffic sign 306, as discussed above with reference to block 108 of method 100. In an exemplary embodiment, memory controller module 202 first determines the distance between exemplary traffic sign 306 and the position of the road configuration indicated by exemplary traffic sign 306 (e.g., based on road rules or regulations). Subsequently, memory controller module 202 uses dead reckoning to track the position of vehicle 12 relative to exemplary traffic sign 306. Then, memory controller module 202 determines the position of vehicle 12 relative to the road configuration indicated by exemplary traffic sign 306.

[0085] Therefore, by using dead reckoning to track the example traffic sign 306, the memory controller module 202 can still "remember" the example traffic sign 306 even if it is no longer detected by the one or more sensors 16. When the vehicle 12 enters the third area 312, the memory controller module 202 determines that the example traffic sign 306 is no longer relevant and removes or deactivates the corresponding record in the medium 22, as discussed in block 112 of the above-mentioned method 100. It should be understood that the above-mentioned method 100... Figure 3 The scenarios discussed are merely illustrative in nature.

[0086] Refer again Figure 2 In a non-restrictive example, the machine learning model controller module 204 is responsible for reducing calls to the machine learning model execution module 206 by maintaining a knowledge base of the meanings of various examples of traffic control devices (i.e., road configurations indicated by various examples of traffic control devices). Boxes 114, 118, and 122 in method 100 are executed by the machine learning model controller module 204.

[0087] In a non-limiting example, to parse a traffic control device, the machine learning model controller module 204 examines medium 22 (or a separate physical / virtual memory) to obtain records of traffic control devices similar to those discussed above with reference to 118. In a non-limiting example, to examine medium 22, the machine learning model controller module 204 compares the embedding of the traffic control device with the embeddings of previously encountered traffic control devices stored in medium 22. In a non-limiting example, a multimodal text and image encoder is used to generate the embedding. In a non-limiting example, a cosine metric is used to compare the embedding.

[0088] If a similar record is identified, the machine learning model controller module 204 will retrieve the similar record instead of executing the machine learning model execution module 206. If no similar record is already stored in the medium 22, the machine learning model controller module 204 uses the machine learning model execution module 206 to determine the road configuration indicated by the traffic control device. Subsequently, the machine learning model controller module 204 saves the information determined by the machine learning model execution module 206 to the medium 22 for subsequent retrieval.

[0089] In an exemplary embodiment, the memory controller module 202 and the machine learning model controller module 204 use independent memory (i.e., independent physical storage devices and / or independent virtual storage containers, such as partitions). The memory of the controller module 202 acts as "short-term memory" for tracking recently encountered traffic control devices and evaluating their relevance. The memory of the machine learning controller module 204 acts as "long-term memory" for maintaining a knowledge base about the meaning of various examples of traffic control devices (i.e., road configurations indicated by various examples of traffic control devices).

[0090] In a non-limiting example, the machine learning model execution module 206 is responsible for executing the machine learning model (e.g., an LLM), as discussed in the reference method above. For example, the machine learning model execution module 206 is responsible for prompting the machine learning model with input information, receiving and parsing the output from the machine learning model, and providing the output to the machine learning model controller module 204. The machine learning model execution module 206 can be implemented using controller 14, an integrated or external graphics processing unit (GPU), another dedicated module / controller / computer of vehicle 12, and / or external devices or servers. The machine learning model execution module 206 executes box 120 in method 100.

[0091] In a non-limiting example, the environment map updater module 208 is responsible for updating the existing environment map based on information from the machine learning model execution module 206 and / or the machine learning model controller module 204. The environment map updater module 208 executes block 124 of method 100. It should be understood that the software architecture discussed above (including the configuration and purpose of the modules and the steps of method 100 executed by each module) is merely illustrative in nature, and various additional and / or alternative software architectures can be used to implement method 100 within the scope of this disclosure.

[0092] The system 10 and method 100 of this disclosure have several advantages. By using the memory controller module 202 and the associated steps of method 100, redundant processing of the same traffic control device can be reduced, and each traffic control device can be directly associated with the corresponding road configuration. By using the machine learning model controller module 204 and the associated steps of method 100, a knowledge base is maintained regarding the meaning of each example of the traffic control device (i.e., the road configuration indicated by each example of the traffic control device), resulting in a reduced frequency of calls to the machine learning model execution module 206, thereby increasing efficiency and speed. In addition, the environmental map updater module 208 and the associated steps of method 100 allow for updating of the environmental map, thereby increasing the accuracy of the environmental map by utilizing information provided by the traffic control devices, and thus improving the performance of the autonomous driving system 18.

[0093] The description in this disclosure is merely exemplary in nature, and variations thereof that do not depart from the spirit of this disclosure are intended to fall within its scope. Such variations should not be considered as departing from the schemes and scope of this disclosure.

Claims

1. A method for adjusting vehicle operation based on a traffic control device, the method comprising: Receive information about traffic control devices in the environment surrounding the vehicle; The relevance state of the traffic control device is determined at least in part based on information about the traffic control device, wherein the relevance state includes one of a relevant state and an irrelevant state; In response to determining that the relevant state is the relevant state, a road configuration indicated by information about the traffic control device is determined; An updated environmental map is generated, at least in part, based on the road configuration indicated by information about the traffic control device and an existing environmental map, wherein the updated environmental map includes information about the road configuration in the environment surrounding the vehicle; and The operation of the vehicle's autonomous driving system is adjusted at least in part based on the updated environment map.

2. The method according to claim 1, wherein, Determining the correlation status of the traffic control device further includes: Determine the novel state of the traffic control device, wherein the novel state includes either a novel state or a non-novel state; In response to determining that the novel state of the traffic control device is the novel state, the relevant state of the traffic control device is determined as the relevant state; and In response to determining that the novel state of the traffic control device is the non-novel state, the correlation state of the traffic control device is determined at least in part based on the position of the vehicle relative to the traffic control device.

3. The method according to claim 2, wherein, Determining the novel state of the traffic control device further includes: Based at least in part on information about the traffic control device and the location of the traffic control device, the non-transitory memory of the vehicle's controller is queried to obtain a matching record corresponding to the traffic control device; In response to determining that the non-transient memory does indeed include the matching record corresponding to the traffic control device, the novel state of the traffic control device is determined to be the non-novel state; and In response to determining that the non-transient memory does not include the matching record corresponding to the traffic control device, the novel state of the traffic control device is determined to be the novel state.

4. The method according to claim 2, wherein, Determining the correlation status of the traffic control device further includes: The location of the road configuration, indicated by information about the traffic control device, is determined at least in part based on road rules; Determine the position of the vehicle relative to the road configuration indicated by information about the traffic control device; In response to determining that the distance between the vehicle's location and the location of the road configuration indicated by information about the traffic control device increases over time, the traffic control device is determined to be in an irrelevant state; and In response to determining that the distance between the location of the vehicle and the location of the road configuration indicated by information about the traffic control device decreases over time, the correlation state of the traffic control device is determined as the correlation state.

5. The method according to claim 1, wherein, Determining the road configuration indicated by information about the traffic control device further includes: The non-transitory memory of the vehicle's controller is queried, at least in part, based on information about the traffic control device, to obtain similar records, wherein the similar records include information similar to the information about the traffic control device; In response to determining that the non-transitory memory does indeed include the similar record, the road configuration indicated by information about the traffic control device is retrieved from the non-transitory memory; and In response to determining that the non-transient memory does not include the similar record, a machine learning model is used, based at least in part on information about the traffic control device, to determine the road configuration indicated by the information about the traffic control device.

6. The method according to claim 5, wherein, Determining the road configuration indicated by information about the traffic control device using a machine learning model also includes: Based at least in part on information about the traffic control device, a machine learning model is used to determine the road configuration indicated by the information about the traffic control device, wherein the machine learning model is a large language model (LLM).

7. The method according to claim 5, wherein, Querying the non-transitory memory to obtain the similar records also includes: Determine a similarity score between information about the traffic control device and information in each of a plurality of records in the non-transitory memory; and In response to determining that the similarity score between information about the traffic control device and information in the similar records is greater than or equal to a predetermined similarity threshold, the similar records among the plurality of records are identified.

8. The method according to claim 1, wherein, Generating the updated environment map also includes: Identify one or more differences between the existing environmental map and the road configuration indicated by information about the traffic control device; Based at least in part on the one or more differences and the road configuration indicated by information about the traffic control device, determine one or more modifications to the existing environmental map; and The existing environment map is modified, at least in part, based on the one or more of the modifications, to generate the updated environment map.

9. The method according to claim 8, wherein, Determining one or more modifications to the existing environment map also includes: Based at least in part on the road configuration indicated by the one or more differences and information about the traffic control device, a rule-based algorithm is used to determine the one or more modifications to the existing environment map.

10. The method according to claim 1, wherein, Receiving information about the traffic control device regarding the environment surrounding the vehicle also includes: Capture an image of the traffic control device, which includes traffic signs.

Images