Glare protection method and apparatus

Abstract

A glare protection method and apparatus. The method comprises: on the basis of the position of a light source at a first moment in an environment where a first vehicle is located and vehicle information, predicting the position of the light source at a second moment, wherein the second moment is later than the first moment, and the vehicle information includes vehicle motion data of a second vehicle, or the vehicle information is used for indicating that a second vehicle is not present in the environment where the first vehicle is located; on the basis of a predicted trajectory of a target vehicle and the position of the light source at the second moment, determining a light spot area formed by the light source on an imaging plane of a light shielding device at the second moment, wherein the target vehicle includes the first vehicle; and outputting shielding information to the light shielding device, wherein the shielding information is used for indicating a light shielding area, which covers the light spot area.

Description

Methods and devices for glare protection

[0001] This application claims priority to Chinese Patent Application No. 202510124967.9, filed on January 26, 2025, entitled "Method and Apparatus for Glare Protection", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence, and more specifically, to a method and apparatus for glare protection. Background Technology

[0003] Glare is a common problem for drivers. It occurs when drivers are suddenly exposed to strong light, such as the high beams of oncoming vehicles at night, intense sunlight in the early morning or evening, road glare, or sunlight at tunnel exits. This can cause eye discomfort, blurred vision, and increase the risk of traffic accidents. Currently, when drivers encounter glare, they typically wear sunglasses, open the sun visor, or manually shield their eyes. However, these traditional coping strategies require manual operation and rely on the driver's reaction time, making it difficult to adapt to changes in light conditions promptly.

[0004] Some proposed solutions involve detecting and then mitigating glare, such as by blocking relevant areas, to reduce its impact. However, these solutions often fail to achieve satisfactory protection. For instance, by the time glare is detected, it may have already begun to affect the driver, and the effectiveness is significantly affected by time delays, making it difficult to achieve adequate protection.

[0005] Therefore, improving the effectiveness of glare protection has become an urgent problem to be solved. Summary of the Invention

[0006] This application provides a method and apparatus for glare protection. This method helps to accurately predict the glare-shielding area, thereby improving the protection effect and preventing the driver from being affected by glare.

[0007] In a first aspect, a method for glare protection is provided, comprising: predicting the position of at least one light source at a second time based on the position of at least one light source in the environment where the first vehicle is located at a first time and vehicle information in the environment where the first vehicle is located, wherein the second time is later than the first time, and the vehicle information includes vehicle motion data of at least one second vehicle, or the vehicle information is used to indicate that there is no second vehicle in the environment where the first vehicle is located; determining a spot area formed by at least one light source on the imaging plane of a glare shielding device at the second time based on the predicted trajectory of at least one target vehicle and the position of at least one light source at the second time, wherein the glare shielding device is disposed on the first vehicle, and the at least one target vehicle includes the first vehicle; and outputting shading information to the glare shielding device, wherein the shading information is used to indicate the shading area, and the shading area covers the spot area.

[0008] According to the solution of this application embodiment, the future position of the light source (e.g., the position of the light source at a first moment) is predicted based on the detected position of the light source and vehicle information (e.g., the position of the light source at a second moment), thereby predicting the future shading area. This can compensate for system latency and motion latency, helping to avoid the protective effect being affected by latency. Simultaneously, by referencing vehicle information when predicting the future position of the light source, it is beneficial to predict the accurate spot area, thus improving the accuracy of the shading area. That is, while blocking the light from the glare source, the visibility of non-spot areas is maintained as much as possible, ensuring a good field of vision for the user and improving the protective effect. In the solution of this application embodiment, the user does not need to manually block the glare, which improves the user experience.

[0009] Optionally, the method may further include: acquiring an image frame of the environment in which the first vehicle is located; performing light source detection based on the image frame to obtain a light source detection result, the light source detection result being used to determine the position of at least one light source at a first moment.

[0010] For example, the position of at least one light source at the first moment can be the position of the at least one light source in the image frame at the first moment, or it can be the position of the at least one light source in three-dimensional space at the first moment.

[0011] Alternatively, the image frame may be obtained by image enhancement of data acquired by a vision sensor.

[0012] In the embodiments of this application, image enhancement can be performed on the acquired image data to improve image quality. This is beneficial for improving the detection effect in glare scenes, thereby obtaining more accurate detection results, such as obtaining a more accurate position of the light source.

[0013] For example, the time interval between the first moment and the second moment can be determined based on relevant parameters of system delay and / or motion delay.

[0014] For example, the shading device can adjust the light transmittance in the shading area according to the shading information so that the light transmittance of the shading area is different from that of the non-shading area.

[0015] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: acquiring the historical trajectory of at least one target vehicle; acquiring map data; and generating a predicted trajectory of at least one target vehicle based on the map data and the historical trajectory of at least one target vehicle.

[0016] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining lane information of drivable lanes in the environment where the first vehicle is located; and generating a predicted trajectory of at least one target vehicle based on map data and the historical trajectory of at least one target vehicle, including: generating a predicted trajectory of at least one target vehicle based on map data, the historical trajectory of at least one target vehicle, and lane information of drivable lanes.

[0017] In the solution of this application embodiment, the introduction of drivable lane lines as reference information for trajectory prediction is beneficial to improving the accuracy of trajectory prediction.

[0018] In conjunction with the first aspect, in certain implementations of the first aspect, where the vehicle information includes vehicle motion data of at least one second vehicle, the at least one target vehicle includes at least one second vehicle, and predicting the position of at least one light source at a second time based on the position of at least one light source at a first time and the vehicle information includes: predicting the position of at least one light source at a second time based on the position of at least one light source at a first time and the predicted trajectory of at least one second vehicle; or, where the vehicle information is used to indicate that the environment in which the first vehicle is located does not contain a second vehicle, the position of at least one light source at a second time is the position of at least one light source at the first time.

[0019] In conjunction with the first aspect, in certain implementations of the first aspect, predicting the position of at least one light source at a second time based on the position of at least one light source at a first time and the predicted trajectory of at least one second vehicle includes: determining a matching relationship between at least one light source and at least one second vehicle based on the position of at least one light source at the first time and vehicle motion data of at least one second vehicle; determining the position of the first light source among at least one light source at the second time based on the predicted trajectory of the first target vehicle among at least one second vehicle, wherein there is a matching relationship between the first target vehicle and the first light source; and / or, the position of the second light source among at least one light source at the second time is the position of the second light source at the first time, wherein there is no matching relationship between the second light source and at least one second vehicle.

[0020] In the scheme of this application embodiment, the light source and the vehicle are matched. The light source matched with the vehicle (such as the first light source) is regarded as a dynamic light source, and the light source not matched with the vehicle (such as the second light source) can be regarded as a static light source. The position of the static light source and the dynamic light source are predicted respectively, which helps to further improve the accuracy of the prediction results.

[0021] In conjunction with the first aspect, in some implementations of the first aspect, the predicted trajectory of at least one target vehicle is the predicted trajectory of at least one target vehicle in three-dimensional space.

[0022] In scenarios such as going up or down slopes, meeting on or off ramps, and uneven road surfaces, vehicles move along the Z-axis. In the solution of this application embodiment, the predicted trajectory is a 3D trajectory, that is, the future 3D position of the vehicle is predicted, which helps to further ensure the accuracy of the shading area.

[0023] In conjunction with the first aspect, in some implementations of the first aspect, the light transmittance of the shaded area is related to the light intensity of at least one light source.

[0024] Optionally, when the light intensity of at least one light source is a first light intensity, the light transmittance of the shaded area is a first light transmittance; when the light intensity of at least one light source is a second light intensity, the light transmittance of the shaded area is a second light transmittance. The first light intensity is greater than the second light intensity, and the first light transmittance is less than the second light transmittance.

[0025] In the solution of this application embodiment, the light transmittance of the shading area can be adjusted according to the light intensity of the light source. This is beneficial for users to experience a relatively consistent brightness under different lighting conditions such as dim light or bright light, which helps to ensure a stable shading effect and thus improves the user experience.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: receiving point cloud data of the environment in which the first vehicle is located; performing vehicle detection based on the point cloud data to obtain a first vehicle detection result, the first vehicle detection result being used to determine vehicle information; and / or, the method further includes: receiving image frames of the environment in which the first vehicle is located; performing vehicle detection based on the image frames to obtain a second vehicle detection result, the second vehicle detection result being used to determine vehicle information.

[0027] In the solution of this application embodiment, point cloud data is introduced. Point cloud data is not affected by glare light sources, has high data quality, and the accuracy of vehicle detection results obtained based on point cloud data is high, which is beneficial to the accuracy of subsequent processing.

[0028] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: obtaining the user's eye position, and determining the light spot area formed by at least one light source on the imaging plane of the shading device at the second moment based on the predicted trajectory of at least one target vehicle and the position of at least one light source at the second moment, including: determining the light spot area formed by at least one light source on the imaging plane of the shading device at the second moment based on the user's eye position, the predicted trajectory of at least one target vehicle and the position of at least one light source at the second moment.

[0029] Optionally, determining the spot area formed by at least one light source on the imaging plane of the shading device at the second moment based on the human eye position, the predicted trajectory of at least one target vehicle, and the position of at least one light source at the second moment includes: determining the light cone curve of at least one light source at the second moment based on the predicted trajectory of the first vehicle and the position of at least one light source at the second moment, wherein the light cone curve of at least one light source at the second moment indicates the position, orientation, and divergence angle of at least one light source in the vehicle coordinate system of the first vehicle at the second moment, and the vehicle coordinate system of the first vehicle is determined based on the predicted trajectory of the first vehicle; and determining the spot area formed by at least one light source on the imaging plane of the shading device at the second moment based on the light cone curve of at least one light source at the second moment and the human eye position.

[0030] In the solution of this application embodiment, parallax registration can be performed according to the position of the human eye to obtain the relative positional relationship between the light source and the human eye, so as to compensate for the inconsistency between the position of the visual sensor and the position of the human eye, which is beneficial to achieve precise occlusion of the human eye.

[0031] In conjunction with the first aspect, in some implementations of the first aspect, the shading information is also used to indicate at least one of the following: the light transmittance of the shading area, the velocity of the center of the shading area, or the acceleration of the center of the shading area.

[0032] In conjunction with the first aspect, in some implementations of the first aspect, when the human eye is in the first position, the light-blocking area is the first region; when the human eye is in the second position, the light-blocking area is the second region. The first position and the second position are different, and the first region and the second region are different.

[0033] In conjunction with the first aspect, in some implementations of the first aspect, when the distance between the first vehicle and at least one second vehicle is a first distance, the ratio between the area of ​​the shading region and the area of ​​the light spot region is a first ratio; when the distance between the first vehicle and at least one second vehicle is a second distance, the ratio between the area of ​​the shading region and the area of ​​the light spot region is a second ratio, the first distance is greater than the second distance, and the first ratio is greater than the second ratio.

[0034] The closer the distance between the first and second vehicles, the more accurate the information about the second vehicle is likely to be. This facilitates a more accurate predicted trajectory for the vehicle, and consequently, a more accurate prediction of the light spot area. In the solution of this application embodiment, when the vehicle distance is close, a smaller shading area can be used to better fit the light spot area, thereby improving the user experience while ensuring the shading effect. When the vehicle distance is far, a larger shading area can be used to achieve full coverage of the light spot area and ensure the shading effect.

[0035] In conjunction with the first aspect, in some implementations of the first aspect, when the vehicle information is determined based on the first vehicle detection result, the ratio between the area of ​​the shading region and the area of ​​the light spot region is the third ratio; when the vehicle information is unrelated to the first vehicle detection result, the ratio between the area of ​​the shading region and the area of ​​the light spot region is the fourth ratio, and the fourth ratio is greater than the third ratio.

[0036] Compared to vehicle detection results based on image data (second vehicle detection results), vehicle detection results based on point cloud data (first vehicle detection results) may be more accurate. This is beneficial for obtaining more accurate predicted vehicle trajectories, and consequently, for predicting more accurate spot areas. In the solution of this application embodiment, when vehicle information is determined based on the first vehicle detection result, a smaller shading area can be used to better fit the spot area, thereby improving user experience while ensuring shading effectiveness. Correspondingly, vehicle detection results based on image data may be affected by glare and have lower accuracy. When vehicle information is unrelated to the first vehicle detection result, a larger shading area can be used to achieve full coverage of the spot area and ensure shading effectiveness.

[0037] In a second aspect, a glare protection device is provided, comprising modules / units for performing the methods of the first aspect and any implementation thereof.

[0038] It should be understood that the extensions, limitations, explanations and descriptions of the relevant content in the first aspect above also apply to the same content in the second aspect.

[0039] Thirdly, an electronic device is provided. The electronic device includes a processor configured to be coupled to a memory, read and execute instructions and / or program code in the memory to perform the methods described in the first aspect or any possible implementation thereof.

[0040] Fourthly, a chip system is provided. The chip system includes logic circuitry for coupling with an input / output interface through which data is transmitted to perform the methods described in the first aspect or any possible implementation thereof.

[0041] Fifthly, a computer program product containing instructions is provided, which, when executed by a computing device, cause the computing device to perform the method as described in the first aspect or any implementation thereof.

[0042] In a sixth aspect, a computer-readable storage medium is provided, including computer program instructions that, when executed by a computing device, perform the method as described in the first aspect or any implementation thereof.

[0043] As examples, these computer-readable storage devices include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), flash memory, electrically EPROM (EEPROM), and hard drive.

[0044] Alternatively, as one implementation method, the aforementioned storage medium can specifically be a non-volatile storage medium. Attached Figure Description

[0045] Figure 1 is a schematic diagram of the functional block diagram of a vehicle according to an embodiment of this application.

[0046] Figure 2 is a schematic diagram of a glare protection process according to an embodiment of this application.

[0047] Figure 3 is a schematic flowchart of a glare protection method according to an embodiment of this application.

[0048] Figure 4 is a schematic diagram illustrating an example of the light source detection process and the vehicle detection process according to an embodiment of this application.

[0049] Figure 5 is a schematic diagram of a trajectory prediction process according to an embodiment of this application.

[0050] Figure 6 is a schematic diagram of a masking information generation process according to an embodiment of this application.

[0051] Figure 7 is a schematic diagram of the three-dimensional position of the light source and its position on the imaging plane according to an embodiment of this application.

[0052] Figure 8 is a schematic diagram of the change process of the light cone in an embodiment of this application.

[0053] Figure 9 is a schematic diagram of the principle of parallax registration according to an embodiment of this application.

[0054] Figure 10 is a schematic flowchart of another glare protection method according to an embodiment of this application.

[0055] Figure 11 is a schematic diagram of a specific example of the glare protection process according to an embodiment of this application.

[0056] Figure 12 is a schematic flowchart of another glare protection method according to an embodiment of this application.

[0057] Figure 13 is a schematic diagram of a system architecture according to an embodiment of this application.

[0058] Figure 14 is a schematic block diagram of an apparatus according to an embodiment of this application.

[0059] Figure 15 is a schematic diagram of the architecture of a computing device according to an embodiment of this application. Detailed Implementation

[0060] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0061] The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” and “the” are intended to include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one,” “at least one,” and “one or more” refer to one, two, or more than two. “First,” “second,” and various numerical designations are merely distinctions for descriptive convenience and are not intended to limit the scope of the embodiments of this application. “And / or” is used to describe the correspondence between corresponding objects, indicating that three relationships can exist. For example, “A and / or B” can represent: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship. The order of the process numbers below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic and should not constitute any limitation on the implementation process of the embodiments of this application. For example, in the embodiments of this application, the words "301", "401", "501" etc. are merely identifiers made for the convenience of description and do not limit the order of execution steps.

[0062] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. In this application, the words "exemplary" or "for example" are used to indicate that something is illustrative, exemplary, or descriptive. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized. In the embodiments of this application, descriptions such as "when," "in the case of," "if," and "if" all refer to the fact that the device will perform a corresponding processing under certain objective circumstances, and are not a limitation on time, nor do they require the device to perform a judgment action during implementation, nor do they imply any other limitations.

[0063] In this application, "for indicating" can include both direct and indirect indication. When describing an indication message as indicating A, it can include whether the indication message directly indicates A or indirectly indicates A, but does not necessarily mean that the indication message carries A.

[0064] Glare is a visual phenomenon referring to the presence of excessively bright objects or extremely high contrast in the field of vision, causing visual discomfort or reducing the ability to observe targets and details. Glare can cause visual interference, making it difficult for people to see objects clearly, and sometimes causing eye discomfort or pain.

[0065] The solutions provided in this application can be applied to scenarios requiring glare protection. For example, glare can be caused by strong light such as the high beams of oncoming vehicles at night, strong sunlight in the early morning or evening, road surface reflections, and sunlight at the exits of underground parking garages or tunnels. The solutions in this application can be applied to these scenarios to achieve glare protection. It should be understood that the above are merely examples, and the solutions in this application can be applied to any other scenario requiring glare protection.

[0066] To facilitate understanding of the solutions in the embodiments of this application, the terms that may be involved in the embodiments of this application will be explained below.

[0067] (1) Neural Networks:

[0068] Neural networks can be composed of neural units, which can refer to units represented by x. s The arithmetic unit that takes an intercept of 1 as input can output the following:

[0069] Where s = 1, 2, ..., n, n is a natural number greater than 1, W s For x s The weights are denoted by b, where b is the bias of the neural unit.

[0070] f represents the activation function of a neural network, used to introduce nonlinear characteristics and convert the input signal into the output signal. The output signal of this activation function can be used as the input to the next layer. For example, the activation function can be ReLU, tanh, or sigmoid.

[0071] A neural network is a network formed by connecting multiple individual neural units, meaning that the output of one neural unit can be the input of another. The input of each neural unit can be connected to the local receptive field of the previous layer to extract features from the local receptive field, which can be a region composed of several neural units.

[0072] (2) Deep Neural Networks:

[0073] A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple hidden layers. Based on the position of the layers, the internal neural network of a DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally, the first layer is the input layer, the last layer is the output layer, and the layers in between are hidden layers. The layers are fully connected, meaning that any neuron in the i-th layer is connected to any neuron in the (i+1)-th layer.

[0074] Although DNNs seem complex, the operation of each layer is actually not complicated. Simply put, it involves the following linear relationship expression: in, It is the input vector. It is the output vector. α is the offset vector, W is the weight matrix (also called coefficients), and α() is the activation function. Each layer is simply an adjustment of the input vector. The output vector is obtained through such a simple operation. Because DNNs have many layers, the coefficients W and the offset vector... The number of these parameters is also relatively large. The definitions of these parameters in DNNs are as follows: Taking the coefficient W as an example: Assuming a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as... The superscript 3 represents the layer number where coefficient W is located, while the subscript corresponds to the third layer index 2 of the output and the second layer index 4 of the input.

[0075] In summary, the coefficient from the k-th neuron in layer L-1 to the j-th neuron in layer L is defined as...

[0076] It's important to note that the input layer does not have a W parameter. In deep neural networks, more hidden layers allow the network to better represent complex real-world situations. Theoretically, the more parameters a model has, the higher its complexity and "capacity," meaning it can perform more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrix of all layers in the trained deep neural network (a weight matrix formed by the vectors W from many layers).

[0077] (3) Loss function:

[0078] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0079] (4) Backpropagation algorithm:

[0080] Backpropagation (BP) is an algorithm used during training to correct the parameters in the initial model, thereby reducing the model's error loss. Specifically, forward propagation of the input signal to the output generates an error loss; this error loss information is then propagated back to update the parameters in the initial model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining optimal model parameters, such as the weight matrix.

[0081] Glare can impair a driver's vision and increase the risk of traffic accidents. Some solutions propose detecting and mitigating glare, such as by blocking relevant areas, to reduce its impact. However, these solutions often fail to provide adequate protection. For example, by the time glare is detected, it may have already begun to affect the driver, and the effectiveness of the protection is significantly affected by time delays, making it difficult to achieve satisfactory results.

[0082] In view of this, the embodiments of this application provide a method for glare protection, which is beneficial for predicting the accurate shading area, thereby improving the protection effect and avoiding the impact of glare on the driver.

[0083] To better illustrate the solutions of the embodiments of this application, the system architecture of the embodiments of this application will be described first below.

[0084] Figure 1 shows a schematic diagram of the functional block diagram of a vehicle 100 according to an embodiment of this application.

[0085] Vehicle 100 may include a perception system 120, a display device 130, and a computing platform 150. The perception system 120 may include several sensors for sensing information about the environment surrounding vehicle 100. For example, the perception system 120 may include a positioning system, which may be a global positioning system (GPS), a BeiDou system, a global navigation satellite system (GNSS) or other positioning systems, an inertial measurement unit (IMU), lidar, millimeter-wave radar, ultrasonic radar, a wheel speed sensor (WSS), and one or more of camera devices.

[0086] Some or all of the functions of vehicle 100 can be controlled by computing platform 150. Computing platform 150 may include processors 151 to 15n (n being a positive integer). A processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, graphics processing unit (GPU) (which can be understood as a type of microprocessor), or digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable. For example, the processor may be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In reconfigurable hardware circuits, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement some or all of the functions of the aforementioned units. In addition, it can also be hardware circuitry designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), etc. Furthermore, the computing platform 150 may also include a memory for storing instructions. Some or all of the processors 151 to 15n can call the instructions in the memory to execute them and achieve the corresponding functions.

[0087] Figure 2 shows a schematic diagram of a glare protection process according to an embodiment of this application.

[0088] The solution in this application embodiment can be used for forward glare protection. In the scenario of forward glare protection, the glare shield can be used to block light emitted from a light source onto the windshield of the vehicle to prevent a large amount of light from suddenly entering the eyes of the user (such as the driver or other passengers). This application embodiment mainly uses forward glare protection as an example for explanation. In other implementations, the glare shield can also be used to block light emitted from a light source onto the rearview mirror to prevent a large amount of light from suddenly entering the user's eyes.

[0089] For example, the light source in the embodiments of this application can be an entity capable of spontaneously generating and emitting light. This light source can be a natural light source. It can also be an artificial light source, such as high beams on a vehicle, high-brightness low beams on a vehicle, streetlights, or flashlights on camera equipment.

[0090] For example, the light source in the solution of this application embodiment can be an object capable of reflecting light. In other words, if a reflective object reflects the light emitted by a luminous object towards the windshield, then the reflective object can also be regarded as the light source in the embodiment of this application.

[0091] The computing platform 150 can be used to perform the following steps.

[0092] 1) Acquire input data. Input data includes image data and vehicle motion data.

[0093] As shown in Figure 2, the computing platform 150 can acquire image data and vehicle motion data from the perception system 120.

[0094] For example, when applied to forward glare protection, the image data may include a forward-view image.

[0095] 2) Detect the light source based on the image data to obtain information about the light source.

[0096] 3) Perform trajectory prediction based on the input data to obtain the predicted trajectory of the vehicle.

[0097] The predicted trajectory of a vehicle can include its own trajectory. Furthermore, the predicted trajectory of a vehicle can also include the predicted trajectories of other vehicles.

[0098] For example, the computing platform 150 can determine the trajectory of the vehicle based on the vehicle's motion data.

[0099] For example, the computing platform 150 can determine the predicted trajectory of another vehicle based on image data.

[0100] For example, the computing platform 150 can also acquire point cloud data from the perception system 120 and determine the predicted trajectory of other vehicles based on the point cloud data.

[0101] Point cloud data refers to a set of vectors in a three-dimensional coordinate system.

[0102] For example, the computing platform 150 can also acquire map data, which can also be used as reference data for the predicted trajectory of the vehicle.

[0103] 4) Predict the light cone based on the information of the light source and the predicted trajectory of the vehicle.

[0104] 5) Generate shading information based on the predicted light cone. The shading information is used to indicate the shading area of ​​the shading device.

[0105] 6) Output shading information to the shading device.

[0106] The vehicle 100 may also include a shading device. The shading device can be used to adjust the light transmittance of the shading area according to the shading information in order to block the light shining on the user.

[0107] The light-shielding device may include a display device 130.

[0108] For example, a light-blocking device could be a sunshade for a transparent LCD screen. This transparent LCD screen could be made of a material with adjustable brightness. Another example is an augmented reality head-up display (ARHUD).

[0109] Alternatively, the light-shielding device can be other than a display device. For example, the light-shielding device can be implemented using dimming glass. Dimming glass, such as electrochromic glass, can vary its light transmittance to block strong light and prevent it from shining directly on the driver.

[0110] For example, a shading device can be a dimmable glass installed on a windshield. Another example is a windshield using dimmable glass, i.e., a dimmable windshield. Yet another example is a sun visor using dimmable glass, i.e., a dimmable glass sun visor.

[0111] The input data may also include other data, such as positioning data from the automated driving system (ADS). This application does not limit this aspect.

[0112] It should be understood that the above are merely examples and do not constitute a limitation on the solutions of the embodiments of this application. For example, in some other implementations, some or all of the steps performed by the computing platform 150 described above may also be performed by the light-shielding device itself.

[0113] For a detailed description of the above process, please refer to Method 300.

[0114] Figure 3 shows a schematic flowchart of a glare protection method according to an embodiment of this application. Exemplarily, the method 300 shown in Figure 3 can be performed by the vehicle 100 shown in Figure 1, for example, by a computing platform 150. The vehicle 100 can be considered as a self-driving vehicle.

[0115] As shown in Figure 3, method 300 may include the following steps.

[0116] 310. Obtain input data. Input data includes image data and vehicle motion data.

[0117] 320, retrieve information about the light source in the image data.

[0118] 330. Based on the input data, trajectory prediction is performed to obtain the predicted trajectory of the vehicle. The predicted trajectory of the vehicle may include the vehicle's own motion trajectory. Further, optionally, the predicted trajectory of the vehicle may also include the predicted trajectories of other vehicles.

[0119] 340. Obtain future information about the light source based on the information of the light source and the predicted trajectory of the vehicle.

[0120] 350, obtain the shading area of ​​the shading device based on the future information of the light source.

[0121] Furthermore, method 300 may also include step 360.

[0122] 360° outputs shading information to the shading device, which is used to indicate the shading area.

[0123] It should be understood that the description of step 310 is for ease of use only and does not imply that different types of input data are acquired simultaneously. For example, step 310 can also be understood as acquiring image data and acquiring vehicle motion data of the vehicle.

[0124] The image data may include image data of the environment surrounding the vehicle. This image data may include one or more frames of images.

[0125] For example, the image data in step 310 can be image data acquired by a vision sensor. For instance, step 310 may include: acquiring image data acquired by the vision sensor.

[0126] As an example, as previously stated, method 300 can be performed by vehicle 100. The visual sensor can be a camera device in perception system 120.

[0127] For example, the visual sensor can be a camera or a digital video recorder (DVR). Image data can be, for instance, real-time images captured by the camera. Image data can be understood as a collection of grayscale values, numerically represented, of each pixel captured using a camera.

[0128] The visual sensor can also be other devices, and this application does not limit the embodiments thereto.

[0129] Furthermore, step 310 may include: performing image enhancement on the image data acquired by the vision sensor to obtain enhanced image data.

[0130] In this case, "image data" in subsequent steps can be understood as "enhanced image data".

[0131] Image enhancement can be achieved in a variety of ways.

[0132] For example, image enhancement can be achieved based on deep learning methods or traditional methods.

[0133] For a specific example of image enhancement, please refer to Method 1000 below.

[0134] Vehicle motion data can include the vehicle's location. For example, vehicle motion data can include the vehicle's location within one or more historical time periods, i.e., one or more historical locations of the vehicle.

[0135] Furthermore, the vehicle motion data may also include one or more of the following: the vehicle's direction of motion, the vehicle's speed, the vehicle's acceleration, the vehicle's angle, or the vehicle's angular velocity.

[0136] For example, the vehicle motion data of the aforementioned vehicle can be the vehicle motion data of the vehicle in the world coordinate system.

[0137] As an example, as previously stated, method 300 can be performed by vehicle 100. Vehicle motion data can be collected by one or more of the following: GPS, gyroscope, GNSS, BeiDou system, IMU, WSS, or accelerometer, etc.

[0138] Step 320 is to detect light sources in the environment surrounding the vehicle from the image data and obtain their relevant information.

[0139] Optionally, step 320 may include: performing light source detection based on image data to obtain information about the light source.

[0140] Alternatively, the light source detection process can also be performed by other equipment besides the vehicle itself.

[0141] The light source detection process in step 320 is described below.

[0142] For example, the image data in step 320 can be image data acquired by a vision sensor or enhanced image data.

[0143] The number of light sources can be one or more. Step 320 can also be understood as detecting light sources based on image data to obtain information about one or more light sources. This application embodiment does not limit the number of light sources. The number of light sources can be determined based on the light source detection results.

[0144] For example, information about the light source may include the location of the light source.

[0145] As mentioned earlier, the image data in step 320 may include one or more frames of images. For ease of description, the following example uses one frame of image data (such as image #1) to illustrate the light source detection process.

[0146] The image data includes image #1. In step 320, light source detection can be performed on image #1 to determine the light sources and their positions in image #1. The number of light sources in image #1 can be determined based on the light source detection results of image #1.

[0147] Light source detection can be achieved in a variety of ways.

[0148] Alternatively, light source detection can be achieved using deep learning methods.

[0149] As one possible implementation, step 320 may include: performing object detection on image #1 using AI model #1 to determine the location of the light source in image #1.

[0150] The AI ​​model in this embodiment can also be replaced by a neural network model. AI model #1 can also be understood as an object detection model.

[0151] As an example, the target detection process for a light source can be implemented through the following steps. Exemplarily, AI model #1 may include: a CNN, a Transformer encoder, a Transformer decoder, and a fully connected network.

[0152] A1) Use CNN to extract features from image #1 to obtain the features of image #1.

[0153] In step A1), features can be extracted from the image using one or more CNNs to obtain the features of image #1.

[0154] For example, as shown in Figure 4, the one or more CNNs can be CNNs in a feature extractor.

[0155] For example, as shown in Figure 4, image #1 can be the input image in Figure 4. Alternatively, image #1 can be the enhanced image in Figure 4. For example, the input image can be enhanced using a CNN in the image enhancement module, and the enhanced image can be used as image #1. Image enhancement can employ a self-supervised image enhancement method.

[0156] A2) The features of image #1 are processed by the Transformer encoder to obtain multiscale features.

[0157] Transformer encoders can be used to model the global relationships between features of an image to obtain a set of multi-scale features.

[0158] A3) The Transformer decoder extracts features based on the key, value, and query to obtain the features of the target. Among them, the key and value are multi-scale features.

[0159] The Transformer decoder can use multi-scale features as keys and values, and with a set of queries, extracts features of the target (such as a light source) from the multi-scale features based on a cross-attention mechanism. The number of queries in this set represents the maximum number of detectable targets. The number of queries in this set can be fixed. This set of queries is learnable.

[0160] The number of targets may be one or multiple.

[0161] A4) The target's features are classified and bounding box regression is performed using a fully connected network to obtain the target's location, size, and category.

[0162] For example, a fully connected network may include a regression head and a classification head, used for bounding box regression and classification of the target's features, respectively.

[0163] The target category can include light sources. Furthermore, the target category can also include other categories such as background.

[0164] Therefore, the position and size of the light source in image #1 can be obtained.

[0165] AI model #1 can be obtained through training.

[0166] The above is just one example of the object detection process. Object detection can also be implemented in other ways; for details, please refer to the example in Method 1000 below, which will not be elaborated here.

[0167] As another possible implementation, step 320 may include: performing semantic segmentation on image #1 using AI model #2 to determine the location of the light source in image #1.

[0168] AI model #2 can also be understood as a semantic segmentation model.

[0169] It should be understood that the numbering of AI Model #1 and AI Model #2 is for descriptive convenience only and does not have a limiting effect.

[0170] It should be understood that the above are just examples. The location of the light source can also be determined by other deep learning models, or by non-deep learning methods. For example, light source detection can also be achieved based on pixel intensity.

[0171] Furthermore, method 300 may also include step 331 (not shown in the figure).

[0172] 331. Perform vehicle detection based on the input data to obtain information about other vehicles in the surrounding environment of your own vehicle. The information about other vehicles can indicate that no other vehicle exists in the surrounding environment of your own vehicle. Alternatively, the information about other vehicles can include the vehicle motion data of other vehicles.

[0173] Step 331 is to detect whether there are other vehicles in the environment around the vehicle and related information about other vehicles.

[0174] The number of other vehicles can be one or more. That is, the information of other vehicles can include the vehicle movement data of one or more other vehicles. This application embodiment does not limit the number of other vehicles. The number of other vehicles can be determined based on the vehicle detection results.

[0175] Vehicle motion data for another vehicle can include the vehicle's location. For example, vehicle motion data for another vehicle can include the vehicle's location within one or more historical time periods, i.e., one or more historical locations of the vehicle.

[0176] Furthermore, the vehicle motion data of another vehicle may also include one or more of the following: the direction of motion of the other vehicle, the speed of the other vehicle, or the acceleration of the other vehicle.

[0177] Information about his car can be determined in several ways.

[0178] The vehicle inspection process will be explained below using two methods (Method 1 and Method 2) as examples.

[0179] Method 1;

[0180] Information about his vehicle can be determined based on image data.

[0181] Optionally, step 331 may include performing vehicle detection based on image data to obtain information about other vehicles.

[0182] As mentioned earlier, image data may include one or more frames of images. The following example, using one frame of image data (such as image #2), illustrates the vehicle detection process.

[0183] The image data includes image #2. In step 331, vehicle detection can be performed on image #2 to determine other vehicles in image #2 and their locations. The number of other vehicles in image #2 can be determined based on the vehicle detection results of image #2.

[0184] The numbering of image #2 is for descriptive convenience only and has no limiting effect.

[0185] Image #2 and image #1 can be the same image or different images.

[0186] The description of image #2 can be found in the description of image #1. To avoid repetition, it will not be repeated here.

[0187] Vehicle detection based on image data can be achieved in various ways. The following provides an example illustrating methods for vehicle detection based on image data.

[0188] Alternatively, vehicle detection can be achieved using deep learning methods.

[0189] For example, the deep learning-based vehicle detection process can refer to the relevant description of the deep learning-based light source detection process (such as object detection model or semantic segmentation model) mentioned above, simply by replacing image #1 with image #2 and replacing "light source" with "vehicle".

[0190] Furthermore, when image #1 and image #2 are the same image, vehicle detection and light source detection can be performed simultaneously.

[0191] As an example, object detection can be performed on an image using AI model #3 to output the category and location of objects in the image.

[0192] The target category can include light sources and vehicles. Furthermore, the target category can also include other categories such as background.

[0193] AI model #3 can also be understood as an object detection model.

[0194] The designation of AI model #3 is for ease of description only and has no other limiting function.

[0195] For example, AI model #3 can adopt the structure of AI model #1, and the target detection process can refer to the relevant descriptions in A1) to A4), replacing the target category with light source, vehicle and background.

[0196] AI model #3 can output the category, location, and size of objects in an image.

[0197] For example, some or all of the targets categorized as light sources in the output of AI model #3 can be considered as light sources in the environment surrounding the vehicle, and their locations can be considered as the locations of the light sources.

[0198] Figure 4 illustrates an example of the light source detection process and vehicle detection process according to an embodiment of this application. The detection process in Figure 4 includes two parts: the light source and vehicle detection process in image data and the vehicle detection process in point cloud data. As shown in Figure 4, the light source and vehicle detection process in image data can be implemented based on an object detection model. This object detection model may include a feature extractor and a glare and vehicle decoder. The feature extractor may include one or more CNNs and an encoder. For example, the encoder may be a Transformer encoder. The glare and vehicle decoder may include a decoder, a regression head, and a classification head. For example, the decoder may be a Transformer decoder.

[0199] Optionally, the object detection model may also include an image enhancement module. The image enhancement module may include one or more CNNs.

[0200] The detailed description of the object detection process in Figure 4 can be found in steps A1) to A4) above. The regression head and classification head are used to perform bounding box regression and classification on the object's features, respectively. The main difference between the scheme shown in Figure 4 and steps A1) to A4) above lies in the object category. As shown in Figure 4, when this object detection model is used for light source and vehicle detection, the object category can include light source, vehicle, and background, etc. In Figure 4, "G" represents light source, "V" represents vehicle, and "B" represents background. If the object detection model is only used for light source detection, "V" in the object detection model in Figure 4 can be replaced with "G", and correspondingly, the "light source and vehicle decoder" can also be replaced with "light source decoder". If the object detection model is only used for vehicle detection, "G" in the object detection model in Figure 4 can be replaced with "V", and correspondingly, the "light source and vehicle decoder" can also be replaced with "vehicle decoder".

[0201] For ease of description, this application embodiment mainly uses all targets of the category of light source as light sources in the environment around the vehicle as an example for illustration, and does not constitute a limitation on the solution of this application embodiment.

[0202] For example, some or all of the targets categorized as vehicles in the output of AI model #3 can be considered as other vehicles in the environment surrounding the self vehicle, and correspondingly, their positions can be considered as the positions of other vehicles.

[0203] For ease of description, this application embodiment mainly uses all targets of the category of vehicle as other vehicles in the surrounding environment of the self vehicle for illustration, and does not constitute a limitation on the solution of this application embodiment.

[0204] If none of the targets in the output of AI model #3 are classified as vehicles, then it can be assumed that no other vehicle exists in the image. In this case, the information about other vehicles can be used to indicate that no other vehicle exists in the environment surrounding your own vehicle.

[0205] AI model #3 can be obtained through training.

[0206] The above are just examples. Other implementations of object detection can be found in the examples below, which will not be described in detail here.

[0207] Alternatively, the image can be semantically segmented using AI model #4 to determine the location of light sources and other vehicles in the image.

[0208] AI model #4 can also be understood as a semantic segmentation model.

[0209] It should be understood that the numbering of AI Model #3 and AI Model #4 is for descriptive convenience only and does not have a limiting effect.

[0210] It should be understood that the above are merely examples, and vehicle detection can also be achieved through other deep learning models. This application does not limit the scope of these examples.

[0211] Method 2;

[0212] The information about the other vehicle can be determined based on point cloud data. In this case, the input data may also include point cloud data.

[0213] Optionally, step 331 may include performing vehicle detection based on point cloud data to obtain information about other vehicles. The number of other vehicles may be determined based on the vehicle detection results.

[0214] Point cloud data can include point cloud data of the vehicle's surrounding environment.

[0215] Point cloud data can be a set of vectors in three-dimensional space.

[0216] For example, the point cloud data obtained in step 310 can be three-dimensional point cloud data.

[0217] For example, the point cloud data in step 310 can be data collected by a sensor (such as a lidar). For instance, step 310 may include acquiring point cloud data collected by a lidar.

[0218] As an example, as previously described, method 300 can be performed by vehicle 100. In this case, the sensor can be light detection and ranging (LiDAR) in perception system 120.

[0219] Alternatively, the point cloud data obtained in step 310 can be preprocessed data collected by the lidar.

[0220] Vehicle detection based on point cloud data can be achieved in various ways. The following is an example illustrating a method for vehicle detection based on point cloud data.

[0221] Alternatively, vehicle detection can be achieved using deep learning methods.

[0222] For example, step 331 may include: performing vehicle detection based on point cloud data using AI model #5 to determine information about other vehicles. The AI ​​model #5 may be a vehicle detection model. For example, as shown in Figure 4, the vehicle detection model may include a view transformer and a vehicle decoder. The view transformer may include PointPillars. The vehicle decoder may include a decoder, a regression head, and a classification head. For example, the decoder may be a Transformer decoder. The vehicle detection process is illustrated below with reference to Figure 4.

[0223] As an example, the vehicle inspection process can be implemented through the following steps.

[0224] B1) Compress point cloud data to a two-dimensional (2D) plane.

[0225] For example, voxelization can be used to compress point cloud data into a bird's-eye view (BEV) plane.

[0226] B2) Extract features from points on the same grid in the 2D plane to obtain a 2D feature map.

[0227] For example, for 3D points in the same grid within the BEV plane, PointNet can be used to extract features to obtain a 2D BEV feature map, or BEV features. As shown in Figure 4, PointPillars can also be used to extract features to obtain BEV features.

[0228] B3) Extract features from the 2D feature map.

[0229] For example, CNNs or Transformers can be used to further extract features from the BEV feature map.

[0230] Step B3) is an optional step.

[0231] B4) The Transformer decoder extracts features based on the key, value, and query to obtain the vehicle's features. The key and value are 2D features.

[0232] The Transformer decoder can use BEV features as keys and values, and a set of queries to extract vehicle features. The number of queries in this set can be fixed. This set of queries is learnable.

[0233] B5) Vehicle features are decoded through a fully connected network to obtain vehicle detection results.

[0234] Vehicle detection results may include the vehicle's position, such as its BEV planar position. Furthermore, vehicle detection results may also include the vehicle's 3D spatial dimensions and / or confidence level.

[0235] Taking Figure 4 as an example, the fully connected network can include a regression head and a classification head, which are used to perform bounding box regression and classification on the features of the vehicle, respectively.

[0236] For example, some or all of the vehicles in the vehicle detection results can be considered as other vehicles in the environment surrounding the vehicle itself.

[0237] Furthermore, as shown in Figure 4, the output of the vehicle decoder can also be transformed for subsequent processing.

[0238] For ease of description, this application embodiment mainly uses all vehicles in the vehicle detection results as other vehicles in the surrounding environment of the self vehicle for illustration, and does not constitute a limitation on the solution of this application embodiment.

[0239] If the vehicle detection result indicates that no vehicle exists, it can be considered that no other vehicle exists in the point cloud data. In this case, the information about other vehicles can be used to indicate that no other vehicle exists in the environment surrounding the vehicle itself.

[0240] Therefore, information about other vehicles can be determined based on point cloud data.

[0241] AI model #5 can be obtained through training.

[0242] The specific object detection and training processes can be found in the examples below, and will not be described in detail here.

[0243] It should be understood that the numbering of AI model #5 is for convenience of description only and has no limiting effect.

[0244] It should be understood that the above are merely examples, and vehicle detection can also be achieved through other deep learning models, or through non-deep learning methods. This application does not limit the scope of these examples.

[0245] The following provides illustrative examples of the use cases for Method 1 and Method 2.

[0246] For example, if the vehicle cannot obtain point cloud data, vehicle detection can be performed using method 1.

[0247] For example, if the vehicle is not equipped with LiDAR, vehicle detection can be performed based on image data.

[0248] For example, if the vehicle is able to acquire image data and point cloud data, the vehicle information can be determined by method 1 and / or method 2.

[0249] For example, vehicle detection is performed using method 1 and method 2 respectively, yielding vehicle detection results based on image data and vehicle detection results based on point cloud data. Information about other vehicles is then determined based on these two vehicle detection results.

[0250] For example, vehicle detection is performed using method 2 to obtain vehicle detection results based on point cloud data. Information about other vehicles is determined based on the vehicle detection results based on point cloud data. In this case, the information about other vehicles is unrelated to the image data.

[0251] The accuracy of vehicle detection results based on image data is greatly affected by image quality. Glare scenes usually have high light intensity, and the quality of the acquired images may be low, which may lead to lower accuracy of the detection results based on image data. Vehicle detection results based on point cloud data are not affected by glare and have higher accuracy, which helps to ensure the accuracy of shading areas.

[0252] Optionally, step 330 may include: performing trajectory prediction based on the input data to obtain the predicted trajectory of the vehicle.

[0253] Alternatively, the trajectory prediction process can also be performed by other devices besides the vehicle.

[0254] The predicted trajectory of a vehicle is the predicted future trajectory of the vehicle.

[0255] The predicted trajectory of a vehicle can be determined in a variety of ways.

[0256] Optionally, step 330 may include: determining the predicted trajectory of the vehicle based on the vehicle's historical trajectory. The vehicle's historical trajectory is determined based on the vehicle's motion data.

[0257] For example, a vehicle's historical trajectory may include the vehicle's location at one or more past time steps, i.e., one or more historical locations of the vehicle.

[0258] Furthermore, the input data may include map data. Step 330 may include: determining the predicted trajectory of the vehicle based on the vehicle's historical trajectory and map data.

[0259] For example, map data may include at least one of high-precision (HD) maps or standard definition (SD) maps.

[0260] SD map can be understood as a map that includes basic road network information.

[0261] HD maps can be understood as maps that include high-precision road and lane information.

[0262] Further, optionally, step 330 may include: generating a predicted trajectory for the vehicle based on the vehicle's historical trajectory, lane information of the drivable lanes, and map data.

[0263] For example, lane information for a drivable lane may include the centerline and / or attribute information of the drivable lane.

[0264] Lane information of drivable lanes can serve as an explicit static scene representation, making the predicted trajectory more consistent with the traffic scenario.

[0265] For example, drivable lanes can be perceived based on SD maps or HD maps.

[0266] Optionally, step 330 may include: generating an anchor trajectory based on the vehicle's motion data and lane information of the drivable lanes; and generating a predicted trajectory for the vehicle based on map data, the vehicle's historical trajectory, and the anchor trajectory.

[0267] The number of anchor tracks can be determined based on the number of drivable lanes. For example, the number of anchor tracks can be equal to the number of drivable lanes.

[0268] In step 330, the predicted trajectory of a vehicle can be the predicted trajectory of a single vehicle (such as the predicted trajectory of the vehicle itself) or the predicted trajectory of multiple vehicles (such as the predicted trajectory of the vehicle itself and the predicted trajectories of other vehicles).

[0269] The following explanation uses the process of determining the predicted trajectories of multiple vehicles as an example. For instance, the predicted trajectories of multiple vehicles can be determined based on their historical trajectories and map data.

[0270] Figure 5 illustrates a schematic diagram of a trajectory prediction process according to an embodiment of this application. The trajectory prediction process will be described exemplarily below with reference to Figure 5. For example, as shown in Figure 5, the input data may include vehicle motion data, image data acquired by a camera, point cloud data acquired by a LiDAR, and a navigation map. The point cloud data is optional. The navigation map may be a navigation SD map.

[0271] C1) The map features are obtained by encoding features based on the map data using a map encoder.

[0272] For example, when the map data includes an HD map, features are encoded based on the HD map using an HD map encoder to obtain HD map features. For instance, as shown in Figure 5, the HD map is optional, and a local HD map can be constructed using image data and point cloud data.

[0273] For example, when the map data includes an SD map, feature encoding is performed based on the SD map using an SD map encoder to obtain SD map features. For instance, SD routes are feature-encoded using an SD map encoder to obtain SD map features. SD routes originate from the SD map. As another example, SD routes and SD route skeleton information are feature-encoded using an SD map encoder to obtain SD map features. SD route skeleton information originates from the SD map.

[0274] Furthermore, as shown in Figure 5, when the map data includes both SD and HD maps, the HD map and SD routes can be aligned using an alignment network (AlignNet), and the processing result can be feature-encoded using an SD map encoder to obtain SD map features.

[0275] C2) The relative information between the vehicle motion data of multiple vehicles is encoded through the agent-agent interaction module to obtain vehicle interaction features.

[0276] For example, as shown in Figure 5, the relative information between the vehicle motion data of these multiple vehicles can be represented by agent trajectories. Agent trajectories can be inferred based on the vehicle motion data, image data, and point cloud data of the vehicle itself.

[0277] C3) The historical trajectories of multiple vehicles are encoded using a temporal module to obtain historical temporal features.

[0278] The temporal module can be implemented using the Temporal Transformer model. For example, the Temporal Transformer encodes the historical trajectory of each vehicle and the agent-agent interaction features (i.e., vehicle interaction features) to extract temporal information, i.e., historical temporal features.

[0279] C4) The historical time series features and map features are fused through the proxy map interaction module to obtain the time series and map interaction features.

[0280] For example, when the map data includes HD maps, the agent map interaction module may include an agent-lane interaction module, which fuses HD map features and historical time-series features to obtain time-series and HD map interaction features.

[0281] For example, when the map data includes SD maps, the agent map interaction module may include an agent-SD interaction module, which fuses SD map features and historical time-series features to obtain time-series and SD map interaction features.

[0282] The routing and route skeleton information provided by the navigation SD map can effectively improve trajectory prediction accuracy, and can also predict a reasonable trajectory even when the HD map is missing. Introducing the agent-SD Interaction module to fuse and extract SD map features and vehicle historical temporal features further enhances trajectory prediction accuracy.

[0283] C5) performs feature fusion through the global interaction module to obtain global features.

[0284] For example, when the map data includes HD maps and SD maps, the global interaction module can be used to perform feature fusion on vehicle interaction features, historical time series features, time series interaction features with HD maps, and time series interaction features with SD maps to obtain global features.

[0285] Alternatively, when the map data includes both HD and SD maps, the global interaction module can be used to fuse vehicle interaction features, historical time-series features, and time-series and HD map interaction features to obtain global features.

[0286] For example, when the map data only includes the SD map, the global interaction module can be used to perform feature fusion on vehicle interaction features, historical time series features, and time series and SD map interaction features to obtain global features.

[0287] C6) The global features are decoded by the decoder to obtain the predicted trajectories of the multiple vehicles.

[0288] Furthermore, the above process may also include step C7).

[0289] C7) The trajectory planner generates an anchor trajectory based on the vehicle motion data of the multiple vehicles and the lane information of the drivable lanes.

[0290] A trajectory planner can also be called a planner or motion planner.

[0291] Furthermore, C7) may include: generating an anchor trajectory by means of a trajectory planner based on the vehicle motion data of the multiple vehicles, lane information of the drivable lanes, and future trajectory speed sampling intervals.

[0292] Furthermore, the trajectory planner can incorporate multi-vehicle motion collision modeling in the cost function design for optimizing sampled trajectories, thereby improving the safety of predicted trajectories.

[0293] If the above process includes step C7), step C6) may include: decoding the global features to obtain a one-stage trajectory; performing feature encoding on the one-stage trajectory and the anchor trajectory respectively to obtain one-stage trajectory features and anchor trajectory features; performing feature fusion on the one-stage trajectory features and the anchor trajectory features; and decoding the fused features to obtain the predicted trajectories of the multiple vehicles.

[0294] Thus, the final predicted trajectory can be called a multimodal trajectory. At different intersections, different predicted trajectories and their probability distributions can be generated based on the available lanes. In one-way lanes, the trajectory converges to a single mode.

[0295] Optionally, the predicted trajectory of the vehicle can be a 3D predicted trajectory, that is, the predicted trajectory of the vehicle in three-dimensional space.

[0296] In the embodiments of this application, a 3D predicted trajectory can be obtained through 3D trajectory prediction.

[0297] Optionally, the vehicle's historical trajectory is a 3D historical trajectory. The 3D historical trajectory can be used as input data for 3D trajectory prediction.

[0298] The 3D predicted trajectory of a vehicle can be determined based on the vehicle's historical 3D trajectory.

[0299] For example, vehicle detection results based on image data can be used to determine the 3D position of other vehicles, thereby obtaining the 3D historical trajectory of other vehicles.

[0300] For example, vehicle detection results based on point cloud data can be used to determine the 3D position of other vehicles, thereby obtaining the 3D historical trajectory of other vehicles.

[0301] Optionally, the 3D location of the drivable lane can be used as input data for 3D trajectory prediction.

[0302] For example, the drivable lane can be input into the trajectory planner in the form of 3D position coordinates. For instance, the 3D predicted trajectory of the multiple vehicles can be determined based on the 3D historical trajectories of the multiple vehicles and the 3D position of the drivable lane.

[0303] Alternatively, the estimated 3D position coordinates of the road surface or lane lines obtained by ADS can be used as input data for 3D trajectory prediction.

[0304] From the existing BEV perspective, we add Z-axis vehicle temporal encoding and lane line feature encoding for 3D perception, add Z-axis constraints to the loss function of the vehicle detection model, and finally decode to generate 3D predicted trajectory.

[0305] In scenarios such as going up or down slopes, meeting on or off ramps, and uneven road surfaces, vehicles move along the Z-axis. In the solution of this application embodiment, the predicted trajectory is a 3D trajectory, that is, the future 3D position of the vehicle is predicted, which helps to further ensure the accuracy of the shading area.

[0306] Alternatively, the predicted trajectory of the vehicle can also be a 2D trajectory, that is, the predicted trajectory of the vehicle in 2D space.

[0307] For example, in the absence of point cloud data, the 2D position of a vehicle can be obtained based on image data, and then the predicted trajectory of the vehicle in 2D space can be predicted. Correspondingly, the predicted trajectory of a light source in 2D space can be obtained.

[0308] Step 340 will be explained below.

[0309] The information about the light source in step 320 can be regarded as information about the light source at a historical moment, or information about the light source at the current moment.

[0310] Step 340 can also be understood as predicting the information of the light source at future moments based on the information of the light source at the current moment (or historical moment) and the predicted trajectory of the vehicle.

[0311] Future moments and historical moments can be understood as relative concepts. Any time after the moment when the light source information is acquired in step 320 can be considered a future moment.

[0312] Optionally, step 340 may include steps 341 to 342 (not shown in the figure).

[0313] 341. Predict the future position of the light source based on the information of the light source and the information of other vehicles.

[0314] For example, step 341 can also be replaced by predicting the future light cone of the light source based on the information of the light source and the information of the other vehicle.

[0315] The future light cone of a light source is the light cone formed by the light source at a future moment.

[0316] The light cone of a light source can be represented by one or more of the following: the position of the light source, the orientation of the light source, or the divergence angle of the light source.

[0317] The orientation of the light source can also be replaced by the orientation of the light cone. The divergence angle of the light source can also be replaced by the divergence angle of the light cone, the range of the light source, or the range of the light cone, etc.

[0318] 342. The future light cone curve is determined based on the future position of the light source and the predicted trajectory of the vehicle. In the embodiments of this application, the future light cone curve can also be referred to as the predicted light cone curve.

[0319] The method for determining the future location of the light source is explained below.

[0320] Step 341 can be understood as predicting the future position of the light source based on information such as the current (or historical) position of the light source and information about other vehicles.

[0321] The light source may include static light sources and / or dynamic light sources.

[0322] For example, if a light source matches a vehicle, meaning the light source is located on that vehicle, the light source can be considered a dynamic light source. The future position of the dynamic light source is determined based on the future position of the matched vehicle, or in other words, based on the predicted trajectory of the matched vehicle. The position of the light source on the matched vehicle is usually relatively fixed compared to the position of the dynamic light source. Obtaining the predicted trajectory of the matched vehicle is equivalent to obtaining the predicted trajectory of the light source on the matched vehicle.

[0323] For example, if a light source does not have a matching vehicle, then the light source can be considered a static light source. The future position and the historical position of a static light source can be the same position.

[0324] Specifically, information about the light source and other vehicles can be used to determine whether the light source is dynamic or static.

[0325] As an example, if the information from another vehicle indicates that there is no other vehicle in the environment around the vehicle, then all light sources in step 320 can be regarded as static light sources.

[0326] As an example, if the information about another vehicle includes its motion data, the dynamic or static light source can be determined through light source-vehicle modal matching. The process of light source-vehicle modal matching is explained below.

[0327] One possible approach is to perform vehicle mode matching based on the position of the light source and the positions of other vehicles.

[0328] For ease of description, let's assume the number of light sources is N. p The number of his cars is N. l N p N is a positive integer. l It is a positive integer. That is, according to N p The position of each light source and N l The location of the other vehicle is determined. p One light source and N l The matching relationship between each vehicle. Each light source corresponds to at most N. l One of his vehicles.

[0329] Specifically, according to N p The position of each light source and N l The positions of each vehicle are combined and matched to determine N. p One light source and N l The matching relationship between each other and other vehicles.

[0330] In the N p Of the light sources, those that are matched with other vehicles can be considered dynamic light sources, while those that are not matched with any other vehicle can be considered static light sources. In other words, light sources matched with other vehicles are dynamic light sources, and the other light sources are static light sources.

[0331] The positions of the light source and the other vehicle used for combined matching can be in the same coordinate system, for example, both in the world coordinate system in three-dimensional space, i.e., three-dimensional positions, or spatial positions. That is, according to N p The three-dimensional position of each light source and N l The three-dimensional position of the vehicle is determined. p One light source and Nl The matching relationship between each other and other vehicles.

[0332] For example, as described above, light source detection can be performed based on image data to obtain a light source detection result. This result can include the position of the light source on the imaging plane (i.e., the image plane or pixel plane) of the visual sensor, specifically the position of the light spot area formed by the light source on the imaging plane. The position of the light source on the imaging plane can be represented by the pixel coordinates of the light source. Before performing combined matching, the pixel coordinates of the light source can be converted to three-dimensional coordinates, i.e., the world coordinates of the light source in three-dimensional space. In other words, the position of the light source can be represented by the three-dimensional coordinates of the light source.

[0333] Figure 7 shows a schematic diagram of the three-dimensional position of the light source and its position on the imaging plane.

[0334] For example, as shown in Figure 7, based on the position of the light spot area formed by the light source (the dynamic and static light sources in Figure 7) in the imaging plane, combined with the principle of rectilinear propagation of light and the pinhole camera model, the position curve of the light source in three-dimensional space is inferred, and the estimated result of the three-dimensional position of the light source is obtained. As shown in Figure 7, this imaging plane can be the imaging plane of the DVR. The matching relationship between the light source and the other vehicle is determined by combining and matching the position curve of the light source in three-dimensional space with the three-dimensional position of the other vehicle.

[0335] As another possible implementation, light source vehicle mode matching can be performed based on the position of the light source and the predicted trajectory of other vehicles, as shown in Figure 6.

[0336] For ease of description, let's assume the number of light sources is N. p The number of predicted trajectories for his car is N. l N p N is a positive integer. l It is a positive integer. That is, according to N p The position of each light source and N l The predicted trajectory of each vehicle is determined. p One light source and N l The matching relationship between the predicted trajectories of individual vehicles.

[0337] Specifically, according to N p The position of each light source and N l The predicted trajectories of each vehicle are combined and matched to determine N. p One light source and N l The matching relationship between the predicted trajectories of individual vehicles.

[0338] In the N pAmong the light sources, those that match the predicted trajectories of other vehicles can be considered dynamic light sources, while those that do not match the predicted trajectories of any other vehicle can be considered static light sources. In other words, light sources that match the predicted trajectories of other vehicles are dynamic light sources, and the other light sources are static light sources.

[0339] The position of the light source used for combined matching and the predicted trajectory of other vehicles can be in the same coordinate system. The description of the same coordinate system can be found in the previous text and will not be repeated here.

[0340] For a detailed description of determining the matching relationship between the light source and other vehicles based on the position of the light source and the predicted trajectory of other vehicles, please refer to Method 1000 below.

[0341] Furthermore, the position of the light source can be used to construct a three-dimensional light cone. Constructing a three-dimensional light cone can also be replaced by descriptions such as constructing a light cone distribution in three-dimensional space, constructing a light field distribution in three-dimensional space, or constructing a three-dimensional light field. In the embodiments of this application, the three-dimensional light cone is simply referred to as a light cone.

[0342] For example, taking a light source as an example, a three-dimensional light cone of the light source can be constructed based on the position of the light source and the range of the light spot area formed by the light source on the image plane.

[0343] For example, the three-dimensional light cone of a light source at time t can be represented by the spatial position, orientation, and extent of the light source at time t. The three-dimensional light cone of the light source at time t can be constructed based on its spatial position at time t and the extent of its spot area on the image plane at time t.

[0344] Step 341 can also be understood as: determining the future three-dimensional light cone of the light source based on the three-dimensional light cone of the light source and the predicted trajectory of the other vehicle.

[0345] Figure 8 shows a schematic diagram of the change process of the light cone.

[0346] For example, as shown in Figure 8, the spatial position, orientation, and range of the light source at time t+Δt are determined based on the historical trajectory of the other vehicle at time t, the orientation of the light source at time t, and the range of the light source at time t.

[0347] Step 342 can also be called light cone prediction.

[0348] Optionally, step 342 may include: projecting the future position and future orientation of the light source into the local vehicle coordinate system according to the future pose of the vehicle to obtain the predicted light cone curve of the light source.

[0349] Specifically, by projecting the future position and orientation of the light source onto the vehicle's coordinate system based on the vehicle's future pose, the position and orientation of the light source in the vehicle coordinate system at future moments can be obtained. The predicted light cone curve of the light source can indicate its future position, future orientation, and divergence angle.

[0350] For example, the divergence angle of the light source can be determined based on image data. For instance, the divergence angle of the light source can be determined based on the spot area of ​​the light source in the image data, or in other words, based on the spot projection formed by the light source on the pixel plane.

[0351] The vehicle's coordinate system can be determined based on the vehicle's predicted trajectory.

[0352] For example, as shown in Figure 6, light cone prediction can be performed on dynamic and static light sources based on the predicted trajectory of the vehicle.

[0353] The method for determining the light cone curve can be found in Method 1000, which will not be described in detail here.

[0354] In the above scheme, steps 341 and 342 can also be understood as determining the predicted light cone curve of the light source based on the current three-dimensional light cone, the predicted trajectory of the vehicle, and the predicted trajectories of other vehicles. This can compensate for time delay and achieve stable tracking of the light cone curve. It is equivalent to constructing a dynamic change model of the light cone over time in three-dimensional space. This model can obtain the optimal estimate of the light cone for future times based on the light cone at historical times, thereby predicting the light spot area at future times and improving the accuracy of the shading area.

[0355] Furthermore, if method 300 includes step 342, step 350 may include: performing parallax registration on the predicted light cone curve based on the human eye position to predict the light spot area formed by the light source on the imaging plane of the light-shielding device. The light-shielding area is determined based on this light spot area.

[0356] For example, as shown in Figure 6, the predicted light cone curve is parallax registered to generate occlusion information.

[0357] The position of the human eye and the position of the vision sensor are usually different. In the solution of this application embodiment, the predicted light cone curve can be parallax registered according to the position of the human eye to obtain the relative positional relationship of the light source to the human eye, so as to compensate for the inconsistency between the position of the vision sensor and the position of the human eye, which is beneficial to achieving precise occlusion of the human eye.

[0358] Figure 9 illustrates the principle of parallax registration. As shown in Figure 9, the positions of the human eye and the DVR are different, and correspondingly, the light spot projection (i.e., the light spot area) formed by the light source on different imaging planes is different. An affine model of the human eye, the light-blocking device, the visual sensor image, and the light cone can be established to determine the mapping position of the light spot on the imaging plane of the light-blocking device.

[0359] Furthermore, in method 300, other information about the light spot region can be calculated based on the predicted light conic curve. For example, the velocity and / or acceleration of the center of the light spot region can be calculated based on the predicted trajectory of the vehicle and the instantaneous rate of change of the predicted light conic curve.

[0360] Optionally, the range of the shading area may include the range of the light spot area.

[0361] This allows for complete coverage of the light spot area, thus ensuring the masking effect.

[0362] Shading devices can be used to adjust the light transmittance of the shaded area based on shading information in order to block light from shining on the user.

[0363] An example of a light-shielding device can be found in the description in Figure 2.

[0364] The shaded area is located on the imaging plane of the shading device. In step 360, shading information can be output to the shading device to control the shading device to adjust the light transmittance in the shaded area so that the light transmittance of the shaded area is different from that of the non-shaded area. For example, the light transmittance of the shaded area can be made less than that of the non-shaded area.

[0365] The non-shaded area can be understood as the area on the imaging plane of the shading device other than the shaded area.

[0366] Furthermore, optionally, the area of ​​the shaded region is related to the method of determining the information of other vehicles.

[0367] When the information about another vehicle is determined based on point cloud data, the ratio between the area of ​​the shading region and the area of ​​the spot region is ratio #1. When the information about another vehicle is determined based on image data, the ratio between the area of ​​the shading region and the area of ​​the spot region is ratio #2. Ratio #1 is less than ratio #2.

[0368] The shading area obtained when determining the vehicle motion data of another vehicle based on point cloud data is closer to the light spot area. In other words, in the same or similar surrounding environment, the area of ​​the shading area obtained when determining the vehicle motion data of another vehicle based on point cloud data can be smaller than the area of ​​the shading area obtained when determining the vehicle motion data of another vehicle based on image data.

[0369] Compared to vehicle detection results based on image data, those based on point cloud data are likely to be more accurate. This allows for more precise prediction of vehicle trajectories, which in turn leads to more accurate prediction of light spot areas. In this case, a smaller shading area can be used to better fit the light spot area, thus improving user experience while ensuring effective shading. Conversely, vehicle detection results based on image data may be affected by glare, resulting in lower accuracy. In this case, a larger shading area can be used to achieve comprehensive coverage of the light spot area and ensure effective shading.

[0370] The above is just one example of implementation. For example, in other implementations, the ratio #1 can also be equal to the ratio #2, or the ratio #1 can also be greater than the ratio #2.

[0371] Furthermore, optionally, the area of ​​the shaded area is related to the distance between the vehicle and other vehicles.

[0372] When the distance between your vehicle and another vehicle is distance #1, the ratio between the area of ​​the shading area and the area of ​​the light spot is ratio #3. When the distance between your vehicle and another vehicle is distance #2, the ratio between the area of ​​the shading area and the area of ​​the light spot is ratio #4. When distance #1 is less than distance #2, ratio #3 is less than ratio #4.

[0373] In other words, the closer the distance between your car and other cars, the closer the shading area will be to the area of ​​the light spot.

[0374] The closer the vehicle is to other vehicles, the more accurate the information about those vehicles is likely to be. This allows for a more accurate prediction of the vehicle's trajectory, which in turn helps in predicting a more precise light spot area. In this case, a smaller shading area can be used, fitting closer to the light spot area, thus improving the user experience while ensuring effective shading. Conversely, when the distance between vehicles is greater, a larger shading area can be used to achieve full coverage of the light spot area and ensure effective shading.

[0375] The above is just one example of implementation. For example, in other implementations, the ratio #3 can also be equal to the ratio #4, or the ratio #3 can also be greater than the ratio #4.

[0376] Furthermore, optionally, the shading information can also be used to indicate the light transmittance of the shading area.

[0377] Alternatively, the light transmittance of the shaded area can also be determined by the shading device itself.

[0378] Optionally, the light transmittance of the shaded area is related to the light intensity of the light source.

[0379] The higher the light intensity of the light source, the less light transmittance the shaded area will have.

[0380] The light intensity of a light source can also be replaced by the brightness of the light source.

[0381] For example, the transmittance of the light-transmitting area can be determined based on the mapping relationship between brightness and transmittance and the brightness of the light source.

[0382] The brightness of a light source can be determined based on image data. For example, the brightness of the light source can be determined based on the brightness within the range of the light source in the image data. For instance, the average brightness within the range of the light source in the image data can be used as the brightness of the light source. Alternatively, the maximum brightness within the range of the light source in the image data can be used as the brightness of the light source.

[0383] This allows the light transmittance of the shaded area to be adjusted according to the brightness of the light source, which helps users experience a more consistent brightness under different lighting conditions, such as dim or bright light, ensuring a stable shading effect and thus improving the user experience.

[0384] Alternatively, the light transmittance of the shaded area can also be a preset fixed value.

[0385] In addition, occlusion information can also be used to indicate other information. For example, occlusion information can also be used to indicate one or more of the following: the light transmittance of the occluded area, the velocity of the center of the occluded area, or the acceleration of the center of the occluded area.

[0386] In the solution of this application embodiment, future light source information (such as the position of the light source) can be predicted based on the currently detected light source information and vehicle information, thereby determining the shading area. This can compensate for system latency and motion latency, helping to avoid the protective effect being affected by latency. Simultaneously, by referencing vehicle information when predicting the future light source position, it is beneficial to predict the accurate spot area, thus improving the accuracy of the shading area. In other words, while blocking the light from the glare source, the visibility of non-spot areas is maintained as much as possible, ensuring a good field of vision for the user and improving the protective effect. In the solution of this application embodiment, the user does not need to manually block the glare, which improves the user experience.

[0387] Furthermore, in the solution of this application embodiment, image enhancement can be performed on the acquired image data to improve image quality, which is beneficial to improving the detection effect in glare scenes, thereby helping to obtain more accurate detection results, such as obtaining a more accurate position of the light source.

[0388] Glare scenarios are quite complex, and the perception of other vehicles based on image data is easily interfered with, making it difficult to obtain accurate vehicle detection results.

[0389] In the solution of this application embodiment, point cloud data is introduced. Point cloud data is not affected by glare light source, has high data quality, and the accuracy of vehicle detection results obtained based on point cloud data is high, which is beneficial to the accuracy of subsequent processing. For example, it is beneficial to improve the accuracy of the matching result between the light source and the vehicle, and to ensure the accuracy of the predicted trajectory of the vehicle, thereby ensuring the accuracy of the predicted spot area, and thus to achieving precise masking and improving the protection effect.

[0390] Data-driven, learning-based trajectory prediction methods primarily rely on attention mechanisms to query key element features. However, the predicted trajectories may deviate from lane lines or fail to meet vehicle kinematic constraints.

[0391] In the solution of this application embodiment, the introduction of drivable lane lines as reference information for trajectory prediction is beneficial to improving the accuracy of trajectory prediction.

[0392] Specifically, a trajectory planner is introduced to model the motion of vehicles based on temporal information. Using drivable lanes as reference lines, the planner essentially generates possible future trajectories from candidate drivable lanes. This helps ensure that the predicted trajectory conforms to the traffic scenario and better reflects the driver's actual decision-making process, thus improving the accuracy of trajectory prediction. For example, traffic scenario representation based on the trajectory planner can be combined with learning-based representation methods. This involves fusing features from the initial trajectory obtained during the planning phase (i.e., the anchor trajectory) and the trajectory obtained through the learning method to obtain the final trajectory. The trajectory obtained in this way is highly interpretable and more accurate.

[0393] Furthermore, in the scheme of this application embodiment, modal matching is performed on the light source and the vehicle. The light source that is matched with the vehicle is regarded as a dynamic light source, and the light source that is not matched with the vehicle can be regarded as a static light source. The light source information is predicted for static light sources and dynamic light sources respectively, which helps to further improve the accuracy of the prediction results.

[0394] Figure 10 shows a schematic flowchart of another glare protection method according to an embodiment of this application. The method 1000 shown in Figure 10 can be regarded as a specific implementation of the method shown in Figure 3.

[0395] As shown in Figure 10, method 1000 may include the following steps.

[0396] (a) Vehicle data collection.

[0397] Step (1) corresponds to step 310 in method 300.

[0398] As shown in Figure 10, vehicle data can include image data, point cloud data, vehicle motion data, and map prior data.

[0399] Point cloud data is optional.

[0400] Image data, point cloud data, and vehicle motion data can be found in the description of Method 300 above, and will not be repeated here.

[0401] For example, prior map data may include the vehicle's planned path and / or lane access attributes in the SD map.

[0402] For example, image data can be acquired through cameras, point cloud data through LiDAR, map prior data through SD maps, and vehicle motion data can be acquired through GNSS, IMU, and / or WSS.

[0403] The above is just an example. In other ways, vehicle data can also be collected through other devices and methods. For details, please refer to step 310. It will not be repeated here.

[0404] For example, in step (a), the vehicle data can be collected in real time.

[0405] (ii) Image data enhancement.

[0406] After obtaining the image data captured by the camera, image enhancement can be performed on the image data.

[0407] For example, image enhancement can be achieved through the following steps.

[0408] 1) Use adaptive histogram equalization to process local region contrast.

[0409] Specifically, the RGB image captured by the camera is converted into a grayscale image, the grayscale image is processed using adaptive histogram equalization, and the processed result is converted back into an RGB image.

[0410] Taking a frame of image captured by a camera as an example, for instance, the RGB image captured by the camera is converted into a grayscale image, the contrast limit of the adaptive histogram equalization and the size of the segmented image grid are set to 2.0 and (8, 8) respectively, and then the grayscale image is processed using adaptive histogram equalization, and the processed result is converted back into an RGB image.

[0411] 2) Normalize the pixels of the image obtained in step 1) and perform gamma correction to suppress the influence of overly bright areas in the image.

[0412] For example, the pixels of the image obtained in step 1) are normalized, and then gamma correction is performed. The gamma coefficient used for gamma correction is 1.2.

[0413] The image enhancement methods described above are merely examples and do not constitute a limitation on the solutions of this application. In other implementations, image enhancement can also be performed in other ways, such as image enhancement based on deep learning.

[0414] Step (ii) is optional. If step (ii) is not included in method 1000, the image data subsequently used for light source detection and vehicle detection can be the image data acquired in step (i).

[0415] (III) Target detection.

[0416] Step (iii) corresponds to steps 320 and 331 in method 300.

[0417] As shown in Figure 10, step (iii) may include light source detection and vehicle detection. Light source detection can be implemented based on enhanced image data. Vehicle detection can be implemented based on enhanced image data or point cloud data.

[0418] The following example illustrates the process of light source detection and vehicle detection based on image data. For ease of description, this example uses only one frame of image and does not limit the solution of this application. The target detection process for other frames of image can be referred to the following description.

[0419] For example, the process of light source detection and vehicle detection based on image data may include the following steps.

[0420] The input image in step (iii) can be regarded as image #1 or image #2 in method 300. In method 1000, only the example of image #1 and image #2 being the same image is used for illustration, which does not constitute a limitation on the solution of the embodiments of this application. In other implementations, image #1 and image #2 can also be different images. For a detailed description, please refer to method 300.

[0421] 1) Extract features from the image to obtain a multi-scale feature map of the image.

[0422] Suppose the input image in step (iii) (e.g., the enhanced image) has dimensions of 320×800×3. For example, a CNN can be used to extract features from the input image to obtain feature maps of three dimensions (i.e., multi-scale feature maps). These three feature maps come from different network layers of the CNN. These three feature maps can include intermediate and final features from the CNN. The dimensions of these three feature maps are: 10×25×128, 20×50×64, and 40×100×32. These three dimensions can respectively cover the detection of targets at different distances (near, medium, and far).

[0423] For example, the feature extraction model can be a standard residual network (ResNet-18) or other CNN structures.

[0424] 2) Perform feature fusion on multi-scale feature maps.

[0425] For example, by transforming the channel count of the feature maps of the above three sizes using three fully connected networks to unify the channel count to 128, the resulting feature map sizes are 10×25×128, 20×50×128, and 40×100×128, respectively. These three 2D feature maps are then flattened into 1D features and combined into a single 5250×128 vector (i.e., the combined 1D feature). Standard self-attention is used to process the combined 1D feature, allowing for interaction and complementarity between the feature maps of different sizes, while the output feature size remains unchanged at 5250×128.

[0426] 3) Extract features using a decoder to obtain the target features.

[0427] For example, the decoder has 100 learnable queries, denoted as 100×128. 100 learnable queries means that it can detect up to 100 targets, namely 100 light sources and vehicles.

[0428] Using standard multi-layer cross-attention, the 5250×128 features obtained in step 2) are used as key and value, and a 100×128 query is used to perform feature query to obtain the 100×128 target features.

[0429] 4) The target features are classified and bounding box regressions are performed by two fully connected layers to obtain the target's category and geometric information.

[0430] Two fully connected layers decode the target's category and geometric information, respectively. The geometric information can include the target's location and size. One fully connected layer decodes a 100×3 vector, representing the predicted probability that each of the 100 targets (i.e., each object) is a vehicle, a light source, or the background. The other fully connected layer decodes a 100×4 vector, representing the center coordinates, length, and width of each target's predicted bounding box. The location of each target can be represented by the center coordinates of its predicted bounding box. The size of each target can be represented by the length and width of its predicted bounding box.

[0431] The neural network model used in the above object detection process is obtained through training. For example, during training, 100 detection results can be Hungarian matched with k ground truth values. For the k matched results, the predicted probabilities of k×3 are learned from the ground truth class using cross-entropy loss, and the geometric information of k×4 is regressed from the geometric information of the ground truth using generalized intersection over union loss (GIoU loss). For the 100-k results that do not match any ground truth values, their predicted probabilities of (100-k)×3 are learned from the background class. k is a positive integer.

[0432] The following example illustrates the process of vehicle detection based on point cloud data.

[0433] For example, the process of vehicle detection based on point cloud data may include the following steps.

[0434] 1) Point cloud data filtering.

[0435] For example, the solution of this application embodiment can be used to protect against glare in front of a vehicle. In this case, the input point cloud data can be filtered, and vehicle detection can be performed based on the filtered point cloud data in subsequent steps.

[0436] Assume the input point cloud data has a size of J×4, where J is the number of points. Each point is represented by four dimensions, such as (x, y, z, intensity). For example, to filter point cloud data from the input point cloud data within a 100m×60m area—100m in front of the vehicle, 30m to the left, and 30m to the right—the J×4 point cloud data is filtered into J'×4 point cloud data based on the aforementioned rectangular range.

[0437] Step 1) is optional.

[0438] 2) Compress the point cloud data to a 2D plane.

[0439] For example, the 100m × 60m rectangular area in step 1) is divided into a 1000 × 600 grid with a spacing of 0.1m. J' × 4 point cloud data is allocated to the grid. Each grid contains a maximum of 128 points. If a grid contains more than 128 points, some points in that grid are randomly discarded to ensure that the total number of points in that grid is 128. If a grid contains fewer than 128 points, it is padded with zeros to ensure that the total number of points in that grid is 128.

[0440] 3) Extract features from points on the same grid in the 2D plane to obtain a 2D feature map.

[0441] For example, for each grid in step 2), a fully connected network is used to process the 128×4 point cloud data into a 128-dimensional vector, resulting in a 1000×600×128 2D point cloud feature. A CNN is then used to further process and compress the 2D point cloud feature, yielding a 125×75×128 BEV feature (i.e., a 2D feature map).

[0442] 4) Feature extraction is performed using a decoder to obtain the target's features.

[0443] For example, the BEV features obtained in step 3) are flattened to 9375×128 and used as input to the decoder. This decoder can be called a point cloud decoder. The point cloud decoder has 30 learnable queries, denoted as 30×128. 30 learnable queries mean that it can detect a maximum of 30 targets, i.e., 30 vehicles.

[0444] Using standard multi-layer cross-attention, the 9375×128 features are used as the key and value, and the 30×128 query is used to perform feature lookup to obtain the 30×128 target features.

[0445] 5) The target features are classified and bounding box regressed by two fully connected layers to obtain the target's category and geometric information.

[0446] Two fully connected layers decode the target's category and geometric information, respectively. The geometric information can include the target's position, size, and orientation. One fully connected layer decodes a 30×2 vector, representing the predicted probability of each of the 30 targets (i.e., each object) being a vehicle or the background. The other fully connected layer decodes a 30×7 vector, representing the center coordinates, length, width, height, and orientation of each target's predicted bounding box. The position of each target can be represented by the center coordinates of its predicted bounding box. The size of each target can be represented by the length, width, and height of its predicted bounding box.

[0447] The neural network model used in the above object detection process is obtained through training. For example, during the training process, 30 detection results are matched with k ground truth values ​​using Hungarian matching. For the k matching results, the k×2 prediction probability is learned from the ground truth category using cross-entropy loss, and the k×7 geometric information is regressed from the ground truth geometric information using GIoU loss. For the 30-k results that do not match ground truth values, their (30-k)×2 category probabilities are learned from the background category.

[0448] It should be understood that the above target detection process is just an example and does not constitute a limitation on the solution of the embodiments of this application. Other implementation methods can refer to method 300 above, which will not be repeated here.

[0449] (iv) Vehicle trajectory prediction.

[0450] Step (iv) corresponds to step 330 in method 300.

[0451] As shown in Figure 10, step (iv) may include trajectory detection of other vehicles and trajectory prediction of the vehicle itself.

[0452] For example, step (iv) may include the following steps.

[0453] 1) Vehicle interaction feature encoding.

[0454] For ease of description, assume there are N vehicles at each time step, and each vehicle's initial state features are a 6-dimensional vector (position, velocity, acceleration). That is, the state of each vehicle can be represented by a 6-dimensional vector (position, velocity, acceleration). At each time step, the states of these N vehicles can be represented by a vector of size (N, 6). A vector of size (a, b) is a vector of size a × b.

[0455] For vehicle i among the N vehicles, calculate the relative information of vehicle i with other vehicles j, generate a relative relationship matrix of shape (N, N, 6), and encode it through a graph neural network (GNN) to obtain vehicle interaction features of size (N, 32).

[0456] 2) Vehicle timing information encoding.

[0457] The historical trajectory of each vehicle is encoded, and its time sequence information is extracted.

[0458] Assuming the past time series has a length of T, meaning that for each vehicle, its historical trajectory includes the vehicle's state over the past T time steps. For these N vehicles, the sequence of vehicle states over the past T time steps can be represented by a vector of size (N, T, 6). For example, encoding the (N, T, 6) vector using a Transformer yields a vehicle time series feature of size (N, 32).

[0459] 3) HD map encoding.

[0460] The HD map obtained from local perception is rasterized to obtain a feature map of size (H, W, C). Here, H and W represent spatial dimensions, and C represents high-definition lane attribute information and traffic control information. A CNN is used to extract the rasterized map features, mapping vehicle positions to the feature map, and outputting HD map features of size (N, 128).

[0461] The vehicle's location can be obtained by detecting the vehicle based on point cloud data or by detecting the vehicle based on image data.

[0462] 4) SD map encoding.

[0463] SD map encoding is similar to HD encoding. It rasterizes the routes and road network skeleton, uses CNN to extract rasterized map features, maps vehicle positions to feature maps, and outputs SD map features of size (N, 64).

[0464] 5) Initial trajectory planning.

[0465] Assume the time step for predicting future trajectories is T', meaning that for each vehicle, its future trajectory includes the vehicle's state for the next T' time steps. For these N vehicles, the positions of the vehicles at the next T time steps can be represented by a vector of (N, T', 2).

[0466] The trajectory planner models the vehicle kinematic parameters of each vehicle based on vehicle-dead reckoning (VDR) and the historical positions of other vehicles. It then samples and predicts future trajectory velocities using a constant turn rate and acceleration (CTRA) motion model to obtain the future trajectory velocity sampling interval. The centerline of the perceived lane is fitted, using the centerline of the drivable lane as the guide line for the trajectory planner. A lattice planner is used as the trajectory planner. The input data includes the initial motion parameters (i.e., initial motion state) of the N vehicles, the centerline of the drivable lane, and the future trajectory velocity sampling interval. The output data is an anchor trajectory of size (N, T', 2), i.e., the initial trajectory.

[0467] 6) Global feature fusion.

[0468] Multi-layer MLP and self-attention are used to fuse vehicle interaction features of size (N, 32), vehicle temporal features of size (N, 32), HD map features of size (N, 128), and SD map features of size (N, 64) to finally obtain a global fused feature of size (N, 256).

[0469] 7) Trajectory decoding.

[0470] A multi-layer MLP is used to perform the first stage of decoding on the global fusion features to obtain a one-stage trajectory of size (N, T', 2). The MLP is then used to encode the one-stage trajectory of size (N, T', 2) to obtain a one-stage trajectory feature of size (N, 64). The anchor trajectory is then feature-encoded by the estimation planner fusion module to obtain an anchor trajectory feature of size (N, 64). The one-stage trajectory feature and the anchor trajectory feature are then fused. The fused feature is then decoded in the second stage to obtain a two-stage trajectory of size (N, T', 2), which is the final output future trajectory (i.e., the predicted trajectory).

[0471] The neural network model in the above process is obtained through training. For example, the neural network model in the above process can be trained based on regression loss and classification loss. The regression loss can be used to constrain the position distribution of each point in the future trajectory, using Laplace loss. The multimodal trajectory probability distribution uses cross-entropy loss. The true value of the model training is the vehicle trajectory of the vehicle's VDR positioning or perception detection at future time.

[0472] The above is just an example. In other implementations, vehicle trajectory prediction can also be achieved through other methods. For a detailed description, please refer to method 300 above, which will not be repeated here.

[0473] It should be understood that if the vehicle detection results obtained in step (iii) indicate that there are no other vehicles around the vehicle, then step (iv) may only include the trajectory prediction of the vehicle.

[0474] (v) Modal matching of light source vehicle.

[0475] In step (v), the matching relationship between the light source and the vehicle detected in step (iii) is determined. Based on this matching relationship, dynamic light sources and static light sources are distinguished, and then the light source model of each light source is constructed, that is, the three-dimensional light cone of each light source is constructed.

[0476] For example, as shown in Figure 10, light source vehicle mode matching can be performed based on the light source detection results and the predicted trajectory of other vehicles.

[0477] For example, step (v) can be implemented by the following steps.

[0478] 1) Three-dimensional light source estimation.

[0479] That is, the estimation of the position of the light source in three-dimensional space.

[0480] Assume that the pixel coordinates of any light source obtained in step (iii) can be represented as P p =(x p ,y p The camera's intrinsic parameter matrix is ​​K, and its extrinsic parameter matrix is ​​M. The coordinates P of the light source in the world coordinate system of three-dimensional space are... w =(x w ,y w ,z w P can satisfy the following formula: w =d·M -1 K -1 P p ;

[0481] Where d represents the depth of the light source relative to the camera.

[0482] The set of pixel coordinates of the detected light source in step (iii) can be represented as: N p Indicates the number of light sources. This represents the pixel coordinates of the i-th light source. N p The set of coordinates of a light source in the world coordinate system of three-dimensional space, that is, the set of three-dimensional light source position curves, can be represented as: or, Let represent the coordinates of the i-th light source in the world coordinate system of three-dimensional space, or the position curve of the i-th three-dimensional light source.

[0483] 2) Combination matching.

[0484] For example, the position of the light source in three-dimensional space is combined and matched with the predicted trajectory of another vehicle to determine the matching relationship between the light source and the other vehicle.

[0485] Assume that the set of predicted trajectories of other vehicles obtained in step (iv) is represented as follows: N l This indicates the number of tracks his car has. Let j represent the j-th trajectory. N c Representing the trajectory The number of points in Representing the trajectory The pose of the j-th point in the array. Let j be the coordinates of the j-th point. This indicates the orientation of the j-th point.

[0486] For any predicted trajectory Extract the segment from time t to time t+Δt. Δt is greater than 0. Calculate the position curve of any three-dimensional light source. The shortest distance d between ij The position curve of the three-dimensional light source is calculated using this method. The shortest distance between the predicted trajectory and all predicted trajectories is obtained. The distance matrix, that is, the distance matrix of the i-th light source. The distance matrix of all three-dimensional light source position curves can be obtained in this way. Where, d i1 express The shortest distance d between the segment and the first predicted trajectory i2 express The shortest distance between the segment and the second predicted trajectory. express With the Nth l The shortest distance between segments of the predicted trajectory. D1 represents the distance matrix of the first light source, and D2 represents the distance matrix of the second light source. Indicates the Nth p Distance matrix of each light source.

[0487] Set the maximum matching tolerance distance d max Ignore d ij >d max (dij The optimal light source vehicle matching pair is obtained by using the Hungarian matching algorithm to match the relationship between ∈D).

[0488] Light sources that match the predicted trajectory of other vehicles are called dynamic light sources. The set of dynamic light sources can be represented as... Where, N m Indicates the number of dynamic light sources. Let represent the i-th dynamic light source. Light sources that do not match the predicted trajectory of any other vehicle are considered static light sources. The set of static light sources can be represented as follows: Where, N n Indicates the number of static light sources. This represents the i-th static light source.

[0489] 3) Construction of three-dimensional light cones.

[0490] That is, to construct light source models for each light source.

[0491] For any dynamic light source Based on the predicted trajectory of other vehicles matched with it Calculate the coordinates of the dynamic light source in three-dimensional space. and orientation And based on the coordinates of the dynamic light source in three-dimensional space Orientation And the light spot projection of the dynamic light source onto the pixel plane, estimating the divergence angle of the dynamic light source. The light source model of this dynamic light source, thus constructed, can be represented as follows: Accordingly, the set of light source models for dynamic light sources can be represented as

[0492] For any static light source Estimate the depth of the static light source based on the two-dimensional matching results between frames. According to depth The divergence angle of the static light source is estimated by comparing the area of ​​the light spot projection on the pixel plane. The light source model of this static light source, thus constructed, can be represented as follows: Correspondingly, the set of light source models for static light sources can be represented as follows:

[0493] (vi) Light cone prediction (i.e., light cone curve prediction).

[0494] That is, predicting the future light cone curve of each light source based on the light source model and the predicted trajectory of the vehicle.

[0495] Suppose we want to predict the light cone curve after a time interval Δt. For example, if the current time is t, we want to predict the light cone curve at time t+Δt. For instance, Δt can be an estimate of system delay and motion delay. This can compensate for errors caused by system delay and motion delay.

[0496] Assume the predicted trajectory of the vehicle obtained in step (vi) is represented as follows: N S Represents trajectory L S The number of points in Represents trajectory L S The pose of the j-th point in the vector. The predicted pose of the vehicle at time t+Δt can be expressed as: Let be the predicted coordinates of the vehicle at time t+Δt. This represents the predicted orientation of the vehicle at time t+Δt. It is based on the set of light source models F for dynamic light sources. m The set F of light source models with static light sources n This allows determining the position and orientation of any light source at time t+Δt. The position and orientation of the light source at time t+Δt are then used to determine the vehicle's predicted pose. By projecting onto the local vehicle coordinate system, the predicted light cone curve F = {(P} can be obtained. i ,Θ i )|(P i ,Θ i )∈(F m ∪F n )}.

[0497] (vii) Parallax registration.

[0498] In step (seven), the predicted light cone curve can be parallax registered based on the human eye position. This human eye position can be the position of the driver's eyes, or the position of the eyes of other users in the vehicle besides the driver. Accordingly, the solution of this application can be used to achieve glare protection for the driver, or it can be used to achieve glare protection for other users.

[0499] Assuming the light-shielding device is a display device, the intrinsic parameter matrix of the display device is represented by K. d The extrinsic parameter matrix of the human eye relative to the vehicle body coordinate system is represented as M. d Combining the predicted light cone curve F obtained in step (vi), for any light source (or any predicted light cone curve) (P) i ,Θ i )∈F, P i Projected onto the human eye coordinate system to obtain The boundary of the light spot projection formed by this light source on the imaging plane of the display device is defined by... For vertices, Θ iLet T(u) be the intersection of the quadratic cone with the divergence angle and the imaging plane. The equation of this intersection is denoted as T(u), where u ∈ [0, 2π]. Therefore, the minimum boundary of the light-blocking region on the display device can be expressed as B. min =K d T(u).

[0500] (viii) Generation of masking information.

[0501] For example, the minimum boundary B obtained in step (vii) min Expansion is performed to obtain the expanded boundary.

[0502] Assume the expansion distance is represented as d B d B >0. Based on the expansion distance d B For the minimum boundary B min Perform dilation to obtain the dilated boundary B. exp For example, the expanded boundary B exp The boundary of the light-shielding area is output to the display device. For example, the minimum bounding curve B of this expanded boundary... ext The boundary of the light-shielding area is output to the display device. Minimum circumscribed curve B ext It can use any closed curve model, such as a rectangle or an ellipse.

[0503] Furthermore, a mapping relationship S(l) between brightness and transmittance of the display device can be constructed. The target transmittance of the display device is determined based on the target brightness and this mapping relationship S(l). For example, the average brightness value l within the light source area obtained in step (iii) is taken as the target brightness, and the average brightness value is determined based on the mapping relationship S(l). Corresponding light transmittance That is, the target transmittance, which is output to the display device.

[0504] Furthermore, based on the vehicle's predicted trajectory and pose... Similarly, based on the instantaneous rate of change of the predicted light cone curve F, information such as the velocity of the center of the shading area and the acceleration of the center of the shading area can be calculated, and this information can be output to the display device.

[0505] The solution shown in Figure 10 is based on 3D trajectory prediction. In other implementations, the solution of this application embodiment can also be implemented based on 2D trajectory prediction. For example, the future trajectory of the light source in 2D space can be predicted based on image data, vehicle motion data, and map data, thereby determining the shading area.

[0506] Figure 11 illustrates a specific example of a glare protection process according to an embodiment of this application. As shown in Figure 11, the scheme can be divided into two parts: a glare protection process based on 2D light source tracking and a glare protection process based on 3D light field construction. Since light field construction also involves a time dimension, 3D light field construction can also be replaced by 4D light field construction.

[0507] As shown in Figure 11, when the LiDAR point cloud input is invalid or the vehicle is not equipped with LiDAR, the glare protection process based on 2D light source tracking can be performed by relying on image data, vehicle motion data and map data (such as SD map). For example, operations such as image enhancement, detection, tracking and prediction of light source and vehicle, 2D light cone prediction, parallax registration, etc., are performed to output occlusion information to the shading device.

[0508] When LiDAR point cloud data is available as input, a glare protection process based on 3D light field construction can be executed by relying on image data, point cloud data, vehicle motion data, and map data. For example, point cloud data can be used for 3D vehicle detection, and tracking and prediction in 3D space can be performed by matching the relationship between 3D and 2D, thereby improving the calculation accuracy of the shading area.

[0509] As shown in Figure 11, in the glare protection process based on 2D light source tracking, image enhancement is performed on the front view image, followed by 2D vehicle detection and 2D light source detection. Based on the detection results, 2D-2D light source-vehicle modal matching (as shown in Figure 11, 2D-2D light-vehicle matching) is performed, matching the light source and vehicle based on their 2D positions. 2D target tracking is then performed based on the light source detection results, vehicle detection results, and matching results. For example, 2D targets may include successfully matched light source-vehicle pairs and unmatched light sources. The 2D target tracking results can be used to determine 2D historical trajectories (such as the historical trajectories of light sources and / or vehicles). The map data undergoes dimensionality transformation, converting 3D map data into 2D map data. 2D trajectory prediction is performed based on the 2D historical trajectories and 2D map data to obtain the predicted 2D trajectory. This predicted 2D trajectory can be used for 2D spot prediction, i.e., predicting the shape and trajectory of the spot area formed by the light source on the camera's imaging plane in the future. Trajectory prediction is performed based on vehicle motion data and map data to obtain the predicted trajectory of the vehicle. Light cone prediction is then performed based on the 2D light spot prediction result and the predicted trajectory of the vehicle, i.e., estimating the light spot position relative to the predicted pose of the vehicle. This light cone prediction can be considered as a 2D light cone prediction. Parallax registration is performed on the light cone prediction result to obtain future information about the light source on the imaging plane of the shading device (the windshield in Figure 11), such as the future trajectory and shape of the light source on the windshield, and occlusion information is generated based on this. Furthermore, as shown in Figure 11, the future trajectory of the vehicle on the windshield can also be obtained, and this information can also be used to generate occlusion information.

[0510] As shown in Figure 11, in the glare protection process based on 3D light field construction, 3D vehicle detection is performed based on point cloud data to obtain the 3D position of the vehicle. 2D-3D light source-vehicle modal matching (as shown in Figure 11) is performed based on the 3D vehicle detection results and 2D light source detection results, i.e., matching the light source and vehicle based on the 2D position of the light source and the 3D position of the vehicle. 3D target tracking is performed based on the 3D vehicle detection results, and the 3D target tracking results can be used to determine the vehicle's 3D historical trajectory. 3D trajectory prediction is performed based on the 3D historical trajectory and 3D map data to obtain the vehicle's predicted 3D trajectory, i.e., the vehicle's future trajectory in 3D space. The predicted 3D trajectory of the light source, i.e., the future trajectory of the light source in 3D space, can be determined based on the vehicle's predicted 3D trajectory and / or the light source's predicted 3D trajectory and the vehicle's predicted trajectory. Light cone prediction (i.e., 3D light cone prediction) is performed based on the vehicle's predicted 3D trajectory and / or the light source's predicted 3D trajectory and the vehicle's predicted trajectory. The light cone prediction results are parallax registered to obtain future information about the light source on the imaging plane of the shading device (the windshield in Figure 11), such as the future trajectory and shape of the light source on the windshield, and shading information is generated based on this. The glare protection process based on 3D light field construction can refer to method 1000. In addition, as shown in Figure 11, the future trajectory of the vehicle on the windshield can also be obtained.

[0511] In addition, the vehicle detection in Figure 11 can be replaced by vehicle and obstacle detection.

[0512] Figure 12 shows a schematic diagram of a glare protection method according to an embodiment of this application. The method 1200 shown in Figure 12 can be implemented based on the method 300 or method 1000 described above. To avoid repetition, some descriptions are omitted appropriately when describing method 1200.

[0513] As shown in Figure 12, method 1200 may include the following steps.

[0514] 1210, predicting the position of at least one light source at a second time point based on the position of at least one light source in the environment where the first vehicle is located at a first time point and vehicle information in the environment where the first vehicle is located, the second time point being later than the first time point. The vehicle information includes vehicle motion data of at least one second vehicle, or the vehicle information is used to indicate that no second vehicle exists in the environment where the first vehicle is located.

[0515] 1220, Based on the predicted trajectory of at least one target vehicle and the position of at least one light source at a second time moment, determine the light spot area formed by at least one light source on the imaging plane of a light-shielding device at a second time moment, wherein the light-shielding device is disposed on a first vehicle. The at least one target vehicle includes the first vehicle.

[0516] 1230, output shading information to the shading device. The shading information is used to indicate the shading area, which covers the light spot area.

[0517] For example, "first vehicle" can be "self-vehicle" in method 300 or method 1000, and the environment in which the first vehicle is located can be the environment surrounding the self-vehicle. The relevant description of the first vehicle can refer to the description of the self-vehicle in method 300 or method 1000.

[0518] Optionally, method 1200 may further include: receiving an image frame of the environment in which the first vehicle is located; performing light source detection based on the image frame to obtain a light source detection result, the light source detection result being used to determine the position of at least one light source at a first moment.

[0519] The position of at least one light source at the first moment can be either the position of the at least one light source in the image frame at the first moment, or the position of the at least one light source in three-dimensional space at the first moment.

[0520] For example, the light source detection results can indicate the light sources in an image frame and their positions within the frame. The position of a light source in the image frame can be represented by its pixel coordinates. For instance, the pixel coordinates of each light source in the light source detection results can be used as the position of the at least one light source at a first moment. Alternatively, the pixel coordinates of each light source can be converted to world coordinates in three-dimensional space, i.e., three-dimensional coordinates, and the three-dimensional coordinates of each light source can be used as the position of the at least one light source at a first moment.

[0521] Alternatively, the image frame may be obtained by image enhancement of data acquired by a vision sensor.

[0522] The terms "first moment" and "second moment" are used only to distinguish different moments and have no limiting effect; the second moment simply needs to be later than the first moment. The first moment can also be replaced with a historical moment, where the position at the first moment is the historical position. Similarly, the second moment can be replaced with a future moment, where the position at the second moment is the future moment. The position of a light source determined by light source detection based on an image frame can be considered the position of the light source at the first moment. For example, the image frame could be an image frame of the environment in which the first vehicle is located at time t, where time t can be considered the first moment.

[0523] The time point can also be replaced with other time description methods such as time steps.

[0524] For example, the image frame used for light source detection can be the image data in method 300 or method 1000. A description of the image frame can be found in the description of "image data" in method 300 or method 1000, and a description of light source detection can be found in step 320 of method 300 or step (iii) of method 1000. For instance, the image frame can be image #1 in method 300 or the input image in method 1000.

[0525] For example, "second vehicle" can be "other vehicle" in method 300 or method 1000, and vehicle information can be information about other vehicles. The relevant description of the second vehicle can refer to the description of other vehicles in method 300 or method 1000.

[0526] Optionally, the vehicle motion data of at least one second vehicle may include the position of the at least one second vehicle.

[0527] For example, the vehicle motion data of the second vehicle may also include other data. The description of the vehicle motion data of the second vehicle may refer to the description of the vehicle motion data of another vehicle in method 300 or method 1000.

[0528] Optionally, method 1200 may further include: receiving an image frame of the environment in which the first vehicle is located; performing vehicle detection based on the image frame to obtain a second vehicle detection result, the second vehicle detection result being used to determine vehicle information.

[0529] For example, the image frame used for vehicle detection and the image frame used for light source detection can be the same image.

[0530] The position of the at least one second vehicle can be the position of the at least one second vehicle in the image frame, or the position of the at least one second vehicle in three-dimensional space.

[0531] For example, the second vehicle can be some or all of the vehicles indicated by the second vehicle detection result. The second vehicle detection result can indicate the vehicles (i.e., the second vehicles) in the image frame and their positions in the image frame. The position of the vehicle in the image frame can be represented by the vehicle's pixel coordinates. For example, the pixel coordinates of some or all of the vehicles in the second vehicle detection result can be used as the position of the at least one second vehicle. Alternatively, the pixel coordinates of each vehicle can be converted to world coordinates in three-dimensional space, i.e., three-dimensional coordinates, and the three-dimensional coordinates of some or all of the vehicles can be used as the position of the at least one second vehicle.

[0532] For example, the second vehicle detection result may indicate that no vehicle was detected, and correspondingly, vehicle information may be used to indicate that the second vehicle does not exist in the environment where the first vehicle is located.

[0533] Alternatively, the image frame may be obtained by image enhancement of data acquired by a vision sensor.

[0534] For example, the image frame used for vehicle detection can be the image data in method 300 or method 1000. A description of the image frame can be found in the description of "image data" in method 300 or method 1000, and a description of vehicle detection can be found in step 331 of method 300 or step (iii) of method 1000. For example, the image frame can be image #2 in method 300 or the input image in method 1000.

[0535] Optionally, method 1200 may further include: receiving point cloud data of the environment in which the first vehicle is located; performing vehicle detection based on the point cloud data to obtain a first vehicle detection result, the first vehicle detection result being used to determine vehicle information.

[0536] The terms "first vehicle detection result" and "second vehicle detection result" are used only to distinguish between different detection results and have no other limiting function. The vehicle detection result determined based on the image frame is the second vehicle detection result, and the vehicle detection result determined based on the point cloud data is the first vehicle detection result.

[0537] For example, the second vehicle may be some or all of the vehicles indicated by the first vehicle detection result. The positions of some or all of the vehicles in the first vehicle detection result may be used as the positions of the at least one second vehicle.

[0538] For example, the first vehicle detection result may indicate that no vehicle was detected, and correspondingly, vehicle information may indicate that there is no second vehicle in the environment where the first vehicle is located.

[0539] For example, the point cloud data can be the point cloud data in method 300 or method 1000. For a description of the point cloud data, please refer to the description of "point cloud data" in method 300 or method 1000. For a description of vehicle detection, please refer to the description of step 331 in method 300 or step (iii) in method 1000.

[0540] Step 1210 can also be understood as predicting the future position of at least one light source based on its current (or historical) position and vehicle information, or determining its future position based on its historical position.

[0541] For example, the time interval between the first moment and the second moment can be determined based on relevant parameters of system delay and / or motion delay.

[0542] Optionally, if the vehicle information indicates that there is no second vehicle in the environment where the first vehicle is located, the position of the at least one light source at the second moment can be the position of the at least one light source at the first moment.

[0543] In this case, the at least one light source can be regarded as a static light source in method 300 or method 1000, and the relevant description of the at least one light source can be referred to the description of a static light source.

[0544] Optionally, if the vehicle information includes vehicle motion data of at least one second vehicle, the at least one target vehicle may include the at least one second vehicle.

[0545] Step 1210 may include: predicting the position of at least one light source at a second time based on the position of at least one light source at a first time and the predicted trajectory of at least one second vehicle.

[0546] Further, optionally, step 1210 may include:

[0547] The matching relationship between at least one light source and at least one second vehicle is determined based on the position of at least one light source at a first moment and the vehicle motion data of at least one second vehicle;

[0548] The position of the first light source in at least one light source at a second time moment is determined based on the predicted trajectory of the first target vehicle in at least one second vehicle, and there is a matching relationship between the first target vehicle and the first light source.

[0549] There is a matching relationship between the first target vehicle and the first light source, meaning the first light source is positioned on the first target vehicle. The positional relationship between the first light source and the first target vehicle is relatively fixed. The pose of the first target vehicle at a second time moment can be determined based on its predicted trajectory, and thus the pose of the first light source at that second time moment can be determined. Alternatively, the predicted trajectory of the first light source can be determined based on the predicted trajectory of the first target vehicle, and thus the pose of the first light source at the second time moment can be determined.

[0550] For example, the pose of the first light source at the second moment may include the position of the first light source at the second moment. Further, the pose of the first light source at the second moment may also include the orientation of the first light source at the second moment, etc.

[0551] Optionally, the position of the second light source in the at least one light source at the second time is the same as the position of the second light source at the first time, and there is no matching relationship between the second light source and the at least one second vehicle.

[0552] The terms "first light source," "second light source," and "first target vehicle" are used for descriptive convenience only and are not intended to be limiting. A light source that has a matching relationship with at least one second vehicle is a first light source, and a vehicle that has a matching relationship with a light source is a first target vehicle. A light source that has no matching relationship with any vehicle is a second light source.

[0553] For example, the second light source can be regarded as a static light source in method 300 or method 1000, and the relevant description of the second light source can refer to the description of the static light source. The first light source can be regarded as a dynamic light source in method 300 or method 1000, and the relevant description of the first light source can refer to the description of the dynamic light source.

[0554] For example, the at least one second vehicle can be understood as all vehicles indicated by the vehicle detection results (first vehicle detection results and / or second vehicle detection results), or the at least one second vehicle can also be understood as a vehicle that has a matching relationship with the at least one light source.

[0555] Optionally, determining the matching relationship between at least one light source and at least one second vehicle based on the position of at least one light source at a first moment and the vehicle motion data of at least one second vehicle may include:

[0556] The matching relationship between at least one light source and at least one second vehicle is determined based on the position of the at least one light source at a first moment and the position of the at least one second vehicle.

[0557] Optionally, determining the matching relationship between at least one light source and at least one second vehicle based on the position of at least one light source at a first moment and the vehicle motion data of at least one second vehicle may include:

[0558] The matching relationship between the at least one light source and the at least one second vehicle is determined based on the position of the at least one light source at a first moment and the predicted trajectory of the at least one second vehicle. The predicted trajectory of the at least one second vehicle is determined based on the vehicle motion data of the at least one second vehicle.

[0559] For details on how to implement step 1210, please refer to the relevant descriptions of step 341 in method 300 or step (v) in method 1000.

[0560] Optionally, method 1200 may further include: acquiring the historical trajectory of at least one target vehicle; acquiring map data; and generating a predicted trajectory of at least one target vehicle based on the map data and the historical trajectory of at least one target vehicle.

[0561] For a detailed description of the historical trajectory, map data, and predicted trajectory, please refer to the description in step 330.

[0562] Optionally, method 1200 may further include: obtaining lane information of drivable lanes in the environment where the first vehicle is located; and generating a predicted trajectory of at least one target vehicle based on map data, the historical trajectory of at least one target vehicle, and the lane information of the drivable lanes.

[0563] For example, lane information for a drivable lane can include the centerline of the drivable lane.

[0564] For example, drivable lanes can be perceived based on SD maps or HD maps.

[0565] Optionally, generating a predicted trajectory for at least one target vehicle based on map data, the historical trajectory of at least one target vehicle, and lane information of the drivable lane may include: generating at least one anchor trajectory based on the vehicle motion data of the at least one target vehicle and the lane information of the drivable lane; and generating a predicted trajectory for at least one target vehicle based on map data, the historical trajectory of at least one target vehicle, and the at least one anchor trajectory.

[0566] Optionally, the predicted trajectory of the at least one target vehicle is the predicted trajectory of the at least one target vehicle in three-dimensional space.

[0567] The specific implementation of determining the predicted trajectory of the target vehicle can refer to the description of step 330 in method 300 or step (iv) in method 1000, where the predicted trajectory of the vehicle is replaced with the predicted trajectory of at least one target vehicle.

[0568] In step 1220, the area of ​​light spot formed by the at least one light source on the imaging plane of the light-shielding device at the second time can be determined based on the predicted trajectory of the first vehicle and the position of the at least one light source at the second time.

[0569] For a detailed description of the shading device, please refer to Method 300 or Method 1000.

[0570] Optionally, method 1200 may further include: obtaining the user's eye position. Step 1220 may include: determining, based on the eye position, the predicted trajectory of at least one target vehicle, and the position of at least one light source at the second time, the area of ​​light spot formed by at least one light source on the imaging plane of the light-shielding device at the second time.

[0571] The location of the light spot is related to the location of the human eye. Correspondingly, the location of the shading area is related to the location of the human eye.

[0572] Optionally, step 1220 may include steps 1221 and 1222 (not shown in the figure).

[0573] 1221. The light cone curve of the at least one light source at the second moment is determined based on the predicted trajectory of the first vehicle and the position of the at least one light source at the second moment.

[0574] The light cone curve of the at least one light source at the second moment indicates the position, orientation, and divergence angle of the at least one light source in the vehicle's coordinate system at the second moment. The vehicle's coordinate system is determined based on the predicted trajectory of the first vehicle.

[0575] 1222, The light spot area formed by the at least one light source on the imaging plane of the light-shielding device at the second time is determined based on the light cone curve of the at least one light source at the second time and the position of the human eye.

[0576] The position and orientation of the at least one light source at the second moment can be the position and orientation of the at least one light source in the world coordinate system. The pose (i.e., predicted pose) of the first vehicle at the second moment can be determined based on the predicted trajectory of the first vehicle. By projecting the position and orientation of the at least one light source at the second moment onto the vehicle coordinate system at the second moment based on the pose of the first vehicle at the second moment, the position and orientation of the at least one light source in the vehicle coordinate system at the second moment can be obtained.

[0577] The specific description of step 1221 can be found in step 342 of method 300 or step (vi) of method 1000.

[0578] Optionally, step 1222 may include: performing parallax registration on the light cone curve of the at least one light source at a second time according to the position of the human eye, so as to determine the light spot area formed by the at least one light source on the imaging plane of the light-shielding device at the second time.

[0579] For a detailed description of step 1222, please refer to step 350 in method 300 or step (vii) in method 1000.

[0580] The shading area is determined based on the light spot area. For example, the shading area may cover the light spot area, that is, the range of the shading area includes the range of the light spot area.

[0581] Optionally, the shading information may also indicate at least one of the following: the light transmittance of the shading area, the center velocity of the shading area, or the acceleration of the center of the shading area.

[0582] Optionally, the light transmittance of the shaded area is related to the light intensity of at least one light source.

[0583] Optionally, when the light intensity of at least one light source is a first light intensity, the light transmittance of the shaded area is a first light transmittance; when the light intensity of at least one light source is a second light intensity, the light transmittance of the shaded area is a second light transmittance. The first light intensity is greater than the second light intensity, and the first light transmittance is less than the second light transmittance.

[0584] In other words, the higher the light intensity of at least one light source, the lower the light transmittance of the shaded area.

[0585] For example, the illumination intensity of at least one light source can be determined by an image frame. For instance, the average brightness of the area within which at least one light source is located in the image frame (such as within the bounding box of the light source) can be used as the illumination intensity of at least one light source.

[0586] For instructions on setting the light transmittance of the shaded area, please refer to the relevant descriptions in Method 300 or Method 1000.

[0587] Optionally, the area of ​​the shaded area is related to the distance between the first vehicle and the at least one second vehicle.

[0588] Optionally, when the distance between the first vehicle and at least one second vehicle is a first distance, the ratio between the area of ​​the shading region and the area of ​​the light spot region is a first ratio; when the distance between the first vehicle and at least one second vehicle is a second distance, the ratio between the area of ​​the shading region and the area of ​​the light spot region is a second ratio, wherein the first distance is greater than the second distance and the first ratio is greater than the second ratio.

[0589] In other words, the closer the first vehicle is to the at least one second vehicle, the smaller the ratio between the area of ​​the shading region and the area of ​​the light spot region, meaning the shading region is more closely aligned with the light spot region.

[0590] When there are multiple second vehicles, the distance between the first vehicle and the at least one second vehicle can be a statistical value of the distances between the second vehicle and all the second vehicles. For example, the statistical value can include any of the following: maximum value, minimum value, or average value, etc.

[0591] For example, the first distance can be distance #2 in method 300, the second distance can be distance #1 in method 300, the first ratio can be ratio #4 in method 300, and the second ratio can be ratio #3 in method 300.

[0592] Optionally, the area of ​​the shaded area is related to the method of determining vehicle information.

[0593] Optionally, when the vehicle information is determined based on the first vehicle detection result, the ratio between the area of ​​the shading area and the area of ​​the light spot area is the third ratio; when the vehicle information is unrelated to the first vehicle detection result, the ratio between the area of ​​the shading area and the area of ​​the light spot area is the fourth ratio, and the fourth ratio is greater than the third ratio.

[0594] In other words, the smaller the ratio between the area of ​​the shading region and the area of ​​the light spot region when the vehicle motion data of the second vehicle is determined based on point cloud data, the closer the shading region is to the light spot region.

[0595] For example, the third ratio can be ratio #1 in method 300, and the fourth ratio can be ratio #2 in method 300.

[0596] For a detailed description of step 1230, please refer to step 360 in method 300 or step (viii) in method 1000.

[0597] The apparatus of the present application embodiment will now be described with reference to Figures 13 to 15. It should be understood that the apparatus described below is capable of performing the methods of the foregoing embodiments of the present application. To avoid unnecessary repetition, repeated descriptions will be appropriately omitted when introducing the apparatus of the present application embodiment.

[0598] The system architecture of the embodiments of this application will be described below.

[0599] Figure 13 illustrates a system architecture 200 provided in an embodiment of this application. In Figure 13, a data acquisition device 260 is used to acquire training data. For example, for a neural network model for target detection in image data in an embodiment of this application, the training data may include sample images and corresponding ground truth values. The ground truth values ​​can be used to indicate the category and geometric information of the target in the sample images. Similarly, for a neural network model for target detection in point cloud data in an embodiment of this application, the training data may include sample point cloud data and corresponding ground truth values. The ground truth values ​​can be used to indicate the category and geometric information of the target in the sample point cloud data.

[0600] After collecting the training data, the data acquisition device 260 stores the training data in the database 230, and the training device 220 trains the target model / rule 201 based on the training data maintained in the database 230.

[0601] The following describes how the training device 220 obtains the target model / rule 201 based on the training data. The training device 220 processes the input raw data and compares the output value with the target value until the difference between the output value of the training device 220 and the target value is less than a certain threshold, thereby completing the training of the target model / rule 201.

[0602] The aforementioned target model / rule 201 can be used to implement the method of the embodiments of this application. Specifically, the target model / rule 201 in the embodiments of this application can be a neural network model. It should be noted that in practical applications, the training data maintained in the database 230 may not all come from the data acquisition device 260; it may also be received from other devices. Furthermore, it should be noted that the training device 220 may not necessarily train the target model / rule 201 entirely based on the training data maintained in the database 230; it may also obtain training data from the cloud or other sources for model training. The above description should not be construed as limiting the embodiments of this application.

[0603] The target model / rule 201 trained by training device 220 can be applied to different systems or devices, such as the execution device 210 shown in Figure 13. The execution device 210 can be a terminal, such as an in-vehicle terminal, or a server or cloud. In Figure 13, the execution device 210 is configured with an input / output (I / O) interface 212 for data interaction with external devices. The client device 240 can input data into the I / O interface 212. The input data in this embodiment may include image data and / or point cloud data, etc.

[0604] During the preprocessing of input data by the execution device 210, for example, by preprocessing through the preprocessing module 213 and / or the preprocessing module 214 (such as image enhancement), or during the calculation and other related processing performed by the calculation module 211 of the execution device 210, the execution device 210 may call data, code, etc. in the data storage system 250 for corresponding processing, or store the data, instructions, etc. obtained from the corresponding processing into the data storage system 250.

[0605] Finally, I / O interface 212 returns the processing result, such as the data processing result obtained above, to client device 240.

[0606] It is worth noting that the training device 220 can generate corresponding target models / rules 201 based on different training data for different objectives or tasks. The corresponding target models / rules 201 can be used to achieve the above objectives or complete the above tasks, thereby providing the user with the required results.

[0607] The client device 240 can also function as a data acquisition terminal, collecting the input data and output results of the input I / O interface 212 as shown in the figure, and storing them in the database 230. Alternatively, data can be collected directly from the I / O interface 212 as new sample data, without going through the client device 240.

[0608] It is worth noting that Figure 13 is only a schematic diagram of a system architecture provided by an embodiment of this application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 13, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0609] Figure 14 shows a schematic block diagram of an apparatus provided in an embodiment of this application. The apparatus 1800 shown in Figure 14 can be used to perform the methods of the embodiments of this application, such as the methods shown in Figure 3, Figure 10, or Figure 12.

[0610] As shown in Figure 14, the device 1800 may include a processing module 1810 and an output module 1820.

[0611] As one possible implementation, the device 1800 can be used to perform the method shown in FIG12.

[0612] Processing module 1810 is used for:

[0613] The position of at least one light source is predicted at a second time based on the position of at least one light source in the environment where the first vehicle is located at a first time and vehicle information in the environment where the first vehicle is located. The second time is later than the first time. The vehicle information includes vehicle motion data of at least one second vehicle, or the vehicle information is used to indicate that there is no second vehicle in the environment where the first vehicle is located.

[0614] Based on the predicted trajectory of at least one target vehicle and the position of at least one light source at a second time, a light spot area formed by at least one light source on the imaging plane of the shading device at the second time is determined. The shading device is disposed on the first vehicle, and the at least one target vehicle includes the first vehicle.

[0615] The output module 1820 is used to output shading information to the shading device. The shading information is used to indicate the shading area, which covers the light spot area.

[0616] Optionally, the processing module 1810 is also used for:

[0617] Obtain the historical trajectory of at least one target vehicle;

[0618] Used to acquire map data;

[0619] Generate a predicted trajectory for at least one target vehicle based on map data and the historical trajectory of at least one target vehicle.

[0620] Optionally, the processing module 1810 is also used for:

[0621] Obtain lane information of the drivable lanes in the environment where the first vehicle is located;

[0622] Generate a predicted trajectory for at least one target vehicle based on map data, the historical trajectory of at least one target vehicle, and lane information of the drivable lanes.

[0623] Optionally, when the vehicle information includes vehicle motion data of at least one second vehicle, the at least one target vehicle includes at least one second vehicle, and the processing module 1810 is specifically configured to: predict the position of at least one light source at a second time based on the position of at least one light source at a first time and the predicted trajectory of at least one second vehicle; or,

[0624] When the vehicle information is used to indicate that there is no second vehicle in the environment where the first vehicle is located, the position of at least one light source at the second moment is the same as the position of at least one light source at the first moment.

[0625] Optionally, the processing module 1810 is specifically used for:

[0626] The matching relationship between at least one light source and at least one second vehicle is determined based on the position of at least one light source at a first moment and the vehicle motion data of at least one second vehicle;

[0627] The position of the first light source among at least one light source is determined at a second time based on the predicted trajectory of the first target vehicle among at least one second vehicle, wherein there is a matching relationship between the first target vehicle and the first light source; and / or,

[0628] The position of the second light source in at least one light source at the second time moment is the same as the position of the second light source at the first time moment, and there is no matching relationship between the second light source and at least one second vehicle.

[0629] Optionally, the predicted trajectory of at least one target vehicle is the predicted trajectory of at least one target vehicle in three-dimensional space.

[0630] Optionally, the light transmittance of the shaded area is related to the light intensity of at least one light source.

[0631] Optionally, the device 1800 also includes:

[0632] The first receiving module is used to receive point cloud data of the environment in which the first vehicle is located; and the processing module 1810 is specifically used to perform vehicle detection based on the point cloud data to obtain the first vehicle detection result, the first vehicle detection result being used to determine vehicle information; and / or,

[0633] Device 1800 also includes:

[0634] The second receiving module is used to receive image frames of the environment where the first vehicle is located; and the processing module 1810 is specifically used for:

[0635] Vehicle detection is performed based on image frames to obtain a second vehicle detection result, which is used to determine vehicle information.

[0636] Optionally, the processing module 1810 is also used for:

[0637] Obtain the user's eye position;

[0638] Based on the position of the human eye, the predicted trajectory of at least one target vehicle, and the position of at least one light source at the second moment, determine the light spot area formed by at least one light source on the imaging plane of the light-shielding device at the second moment.

[0639] Optionally, the shading information is also used to indicate at least one of the following: the light transmittance of the shading area, the velocity of the center of the shading area, or the acceleration of the center of the shading area.

[0640] For a detailed description, please refer to Method 1200 above; it will not be repeated here.

[0641] Each module in device 1800 can be implemented in software or in hardware.

[0642] It should be noted that the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. In other embodiments, the processing module 1810 can be used to execute any step in the method described above, and other modules can be used to implement any step described above. The steps that each module is responsible for implementing can be specified as needed. By having each module implement different steps described above, all functions of the device 1800 can be achieved.

[0643] Figure 15 is a schematic block diagram of the apparatus provided in an embodiment of this application. The apparatus 1900 may include a processor 1910, a transceiver 1920, and a memory 1930. The processor 1910, transceiver 1920, and memory 1930 are connected via internal interconnection paths. The memory 1930 is used to store instructions, and the processor 1910 is used to execute the instructions stored in the memory 1930 to receive / send data via the transceiver 1920. Optionally, the memory 1930 may be coupled to the processor 1910 via an interface or integrated with the processor 1910.

[0644] It should be noted that the transceiver 1920 mentioned above may include, but is not limited to, transceiver devices such as input / output interfaces, to enable communication between device 1900 and other devices or communication networks.

[0645] The memory 1930 can be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM).

[0646] In one implementation, the processor 1910 can be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), microprocessor, or digital signal processor (DSP). In another implementation, the processor 1910 can implement certain functions through the logical relationships of hardware circuits. These logical relationships can be fixed or reconfigurable. For example, the processor 1910 can be a hardware circuit implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA). In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement some or all of the functions of the aforementioned units.

[0647] This application also provides an electronic device, which may include the above-described device 1800 or device 1900.

[0648] This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0649] This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.

[0650] This application also provides a chip, including circuitry, for performing the methods described in the above embodiments.

[0651] In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software. The method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.

[0652] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0653] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0654] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0655] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0656] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0657] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0658] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be covered.

Claims

1. A method for glare protection, characterized in that, include: The position of at least one light source in the environment where the first vehicle is located is predicted at a second time based on the position of at least one light source in the environment at a first time and vehicle information in the environment where the first vehicle is located. The second time is later than the first time. The vehicle information includes vehicle motion data of at least one second vehicle, or the vehicle information is used to indicate that there is no second vehicle in the environment where the first vehicle is located. The light spot area formed by the at least one light source on the imaging plane of the shading device at the second time moment is determined based on the predicted trajectory of at least one target vehicle and the position of the at least one light source at the second time moment. The shading device is disposed on the first vehicle, and the at least one target vehicle includes the first vehicle. The shading device outputs shading information, which indicates the shading area, and the shading area covers the light spot area.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the historical trajectory of the at least one target vehicle; Obtain map data; The predicted trajectory of the at least one target vehicle is generated based on the map data and the historical trajectory of the at least one target vehicle.

3. The method according to claim 2, characterized in that, The method further includes: Acquire lane information of drivable lanes in the environment where the first vehicle is located; and generate a predicted trajectory for the at least one target vehicle based on the map data and the historical trajectory of the at least one target vehicle, including: The predicted trajectory of the at least one target vehicle is generated based on the map data, the historical trajectory of the at least one target vehicle, and the lane information of the drivable lane.

4. The method according to any one of claims 1 to 3, characterized in that, When the vehicle information includes vehicle motion data of at least one second vehicle, the at least one target vehicle includes the at least one second vehicle, and the prediction of the position of the at least one light source at a second time based on the position of the at least one light source at a first time and the vehicle information includes: The position of the at least one light source at the second time is predicted based on the position of the at least one light source at the first time moment and the predicted trajectory of the at least one second vehicle; or, When the vehicle information indicates that there is no second vehicle in the environment where the first vehicle is located, the position of the at least one light source at the second time is the same as the position of the at least one light source at the first time.

5. The method according to claim 4, characterized in that, The step of predicting the position of the at least one light source at a second time based on the position of the at least one light source at a first time and the predicted trajectory of the at least one second vehicle includes: The matching relationship between the at least one light source and the at least one second vehicle is determined based on the position of the at least one light source at a first moment and the vehicle motion data of the at least one second vehicle; The position of the first light source among the at least one light source at the second time moment is determined based on the predicted trajectory of the first target vehicle among the at least one second vehicle, wherein the first target vehicle and the first light source have the matching relationship; and / or, The position of the second light source in the at least one light source at the second time moment is the same as the position of the second light source at the first time moment, and there is no matching relationship between the second light source and the at least one second vehicle.

6. The method according to any one of claims 1 to 5, characterized in that, The predicted trajectory of the at least one target vehicle is the predicted trajectory of the at least one target vehicle in three-dimensional space.

7. The method according to any one of claims 1 to 6, characterized in that, The light transmittance of the light-shielding area is related to the light intensity of the at least one light source.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Receive point cloud data of the environment in which the first vehicle is located; Vehicle detection is performed based on the point cloud data to obtain a first vehicle detection result, which is used to determine the vehicle information; and / or, The method further includes: Receive image frames of the environment in which the first vehicle is located; Vehicle detection is performed based on the image frame to obtain a second vehicle detection result, which is used to determine the vehicle information.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Obtain the user's eye position, and Determining the light spot area formed by the at least one light source on the imaging plane of the shading device at the second time moment based on the predicted trajectory of the at least one target vehicle and the position of the at least one light source at the second time moment includes: The spot area formed by the at least one light source on the imaging plane of the light-shielding device at the second moment is determined based on the position of the human eye, the predicted trajectory of the at least one target vehicle, and the position of the at least one light source at the second moment.

10. The method according to any one of claims 1 to 9, characterized in that, The shading information is also used to indicate at least one of the following: the light transmittance of the shading area, the velocity of the center of the shading area, or the acceleration of the center of the shading area.

11. A glare protection device, characterized in that, include: Processing module, used for: The position of at least one light source in the environment where the first vehicle is located is predicted at a second time based on the position of at least one light source in the environment at a first time and vehicle information in the environment where the first vehicle is located. The second time is later than the first time. The vehicle information includes vehicle motion data of at least one second vehicle, or the vehicle information is used to indicate that there is no second vehicle in the environment where the first vehicle is located. The light spot area formed by the at least one light source on the imaging plane of the shading device at the second time moment is determined based on the predicted trajectory of at least one target vehicle and the position of the at least one light source at the second time moment. The shading device is disposed on the first vehicle, and the at least one target vehicle includes the first vehicle. An output module is used to output shading information to the shading device, the shading information being used to indicate the shading area, the shading area covering the light spot area.

12. The apparatus according to claim 11, characterized in that, The processing module is also used for: Obtain the historical trajectory of the at least one target vehicle; Obtain map data; The predicted trajectory of the at least one target vehicle is generated based on the map data and the historical trajectory of the at least one target vehicle.

13. The apparatus according to claim 12, characterized in that, The processing module is also used for: Obtain lane information of the drivable lanes in the environment where the first vehicle is located; The predicted trajectory of the at least one target vehicle is generated based on the map data, the historical trajectory of the at least one target vehicle, and the lane information of the drivable lane.

14. The apparatus according to any one of claims 11 to 13, characterized in that, When the vehicle information includes vehicle motion data of at least one second vehicle, the at least one target vehicle includes the at least one second vehicle, and the processing module is specifically used for: The position of the at least one light source at the second time is predicted based on the position of the at least one light source at the first time moment and the predicted trajectory of the at least one second vehicle; or, When the vehicle information indicates that there is no second vehicle in the environment where the first vehicle is located, the position of the at least one light source at the second time is the same as the position of the at least one light source at the first time.

15. The apparatus according to claim 14, characterized in that, The processing module is specifically used for: The matching relationship between the at least one light source and the at least one second vehicle is determined based on the position of the at least one light source at a first moment and the vehicle motion data of the at least one second vehicle; The position of the first light source among the at least one light source at the second time moment is determined based on the predicted trajectory of the first target vehicle among the at least one second vehicle, and the matching relationship exists between the first target vehicle and the first light source; And / or, The position of the second light source in the at least one light source at the second time moment is the same as the position of the second light source at the first time moment, and there is no matching relationship between the second light source and the at least one second vehicle.

16. The apparatus according to any one of claims 11 to 15, characterized in that, The predicted trajectory of the at least one target vehicle is the predicted trajectory of the at least one target vehicle in three-dimensional space.

17. The apparatus according to any one of claims 11 to 16, characterized in that, The light transmittance of the light-shielding area is related to the light intensity of the at least one light source.

18. The apparatus according to any one of claims 11 to 17, characterized in that, The device further includes: The first receiving module is configured to receive point cloud data of the environment in which the first vehicle is located; and the processing module is specifically configured to: Vehicle detection is performed based on the point cloud data to obtain a first vehicle detection result, which is used to determine the vehicle information; and / or, The device further includes: The second receiving module is used to receive image frames of the environment in which the first vehicle is located; and the processing module is specifically used for: Vehicle detection is performed based on the image frame to obtain a second vehicle detection result, which is used to determine the vehicle information.

19. The apparatus according to any one of claims 11 to 18, characterized in that, The processing module is also used for: Obtain the user's eye position; The spot area formed by the at least one light source on the imaging plane of the light-shielding device at the second moment is determined based on the position of the human eye, the predicted trajectory of the at least one target vehicle, and the position of the at least one light source at the second moment.

20. The apparatus according to any one of claims 11 to 19, characterized in that, The shading information is also used to indicate at least one of the following: the light transmittance of the shading area, the velocity of the center of the shading area, or the acceleration of the center of the shading area.

21. A computing device, characterized in that, It includes a processor and a memory, the processor being configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1 to 10.

22. A computer program product containing instructions, characterized in that, When the instructions are executed by the computing device, the computing device performs the method as described in any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a computing device, cause the computing device to perform the method as described in any one of claims 1 to 10.

Images