A method, system, device and medium for controlling anti-glare of projection light machine

Abstract

The present application discloses a method, system, device and medium for controlling the anti-glare of a projection optical machine, and relates to the field of intelligent vehicle driving. The method comprises: performing target detection on the image of the vehicle's surrounding environment to determine traffic sign information and anti-glare target information; determining the icon to be projected according to the traffic sign information and the anti-glare target information; if there is no driver's instruction, controlling the projection optical machine to project the icon to be projected onto the ground in front of the vehicle; if there is a driver's instruction, using a pre-trained large language model to parse the driver's instruction and the state of the projection optical machine respectively, obtaining quantitative constraint information and the quantitative projection optical machine state, generating a final projection icon in combination with the icon to be projected, and controlling the projection optical machine to project the final projection icon onto the ground in front of the vehicle. While reducing the cognitive load of the driver, the present application realizes the anti-glare projection of traffic signs, improves driving safety, and ensures the visual safety of other traffic participants.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle driving, and in particular to a projection optical machine anti-glare control method, system, device and medium based on large-model driving instruction analysis. Background Art

[0002] In the daily driving process of drivers, they often ignore traffic signs and violate traffic rules. It is even not uncommon for traffic accidents to occur after the driver is distracted by checking traffic signs. The current methods to solve such problems are mostly to identify traffic signs through cameras, and then use projectors to project their icons onto the ground in front of the car to display the traffic signs. The projection position is exactly at the point where the driver's attention is focused during normal driving, thereby solving the problem of drivers ignoring traffic signs or being distracted by traffic signs.

[0003] However, this method still has defects. The projection light machine used to project traffic signs has high power and brightness. If it directly shines on the retinas of pedestrians or other drivers, it may cause them to feel strong dazzle and even damage their eyesight.

[0004] In addition, vehicles often encounter some complex traffic scenes during nighttime driving, and at this time, it is necessary to screen and prioritize the anti-glare targets. For example, when there are many oncoming vehicles on the right side of the projection light machine's irradiation direction, it is necessary to focus on shielding the right side of the projection light machine's irradiation range, or lower the brightness of the right side of the projection light machine, etc. However, in complex traffic scenes, traditional machine learning and other methods are not capable of identifying and judging them, so the driver needs to provide auxiliary instructions, that is, the driver needs to directly issue instructions to the anti-glare control algorithm, which requires the driver to have relevant professional qualities, and is not applicable at the user level. Summary of the invention

[0005] The purpose of this application is to provide a projection optical machine anti-glare control method, system, device and medium, which can achieve anti-glare projection of traffic signs while reducing the cognitive load of the driver.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a method for controlling the anti-glare of a projection optical machine, comprising:

[0008] Collect images of the vehicle's surroundings in real time;

[0009] Performing target detection on the vehicle surrounding environment image to determine traffic sign information and anti-glare target information in the vehicle surrounding environment image;

[0010] Determining an icon to be projected according to the traffic sign information and the anti-glare target information;

[0011] Real-time detection of driver instructions;

[0012] If there is no driver instruction, controlling the projection light machine to project the icon to be projected onto the ground in front of the vehicle;

[0013] If there is a driver instruction, the pre-trained large language model is used to analyze the driver instruction and the state of the projection optical machine respectively to obtain quantitative constraint information and the quantitative projection optical machine state;

[0014] A final projection icon is generated according to the quantified constraint information, the quantified projection optical machine state and the icon to be projected, and the projection optical machine is controlled to project the final projection icon onto the ground in front of the vehicle.

[0015] In a second aspect, the present application provides a projection optical machine anti-glare control system, comprising:

[0016] An image acquisition module, used for acquiring images of the vehicle's surroundings in real time;

[0017] A target detection module, connected to the image acquisition module, for performing target detection on the vehicle surrounding environment image, and determining traffic sign information and anti-glare target information in the vehicle surrounding environment image;

[0018] An icon generation module, connected to the target detection module, for determining an icon to be projected according to the traffic sign information and the anti-glare target information;

[0019] A command detection module, used to detect in real time whether there is a driver command;

[0020] A first projection module, connected to the instruction detection module and the icon generation module respectively, for controlling the projection light machine to project the icon to be projected onto the ground in front of the vehicle when there is no driver instruction;

[0021] An instruction parsing module, connected to the instruction detection module, is used to parse the driver's instruction and the state of the projection optical machine respectively by using a pre-trained large language model when there is a driver's instruction, so as to obtain quantitative constraint information and a quantitative projection optical machine state;

[0022] The second projection module is connected to the instruction parsing module and the icon generating module respectively, and is used to generate a final projection icon according to the quantitative constraint information, the quantitative projection optical machine state and the icon to be projected, and control the projection optical machine to project the final projection icon onto the ground in front of the vehicle.

[0023] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned projection light machine anti-glare control method.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned projection light machine anti-glare control method when executed by a processor.

[0025] According to the specific embodiments provided in this application, this application has the following technical effects:

[0026] The present application provides a method, system, device and medium for controlling anti-glare of a projection optical machine. By performing target detection on an image of the vehicle's surrounding environment, traffic sign information and anti-glare target information are determined, and an icon to be projected is determined based on the traffic sign information and the anti-glare target information. When there is no driver's instruction, the projection optical machine projects the icon to be projected onto the ground in front of the vehicle. When the driver's instruction is received, a large language model is used to parse the driver's instruction and the state of the projection optical machine. The multi-round dialogue and context understanding capabilities of the large language model can maintain consistency in continuous instructions, ensure the accuracy and coherence of operations, and the driver can control the projection optical machine without complex operations, which greatly reduces the operating burden, improves driving safety, and ensures the visual safety of other traffic participants. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 This is a diagram of the application environment of a projection light machine anti-glare control method in one embodiment of the present application.

[0029] Figure 2 An overall flow chart of a projection optical machine anti-glare control method provided in one embodiment of the present application.

[0030] Figure 3 A detailed flow chart of a projection optical machine anti-glare control method provided in one embodiment of the present application.

[0031] Figure 4 Schematic diagram of the logic of the anti-glare control algorithm in one embodiment of the present application.

[0032] Figure 5Schematic diagram of the internal processing process of a large language model in one embodiment of the present application.

[0033] Figure 6 A schematic diagram of a module of a projection optical machine anti-glare control system provided in one embodiment of the present application.

[0034] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0036] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0037] First, some technical terms involved in the embodiments of the present application are introduced.

[0038] (1) Projection machine: The projection machine includes a light source, an optical lens, a reflector and an image processing unit, and is used to project images or information onto a specific surface, such as a wall, a screen or the ground, through optical projection technology. It can be installed in front of the vehicle hood to project images onto the ground in front of the vehicle.

[0039] (2) Anti-glare control algorithm: used to control the brightness and direction of the projection light machine to reduce the possibility of it causing glare to pedestrians or other traffic participants. The algorithm can be dynamically adjusted based on factors such as vehicle speed and ambient lighting.

[0040] (3) Anti-glare targets: at least one of pedestrians or drivers of other vehicles.

[0041] (4) Large Language Model: Large Language Model (LLM) is an artificial intelligence model based on deep learning technology, which is specially used to process and generate natural language text. It is able to understand and generate language expressions with complex semantics and grammar through large-scale data training. Representative models include the Generative Pre-trained Transformer (GPT) series models developed by OpenAI and the Bidirectional Encoder Representations from Transformers (BERT) model developed by Google.

[0042] The anti-glare control method for projection light machine provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, or it can be integrated on the server 104, or it can be placed on the cloud or other servers. The terminal 102 can send the driver's instructions and the vehicle's surrounding environment image to the server 104, and the server 104 generates the icon to be projected or the final projection icon based on the received driver's instructions and the vehicle's surrounding environment image. The server 104 can feedback the icon to be projected or the final projection icon to the terminal 102. In addition, in some embodiments, the projection light machine anti-glare control method can also be implemented by the server 104 or the terminal 102 alone.

[0043] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0044] In an exemplary embodiment, Figures 2 to 4 As shown, a projection light machine anti-glare control method is provided, which is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 207.

[0045] Step 201, collecting images of the vehicle's surroundings in real time.

[0046] In an exemplary embodiment, a high-performance image sensor is used to capture images of the vehicle's surrounding environment. The image sensor should also have low-light performance to ensure that clear images can be captured at night or in poor lighting conditions.

[0047] Since the projection light machine anti-glare control method provided in this application is mainly used for nighttime driving, in order to improve the accuracy of subsequent target detection, the vehicle surrounding environment image is first enhanced before step 202. Specifically, image enhancement technology such as Retinex can be used for low-light image processing.

[0048] Step 202: Target detection is performed on the vehicle surrounding environment image to determine traffic sign information and anti-glare target information in the vehicle surrounding environment image. The traffic sign information includes the type, quantity and confidence of traffic signs. The anti-glare target information includes the quantity, type, location and confidence of anti-glare targets.

[0049] In an exemplary embodiment, a deep learning algorithm such as the YOLO (You Only Look Once) series or other appropriate algorithms may be used to detect traffic signs to obtain the type and quantity of traffic signs on the road ahead of the vehicle and the confidence level of the detection results. Specifically, a pre-trained YOLOv4 network is used to perform target detection on the vehicle surrounding environment image to determine the traffic sign information in the vehicle surrounding environment image.

[0050] A pre-trained Faster Region-based Convolutional Neural Network (Faster R-CNN) network is used to perform target detection on the vehicle surrounding environment image to determine anti-glare target information in the vehicle surrounding environment image.

[0051] Step 203: determining an icon to be projected according to the traffic sign information and the anti-glare target information.

[0052] In an exemplary embodiment, step 203 includes the following steps 31 to 35 .

[0053] Step 31 , determining whether the confidence level of the traffic sign in the vehicle surrounding environment image is greater than a first set threshold.

[0054] Step 32: If the confidence level of the traffic sign in the vehicle surrounding environment image is less than or equal to a first set threshold, the icon to be projected is a transparent icon.

[0055] Step 33: If the confidence of the traffic sign in the vehicle surrounding environment image is greater than a first set threshold, an icon of the traffic sign is generated, and it is determined whether the confidence of the anti-glare target in the vehicle surrounding environment image is greater than a second set threshold.

[0056] Specifically, if the confidence of the detected traffic sign is higher than 0.75, the icon to be projected can be generated based on the icon corresponding to the traffic sign; if there are multiple traffic signs in the vehicle's surrounding environment image at the same time, and their detection confidence levels are all higher than 0.75, the icons corresponding to each traffic sign are generated in sequence in a certain order and time interval to be projected.

[0057] Step 34: if the confidence level of the anti-glare target in the vehicle surrounding environment image is greater than a second set threshold, a masking frame is generated on the icon of the traffic sign according to the position of the anti-glare target to obtain the icon to be projected.

[0058] Specifically, if the confidence of the detected anti-glare target is higher than 0.75, the projection icon of the projection light machine needs to be partially masked by covering the corresponding area pixels of the projection icon with a black masking frame. The position of the masking frame needs to be obtained after coordinate transformation according to the position of the anti-glare target.

[0059] First determine the coordinate transformation matrix , if in the vehicle coordinate system, the relative translation vector between the camera and the projection optical machine is , the relative rotation angle is ,in, is the relative translation distance of the x-axis, is the relative translation distance of the y axis, is the relative translation distance of the z axis, is the relative rotation angle about the x-axis, is the relative rotation angle about the y axis, is the relative rotation angle of the z-axis, then the translation transformation matrix for: .

[0060] Rotation transformation matrix around the x-axis for: .

[0061] Rotation transformation matrix around the y axis for: .

[0062] Rotation transformation matrix around the z-axis for: .

[0063] Get the total coordinate transformation matrix : .

[0064] According to the obtained coordinate transformation matrix , if the corner point coordinates of the detection frame of the anti-glare target are and , then the coordinates of the two corner points of the masking box are: .in, is the x-axis coordinate of the upper left corner of the detection frame of the anti-glare target, The y-axis coordinate of the upper left corner of the detection frame of the anti-glare target, is the x-axis coordinate of the lower right corner of the detection frame of the anti-glare target, is the y-axis coordinate of the lower right corner of the detection frame of the anti-glare target, is the x-axis coordinate of the upper left corner of the mask box, The y-axis coordinate of the upper left corner of the mask box.

[0065] The initial opacity of the mask frame is 100%, and the opacity of the mask frame can be adjusted by the driver's command.

[0066] Step 35: If the confidence level of the anti-glare target in the vehicle surrounding environment image is less than or equal to a second set threshold, the icon to be projected is the icon of the traffic sign.

[0067] Step 204: Detect in real time whether there is a driver instruction.

[0068] Step 205: If there is no driver instruction, control the projection light machine to project the icon to be projected onto the ground in front of the vehicle.

[0069] Step 206: If there is a driver instruction, a pre-trained large language model is used to analyze the driver instruction and the state of the projection optical machine to obtain quantitative constraint information and a quantitative projection optical machine state.

[0070] The present application parses the driver's instructions through a pre-trained large language model and converts them into constraints, so that the driver can adjust the anti-glare control algorithm of the projection light machine through straightforward language instructions. The anti-glare control algorithm performs anti-glare control on the projection light machine according to the generated constraints, thereby achieving the purpose of anti-glare control for pedestrians and vehicles.

[0071] In an exemplary embodiment, Figure 5As shown, in step 206, the driver's instruction is first parsed by a decoder to obtain a constraint information text. According to the state of the projection light machine, a preliminary state text is generated by a template generation method. The preliminary state text and the state of the projection light machine are used as prompt words, and a pre-trained large language model (such as GPT4) is used to generate a projection light machine state text. Then, an encoder is used to encode the constraint information text and the projection light machine state text respectively to obtain quantized constraint information and quantized projection light machine state.

[0072] Wherein, the decoder and the encoder are both pre-trained large language models.

[0073] The large language model as a decoder is mainly used to parse and remove redundancy from the driver's natural language instructions. Its function is to understand the original semantic content of the driver's instructions and extract the constraint information related to the projection light machine anti-glare control algorithm. For example, when the driver says "lower the brightness on the right", the decoder will analyze the key semantic components such as the operation (lower the brightness) and the object (right area) contained in the instruction, and convert the complex natural language instructions into more concise and clear language constraints, providing a basis for subsequent processing.

[0074] The large language model as an encoder focuses on encoding the de-redundant constraint information text to obtain the encoded form constraints of the driver's instructions. And through the contrast loss fine-tuning, it is ensured that constraints with the same semantics have similar embeddings. At the same time, the state of the projection light machine is also encoded using the encoder and compared with the encoded state of the driver's constraints to determine whether the state of the projection light machine violates the constraints contained in the driver's instructions, and then decide whether the projection light machine needs to be controlled.

[0075] In this application, the driver's instructions are parsed and de-redundanted through a decoder to analyze whether the driver's instructions contain constraints on the projection light machine anti-glare control algorithm. If the constraint information is contained, the de-redundant constraint information text is organized into a natural language constraint pair and input into the encoder. Among them, the natural language constraint pair is a group of multiple expressions of constraints with the same or similar meanings. Each natural language constraint pair represents a different expression of the same or similar semantics. For example, for the semantics of "lowering the brightness", there may be multiple expressions such as "dim the light" and "reduce the intensity of the light".

[0076] When the de-redundant constraint information text is input into the encoder, in order to ensure that constraints with the same semantics have similar embeddings, the contrast loss is used to fine-tune the encoder so that constraints with the same semantics can have similar embeddings after encoding, thereby improving the accuracy and robustness of parsing driver instructions. For a batch of natural language constraint pairs, the contrast loss can be calculated by the following formula.

[0077] .

[0078] .

[0079] in, is the comparison loss value, n is the number of batches of natural language constraint pairs, For the i The decision value of a batch of natural language constraint pairs, when the i The meaning of the natural language constraint pair is to prohibit the projection light machine from making corresponding actions The value of is 1, otherwise it is 0. and For the i Two natural language constraints in a batch of natural language constraint pairs, for and The cosine similarity of .

[0080] Step 207 , generating a final projection icon according to the quantified constraint information, the quantified projection optical machine state and the icon to be projected, and controlling the projection optical machine to project the final projection icon onto the ground in front of the vehicle.

[0081] In an exemplary embodiment, step 207 includes the following steps 71 and 72 .

[0082] Step 71, calculating the similarity between the quantized constraint information and the quantized projection optical machine state.

[0083] Step 72: If the similarity is greater than a set similarity threshold, a brightness mask frame is added to the icon to be projected to obtain a final projection icon; otherwise, the final projection icon is the icon to be projected.

[0084] Specifically, we first use the formula Determine the control decision value ,in, To quantify the constraint information, The table is used to quantify the state of the projection light machine. A is to set the similarity threshold. If Indicates that the projection optical machine does not meet the constraint state at this time and needs to be controlled. It means that the projection optical machine meets the constraint state at this time and does not need to be controlled.

[0085] Furthermore, the projection light machine anti-glare control algorithm is used to determine whether there is an anti-glare target within the illumination range of the projection light machine. If not, the projected icon will not be adjusted. If so, steps 71 and 72 are used to determine whether a brightness masking frame needs to be added to the projected icon.

[0086] This application utilizes the multi-round dialogue and context understanding capabilities of the large language model to maintain consistency in continuous instructions and ensure the accuracy and coherence of operations. Secondly, the semantic understanding ability of the large language model can handle synonyms, fuzzy expressions, and context dependencies to ensure high accuracy of instruction parsing. In addition, through intuitive voice input, the driver can control the projection light machine without complex operations, which greatly reduces the operating burden and improves driving safety. Finally, the anti-glare control algorithm adjusts the brightness and direction of the projection beam in real time according to the generated constraints to ensure the visual safety of other traffic participants while improving the driving safety of the vehicle itself.

[0087] The projection light machine anti-glare control method provided in this application is applicable to various types of vehicles, including but not limited to private cars, commercial vehicles and self-driving vehicles. It can be flexibly applied to various road environments such as urban roads, highways and rural roads, and play a role in intelligent driving assistance. It improves the safety and driving experience of drivers in complex traffic environments. Through an efficient anti-glare control algorithm, it can dynamically adapt to different traffic scenarios and provide a safer and more comfortable driving environment.

[0088] The following describes the specific implementation process by taking the application of the projection optical machine anti-glare control method on a certain household sports utility vehicle (SUV) as an example.

[0089] (1) A low-light complementary metal oxide semiconductor (CMOS) camera is installed at the front of the car to ensure that it has sufficient field of view in the direction of the car's movement. Image information from the front camera of the car can also be obtained by reading the vehicle's Controller Area Network (CAN) signal. This enables the sensor to collect high-quality images of the surrounding environment in real time at night or under poor lighting conditions.

[0090] (2) For the video acquired from the camera, the Retinex image enhancement algorithm is used to process the low-light image in real time by extracting frames. By analyzing the input image, the corresponding illumination weight matrix is ​​generated to guide the adaptive initialization of the illumination component. Then, under the constraint of structured illumination, the initial illumination component is optimized and smoothed by the introduced equivalent objective function. Subsequently, nonlinear illumination adjustment is performed, and finally, the Retinex theory is combined to achieve comprehensive enhancement of the low-light image.

[0091] In an exemplary embodiment, for the collected video stream, fixed interval frame extraction is first performed to improve the processing speed, and the specific formula is: ;in, For the m The index of the extracted frame, m The order of extracting frames, starting from 0, is the frame rate of the video, is the frame extraction interval, and round indicates rounding operation.

[0092] The downsampling-based adaptive Retinex nighttime fast image enhancement algorithm is used to process the extracted images in real time. The Retinex theory is based on human visual perception and color constancy in natural scenes. Its basic idea is to decompose the observed image into the product of two parts: the illumination component and the reflection component: ;in, is the original low-light image input, is the illumination component, is the reflection component, For pixels, the operator Represents an element-wise multiplication operation.

[0093] Specifically, adaptive illumination initialization is first performed: the video stream is frame-extracted, the extracted color image frames are downsampled by the nearest neighbor interpolation method, and then converted from the RGB color space to the HSV color space, and the brightness channel V is taken as the initial illumination component. Finally, local blocks are introduced to highlight the edge structure information of the image. The specific formula is as follows:

[0094] ;

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, is the initial illumination component, is the V component of the image in the HSV color space, is the red, green, or blue color channel, is the c channel component of the image in the RGB color space, is the downsampling ratio, is the lighting weight matrix, , For The local block centered on is 3×3 in size. Adjust parameters for lighting, , , , are the indices of the image pixels, is the downsampled image, is the original image before downsampling.

[0100] Then, we optimize the illumination under low light conditions: the ideal illumination component optimization algorithm is to smooth the spatial texture as much as possible while maintaining the edge structure, and the detailed information of such images should be reflected in the reflection component. According to the Retinex theoretical model, the accurate estimation of the illumination component will directly affect the reflection component, so the optimization problem of the illumination component can be further simplified as follows: ;in, is the initial illumination component, and same, To optimize the lighting component, is the structural weight matrix, It is a first-order derivative filter, which is specifically divided into (horizontally) and (vertical direction) two parts, and are 1-norm and standard norm respectively, is a balance parameter. is the data fidelity term, used to constrain the difference, represents the regularization term, which is used to limit the scope of the solution space. Used to balance the fidelity term and the regularization term.

[0101] Structural Weight Matrix It includes two parts: horizontal and vertical:

[0102] ;

[0103] ;

[0104] in, is the horizontal structural weight matrix, is the vertical weight matrix, The standard deviation is The Gaussian kernel function, , Represents pixels and pixels The spatial Euclidean distance of is a small constant to avoid the denominator being 0. It represents the absolute value operation.

[0105] Then adjust the video stream for low-light conditions: For low-light image enhancement tasks, improving image brightness is one of the core issues to be solved, so before outputting the enhancement results, perform gamma correction on the optimized illumination components to achieve nonlinear adjustment of image brightness and further improve the enhancement results. The specific formula is ;in, is the illumination component after gamma correction, To set the constant, its value can be adjusted according to the actual situation, temporarily set to 0.8, and then based on the original low-light image obtained by video frame extraction And the gamma-corrected illumination component Make changes, and finally get the output image, then perform nearest neighbor interpolation upsampling, restore the image size, and combine the Retinex model to get the enhanced result : .

[0106] (3) In the traffic sign detection stage, the YOLOv4 network is used. YOLOv4 has high detection accuracy and real-time performance, and is suitable for real-time traffic sign detection applications. First, the YOLOv4 network is pre-trained using the traffic sign dataset TT100k. Then, the pre-trained YOLOv4 network is combined with a specific traffic sign dataset and fine-tuned to improve the detection capability of traffic signs. The pre-trained and fine-tuned YOLOv4 network is deployed to the industrial computer. The trained weight file is loaded into the model and the model is inferred through the deep learning framework PyTorch.

[0107] (4) In the anti-glare target detection stage, the Faster R-CNN network is used. First, through the data set of night environment, the Faster R-CNN network can be trained by focusing on the image samples of vehicles and pedestrians in the training set, which helps to better identify anti-glare targets in night environment. The trained Faster R-CNN network is deployed to the industrial computer, the trained weight file is loaded, and real-time target detection is performed through the deep learning framework PyTorch. The Faster R-CNN network detects other vehicles and pedestrians in the video stream in real time and obtains their location, type and confidence.

[0108] (5) Analyze the vehicle and pedestrian position information obtained in the anti-glare target detection stage to determine whether their positions are within the illumination range of the projection light machine. If there is an anti-glare target within the illumination range of the projection light machine, a mask frame must be added to the corresponding position of the icon to be projected. At the same time, if the driver issues relevant instructions, such as "lower the upper left projection brightness", the driver's instructions are converted into quantitative constraints, and the cosine similarity between the quantitative constraint information and the quantitative projection light machine state is calculated. If the cosine similarity of the two is greater than the set similarity threshold (0.9), it is considered that the projection light machine state and the constraint are too similar, that is, the projection light machine does not meet the constraint state at this time. , it needs to be controlled, and a brightness mask box is added to the upper left corner of the icon to be projected; otherwise , indicating that the projection optical machine meets the constraint state and does not need to be controlled.

[0109] (6) The projection light machine projects the projection icon onto the ground in front of the vehicle.

[0110] In summary, the present application has the following beneficial effects.

[0111] ①Instruction parsing based on large language model.

[0112] The driver's natural language instructions are parsed using an advanced large language model and converted into specific control commands and parameter constraints. This method does not require the driver to have professional technical knowledge. The driver can identify and perform corresponding adjustments through simple language instructions, such as "lower the brightness on the right" or "cover the opposite lane." The advantages of the large language model in multi-round dialogue, semantic understanding, and context association are fully utilized. When faced with various complex driving scenarios and diverse instructions, it can still accurately understand and execute, thereby improving the level of intelligence and user experience.

[0113] ② Dynamic anti-glare control based on driver instructions.

[0114] Under normal conditions, the anti-glare control strategy is automatically executed, and the road conditions and other traffic participants are monitored in real time through sensor and camera data. When receiving driver instructions, the large language model parses the driver's instructions and generates corresponding control constraints, such as adjusting the beam direction, changing the brightness, or setting a shielding area to ensure the safety and effectiveness of the projected beam in a dynamic traffic environment.

[0115] ③User-friendly interaction method.

[0116] It provides an intuitive voice input method, allowing the driver to control the anti-glare function of the projection light machine through simple language commands. At the same time, the large language model supports multiple rounds of dialogue and can maintain contextual consistency in continuous commands. For example, the driver can first say "lower the brightness on the right side" and then say "lower a little bit", and the large language model can understand and perform continuous adjustments. At the same time, the large language model can handle synonyms, fuzzy expressions and context dependencies to ensure the accuracy of command parsing. For example, it can recognize that "lower the brightness a little" and "dim the light" have the same operational intention. It reduces the driver's operating burden, improves driving safety, and enables the driver to easily intervene in the anti-glare control algorithm, ensuring the visual safety of other traffic participants while improving the driving safety of the vehicle itself.

[0117] Based on the same inventive concept, the embodiment of the present application also provides a projection light machine anti-glare control system for implementing the projection light machine anti-glare control method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more projection light machine anti-glare control system embodiments provided below can refer to the limitations of the projection light machine anti-glare control method above, and will not be repeated here.

[0118] In an exemplary embodiment, Figure 6 As shown, a projection optical machine anti-glare control system is provided, including: an image acquisition module 601, a target detection module 602, an icon generation module 603, an instruction detection module 604, a first projection module 605, an instruction parsing module 606 and a second projection module 607.

[0119] The image acquisition module 601 is used to acquire images of the vehicle's surrounding environment in real time.

[0120] The target detection module 602 is connected to the image acquisition module 601 , and is used to perform target detection on the vehicle surrounding environment image, and determine the traffic sign information and anti-glare target information in the vehicle surrounding environment image.

[0121] The icon generation module 603 is connected to the target detection module 602, and the icon generation module 603 is used to determine the icon to be projected according to the traffic sign information and the anti-glare target information.

[0122] The instruction detection module 604 is used to detect in real time whether there is a driver instruction.

[0123] The first projection module 605 is connected to the instruction detection module 604 and the icon generation module 603 respectively. The first projection module 605 is used to control the projection light machine to project the icon to be projected onto the ground in front of the vehicle when there is no driver instruction.

[0124] The instruction parsing module 606 is connected to the instruction detection module 604. The instruction parsing module 606 is used to use a pre-trained large language model to parse the driver's instructions and the state of the projection light machine respectively when there are driver instructions, so as to obtain quantitative constraint information and quantitative projection light machine state.

[0125] The second projection module 607 is connected to the instruction parsing module 606 and the icon generating module 603 respectively. The second projection module 607 is used to generate a final projection icon according to the quantitative constraint information, the state of the quantitative projection optical machine and the icon to be projected, and control the projection optical machine to project the final projection icon onto the ground in front of the vehicle.

[0126] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store vehicle surrounding environment images and driver instructions. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a projection light machine anti-glare control method is implemented.

[0127] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0129] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0131] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0132] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0133] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for controlling the glare prevention of a projection light machine, characterized in that: The anti-glare control method for a projection optical machine comprises: Collect images of the vehicle's surroundings in real time; Performing target detection on the vehicle surrounding environment image to determine traffic sign information and anti-glare target information in the vehicle surrounding environment image; Determining an icon to be projected according to the traffic sign information and the anti-glare target information; Real-time detection of driver instructions; If there is no driver instruction, controlling the projection light machine to project the icon to be projected onto the ground in front of the vehicle; If there is a driver instruction, a pre-trained large language model is used to parse the driver instruction and the state of the projection light machine respectively to obtain quantized constraint information and quantized projection light machine state, specifically including: using a decoder to parse the driver instruction to obtain constraint information text; according to the state of the projection light machine, a template generation method is used to generate a preliminary state text; using the preliminary state text and the state of the projection light machine as prompt words, and using a pre-trained large language model to generate a projection light machine state text; using an encoder to encode the constraint information text and the projection light machine state text respectively to obtain quantized constraint information and quantized projection light machine state; the decoder and the encoder are both pre-trained large language models; A final projection icon is generated according to the quantified constraint information, the quantified projection optical machine state and the icon to be projected, and the projection optical machine is controlled to project the final projection icon onto the ground in front of the vehicle.

2. The anti-glare control method for projection light machine according to claim 1, characterized in that: Before performing target detection on the vehicle surrounding environment image, the projection light machine anti-glare control method further includes: Performing image enhancement on the vehicle surrounding environment image.

3. The anti-glare control method for projection light machine according to claim 1, characterized in that: Performing target detection on the vehicle surrounding environment image to determine traffic sign information and anti-glare target information in the vehicle surrounding environment image specifically includes: Using a pre-trained YOLOv4 network to perform target detection on the vehicle surrounding environment image to determine traffic sign information in the vehicle surrounding environment image; A pre-trained Faster R-CNN network is used to perform target detection on the vehicle surrounding environment image to determine anti-glare target information in the vehicle surrounding environment image.

4. The anti-glare control method for a projection light machine according to claim 1, characterized in that: The traffic sign information includes the type, quantity and confidence level of the traffic sign; the anti-glare target information includes the position and confidence level of the anti-glare target.

5. The anti-glare control method for a projection light machine according to claim 4, characterized in that: Determining an icon to be projected according to the traffic sign information and the anti-glare target information specifically includes: Determining whether the confidence level of the traffic sign in the vehicle surrounding environment image is greater than a first set threshold; If the confidence level of the traffic sign in the vehicle surrounding environment image is less than or equal to a first set threshold, the icon to be projected is a transparent icon; If the confidence level of the traffic sign in the vehicle surrounding environment image is greater than a first set threshold, an icon of the traffic sign is generated, and it is determined whether the confidence level of the anti-glare target in the vehicle surrounding environment image is greater than a second set threshold; If the confidence level of the anti-glare target in the vehicle surrounding environment image is greater than a second set threshold, a masking frame is generated on the icon of the traffic sign according to the position of the anti-glare target to obtain the icon to be projected; If the confidence level of the anti-glare target in the vehicle surrounding environment image is less than or equal to a second set threshold, the icon to be projected is the icon of the traffic sign.

6. The anti-glare control method for projection light machine according to claim 1, characterized in that: Generating a final projection icon according to the quantified constraint information, the quantified projection optical machine state and the icon to be projected specifically includes: Calculating the similarity between the quantized constraint information and the quantized projection optical machine state; If the similarity is greater than a set similarity threshold, a brightness masking frame is added to the icon to be projected to obtain a final projection icon; otherwise, the final projection icon is the icon to be projected.

7. A projection light machine anti-glare control system, applied to the projection light machine anti-glare control method according to any one of claims 1 to 6, characterized in that: The projection light machine anti-glare control system comprises: An image acquisition module, used for acquiring images of the vehicle's surroundings in real time; A target detection module, connected to the image acquisition module, for performing target detection on the vehicle surrounding environment image, and determining traffic sign information and anti-glare target information in the vehicle surrounding environment image; An icon generation module, connected to the target detection module, for determining an icon to be projected according to the traffic sign information and the anti-glare target information; A command detection module, used to detect in real time whether there is a driver command; A first projection module, connected to the instruction detection module and the icon generation module respectively, for controlling the projection light machine to project the icon to be projected onto the ground in front of the vehicle when there is no driver instruction; An instruction parsing module, connected to the instruction detection module, is used to parse the driver's instruction and the state of the projection optical machine respectively by using a pre-trained large language model when there is a driver's instruction, so as to obtain quantitative constraint information and a quantitative projection optical machine state; The second projection module is connected to the instruction parsing module and the icon generating module respectively, and is used to generate a final projection icon according to the quantitative constraint information, the quantitative projection optical machine state and the icon to be projected, and control the projection optical machine to project the final projection icon onto the ground in front of the vehicle.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the projection light machine anti-glare control method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the projection light machine anti-glare control method described in any one of claims 1 to 6 is implemented.

Images