Vehicle control method and system and computer equipment

Abstract

The invention provides a vehicle control method and system and computer equipment, and the method comprises the steps: obtaining multi-modal data of a first target vehicle, determining lamp language features based on the multi-modal data, mapping the lamp language features into lamp language semantic tags through employing a pre-obtained vehicle lamp language coding table, and storing the lamp language semantic tags in a server; and the second target vehicle is controlled according to the lamp language semantic tag, so that the first target vehicle executes or stops a driving intention corresponding to the lamp language semantic tag of the first target vehicle based on a control result of the second target vehicle. According to the method, the multi-modal data is acquired to identify the lamp signal, the problem that a single sensor is easily interfered by the environment to cause misjudgment can be solved, and meanwhile, the driving behavior or driving intention of the vehicle is predicted according to the lamp characteristics, so that a coping strategy is formed in advance, sudden or high-risk situations are effectively avoided, the traffic accident probability is reduced, and the driving safety is improved. And multi-vehicle bidirectional cooperative interaction can be realized in combination with the information of the traffic participants.

Description

Technical Field

[0001] This application relates to the technical field of vehicles, and particularly to a vehicle control method and system, and a computer device. Background Art

[0002] When performing headlight signal recognition, related technologies mainly rely on a single sensor (such as a camera, radar, etc.) for recognition, which is easily affected by environmental interference, such as the interference of billboard lights, rainy and foggy weather, etc., resulting in misjudgment. Moreover, for headlight signals in complex light language scenarios, such as continuous flashing, multi-vehicle interaction scenarios, etc., related technologies are also difficult to parse in real time. At the same time, when the signal lights or headlights are blocked by objects such as trees or the vehicle in front in some scenarios, related technologies are difficult to enhance the driver's perception through auxiliary means. In addition, related technologies only support one-way light language recognition, such as the vehicle recognizing the signals of the vehicle in front unidirectionally, lacking multi-vehicle two-way interaction capabilities (such as feedback from the vehicle in front, multi-vehicle formation negotiation), and it is difficult to achieve collaborative decision-making in complex scenarios; and when multiple vehicles send light languages simultaneously, related technologies cannot solve the signal conflict problem, such as the conflict between overtaking requests and yielding instructions. Summary of the Invention

[0003] In view of the above-mentioned disadvantages of the prior art, the purpose of this application is to provide a vehicle control method and system, and a computer device, which are used to solve the technical problems existing in related technologies.

[0004] To achieve the above purpose and other related purposes, this application provides a vehicle control method, including the following steps: Obtain multi-modal data of a first target vehicle, where the multi-modal data includes headlight image information, vehicle position information, and environmental information, and the environmental information includes road information and traffic participant information; Determine the light language feature of the first target vehicle based on the multi-modal data, where the light language feature includes headlight flashing frequency, headlight color, headlight area, and light language intention; Use a pre-obtained headlight light language coding table to map the light language feature to a light language semantic label, and control a second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the control result of the second target vehicle; where the first target vehicle and the second target vehicle are within a preset range.

[0005] In an embodiment of this application, the process of controlling the second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the control result of the second target vehicle includes: Perform headlight control on the second target vehicle according to the semantic label of the headlight signal, so that the first target vehicle executes or aborts the driving intention corresponding to the semantic label of the headlight signal based on the headlight control result of the second target vehicle; Or, Perform headlight and vehicle speed control on the second target vehicle according to the semantic label of the headlight signal, so that the first target vehicle executes or aborts the driving intention corresponding to the semantic label of the headlight signal based on the headlight and vehicle speed control results of the second target vehicle.

[0006] In an embodiment of the present application, the process of performing control on the second target vehicle according to the semantic label of the headlight signal, so that the first target vehicle executes or aborts the driving intention corresponding to the semantic label of the headlight signal based on the control result of the second target vehicle includes: Use the headlight signal coding table as the optical coding interaction protocol for the sending vehicle and the receiving vehicle; wherein, the sending vehicle and the receiving vehicle are determined from the traffic participant information, the first target vehicle, and the second target vehicle; The receiving vehicle identifies the optical signal parameters, and generates a feedback code or a warning code according to the identification result of the optical signal parameters; wherein, the optical signal parameters are obtained by the sending vehicle mapping the optical code representing the driving intention of the sending vehicle according to the headlight signal coding rule; The sending vehicle analyzes the feedback code or the warning code, and executes the driving intention according to the analysis result of the feedback code, or aborts the driving intention according to the analysis result of the warning code.

[0007] In an embodiment of the present application, the process of the receiving vehicle identifying the optical signal parameters and generating a feedback code or a warning code according to the identification result of the optical signal parameters includes: The receiving vehicle captures headlight signals from the optical signal parameters, including extracting the headlight flashing frequency and the headlight color; and, The receiving vehicle performs point cloud data verification on the optical signal parameters, including positioning the sending vehicle through point cloud data and verifying the movement trajectory of the sending vehicle; The receiving vehicle performs multispectral fusion on the optical signal parameters, including collecting visible light and near-infrared band images and filtering ambient light through band differences; Generate the feedback code or the warning code according to the headlight signal capture result, the point cloud data verification result, and the multispectral fusion result.

[0008] In one embodiment of the present application, the method further includes: competing for a time window according to the priorities of the sending vehicle and the receiving vehicle, and transmitting an encoding instruction according to the result of the time window competition; wherein, the encoding instruction includes: the optical signal parameter, the feedback encoding or the warning encoding; And / or, within a preset time window, using the vehicle with the highest priority among the traffic participant information, the first target vehicle, and the second target vehicle as the sending vehicle, and the remaining vehicles as receiving vehicles, and only allowing the vehicle with the highest priority to send the optical signal parameter.

[0009] In one embodiment of the present application, the process of obtaining the vehicle light signal encoding table includes: Defining the meaning of the light signal according to the vehicle light category, the vehicle position relationship, and the number of vehicle light flashes; and, Determining the vehicle light signal encoding rule according to the pre-determined or real-time determined vehicle light signal field, the encoding length of the vehicle light signal field, and the description of the vehicle light signal field, and obtaining the complete encoding of the vehicle light signal based on the vehicle light signal encoding rule; wherein, the vehicle light signal field includes signal type, flashing mode, number of flashes, light source direction, color identification, priority, and preset reserved bits; Associating the definition of the light signal meaning, the vehicle light signal field, and the complete encoding of the vehicle light signal to form a vehicle light signal encoding table.

[0010] In one embodiment of the present application, the process of controlling the second target vehicle according to the light signal semantic label includes: Displaying the light signal semantic label through a target display device pre-configured in the second target vehicle; wherein, the target display device is within the field of vision of the driver of the second target vehicle; Sending an audio alarm to the second target vehicle according to the light signal semantic label and the occlusion information of the first target vehicle; and when the distance between the first target vehicle and the second target vehicle is less than a preset distance, performing a braking control on the second target vehicle and controlling the brake light of the second target vehicle to flash; wherein, the occlusion information is identified by an enhanced display device.

[0011] In one embodiment of the present application, the process of determining the light signal feature of the first target vehicle based on the multi-modal data includes: Calculating the vehicle light brightness change frequency based on the pixel change of the vehicle light image information to obtain the flashing frequency of the first target vehicle; and, Segment the headlight image information into a red region and a yellow region according to a preset color space, and obtain the headlight color of the first target vehicle based on the segmentation results of the red region and the yellow region; wherein, the headlight color corresponding to the segmentation result of the red region is red, and the headlight color corresponding to the segmentation result of the yellow region is yellow; and, Predict the light signal intention of the first target vehicle based on the headlight image information within a preset time period and the vehicle movement trajectory of the first target vehicle within a preset continuous time period; and, When the headlight in the headlight image information is blocked, predict the headlight position of the first target vehicle using the point cloud data of the first target vehicle; and, Project the three-dimensional data frame of the point cloud data onto a two-dimensional image to locate the headlight area of the first target vehicle.

[0012] This application also provides a vehicle control system, which includes: A data acquisition module for acquiring multi-modal data of a first target vehicle, where the multi-modal data includes headlight image information, vehicle position information, and environmental information, and the environmental information includes road information and traffic participant information; A light signal feature module for determining the light signal features of the first target vehicle according to the multi-modal data, where the light signal features include headlight flashing frequency, headlight color, headlight area, and light signal intention; A vehicle control module for mapping the light signal features to light signal semantic labels using a pre-obtained headlight light signal coding table, and controlling a second target vehicle according to the light signal semantic labels, so that the first target vehicle executes or aborts a driving intention corresponding to the light signal semantic label based on the control result of the second target vehicle; wherein, the first target vehicle and the second target vehicle are within a preset range.

[0013] This application also provides a computer device, including a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the vehicle control method described in any one of the above.

[0014] This application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the vehicle control method described in any one of the above.

[0015] As described above, the present application provides a vehicle control method, a system, and a computer device, which have the following beneficial effects: By obtaining the multi-modal data of the first target vehicle, determining the light language characteristics of the first target vehicle based on the multi-modal data, mapping the light language characteristics to light language semantic labels using a pre-obtained vehicle light language coding table, and controlling the second target vehicle according to the light language semantic labels, the first target vehicle can execute or abort the driving intention corresponding to the light language semantic label of the first target vehicle based on the control result of the second target vehicle; wherein, the multi-modal data includes vehicle light image information, vehicle position information, and environmental information, the environmental information includes road information and traffic participant information, and the light language characteristics include vehicle light flashing frequency, vehicle light color, vehicle light area, and light language intention; the first target vehicle and the second target vehicle are within a preset range. It can be seen that when the present application performs vehicle control, by obtaining multi-modal data for light language signal recognition, it can solve the problem of misjudgment easily caused by a single sensor being affected by the environment. At the same time, based on the light language characteristics of the first target vehicle, the driving behavior or driving intention of the first target vehicle can be predicted, so that the second target vehicle within the preset range of the first target vehicle can form a coping strategy in advance, effectively avoiding the occurrence of sudden or high-risk situations and reducing the probability of traffic accidents. In addition, the present application can also achieve multi-vehicle two-way collaborative interaction by combining traffic participant information, and when multiple vehicles send light languages simultaneously, it can solve the signal conflict problem in a priority manner, realizing collaborative decision-making in complex scenarios and reducing vehicle driving misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic flowchart of a vehicle control method provided by an embodiment of the present application; Figure 2 It is a schematic flowchart during vehicle collaborative interaction control provided by an embodiment of the present application; Figure 3 It is a schematic flowchart of a vehicle control method provided by another embodiment of the present application; Figure 4 It is a schematic hardware structure diagram of a vehicle control system provided by an embodiment of the present application; Figure 5 It is a schematic hardware structure diagram of a computer device suitable for implementing one or more embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] The following uses specific specific examples to illustrate the implementation manners of the present application. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It can be understood that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. In addition, it can be understood that the drawings provided in the following embodiments only illustrate the basic concept of the present application schematically. Therefore, only the components related to the present application are shown in the drawings, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0018] Figure 1 The flowchart of a vehicle control method is shown. Specifically, in an exemplary embodiment, as Figure 1 shown, this embodiment provides a vehicle control method, and the method includes the following steps: S110, obtain multimodal data of a first target vehicle, where the multimodal data includes headlight image information, vehicle position information, and environmental information, and the environmental information includes road information and traffic participant information; S120, determine the light language feature of the first target vehicle based on the multimodal data, where the light language feature includes the headlight flashing frequency, headlight color, headlight area, and light language intention; S130, use the pre-obtained headlight light language coding table to map the light language feature to a light language semantic label, and control a second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts the driving intention corresponding to the light language semantic label based on the control result of the second target vehicle; wherein, the first target vehicle and the second target vehicle are within a preset range.

[0019] In some embodiments, the first target vehicle and the second target vehicle may be vehicles including an assisted driving function or an intelligent driving function. The first target vehicle and the second target vehicle may be fuel vehicles, new energy vehicles, or hybrid vehicles. As some examples, the first target vehicle may be referred to as another vehicle, the leading vehicle, or the surrounding vehicle, and the second target vehicle may be referred to as the self-vehicle, the own vehicle, or the following vehicle.

[0020] In some embodiments, the preset range can be selected or set according to the actual scenario, and specific limitations thereto are not provided herein. For example, the preset range can be selected or set according to the shooting range of the image capturing device in the first target vehicle and / or the second target vehicle, or according to the sensing range of the Light Laser Detection and Ranging (LiDAR) in the first target vehicle and / or the second target vehicle, or can also be selected or set according to the shooting range of the image capturing device and / or the sensing range of the LiDAR of other vehicles in the traffic participant information.

[0021] In some exemplary embodiments, before acquiring the multimodal data of the first target vehicle, it may further include: optically encoding the driving intention of the first target vehicle through a vehicle light signal encoding table, and controlling the vehicle lights of the first target vehicle according to the optical encoding result of the driving intention of the first target vehicle. It can be seen therefrom that before acquiring the multimodal data of the first target vehicle, the vehicle lights are controlled according to the driving intention of the first target vehicle, so that the subsequent acquired vehicle light image information of the first target vehicle is vehicle light image information associated with the driving intention.

[0022] In some exemplary embodiments, the process of obtaining multimodal data of a first target vehicle may include: capturing the vehicle lights of the first target vehicle through an image capturing device determined in advance or in real time to obtain the vehicle light image information of the first target vehicle. Among them, the image capturing device includes, but is not limited to, an in-vehicle camera. For example, infrared images or RGB images corresponding to vehicle lights such as high beams, tail lights, and turn signals in the first target vehicle can be captured by the in-vehicle camera. And, the point cloud data of the first target vehicle is collected through a lidar determined in advance or in real time, and the vehicle position information of the first target vehicle and the vehicle light area in the first target vehicle are determined based on the point cloud data of the first target vehicle. For example, the three-dimensional position of the first target vehicle can be determined through the point cloud data of the first target vehicle. At the same time, the vehicle light area in the first target vehicle can be determined according to the point cloud density of the vehicle light area. Determining the vehicle light area through point cloud data is also applicable to vehicle lights in occlusion scenarios. The road information, traffic participant information, etc. can also be captured through the image capturing device, and the point cloud data of the road information, traffic participant information, etc. can be obtained through the lidar, so as to obtain environmental information such as road information and traffic participant information through the image capturing device and the lidar in the first target vehicle. Among them, the environmental information may also include vehicle control information such as the steering wheel angle of the first target vehicle, the throttle opening degree of the first target vehicle, and the brake opening degree of the first target vehicle. The traffic participant information may include other vehicles, people, obstacles, etc. within a preset range. Among them, when obtaining the multimodal data of the first target vehicle, an augmented reality (AR) device can also be introduced for auxiliary recognition. The specific process of the augmented reality device for auxiliary recognition can refer to related technologies and will not be elaborated here.

[0023] In some exemplary embodiments, after obtaining the multimodal data of the first target vehicle, it may further include: preprocessing the multimodal data, including: filtering the vehicle light image information to filter out the noise in the vehicle light image information in rain, fog, or at night; and aligning the point cloud data with the image coordinate system of the image capturing device to obtain fusion data formed by the vehicle light image information and the point cloud data; and synchronizing the signals of the multimodal data to align the timestamps of the multimodal data. Specifically, when performing image denoising, Gaussian filtering can be used to eliminate the noise in the vehicle light image information in rain, fog, or low light at night. When performing signal synchronization, the multimodal data can be aligned through timestamps so that the signal error is less than or equal to 10 ms. As an example, the light language feature of the first target vehicle can be determined based on the multimodal data before preprocessing. As another example, the light language feature of the first target vehicle can also be determined based on the multimodal data after preprocessing. As yet another example, the light language feature of the first target vehicle can be determined based on the multimodal data before preprocessing and the multimodal data after preprocessing at the same time.

[0024] In some exemplary embodiments, the process of determining the light language features of a first target vehicle based on multimodal data includes: calculating the headlight brightness change frequency based on the pixel changes of the headlight image information to obtain the flashing frequency of the first target vehicle; and segmenting the headlight image information into a red area and a yellow area according to a preset color space, and obtaining the headlight color of the first target vehicle based on the red area segmentation result and the yellow area segmentation result; wherein the headlight color corresponding to the red area segmentation result is red, and the headlight color corresponding to the yellow area segmentation result is yellow; and predicting the light language intention of the first target vehicle based on the headlight image information within a preset time period and the vehicle movement trajectory of the first target vehicle in a preset continuous time period; and when the headlight in the headlight image information is blocked, predicting the headlight position of the first target vehicle by using the point cloud data of the first target vehicle; and projecting the three-dimensional data frame of the point cloud data onto a two-dimensional image to locate the headlight area of the first target vehicle. Specifically, as an example, the pixel changes of the headlight image information can be determined based on the optical flow method, and then the headlight brightness change frequency can be calculated according to the pixel changes of the headlight image information to obtain the flashing frequency of the first target vehicle; wherein the unit of the flashing frequency can be Hz. As an example, the HSV color space can be used to segment the headlight image information into a red area and a yellow area. The red area segmentation result is used to represent the braking vehicle, and the corresponding headlight color is red; the yellow area segmentation result is used to represent the turn signal, and the corresponding headlight color is yellow. As an example, the preset time period can be selected or set according to the actual scenario, and no specific numerical limit is provided here. For example, the preset time period can be determined according to the headlight flashing frequency. As an example, when the headlight in the headlight image information is blocked, for example, the headlight in the headlight image captured by the on-vehicle camera is blocked by a tree or the like, the headlight position can be inferred by using the density distribution of the point cloud data, and at the same time, the three-dimensional data frame generated by the point cloud data can be projected onto a two-dimensional image to accurately locate the headlight area. As an example, when determining the light language features, the long short-term memory network (LSTM) can also be used to model the light language signals of consecutive frames to identify dynamic patterns, such as "double flash warning" is intermittent flashing at 2 Hz.

[0025] In some exemplary embodiments, the process of mapping light language features to light language semantic tags may include: defining the meaning of light language according to the vehicle light category, vehicle position relationship, and the number of times the vehicle light blinks; and determining the vehicle light language coding rule based on the pre-determined or real-time determined vehicle light language fields, the coding length of the vehicle light language fields, and the description of the vehicle light language fields, and obtaining the complete vehicle light language coding based on the vehicle light language coding rule; wherein, the vehicle light language fields include signal type, blinking mode, number of blinks, light source direction, color identification, priority, and preset reserved bits; associating the definition of light language meaning, the vehicle light language fields, and the complete vehicle light language coding to form a vehicle light language coding table; and using the vehicle light language coding table to match the light language features and map the light language features to light language semantic tags.

[0026] Specifically, as an example, when defining the meaning of light language according to the vehicle light category, vehicle position relationship, and the number of times the vehicle light blinks, a vehicle light language interpretation table can be obtained, as shown in Table 1 below.

[0027] Table 1 Vehicle Light Language Interpretation Table

[0028] When determining the vehicle light language coding rule based on the pre-determined or real-time determined vehicle light language fields, the coding length of the vehicle light language fields, and the description of the vehicle light language fields, a vehicle light language coding rule table can be obtained, as shown in Table 2 below.

[0029] Table 2 Vehicle Light Language Coding Rule Table

[0030] Based on the vehicle light language coding rule, obtaining the complete vehicle light language coding, and at the same time associating the definition of light language meaning, the vehicle light language fields, and the complete vehicle light language coding to form a vehicle light language coding table, as shown in Table 3 below.

[0031] Table 3 Vehicle Light Language Coding Table

[0032] In some exemplary embodiments, the vehicle control method may further include: controlling the vehicle lights of a second target vehicle according to a light language semantic tag; after the vehicle lights of the second target vehicle are controlled, obtaining multi-modal data of the second target vehicle by a first target vehicle based on the vehicle light control result of the second target vehicle, determining the light language feature of the second target vehicle based on the multi-modal data of the second target vehicle, mapping the light language feature of the second target vehicle to a light language semantic tag by using a vehicle light language coding table, and then the first target vehicle controlling itself according to the light language semantic tag of the second target vehicle, executing or aborting a driving intention corresponding to the light language semantic tag of the first target vehicle; and after the first target vehicle executes or aborts the driving intention corresponding to the light language semantic tag of the first target vehicle, the first target vehicle controls its vehicle lights again, so that after the second target vehicle obtains the multi-modal data of the first target vehicle again, it can confirm whether the first target vehicle executes or aborts its driving intention. As an example, when two vehicles are in an overtaking scenario, the vehicle in front is used as the first target vehicle and the vehicle behind is used as the second target vehicle. When the vehicle behind detects that the vehicle in front is slower than its own speed and overtaking is allowed on the current road, the vehicle behind triggers and generates an overtaking driving intention, and controls the vehicle lights according to the overtaking driving intention, controlling the right turn signal of the vehicle behind to flash at 3 Hz, and at the same time detecting the distance from the vehicle in front and the position of the vehicle behind and to the side by using a lidar. At the same time, the vehicle in front can determine the light language feature of the vehicle behind according to the multi-modal data when the right turn signal of the vehicle behind flashes at 3 Hz, and then the vehicle behind maps the light language feature to a light language semantic tag by using the vehicle light language coding table, and then controls itself according to the light language semantic tag of the vehicle behind, for example, turning on the left turn signal and the hazard lights of the vehicle in front, and synchronously reducing the vehicle speed of the vehicle in front by 10%. The vehicle behind can determine that the vehicle in front allows it to execute the overtaking driving intention according to the vehicle light signals when the vehicle in front turns on the left turn signal and the hazard lights and synchronously reduces the vehicle speed by 10%. At this time, the vehicle behind can increase its speed and maintain the condition that the right turn signal flashes at 3 Hz, the lidar detects the distance from the vehicle in front and the position of the vehicle behind and to the side until the vehicle behind overtakes the vehicle in front and the distance from the vehicle in front is greater than or equal to the safety distance set by the vehicle behind. At this time, the vehicle behind controls the tail lights to display a vehicle light signal indicating that the overtaking is completed (for example, "the tail lights display √") so that the vehicle in front knows that the vehicle behind has completed the overtaking. At the same time, the vehicle in front stops decelerating and resumes to the vehicle speed before deceleration. It can be seen that after the first target vehicle initiates a driving intention through the vehicle lights, the second target vehicle recognizes the vehicle lights of the first target vehicle and then controls its own vehicle lights, so that the first target vehicle can know whether the second target vehicle agrees to execute the corresponding driving intention according to the vehicle light control result of the second target vehicle, so that the first target vehicle and the second target vehicle can achieve cooperative control through the vehicle lights.

[0033] In some exemplary embodiments, the vehicle control method may further include: controlling the vehicle lights of a second target vehicle according to a light language semantic tag; after the second target vehicle completes the control of the vehicle lights and speed, obtaining multi-modal data of the second target vehicle by a first target vehicle based on the vehicle light control result and speed control result of the second target vehicle, determining the light language feature of the second target vehicle based on the multi-modal data of the second target vehicle, and mapping the light language feature of the second target vehicle to a light language semantic tag by using a vehicle light language coding table. Then, the first target vehicle controls itself according to the light language semantic tag of the second target vehicle, and executes or aborts a driving intention corresponding to the light language semantic tag of the first target vehicle; and after the first target vehicle executes or aborts the driving intention corresponding to the light language semantic tag of the first target vehicle, the first target vehicle controls its vehicle lights again, so that after the second target vehicle obtains the multi-modal data of the first target vehicle again, it can confirm whether the first target vehicle executes or aborts its driving intention. It can be seen that after the first target vehicle initiates a driving intention through the vehicle lights, the second target vehicle identifies the vehicle lights of the first target vehicle, and then controls its own vehicle lights and speed, so that the first target vehicle can know whether the second target vehicle agrees to execute the corresponding driving intention according to the vehicle light control result and speed control result of the second target vehicle, so as to realize cooperative control between the first target vehicle and the second target vehicle through the vehicle lights.

[0034] In some exemplary embodiments, the process of controlling a second target vehicle according to a light signal semantic tag may include: displaying the light signal semantic tag through a target display pre-configured in the second target vehicle; wherein, the target display is within the field of view of the driver of the second target vehicle; sending an audio warning to the second target vehicle according to the light signal semantic tag and the occlusion information of the first target vehicle; and, when the distance between the first target vehicle and the second target vehicle is less than a preset distance, performing a braking control on the second target vehicle and controlling the brake lights of the second target vehicle to flash; wherein, the occlusion information is obtained by identifying through an enhanced display device. As an example, if the first target vehicle and the second target vehicle are vehicles including an assisted driving function or an intelligent driving function, when the vehicle activates the assisted driving function or the intelligent driving function, the process of controlling the second target vehicle according to the light signal semantic tag may be to highlight and display the light signal semantic tag of the first target vehicle in the driver's field of view through a HUD (Head Up Display, head-up display, hereinafter referred to as HUD) pre-configured in the second target vehicle. For example, in the HUD, the vehicle with "emergency braking" is marked with a red box. At the same time, an audio warning is sent to the second target vehicle according to the light signal semantic tag and the occlusion information of the first target vehicle. When sending an audio warning to the second target vehicle, a hierarchical prompt may be triggered according to the semantic urgency. For example, a "beep" sound is a normal warning, and a continuous beep is a collision risk warning. Wherein, the occlusion information is obtained by identifying through an enhanced display device. The working principle of the enhanced display device here is the same as that of the enhanced display device in some of the foregoing embodiments, so the process of obtaining the occlusion information by identifying through the enhanced display device will not be described in detail here. When the second target vehicle identifies that the distance from the first target vehicle is less than the preset distance, AEB (Automatic Emergency Braking, automatic emergency braking, hereinafter referred to as AEB) is triggered to perform a braking control on the second target vehicle. Wherein, the preset distance can be selected or set according to the actual scenario, and no specific numerical limit is provided here. For example, the preset distance may be the safety distance of the second target vehicle. At the same time, control the brake lights of the second target vehicle to flash, so that the second target vehicle sends a light signal to the vehicle located behind, that is, when the second target vehicle brakes, the brake lights are synchronously triggered to flash at a set flashing frequency. In addition, the second target vehicle can also display the light signal parsing result of the first target vehicle on the in-vehicle screen and provide options of "confirm" or "ignore" to the user for interactive feedback, so as to prevent accidental touch. It can be seen that to control the second target vehicle, the point cloud data and the image fusion technology can be combined to accurately locate the light signal area of the target vehicle, and then combined with the traffic participant status information to trigger a warning or an intelligent driving action. The light signal parsing result is shown in Table 4 below.

[0035] Table 4 Light Signal Parsing Result

[0036] In some exemplary embodiments, when mapping the light language features to light language semantic labels, semantic classification can also be performed, including: using neural network architectures such as the Transformer architecture to fuse headlight image information and point cloud data. Specifically, the headlight image information is input into a Convolutional Neural Networks (CNN for short), and image features are output through the convolutional neural network; and the point cloud data is input into PointNet, and point cloud features are output through PointNet. Then, the image features and point cloud features are used as the input of the Transformer architecture, and the light language label probability distribution is output through the Transformer architecture. For example, the probability distribution of the light language label of "emergency braking" is 0.92, and the probability distribution of the light language label of "left turn reminder" is 0.05. Finally, the classification confidence threshold is adjusted according to environmental conditions (such as visibility). For example, the probability distribution of the rainy day threshold is increased from 0.8 to 0.9. Among them, the core of the Transformer architecture is based on an encoder-decoder structure, and realizes parallel processing of sequence data and global dependency modeling through the self-attention mechanism. PointNet is a network that directly processes point cloud data and can achieve classification and segmentation of point cloud data.

[0037] In some exemplary embodiments, the vehicle control method may further include: selecting two vehicles from traffic participant information, a first target vehicle, and a second target vehicle, and designating one vehicle as the sending vehicle and the other as the receiving vehicle; using a pre-obtained headlight signal encoding table as the optical encoding interaction protocol for the sending vehicle and the receiving vehicle; mapping, by the sending vehicle, an optical encoding representing the driving intention of the sending vehicle according to the headlight signal encoding rule to obtain optical signal parameters; identifying, by the receiving vehicle, the optical signal parameters and generating a feedback encoding or a warning encoding according to the identification result of the optical signal parameters; parsing, by the sending vehicle, the feedback encoding or the warning encoding and executing the driving intention according to the parsing result of the feedback encoding, or aborting the driving intention according to the parsing result of the warning encoding. Among them, the traffic participant information may include other vehicles, persons, obstacles, etc. within a preset range of the first target vehicle and the second target vehicle. Among them, the process of identifying, by the receiving vehicle, the optical signal parameters and generating a feedback encoding or a warning encoding according to the identification result of the optical signal parameters may include: capturing headlight signals from the optical signal parameters by the receiving vehicle, including extracting the headlight flashing frequency and the headlight color; and verifying, by the receiving vehicle, the point cloud data from the optical signal parameters, including locating the sending vehicle through the point cloud data and verifying the movement trajectory of the sending vehicle; performing multispectral fusion by the receiving vehicle from the optical signal parameters, including collecting visible light and near-infrared band images and filtering out ambient light interference through band differences; generating the feedback encoding or the warning encoding according to the headlight signal capture result, the point cloud data verification result, and the multispectral fusion result. In addition, in some examples, when transmitting encoding instructions such as optical signal parameters, feedback encoding, or warning encoding, the priorities of the sending vehicle and the receiving vehicle may also be obtained, and time window competition may be performed according to the priorities, and the encoding instructions may be transmitted according to the time window competition result; among them, the encoding instructions include but are not limited to optical signal parameters, feedback encoding, or warning encoding. In some other examples, within a preset time window, the vehicle with the highest priority among the traffic participant information, the first target vehicle, and the second target vehicle is designated as the sending vehicle, and the remaining vehicles are designated as the receiving vehicles, and within the preset time window, only the vehicle with the highest priority is allowed to send optical signal parameters.

[0038] In some examples, the sending vehicle and the receiving vehicle can perform cooperative interaction control using the encoded instruction multi-vehicle interaction protocol. For example, basic instruction encoding can be performed according to predefined light signal encoding rules such as the light signal encoding rules in the vehicle light signal encoding table as the encoded instruction multi-vehicle interaction protocol; or the basic instruction encoding can be extended to support dot matrix LED (Light-Emitting Diode) display of dynamic graphics (such as arrows, numbers, etc.), and then the graphic semantics can be recognized through a convolutional neural network as the encoded instruction multi-vehicle interaction protocol.

[0039] As an example, for two adjacent vehicles in the front and rear, the rear vehicle can be used as the sending vehicle and the front vehicle can be used as the receiving vehicle. The process when the sending vehicle and the receiving vehicle perform vehicle cooperative interaction control can be as Figure 2As shown. Specifically, the sending vehicle uses image capture devices, lidar, etc. in the vehicle for environmental perception, and then forms the driving intention of the sending vehicle (such as an overtaking request, etc.) according to the environmental perception result. Among them, the specific process of the sending vehicle forming the driving intention according to the environmental perception result can refer to other related technologies, which will not be elaborated here. For example, when the sending vehicle and the receiving vehicle are both in the second lane, and the environmental perception result of the sending vehicle includes that there are no other vehicles, pedestrians and other objects within 200 meters before and after the parallel position of the sending vehicle in the first lane, at this time, the sending vehicle can form an overtaking request driving intention according to this environmental perception result. Then, the sending vehicle uses the light signal encoding rules in the encoding rule library to map the corresponding driving intention to optical signal parameters, and synchronously controls the vehicle lights according to the optical signal parameters corresponding to the driving intention, such as turning on the left turn signal. Among them, the light signal encoding rules in the encoding rule library are obtained based on the vehicle light signal encoding table, which will not be elaborated here. When the sending vehicle and the receiving vehicle perform vehicle-to-vehicle collaborative interaction control, there may be conflicts in the encoding instruction signals when the two vehicles send optical signal parameters, feedback encoding or warning encoding and other encoding instructions to each other. At this time, time window preemption is required. When performing time window preemption, the current time window can be competed according to the vehicle priority. In addition, within the preset time window, only the vehicle with the highest priority is allowed to send encoding instructions. Among them, the vehicle priority includes but is not limited to vehicle type, lane position, special vehicle's dedicated alarm light sound and light prompt, etc. For example, the priority of an ambulance is higher than that of other ordinary vehicles, and the ambulance can interrupt the current time window. If there is no conflict in the encoding instruction signal, the corresponding vehicle can send the corresponding encoding instruction in turn according to the time window of 100 ms / segment. As an example, the time window preemption mechanism can be as shown in Table 5 below. In Table 5, the priority of vehicle C is greater than that of vehicle A and vehicle B, and the priority of vehicle A is greater than that of vehicle B. When vehicle C interrupts the current time window at 150 ms, vehicle B that originally preempted the 100 ms - 200 ms time window stops sending encoding instructions. At the same time, since each vehicle sends encoding instructions in turn according to the time window of 100 ms / segment, vehicle C occupies the 150 ms - 250 ms time window to send encoding instructions. At this time, vehicle C can be a special vehicle, such as an ambulance; vehicle A and vehicle B can be ordinary vehicles.

[0040] Table 5 Time Window Preemption Mechanism

[0041] The receiving vehicle then performs signal capture, including: taking an image of the headlights of the sending vehicle through an in-vehicle camera, and then extracting headlight features such as the flashing frequency and headlight color of the sending vehicle from the captured headlight image. Then the receiving vehicle performs LiDAR calibration, including: locating the signal source vehicle through LiDAR point cloud data and verifying the motion trajectory of the signal source vehicle; for example, the motion trajectory of the signal source vehicle can be verified by verifying the acceleration and steering angle of the signal source vehicle. By performing point cloud data inspection, when the optical signal of the receiving vehicle is partially blocked, the motion trajectory of the signal source vehicle can be analyzed through point cloud data calibration, and its driving intention can be speculated, so as to identify whether the signal source vehicle and the sending vehicle are the same vehicle. Then the receiving vehicle performs multispectral fusion, including: collecting image features of visible light and near-infrared band images, and then filtering out environmental light interference (such as billboard reflections) through band differences, there is: , where represents the image features after multispectral fusion, represents the image features of the visible light band (400 - 700nm) image, represents the image features of the near-infrared band (800 - 1000nm) image, α represents the dynamic weight, ; and the dynamic weight can be automatically adjusted according to the environmental light intensity. For example, in a night environment, can be 0.3. At this time, the image features of the near-infrared band image can be preferentially used.

[0042] Then the receiving vehicle performs multimodal perception recognition through the image features after multispectral fusion and the verified LiDAR trajectory data; and performs semantic parsing, including: inputting the multimodal perception recognition result into a CNN classification model, and outputting corresponding instruction labels such as light language semantic labels (such as overtaking request) through the CNN classification model; then performing feedback generation, including: generating a feedback code or a warning code after making a decision according to the light language semantic label, and then transmitting the feedback code or the warning code to the sending vehicle. Among them, the process during semantic parsing can be the semantic classification process described in some of the above embodiments, which will not be elaborated here.

[0043] The sending vehicle performs environmental perception again to confirm whether to execute or abort the driving intention according to the environmental perception result. If the environmental perception result meets the execution conditions of the driving intention, the CNN classification model is used to perform semantic parsing on the feedback code or the warning code, and output corresponding instruction labels such as light language semantic labels (such as allowing overtaking, rejecting overtaking). At the same time, the sending vehicle performs corresponding collaborative actions according to the semantic parsing result. For example, when overtaking is allowed, the sending vehicle performs acceleration overtaking control; when overtaking is rejected, the sending vehicle maintains a vehicle distance from the receiving vehicle.

[0044] It can be seen from this that by combining traffic participant information to achieve multi-vehicle two-way collaborative interaction, collaborative decision-making can be carried out in complex scenarios, reducing misjudgment of vehicle driving. And when multi-vehicle collaborative interaction is carried out, if multiple vehicles send light signals simultaneously, the signal conflict problem between vehicles can be solved in the way of vehicle priority. Therefore, in the scenario without V2X (Vehicle to Everything, vehicle wireless communication, hereinafter referred to as V2X) or cellular network, multi-vehicle two-way collaborative interaction can be achieved by combining traffic participant information and light signal recognition.

[0045] In another exemplary embodiment, as Figure 3 shown, a vehicle control method is provided, including the following steps: After the target vehicle activates the assisted driving function or intelligent driving function, the light signal, position information and environmental information of the target vehicle are obtained in real time through the on-vehicle camera and lidar of the target vehicle. The environmental information includes road information, traffic participant information and vehicle control information. The vehicle control information includes steering wheel angle, throttle opening degree and brake opening degree, etc.; at the same time, an augmented reality device is introduced to assist in identifying the blocked signal lights. Among them, the light signal, position information and environmental information of the target vehicle can be obtained from the headlight image captured by the on-vehicle camera and / or the headlight point cloud data formed by the lidar. As an example, for example, the on / off state and on / off time of headlights, taillights and turn signals in the target vehicle can be determined through the headlight image, and the three-dimensional position of the target vehicle can also be determined through the lidar point cloud data. At the same time, then the headlight area or headlight position is determined from the three-dimensional position of the target vehicle according to the headlight area point cloud density. For the headlights in the occlusion scenario, an AR device can be introduced to assist in identifying the blocked signal lights, such as generating a prompt image through an image completion algorithm. Among them, the light signal includes but is not limited to headlight category, on / off state and on / off time of the headlight, headlight area or headlight position, etc. In addition, the process of obtaining the road information and traffic participant information in the environmental information from the headlight image captured by the on-vehicle camera and / or the headlight point cloud data formed by the lidar can refer to some of the above embodiments, and will not be elaborated here.

[0046] Input the camera image obtained by the in-vehicle camera, the lidar point cloud obtained by the lidar, and the prompt image obtained by the AR device into a convolutional neural network, and perform feature extraction through attention mechanisms such as spatial attention mechanism, channel attention mechanism, and spatio-temporal attention mechanism to obtain light language features such as the flashing frequency of vehicle lights, the color of vehicle lights, and the light language intention for representing the trajectory; then perform vehicle light semantic matching according to the light language features, and map the light language features to vehicle light language semantic labels such as emergency braking, left turn lane change, yielding, high beam switching, etc. Moreover, perform feature fusion extraction on the camera image obtained by the in-vehicle camera and the lidar point cloud obtained by the lidar, and then form a vectorized map according to the feature fusion extraction result.

[0047] Combine the vehicle light language semantic labels and the vectorized map to form a general regulation and control framework for the target vehicle; this general regulation and control framework can be used for vehicle prediction, vehicle planning, vehicle control, etc. of the target vehicle.

[0048] In summary, the present application provides a vehicle control method. By obtaining multi-modal data of a first target vehicle, then determining the light language features of the first target vehicle based on the multi-modal data, and then using a pre-obtained vehicle light language coding table to map the light language features to light language semantic labels, and controlling a second target vehicle according to the light language semantic labels, so that the first target vehicle executes or aborts the driving intention corresponding to the light language semantic label of the first target vehicle based on the control result of the second target vehicle; wherein, the multi-modal data includes vehicle light image information, vehicle position information, and environmental information, the environmental information includes road information and traffic participant information, and the light language features include the flashing frequency of vehicle lights, the color of vehicle lights, the vehicle light area, and the light language intention; the first target vehicle and the second target vehicle are within a preset range. It can be seen from this that when performing vehicle control, by obtaining multi-modal data for light language signal recognition, the problem that a single sensor is easily interfered by the environment and misjudged can be solved. At the same time, according to the light language features of the first target vehicle, the driving behavior or driving intention of the first target vehicle can be predicted, so that the second target vehicle within the preset range of the first target vehicle can form a coping strategy in advance, thereby effectively avoiding the occurrence of sudden or high-risk situations and reducing the probability of traffic accidents. Moreover, this method can also combine traffic participant information to achieve multi-vehicle two-way collaborative interaction. When multiple vehicles send light languages simultaneously, the signal conflict problem can be solved in a priority manner, and collaborative decision-making can be achieved in complex scenarios, reducing vehicle driving misjudgment. Therefore, the vehicle control method described in this method can not only realize vehicle collaborative interaction control in traffic scenarios with V2X or cellular networks, but also even in traffic scenarios without V2X or cellular networks, vehicle collaborative interaction control can still be achieved by performing vehicle light control between vehicles.

[0049] In another exemplary embodiment of the present application, as Figure 4As shown in the figure, this embodiment also provides a vehicle control system, including: A data acquisition module 410, configured to obtain multimodal data of a first target vehicle, where the multimodal data includes headlight image information, vehicle position information, and environmental information, and the environmental information includes road information and traffic participant information; A light signal feature module 420, configured to determine the light signal features of the first target vehicle according to the multimodal data, where the light signal features include headlight flashing frequency, headlight color, headlight area, and light signal intention; A vehicle control module 430, configured to map the light signal features to light signal semantic labels, and control a second target vehicle according to the light signal semantic labels; where the first target vehicle and the second target vehicle are within a preset range.

[0050] It can be understood that the vehicle control system provided in the above embodiment and the vehicle control method provided in the above embodiment belong to the same concept. The specific manner of performing operations in the vehicle control method has been described in detail in the above embodiment, and will not be elaborated here. In practical applications, the vehicle control system provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the vehicle control system into different functional modules, and then implement all or part of the functions of the corresponding functional modules through the vehicle control method described in the above embodiment. For example, all or part of the functions of the data acquisition module 410 can be implemented or executed through the relevant step processes of step S110, all or part of the functions of the light signal feature module 420 can be implemented or executed through the relevant step processes of step S120, and all or part of the functions of the vehicle control module 430 can be implemented or executed through the relevant step processes of step S130. For the specific implementation or execution process, refer to the above embodiment, and no specific elaboration will be made here.

[0051] Therefore, the present application provides a vehicle control system, which obtains multimodal data of a first target vehicle through a data acquisition module, and then determines the light language features of the first target vehicle based on the multimodal data through a light language feature module, and then uses a pre-obtained light language coding table of the vehicle light through a vehicle control module to map the light language features into light language semantic labels, and controls the second target vehicle according to the light language semantic labels, so that the first target vehicle executes or terminates the driving intention corresponding to the light language semantic label of the first target vehicle based on the control result of the second target vehicle; wherein, the multimodal data includes headlight image information, vehicle position information and environmental information, the environmental information includes road information and traffic participant information, and the light language features include headlight flashing frequency, headlight color, headlight area and light language intention; the first target vehicle and the second target vehicle are located within a preset range. As can be seen, when controlling vehicles, this system acquires multimodal data to identify light signals, resolving the problem of single sensors being susceptible to environmental interference and resulting in misjudgments. Furthermore, based on the light signal characteristics of the first target vehicle, the system can predict the first target vehicle's driving behavior or intention, allowing a response strategy to be formed in advance for a second target vehicle within a preset range of the first target vehicle, effectively avoiding sudden or high-risk situations and reducing the probability of traffic accidents. Furthermore, this system can also combine information from traffic participants to achieve two-way collaborative interaction among multiple vehicles. When multiple vehicles simultaneously transmit light signals, signal conflicts can be resolved based on priority, enabling collaborative decision-making in complex scenarios and reducing vehicle driving misjudgments. Therefore, the vehicle control method described in this system can achieve collaborative interactive control of vehicles not only in traffic scenarios with V2X or cellular networks, but also in traffic scenarios without V2X or cellular networks, by controlling vehicle lights between vehicles.

[0052] In another exemplary embodiment of the present application, a computer device is further provided. The computer device may include a memory, a processor, and a computer program stored in the memory. The processor executes the computer program so that the computer device performs Figure 1 The steps of the vehicle control method. Figure 5 FIG1 shows a schematic diagram of the structure of a computer device 1000. Figure 5 As shown, the computer device 1000 includes: a processor 1010 , a memory 1020 , a power supply 1030 , a display unit 1040 , and an input unit 1060 .

[0053] The processor 1010 is the control center of the computer device 1000. It connects various components using various interfaces and lines, and performs various functions of the computer device 1000 by running or executing computer programs / instructions stored in the memory 1020, thereby monitoring the computer device 1000 as a whole. In the embodiment of the present application, when the processor 1010 calls the computer program stored in the memory 1020, it executes the following Figure 1 The steps of the vehicle control method are as follows. Optionally, processor 1010 may include one or more processing units; preferably, processor 1010 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and applications, and the modem processor primarily processes wireless communications. In some embodiments, the processor and memory may be implemented on a single chip; in some embodiments, they may also be implemented on separate chips.

[0054] The memory 1020 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, various applications, and the like; the data storage area may store instruction data and the like generated based on the use of the computer device 1000. Furthermore, the memory 1020 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0055] The computer device 1000 also includes a power supply 1030 (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 1010 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0056] The display unit 1040 can be used to display information input by the user or information provided to the user, as well as various menus of the computer device 1000. In the embodiment of the present application, it is mainly used to display the display interface of each application in the computer device 1000 and objects such as text and images displayed on the display interface. The display unit 1040 may include a display panel 1050. The display panel 1050 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0057] The input unit 1060 can be used to receive information such as numbers or characters input by the user. The input unit 1060 can include a touch panel 1070 and other input devices 1080. Among them, the touch panel 1070, also known as a touch screen, can collect touch operations of the user on or near it (such as operations of the user using any suitable object or accessory such as a finger, a stylus, etc. on or near the touch panel 1070).

[0058] Specifically, the touch panel 1070 can detect the touch operation of the user, detect the signals brought by the touch operation, convert these signals into contact coordinates, send them to the processor 1010, and receive and execute the commands sent by the processor 1010. In addition, the touch panel 1070 can be implemented in multiple types such as resistive, capacitive, infrared, and surface acoustic wave. The other input devices 1080 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0059] Of course, the touch panel 1070 can cover the display panel 1050. After the touch panel 1070 detects a touch operation on or near it, it is transmitted to the processor 1010 to determine the type of touch event. Subsequently, the processor 1010 provides a corresponding visual output on the display panel 1050 according to the type of touch event. Although in Figure 5 the touch panel 1070 and the display panel 1050 are implemented as two independent components to realize the input and output functions of the computer device 1000, in some embodiments, the touch panel 1070 and the display panel 1050 can be integrated to realize the input and output functions of the computer device 1000.

[0060] The computer device 1000 can also include one or more sensors, such as a pressure sensor, a gravitational acceleration sensor, a proximity light sensor, etc. Of course, according to the needs in specific applications, the above computer device 1000 can also include other components such as a camera.

[0061] In another exemplary embodiment of the present application, a computer-readable storage medium is also provided. The computer program / instructions are stored in the storage medium. When the computer program / instructions are executed by the processor, the above device can execute the steps of the vehicle control method as described in Figure 1 this application.

[0062] Those skilled in the art can understand that Figure 5The computer device is only an example and does not limit the device. The device may include more or fewer components than shown in the figure, or combine some components, or different components. For the convenience of description, the above parts are divided into various modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of the various modules (or units) can be implemented in the same or multiple software or hardware. For example, as some examples, the aforementioned computer device may be a vehicle, a vehicle-mounted computer, etc.

[0063] Those skilled in the art can understand that this application can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be applied to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, so that the instructions executed by the processors of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0064] It can be understood that although terms such as first and second may be used to describe target vehicles, etc. in the embodiments of this application, these terms are only used to distinguish target vehicles from each other. For example, without departing from the scope of the embodiments of this application, the first target vehicle can also be called the second target vehicle, and similarly, the second target vehicle can also be called the first target vehicle.

[0065] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A vehicle control method, characterized in that, The method includes the following steps: Obtain multimodal data of a first target vehicle, where the multimodal data includes headlight image information, vehicle position information, and environmental information, and the environmental information includes road information and traffic participant information; Determine the light language features of the first target vehicle based on the multimodal data, where the light language features include headlight flashing frequency, headlight color, headlight area, and light language intention; Use a pre-obtained headlight light language coding table to map the light language features to light language semantic labels, and control a second target vehicle according to the light language semantic labels, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the control result of the second target vehicle; where the first target vehicle and the second target vehicle are within a preset range.

2. The vehicle control method according to claim 1, characterized in that The process of controlling the second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the control result of the second target vehicle includes: Control the headlights of the second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the headlight control result of the second target vehicle; Or, Control the headlights and vehicle speed of the second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the headlight and vehicle speed control results of the second target vehicle.

3. The vehicle control method according to claim 1, wherein The process of controlling the second target vehicle according to the light language semantic label, so that the first target vehicle executes or aborts a driving intention corresponding to the light language semantic label based on the control result of the second target vehicle includes: Use the headlight light language coding table as the optical coding interaction protocol between the sending vehicle and the receiving vehicle; where the sending vehicle and the receiving vehicle are determined from the traffic participant information, the first target vehicle, and the second target vehicle; Identify optical signal parameters through the receiving vehicle, and generate a feedback coding or a warning coding according to the identification result of the optical signal parameters; where the optical signal parameters are obtained by the sending vehicle mapping an optical coding representing the driving intention of the sending vehicle according to the headlight light language coding rule; Parse the feedback coding or the warning coding through the sending vehicle, and execute the driving intention according to the parsing result of the feedback coding, or abort the driving intention according to the parsing result of the warning coding.

4. The vehicle control method according to claim 3, characterized in that, The process of identifying optical signal parameters through the receiving vehicle, and generating a feedback coding or a warning coding according to the identification result of the optical signal parameters includes: Capture headlight signals from the optical signal parameters through the receiving vehicle, including extracting the headlight flashing frequency and the headlight color; and, Verify the point cloud data from the optical signal parameters through the receiving vehicle, including locating the sending vehicle through the point cloud data and verifying the movement trajectory of the sending vehicle. Multi - spectral fusion is performed from the optical signal parameters by the receiving - end vehicle, including collecting visible - light and near - infrared - band images and filtering ambient light through band differences; The feedback code or the warning code is generated according to the headlight signal capture result, the point - cloud data verification result, and the multi - spectral fusion result.

5. The vehicle control method according to claim 3 or 4, characterized in that The method further includes: competing for a time window according to the priorities of the sending - end vehicle and the receiving - end vehicle, and transmitting a coding instruction according to the time - window competition result; wherein, the coding instruction includes: the optical signal parameters, the feedback code or the warning code; And / or, within a preset time window, the vehicle with the highest priority among the traffic participant information, the first target vehicle, and the second target vehicle is used as the sending - end vehicle, and the remaining vehicles are used as receiving - end vehicles, and only the vehicle with the highest priority is allowed to send the optical signal parameters.

6. The vehicle control method according to claim 1, wherein The process of obtaining the headlight signal - language coding table includes: Defining the meaning of the signal language according to the headlight category, the vehicle position relationship, and the number of headlight flashes; and, Determining the headlight signal - language coding rule according to the pre - determined or real - time determined headlight signal - language field, the coding length of the headlight signal - language field, and the description of the headlight signal - language field, and obtaining the complete headlight signal - language code based on the headlight signal - language coding rule; wherein, the headlight signal - language field includes signal type, flash mode, number of flashes, light - source direction, color identifier, priority, and preset reserved bits; Associating the definition of the signal - language meaning, the headlight signal - language field, and the complete headlight signal - language code to form the headlight signal - language coding table.

7. The vehicle control method according to claim 1 or 6, characterized in that The process of controlling the second target vehicle according to the signal - language semantic label includes: Displaying the signal - language semantic label through a target display pre - configured in the second target vehicle; wherein, the target display is within the field of view of the driver of the second target vehicle; Sending an audio warning to the second target vehicle according to the signal - language semantic label and the occlusion information of the first target vehicle; and when the distance between the first target vehicle and the second target vehicle is less than a preset distance, performing a braking control on the second target vehicle and controlling the brake lights of the second target vehicle to flash; wherein, the occlusion information is identified by an enhanced display device.

8. The vehicle control method according to claim 1, wherein The process of determining the signal - language feature of the first target vehicle based on the multi - modal data includes: Calculating the headlight brightness change frequency based on the pixel change of the headlight image information to obtain the flashing frequency of the first target vehicle; and, Segmenting the headlight image information into a red area and a yellow area according to a preset color space, and obtaining the headlight color of the first target vehicle based on the red - area segmentation result and the yellow - area segmentation result; wherein, the headlight color corresponding to the red - area segmentation result is red, and the headlight color corresponding to the yellow - area segmentation result is yellow; and, Predicting the signal - language intention of the first target vehicle based on the headlight image information within a preset time period and the vehicle movement trajectory of the first target vehicle within a preset continuous time period; and, When the vehicle lamp in the vehicle lamp image information is blocked, predicting the position of the vehicle lamp of the first target vehicle by using the point cloud data of the first target vehicle; and, Projecting the three-dimensional data frame of the point cloud data into a two-dimensional image to locate the vehicle lamp area of the first target vehicle.

9. A vehicle control system, characterized in that, The system includes: A data acquisition module for acquiring multi-modal data of a first target vehicle, the multi-modal data including vehicle lamp image information, vehicle position information, and environmental information, and the environmental information including road information and traffic participant information; A lamp language feature module for determining the lamp language features of the first target vehicle according to the multi-modal data, the lamp language features including the vehicle lamp flashing frequency, the vehicle lamp color, the vehicle lamp area, and the lamp language intention; A vehicle control module for mapping the lamp language features to lamp language semantic labels by using a pre-obtained vehicle lamp language coding table, and controlling a second target vehicle according to the lamp language semantic labels, so that the first target vehicle executes or aborts a driving intention corresponding to the lamp language semantic label based on the control result of the second target vehicle; wherein, the first target vehicle and the second target vehicle are within a preset range.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory, and the processor executes the computer program to implement the steps of the vehicle control method according to any one of claims 1 to 7.

Images