Vehicle automatic control method and system, electronic equipment and storage medium

Through multi-sensor fusion technology and adaptive weighting algorithm, combined with V2V communication, the intelligent wiper and driving speed control of the vehicle in extreme weather is realized, solving the problems of insufficient safety and traffic efficiency of vehicles in the existing technology, and improving the Traffic safety and efficiency in extreme weather.

CN119058720APending Publication Date: 2024-12-03CHINA FAW CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411152161.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing vehicle autonomous driving technology has insufficient safety and traffic efficiency in extreme weather such as rain and snow, and it has failed to effectively combine wipers and vehicle networking technology, resulting in an increase in the risk of traffic safety accidents.

Method used

Using multi-sensor fusion technology, the rainfall information is obtained through visual sensors and infrared rainfall sensors, and the adaptive weighted fusion algorithm is used to perform accurate measurements, control the wipers and vehicle driving speed, and combine V2V communication technology to transmit information to rear vehicles to realize intelligent rain-day driving decisions. .

Benefits of technology

It improves the driving safety and traffic efficiency of vehicles in extreme weather, reduces the occurrence of traffic accidents, and ensures the orderly operation of vehicles and traffic flows.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119058720A_ABST
    Figure CN119058720A_ABST
Patent Text Reader

Abstract

The invention discloses a vehicle automatic control method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining the first image information of a front windshield of a current vehicle, and determining the first rainfall information of the front windshield according to the first image information; acquiring environment illumination information of the current vehicle, and acquiring second rainfall information of a front windshield through an infrared rainfall sensor; performing adaptive weighted fusion on the first rainfall information and the second rainfall information according to the environment illumination information to obtain target rainfall information; and according to the target rainfall information, a windshield wiper of the current vehicle is controlled, the running speed of the current vehicle is adjusted, and then the target rainfall information and the adjusted running speed are sent to a rear vehicle. The vehicle driving safety and the traffic efficiency are improved, and the method can be applied to the technical field of vehicle control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle control technology, and in particular to a vehicle automatic control method, system, electronic equipment and storage medium. Background Art

[0002] At this stage, autonomous driving and vehicle networking technologies are becoming more and more mature, and the safety issues of vehicle autonomous driving are particularly important. Most of the existing technologies only consider the autonomous driving of vehicles in normal weather, but traffic accidents are more likely to occur in extreme weather such as rain and snow. In addition, the existing vehicle autonomous driving only controls the speed and gear of the vehicle, and does not take into account the actual situation that the wipers need to be turned on in rainy days. It is also not combined with vehicle networking technology, which has certain limitations and affects the safety of vehicle driving and traffic efficiency. The above problems need to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to solve one of the technical problems existing in the prior art to at least a certain extent.

[0004] To this end, an object of an embodiment of the present invention is to provide a vehicle automatic control method, which improves the safety of vehicle driving and traffic efficiency.

[0005] Another object of an embodiment of the present invention is to provide a vehicle automatic control system.

[0006] In order to achieve the above technical objectives, the technical solutions adopted by the embodiments of the present invention include:

[0007] In one aspect, an embodiment of the present invention provides a vehicle automatic control method, comprising the following steps:

[0008] Acquire first image information of a front windshield of a current vehicle, and determine first rainfall information of the front windshield according to the first image information;

[0009] Acquire the ambient light information of the current vehicle, and acquire the second rainfall information of the front windshield through an infrared rainfall sensor;

[0010] Adaptively weighting and fusing the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information;

[0011] The windshield wipers of the current vehicle are controlled according to the target rainfall information, and the driving speed of the current vehicle is adjusted, and then the target rainfall information and the adjusted driving speed are sent to the rear vehicle.

[0012] Further, in one embodiment of the present invention, the acquiring of first image information of a front windshield of the current vehicle and determining first rainfall information of the front windshield according to the first image information specifically includes:

[0013] Acquiring the first image information by using a camera installed in the cockpit of the current vehicle;

[0014] Performing image enhancement and normalization processing on the first image information to obtain second image information of a preset pixel size;

[0015] The second image information is input into a pre-trained raindrop recognition model to obtain target pixel points containing raindrops in the second image information, and then the first rainfall information is determined according to the pixel ratio of the target pixel points in the second image information.

[0016] Furthermore, in one embodiment of the present invention, the raindrop recognition model is pre-trained by the following steps:

[0017] Acquire a plurality of preset raindrop recognition sample images, and determine a pixel label matrix of each of the raindrop recognition sample images;

[0018] Generate a training data set according to the raindrop recognition sample image and the pixel label matrix;

[0019] Inputting the training data set into a pre-built bilateral semantic segmentation network for training to obtain the trained raindrop recognition model;

[0020] Among them, the bilateral semantic segmentation network includes a detail branch network, a dual attention module, a semantic branch network, a feature fusion module and a segmentation module.

[0021] Further, in one embodiment of the present invention, the step of inputting the training data set into a pre-built bilateral semantic segmentation network for training specifically includes:

[0022] Extracting features of the raindrop recognition sample image through the detail branch network to obtain a first spatial feature;

[0023] Calculating a channel attention feature map and a spatial attention feature map corresponding to the first spatial feature through the dual attention module, and then fusing the channel attention feature map and the spatial attention feature map to obtain a second spatial feature;

[0024] Extracting features of the raindrop recognition sample image through the semantic branch network to obtain a first semantic feature;

[0025] Performing feature fusion on the second spatial feature and the first semantic feature through a feature fusion module to obtain a target fusion feature;

[0026] Determining a predicted semantic segmentation image according to the target fusion features by the segmentation module;

[0027] A loss value is determined according to the predicted semantic segmentation image and the pixel label matrix, and the parameters of the bilateral semantic segmentation network are updated according to the loss value to obtain the trained raindrop recognition model.

[0028] Further, in an embodiment of the present invention, the step of adaptively weighting and fusing the first rainfall information and the second rainfall information according to the ambient light information to obtain the target rainfall information specifically includes:

[0029] Establishing a fuzzy control model of ambient light and a first weight and a second weight, wherein the first weight is a reliability weight of the first rainfall information, and the second weight is a reliability weight of the second rainfall information;

[0030] Determine the first weight and the second weight according to the fuzzy control model and the ambient lighting information;

[0031] The first rainfall information and the second rainfall information are weightedly summed according to the first weight and the second weight to obtain the target rainfall information.

[0032] Further, in one embodiment of the present invention, the windshield wipers of the current vehicle are controlled according to the target rainfall information, and the driving speed of the current vehicle is adjusted, and then the target rainfall information and the adjusted driving speed are sent to the rear vehicle, which specifically includes:

[0033] Determining a current rainfall level according to the target rainfall information, and determining a target wiper gear position and a target vehicle speed threshold according to the current rainfall level;

[0034] controlling the wiper according to the target wiper gear, and adjusting the driving speed according to the target vehicle speed threshold;

[0035] The target rainfall information and the adjusted driving speed are sent to the rear vehicle through V2V communication technology.

[0036] Furthermore, in one embodiment of the present invention, the vehicle automatic control method further comprises the following steps:

[0037] When the current rainfall level changes, determining whether the current vehicle performs windshield wiper control and driving speed adjustment;

[0038] If the current vehicle does not perform windshield wiper control and / or driving speed adjustment, obtain the real-time vehicle condition data of the current vehicle, and upload the real-time vehicle condition data to the traffic platform.

[0039] On the other hand, an embodiment of the present invention provides a vehicle automatic control system, including:

[0040] A first information acquisition module, used to acquire first image information of a front windshield of a current vehicle, and determine first rainfall information of the front windshield according to the first image information;

[0041] A second information acquisition module is used to acquire the ambient light information of the current vehicle and acquire the second rainfall information of the front windshield through an infrared rainfall sensor;

[0042] an information fusion module, configured to perform adaptive weighted fusion on the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information;

[0043] The vehicle control module is used to control the wipers of the current vehicle according to the target rainfall information and adjust the driving speed of the current vehicle, and then send the target rainfall information and the adjusted driving speed to the rear vehicle.

[0044] On the other hand, an embodiment of the present invention provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein the program, when executed by the processor, realizes the vehicle automatic control method as described above.

[0045] On the other hand, an embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the vehicle automatic control method as described above.

[0046] The advantages and beneficial effects of the present invention will be partly given in the following description, partly become apparent from the following description, or be understood through the practice of the present invention:

[0047] The embodiment of the present invention obtains first image information of the front windshield of the current vehicle, determines first rainfall information of the front windshield according to the first image information, obtains ambient light information of the current vehicle, obtains second rainfall information of the front windshield through an infrared rainfall sensor, performs adaptive weighted fusion on the first rainfall information and the second rainfall information according to the ambient light information, obtains target rainfall information, controls the wiper of the current vehicle according to the target rainfall information, and adjusts the driving speed of the current vehicle, and then sends the target rainfall information and the adjusted driving speed to the rear vehicle. The embodiment of the present invention adopts multi-sensor fusion technology, performs adaptive weighted fusion on the rainfall information obtained based on image recognition and the rainfall information obtained by the infrared rainfall sensor according to the ambient light information, realizes accurate measurement of rainfall, thereby accurately controlling the wiper and adjusting the driving speed of the vehicle, improving the driving safety of the vehicle, and transmitting the rainfall information and the driving speed of the vehicle to the rear vehicle at the same time, so that the rear vehicle can make a quick decision according to the acquired rainfall information and the speed of the front vehicle, further improving the driving safety of the vehicle, and also improving the traffic efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solution in the embodiments of the present invention, the following introduction is made to the drawings required for use in the embodiments of the present invention. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solution of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A flow chart of the steps of the vehicle automatic control method provided by an embodiment of the present invention;

[0050] Figure 2 A step flow chart of step S101 provided in an embodiment of the present invention;

[0051] Figure 3 A flowchart of the steps of pre-training a raindrop recognition model provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of a bilateral semantic segmentation network provided by an embodiment of the present invention;

[0053] Figure 5 A step flow chart of step S203 provided in an embodiment of the present invention;

[0054] Figure 6 A step flow chart of step S103 provided in an embodiment of the present invention;

[0055] Figure 7A step flow chart of step S104 provided in an embodiment of the present invention;

[0056] Figure 8 Another step flow chart of the vehicle automatic control method provided by an embodiment of the present invention;

[0057] Fig. 9 A schematic diagram of instructions for the vehicle automatic control method provided by an embodiment of the present invention;

[0058] Fig.10 A schematic diagram of the structure of a vehicle automatic control system provided by an embodiment of the present invention;

[0059] Fig.11 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention;

[0060] Fig.12 A schematic diagram of the structure of a storage medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limitations on the present application. It should be noted that, although the functional module division is performed in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order from the module division in the system schematic diagram or the flow chart. For the step numbers in the following embodiments, they are only set for the convenience of explanation, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0062] In the description of the present invention, the meaning of "a plurality" is two or more. If there is a description of the first or the second, it is only for the purpose of distinguishing the technical features, and it cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features. In addition, unless otherwise defined, all technical and scientific terms used in this document have the same meaning as those commonly understood by technicians in the technical field of this application. The terms used in this document are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0063] At present, the technology of autonomous driving and Internet of Vehicles is becoming more and more mature, and the safety of vehicle autonomous driving is particularly important. Most of the existing technologies only consider the autonomous driving of vehicles in normal weather, but traffic accidents are more likely to occur in extreme weather such as rain and snow. In addition, the existing automatic driving of vehicles only controls the speed and gear of the vehicle, and does not take into account the actual situation that the wipers need to be turned on in rainy days, nor does it combine with Internet of Vehicles technology. There are certain limitations, which affect the safety of vehicle driving and traffic efficiency.

[0064] The present invention proposes a vehicle control method that can be used in rainy weather environments, which not only takes into account the control of vehicle acceleration and deceleration due to weather factors, but also takes into account the intelligent control of wipers. At present, traditional wipers are mainly controlled manually. Drivers manually switch on and off in rainy and snowy days, and adjust the wiper gear, which easily leads to distraction. The present invention recognizes the amount of rainfall, intelligently and automatically turns on and adjusts to the corresponding gear, and automatically adjusts the appropriate vehicle speed according to the rainfall level to achieve vehicle decision control. At the same time, combined with V2V communication technology, various vehicle decision behaviors are transmitted to the rear vehicles in a timely manner, so that the rear vehicles can make appropriate gear speed and wiper mode adjustments, reduce the calculation amount of the rear vehicles, and prevent accidents caused by the failure of the rear vehicles to reduce their speed in time due to the sudden deceleration of the front vehicles, so as to achieve orderly automatic driving traffic flow.

[0065] In extreme weather, the windshield is blurry, and the driver manually turns on the wipers, which can easily lead to distraction. In addition, the current automatic driving system is not capable of processing information in complex situations, which can easily cause traffic accidents. The present invention proposes a method for intelligently controlling wipers and vehicle speed, which identifies the amount of rain based on a convolutional neural network, fully considers the problem of inaccurate rain identification caused by changes in light intensity during the day and night, and uses multi-sensor fusion technology combined with an adaptive weighted algorithm to achieve control of vehicle wipers, and control the acceleration and deceleration of the vehicle according to the identified rainfall level. At the same time, combined with V2V communication technology, information is transmitted to the vehicle behind, and through communication between vehicles, each vehicle has better decision-making control to reduce accidents.

[0066] The vehicle automatic control method provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a set-top box, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle automatic control method, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0068] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0069] The embodiment of the present invention adopts multi-sensor fusion technology, through the visual sensor, infrared rain sensor and light sensor installed in the vehicle, combined with the weighted algorithm and image recognition algorithm, to identify the amount of rain, open the wipers of different gears according to the different amounts of rain, and control the vehicle to accelerate, decelerate, brake, etc. Synchronously send the corresponding information to the rear vehicle to avoid collisions caused by inaccurate radar detection due to rainy and snowy weather, deceleration or emergency braking of the front vehicle.

[0070] like Figure 1 FIG. 1 is a flow chart showing a method for automatically controlling a vehicle according to an embodiment of the present invention. Figure 1 The embodiment of the present invention provides a vehicle automatic control method, which specifically includes the following steps:

[0071] S101. Acquire first image information of a front windshield of a current vehicle, and determine first rainfall information of the front windshield according to the first image information.

[0072] Specifically, the embodiment of the present invention collects image information of the front windshield from inside the vehicle through a visual sensor, and then determines first rainfall information of the front windshield based on image recognition technology.

[0073] like Figure 2 FIG. 1 is a flowchart of step S101 provided in an embodiment of the present invention, referring to FIG. Figure 2 As an optional implementation, first image information of the front windshield of the current vehicle is obtained, and first rainfall information of the front windshield is determined according to the first image information, which specifically includes:

[0074] S1011, acquiring first image information through a camera installed in a cockpit of the current vehicle;

[0075] S1012, performing image enhancement and normalization processing on the first image information to obtain second image information of a preset pixel size;

[0076] S1013: input the second image information into a pre-trained raindrop recognition model to obtain target pixel points containing raindrops in the second image information, and then determine the first rainfall information according to the pixel ratio of the target pixel points in the second image information.

[0077] Specifically, the data collected by the camera device is preprocessed, including image enhancement, flipping, normalization and standardization processing, to form a 512X512 pixel image, which is then input into a pre-trained raindrop recognition model for recognition, and the first rainfall information is determined according to the number of pixels occupied by raindrops.

[0078] like Figure 3FIG. 1 is a flowchart of a step of pre-training a raindrop recognition model provided by an embodiment of the present invention, referring to FIG. Figure 3 , further as an optional implementation, the raindrop recognition model is pre-trained by the following steps:

[0079] S201, obtaining a plurality of preset raindrop recognition sample images, and determining a pixel label matrix of each raindrop recognition sample image;

[0080] S202, generating a training data set according to the raindrop recognition sample image and the pixel label matrix;

[0081] S203, inputting the training data set into a pre-built bilateral semantic segmentation network for training to obtain a trained raindrop recognition model;

[0082] Among them, the bilateral semantic segmentation network includes a detail branch network, a dual attention module, a semantic branch network, a feature fusion module and a segmentation module.

[0083] Specifically, Figure 4 The figure shows a schematic diagram of the structure of the bilateral semantic segmentation network provided by an embodiment of the present invention. The present invention adopts an improved bilateral semantic segmentation algorithm, and adds a dual-dimensional attention mechanism, namely a channel and spatial attention mechanism, to the convolutional layer, thereby improving the learning efficiency of the network. The image is classified at the pixel level, and the data set is divided into a test set and a training set. The semantic segmentation algorithm is used for training, and the operation is continuously iterated to form a trained network to identify the rainfall and judge the rainfall amount. The greater the rainfall, the greater the number of pixels occupied by raindrops. The bilateral semantic segmentation network of the embodiment of the present invention is mainly composed of four parts, namely a detail branch, a semantic branch, a feature fusion module and a segmentation module.

[0084] like Figure 5 FIG. 1 is a flow chart of step S203 provided in an embodiment of the present invention, referring to FIG. Figure 5 As an optional implementation, the training data set is input into a pre-built bilateral semantic segmentation network for training, which specifically includes:

[0085] S2031, extracting features of the raindrop recognition sample image through a detail branch network to obtain a first spatial feature;

[0086] S2032, calculating the channel attention feature map and the spatial attention feature map corresponding to the first spatial feature through the dual attention module, and then fusing the channel attention feature map and the spatial attention feature map to obtain the second spatial feature;

[0087] S2033, extracting features of the raindrop recognition sample image through a semantic branch network to obtain a first semantic feature;

[0088] S2034, performing feature fusion on the second spatial feature and the first semantic feature through a feature fusion module to obtain a target fusion feature;

[0089] S2035, determining a predicted semantic segmentation image according to the target fusion features through a segmentation module;

[0090] S2036. Determine a loss value based on the predicted semantic segmentation image and the pixel label matrix, and update the parameters of the bilateral semantic segmentation network based on the loss value to obtain a trained raindrop recognition model.

[0091] Specifically, the detail branch is used to extract spatial information from the image. It consists of multiple convolutional layers, which can downsample the input image to 1 / 8 of the original image and generate output features through the h-swish activation function. The h-swish activation function has a better optimization effect than the ReLU activation function, and the expression is as follows:

[0092]

[0093] The embodiment of the present invention inserts a dual-dimensional attention module after the detail branch network. This attention mechanism replaces the traditional global average pooling. The steps are as follows: first, the maximum pooling and average pooling operations are applied to each channel of the input feature map to obtain features of the same size as the input feature map, and the h-swish activation function is used to calculate the weights to obtain the channel attention map. The weights in the feature map are adjusted through the neural network, and the weighted formula is used to fuse the obtained weights and the original features to extract the key information in the image. The spatial dimension attention also uses the maximum pooling and average pooling operations on the input features at the spatial position to obtain two spatial attention feature maps. The h-swish function is also used to obtain the weight information of different spatial positions to obtain the spatial attention map, and it is weighted and fused, and global context information is established from the channel and spatial dimensions. The formula is as follows:

[0094] Mc(F)=σ(MLP(AvgPool(F);MaxPool(F)))

[0095] Ms(F)=σ(f 3 × 3 [AvgPool(F);MaxPool(F)])

[0096] Among them, M c (F), M s (F) represents the channel attention parameter matrix and the spatial attention parameter matrix, σ represents the activation function, f represents the convolutional layer, [AvgPool(F); MaxPool(F)] represents the pooled feature map, and MLP represents a neural network algorithm.

[0097] The semantic branch is used to extract high-level semantic information. It is mainly composed of three parts, namely the Stem module, the GE module, and the CE module. The main steps are as follows: After the image is input, it is extracted through the Stem module, that is, through the 3X3 convolution layer. Then it passes through the GE module to perform deep convolution operations to make the model lightweight. Finally, the CE module is used for global average pooling.

[0098] The feature fusion module fuses the feature information extracted from the semantic branch and the detail branch, and then determines the predicted semantic segmentation image based on the target fusion features through the segmentation module. The loss value is determined based on the predicted semantic segmentation image and the pixel label matrix, and the parameters of the bilateral semantic segmentation network are updated based on the loss value to obtain a trained raindrop recognition model.

[0099] S102: Acquire ambient light information of the current vehicle, and acquire second rainfall information of the front windshield through an infrared rainfall sensor.

[0100] Specifically, the data acquisition unit of the embodiment of the present invention includes a visual sensor (camera), a light sensor and an infrared rain sensor. The visual sensor is used to collect image information of the front windshield to identify rainfall information. The infrared rain sensor is used to directly collect rainfall information on the front windshield. Different from the traditional recognition by only a single sensor, the present invention combines multiple sensors and fuses the data collected by multiple sensors, which is more accurate for detecting the amount of rainfall.

[0101] S103 , adaptively weighted fusion the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information.

[0102] like Figure 6 FIG. 1 is a flow chart of step S103 provided in an embodiment of the present invention, referring to FIG. Figure 6 As an optional implementation, the first rainfall information and the second rainfall information are adaptively weighted fused according to the ambient light information to obtain the target rainfall information, which specifically includes:

[0103] S1031, establishing a fuzzy control model of ambient light, a first weight, and a second weight, wherein the first weight is a reliability weight of the first rainfall information, and the second weight is a reliability weight of the second rainfall information;

[0104] S1032, determining a first weight and a second weight according to the fuzzy control model and the ambient light information;

[0105] S1033. Perform weighted summation on the first rainfall information and the second rainfall information according to the first weight and the second weight to obtain target rainfall information.

[0106] Specifically, the embodiment of the present invention fuses the data recognized by the two sensors. During the day, the infrared rain sensor is easily affected by external light sources and cannot recognize accurately; at night, the light is dim and the data collected by the visual sensor is difficult to effectively mark and recognize. Therefore, the embodiment of the present invention adopts an adaptive weighted algorithm to fuse the data of the two sensors to give full play to their respective advantages. When the light intensity is strong, that is, the light intensity is greater than 400 watts / square centimeter, it is regarded as daytime and the visual sensor is mainly used; when the light intensity is weak, that is, the light intensity is less than 400 watts / square centimeter, it is regarded as night and the infrared rain sensor is mainly used. The estimation of rainfall size is achieved through the adaptive weighted algorithm. The weighted average algorithm is as follows:

[0107] The measurement value of sensor i at time k is expressed as x i (k), then the sensor measurement equation is expressed as:

[0108] x i (k) = Z i (k)+ε i (k)

[0109] Among them, Z i (k) represents the estimated value of sensor i at time k, ε i (k) represents noise.

[0110] The n sensors are data-fused, and the weight of sensor i is ω i , the fusion result is:

[0111]

[0112] in,

[0113] The embodiment of the present invention transmits the data recognized by the two sensors to an adaptive weighted algorithm, and can obtain the proportions of different sensors according to the different weight proportions of the camera and the infrared rain sensor, i.e., ω, by establishing a mathematical model of light intensity, rainfall amount and reliability, based on fuzzy control theory, and using the fuzzy control toolbox in MATLAB software. The proportions are then brought into the above formula to determine the rainfall amount, which is divided into four levels: 0, 1, 2, and 3, i.e., no rain, light rain, moderate rain, and heavy rain.

[0114] S104: Controlling the windshield wipers of the current vehicle according to the target rainfall information, and adjusting the driving speed of the current vehicle, and then sending the target rainfall information and the adjusted driving speed to the rear vehicle.

[0115] Specifically, the embodiment of the present invention sends the above information to the decision-making unit and executes functions such as issuing instructions, reminding and controlling vehicles. The present invention not only performs automatic driving control on the entire traffic flow, but also takes into account the situation where the automatic driving system fails and fails to respond in time.

[0116] like Figure 7 FIG. 1 is a flowchart of step S104 provided in an embodiment of the present invention, referring to FIG. Figure 7 As an optional implementation, driving decision information is generated according to the vehicle driving state and the congestion cause, which specifically includes:

[0117] S1041, controlling the wipers of the current vehicle according to the target rainfall information, and adjusting the driving speed of the current vehicle, and then sending the target rainfall information and the adjusted driving speed to the rear vehicle, which specifically includes:

[0118] S1042, determining a current rainfall level according to the target rainfall information, and determining a target wiper gear position and a target vehicle speed threshold according to the current rainfall level;

[0119] S1043, controlling the wiper according to the target wiper gear, and adjusting the driving speed according to the target vehicle speed threshold;

[0120] S1044. Send the target rainfall information and the adjusted driving speed to the rear vehicle through V2V communication technology.

[0121] Specifically, Fig. 9 The figure shows a schematic diagram of instructions for the vehicle automatic control method provided by an embodiment of the present invention. When the decision unit receives a rainfall level of 0, that is, when it is not raining, instruction A is sent through the CAN bus. After the wiper motor receives instruction A, the wiper is placed in the stop position, and other controllers in the vehicle do not perform any control on the vehicle after receiving this instruction, and maintain a normal driving state.

[0122] When the decision unit receives the information that the rainfall level is 1, it means that it is drizzling, and the automatic driving system immediately sends instruction B to control the wipers to the low-frequency gear state. After receiving the message of instruction B, the controllers in the vehicle, such as the gearbox and electronic stability system, immediately make appropriate gear throttle adjustments to control the vehicle to drive safely. When the rainfall level is 1, the automatic driving system controls the vehicle speed to ≤80km / h to prevent inaccurate obstacle recognition due to rainy days and accidents caused by too fast speed.

[0123] When the decision unit receives information that the rainfall level is 2, that is, the state of moderate rain, it immediately issues instruction C to adjust the wipers to the medium-frequency gear, and the automatic driving system controls the vehicle speed to ≤60km / h. Similarly, when the decision unit receives information that the rainfall level is 3, that is, the state of heavy rain, it issues instruction D to adjust the wipers to the high-frequency gear, and the automatic driving system controls the vehicle speed to ≤40km / h. In the above cases, the gear adjustment information of the vehicle is synchronously sent to the rear vehicle so that the rear vehicle can respond in time and make reasonable gear speed adjustments.

[0124] Since the rainfall is constantly changing in actual situations and cannot remain unchanged, the above situations often switch alternately in real scenes. When it is recognized that the current weather state changes from heavy rain to light rain, command B is sent immediately, and the automatic driving system controls the wiper motor to change the wiper from medium frequency to low frequency gear, and controls the vehicle to accelerate in time, rather than maintaining the previous command D state. Similarly, if other different situations are recognized, different instructions are switched to perform corresponding operations. The information on the gear adjustment of this vehicle is sent to the rear vehicle synchronously. Through the message forwarding of each vehicle, the entire traffic flow can be controlled so that the entire traffic flow is continuously adjusted at an appropriate speed, which is not static and can reduce the travel time. The V2V communication technology used in the embodiment of the present invention can also reduce the probability of collision between the two vehicles in the automatic driving scenario because the front vehicle constantly adjusts the gear and speed, and the rear vehicle has no time to operate.

[0125] like Figure 8 FIG. 1 is another flow chart of the steps of the vehicle automatic control method provided by the embodiment of the present invention, referring to FIG. Figure 8 As an optional implementation, the vehicle automatic control method further includes the following steps:

[0126] S105, when the current rainfall level changes, determining whether the current vehicle is to perform windshield wiper control and adjust the driving speed;

[0127] S106: If the current vehicle does not perform windshield wiper control and / or driving speed adjustment, obtain the real-time vehicle condition data of the current vehicle, and upload the real-time vehicle condition data to the traffic platform.

[0128] Specifically, if the autonomous driving system fails and the vehicle does not execute the corresponding command operations according to different rainfall levels, it is divided into the following situations. Since the vehicle does not perform any control when the rainfall level is 0, the autonomous driving failure in this case is not considered.

[0129] When the decision unit receives information about rainfall levels 1, 2, and 3, but the vehicle does not execute the corresponding instructions, it determines that the vehicle's automatic driving system has failed, and immediately sends a message to the traffic police center through the communication unit, including data such as the current vehicle location, license plate number, gear position, throttle status, and the amount of rainfall at the current location. After receiving the vehicle failure message, the backend system of the traffic police center directly performs forced braking to prevent the automatic driving system from being out of control and causing a traffic accident. At the same time, through the information forwarding of each node vehicle, subsequent vehicles can also understand the road conditions and weather conditions ahead in real time, so as to choose a reasonable travel route and save travel time.

[0130] In some optional embodiments, the vehicle communication unit includes a 5G communication module, and the vehicle information can be uploaded to the cloud server through the 5G module, which is convenient for recording and querying vehicle information data and the current rainfall amount at different locations. Through the uploaded vehicle information, the weather conditions at different locations and the complete vehicle traffic flow can be calculated and drawn, which is convenient for management. The data can also be stored to generate a long-term vehicle automatic driving status monitoring curve. Through the information forwarding of multiple vehicle nodes, the purpose of information sharing and real-time adjustment of vehicle speed can be achieved, which has a better control effect on the entire traffic flow and can also save people's time traveling on rainy days.

[0131] The method steps of the embodiment of the present invention are described above. It can be understood that the present invention first uses infrared rain sensors and visual sensors to collect rainfall data, respectively collecting data of no rain, light rain, moderate rain and heavy rain to establish a data set; and uses an improved bilateral semantic segmentation algorithm to identify the pixels in the image to determine the amount of rainfall; then the rainfall levels obtained by the two sensors are combined with the light intensity factor, and adaptively weighted based on fuzzy control theory to finally obtain the rainfall estimate of the two sensors fusion, thereby obtaining 4 rainfall levels; finally, according to the four rainfall levels, the vehicle's automatic driving system is controlled to perform corresponding wiper and acceleration and deceleration operations; while taking into account the automatic In the event of a failure in the automatic driving system, the communication unit is used to contact the background traffic center for forced braking. This operation can reduce the occurrence of autonomous driving traffic accidents. At the same time, combined with V2V communication technology, the gear adjustment information of the vehicle is sent to the rear vehicle so that the rear vehicle can respond in time and make reasonable gear and speed adjustments to prevent the front and rear vehicles from colliding due to the deceleration of the front vehicle and inaccurate obstacle recognition in rainy days. Through the information forwarding of vehicles at each node, subsequent vehicles can also understand the road conditions and weather conditions ahead in real time, so as to achieve the purpose of information sharing and real-time adjustment of vehicle speed, and have a better control effect on the entire traffic flow.

[0132] The embodiment of the present invention adopts multi-sensor fusion technology to perform adaptive weighted fusion of rainfall information obtained based on image recognition and rainfall information obtained by infrared rainfall sensors according to ambient lighting information, thereby achieving accurate measurement of rainfall, so that the wipers can be accurately controlled and the vehicle speed can be adjusted, thereby improving the safety of vehicle driving. At the same time, the rainfall information and the vehicle speed are transmitted to the rear vehicle, so that the rear vehicle can make a quick decision based on the obtained rainfall information and the speed of the front vehicle, thereby further improving the safety of vehicle driving and improving traffic efficiency.

[0133] like Fig.10 FIG. 1 is a schematic diagram of the structure of a vehicle automatic control system provided by an embodiment of the present invention, referring to FIG. Fig.10 , an embodiment of the present invention provides a vehicle automatic control system, including:

[0134] A first information acquisition module, used to acquire first image information of a front windshield of the current vehicle, and determine first rainfall information of the front windshield according to the first image information;

[0135] The second information acquisition module is used to acquire the current ambient light information of the vehicle and acquire the second rainfall information of the front windshield through the infrared rainfall sensor;

[0136] An information fusion module is used to perform adaptive weighted fusion on the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information;

[0137] The vehicle control module is used to control the wipers of the current vehicle according to the target rainfall information and adjust the driving speed of the current vehicle, and then send the target rainfall information and the adjusted driving speed to the rear vehicle.

[0138] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0139] The embodiment of the present invention further provides an electronic device, the electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, and the program is executed by the processor to realize the above-mentioned vehicle automatic control method. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0140] like Fig.11 FIG. 1 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention, referring to FIG. Fig.11 , an embodiment of the present invention provides an electronic device, including:

[0141] The processor 1101 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention;

[0142] The memory 1102 may be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 may store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1102, and the processor 1101 calls and executes the vehicle automatic control method of the embodiment of the present invention;

[0143] Input / output interface 1103, used to implement information input and output;

[0144] The communication interface 1104 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0145] A bus 1105 that transmits information between various components of the device (e.g., the processor 1101, the memory 1102, the input / output interface 1103, and the communication interface 1104);

[0146] The processor 1101 , the memory 1102 , the input / output interface 1103 and the communication interface 1104 are connected to each other in communication within the device via the bus 1105 .

[0147] like Fig.12 FIG. 1 is a schematic diagram of a storage medium according to an embodiment of the present invention. Fig.12 An embodiment of the present invention further provides a storage medium, which is a computer-readable storage medium used for computer-readable storage. The storage medium stores one or more programs 1201, and the one or more programs 1201 can be executed by one or more processors to implement the above-mentioned vehicle automatic control method.

[0148] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The embodiment of the present invention further provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes Figure 1 The method shown.

[0150] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the above-mentioned boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. The optional embodiment is expected, wherein the order of various operations is changed and the sub-operation described as a part of a larger operation is performed independently.

[0151] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified to the contrary, one or more of the above-mentioned functions and / or features can be integrated into a single physical device and / or software module, or one or more functions and / or features can be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0152] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the above methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0153] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0154] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the above-mentioned program is printed, since the above-mentioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or processing in other suitable ways as necessary, and then stored in a computer memory.

[0155] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0156] In the above description of this specification, the description with reference to the terms "one embodiment / example", "another embodiment / example" or "certain embodiments / examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0157] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0158] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A vehicle automatic control method, characterized in that: The following steps are involved: Acquire first image information of a front windshield of a current vehicle, and determine first rainfall information of the front windshield according to the first image information; Acquire the ambient light information of the current vehicle, and acquire the second rainfall information of the front windshield through an infrared rainfall sensor; Adaptively weighting and fusing the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information; The windshield wipers of the current vehicle are controlled according to the target rainfall information, and the driving speed of the current vehicle is adjusted, and then the target rainfall information and the adjusted driving speed are sent to the rear vehicle.

2. A vehicle automatic control method according to claim 1, characterized in that: The step of acquiring first image information of the front windshield of the current vehicle and determining first rainfall information of the front windshield according to the first image information specifically includes: Acquiring the first image information by using a camera installed in the cockpit of the current vehicle; Performing image enhancement and normalization processing on the first image information to obtain second image information of a preset pixel size; The second image information is input into a pre-trained raindrop recognition model to obtain target pixel points containing raindrops in the second image information, and then the first rainfall information is determined according to the pixel ratio of the target pixel points in the second image information.

3. A vehicle automatic control method according to claim 2, characterized in that: The raindrop recognition model is pre-trained by the following steps: Acquire a plurality of preset raindrop recognition sample images, and determine a pixel label matrix of each of the raindrop recognition sample images; Generate a training data set according to the raindrop recognition sample image and the pixel label matrix; Inputting the training data set into a pre-built bilateral semantic segmentation network for training to obtain the trained raindrop recognition model; Among them, the bilateral semantic segmentation network includes a detail branch network, a dual attention module, a semantic branch network, a feature fusion module and a segmentation module.

4. A vehicle automatic control method according to claim 3, characterized in that: The step of inputting the training data set into a pre-built bilateral semantic segmentation network for training specifically includes: Extracting features of the raindrop recognition sample image through the detail branch network to obtain a first spatial feature; Calculating a channel attention feature map and a spatial attention feature map corresponding to the first spatial feature through the dual attention module, and then fusing the channel attention feature map and the spatial attention feature map to obtain a second spatial feature; Extracting features of the raindrop recognition sample image through the semantic branch network to obtain a first semantic feature; Performing feature fusion on the second spatial feature and the first semantic feature through a feature fusion module to obtain a target fusion feature; Determining a predicted semantic segmentation image according to the target fusion features by the segmentation module; A loss value is determined according to the predicted semantic segmentation image and the pixel label matrix, and the parameters of the bilateral semantic segmentation network are updated according to the loss value to obtain the trained raindrop recognition model.

5. A vehicle automatic control method according to claim 1, characterized in that: The step of adaptively weighting and fusing the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information specifically includes: Establishing a fuzzy control model of ambient light and a first weight and a second weight, wherein the first weight is a reliability weight of the first rainfall information, and the second weight is a reliability weight of the second rainfall information; Determine the first weight and the second weight according to the fuzzy control model and the ambient lighting information; The first rainfall information and the second rainfall information are weightedly summed according to the first weight and the second weight to obtain the target rainfall information.

6. A vehicle automatic control method according to any one of claims 1 to 5, characterized in that: The controlling of the windshield wiper of the current vehicle according to the target rainfall information and adjusting the driving speed of the current vehicle, and then sending the target rainfall information and the adjusted driving speed to the rear vehicle, specifically includes: Determining a current rainfall level according to the target rainfall information, and determining a target wiper gear position and a target vehicle speed threshold according to the current rainfall level; controlling the wiper according to the target wiper gear, and adjusting the driving speed according to the target vehicle speed threshold; The target rainfall information and the adjusted driving speed are sent to the rear vehicle through V2V communication technology.

7. A vehicle automatic control method according to claim 6, characterized in that: The vehicle automatic control method further comprises the following steps: When the current rainfall level changes, determining whether the current vehicle performs windshield wiper control and driving speed adjustment; If the current vehicle does not perform windshield wiper control and / or driving speed adjustment, obtain the real-time vehicle condition data of the current vehicle, and upload the real-time vehicle condition data to the traffic platform.

8. A vehicle automatic control system, characterized in that: include: A first information acquisition module, used to acquire first image information of a front windshield of a current vehicle, and determine first rainfall information of the front windshield according to the first image information; A second information acquisition module is used to acquire the ambient light information of the current vehicle and acquire the second rainfall information of the front windshield through an infrared rainfall sensor; an information fusion module, configured to perform adaptive weighted fusion on the first rainfall information and the second rainfall information according to the ambient light information to obtain target rainfall information; The vehicle control module is used to control the wipers of the current vehicle according to the target rainfall information and adjust the driving speed of the current vehicle, and then send the target rainfall information and the adjusted driving speed to the rear vehicle.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the steps of the vehicle automatic control method as described in any one of claims 1 to 7 are realized.

10. A storage medium, the storage medium being a computer-readable storage medium, used for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the vehicle automatic control method as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Vehicle control method and device based on target object outside vehicle, electronic equipment and storage medium

    CN119502918A

  • Binocular image generation and automatic driving simulation method and related equipment

    CN120525942A

  • Windscreen wiper control method and system for commercial vehicle

    CN121553067A