Intelligent driving method and related device

Abstract

An intelligent driving method and a related device are applied to the field of intelligent driving. On one hand, images in the driving process are collected through the two side cameras located on the side of the intelligent driving terminal and the rear camera located on the rear side of the intelligent driving terminal, left and right cameras in an electronic rearview mirror system do not need to be redeployed, and the vehicle body shape can be improved. And on the other hand, the collected images are processed and displayed by the multiplexing automatic driving controller, so that the function of an electronic rearview mirror is realized, and the cost of the whole vehicle is saved. In general, under the condition that the rearview function is achieved, the cost can be greatly saved, and the automobile body shape is improved. In addition, in combination with the high-performance sensing and computing capabilities of the automatic driving controller, richer experience can be provided for the user, and the driving safety is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and particularly to an intelligent driving method and related devices. Background Art

[0002] The safe driving of a vehicle requires many functions to ensure, such as ensuring the safe driving of the vehicle through the rear-view function. The rear-view function can provide the driver with a visual field picture of the current environment, so as to facilitate the driver to observe the situation of oncoming vehicles from behind and make decisions in a timely manner. The rear-view function can be implemented by solutions such as optical rear-view mirrors and electronic rear-view mirrors. One display mode of an electronic rear-view mirror is to monitor the field of view of the physical rear-view mirror through a camera, transmit it to the electronic rear-view mirror controller through a video interface, and the electronic rear-view mirror controller sends it to an independent display screen for display through the video interface.

[0003] However, due to the large number of sensors deployed on the current intelligent driving terminal, these sensors are deployed around the vehicle body, and installing an electronic rear-view mirror will cause the positions of the sensors on the intelligent driving terminal to overlap with the camera of the electronic rear-view mirror, resulting in redundant vehicle body styling and high costs.

[0004] Therefore, how to save costs and improve the vehicle body styling while implementing the rear-view function is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides an intelligent driving method and related devices, which can save costs and improve the vehicle body styling while implementing the rear-view function.

[0006] In a first aspect, this application provides an intelligent driving method, which is applied to an autonomous driving controller. The autonomous driving controller is included in an intelligent driving terminal, and the intelligent driving terminal further includes two side cameras located on the side of the intelligent driving terminal and a rear camera located behind the intelligent driving terminal. The method includes:

[0007] Obtain a first image and a second image respectively collected by the two side cameras, and process the first image and the second image to obtain a target image. The target image is used to indicate the field of view picture of the rear-view mirror angle of the intelligent driving terminal, and the target image is used for intelligent driving.

[0008] Optionally, the intelligent driving terminal further includes a rear camera located behind the intelligent driving terminal. The processing the first image and the second image to obtain a target image includes: obtaining a third image collected by the rear camera; processing the first image, the second image and the third image to obtain the target image.

[0009] Among them, the autonomous driving controller can be connected to multiple sensors installed in the vehicle. The sensors include but are not limited to millimeter-wave radar, cameras, lidar, global positioning system (GPS), etc. The autonomous driving controller can obtain the data collected by these sensors and process them through algorithms to obtain driving decisions and output. The autonomous driving controller has high computing power and can obtain rich perception information. Therefore, the autonomous driving controller can be used to implement the rearview function.

[0010] The two side cameras can collect the field-of-view images on the left and right sides of the intelligent driving terminal, and the rear camera can collect the field-of-view image on the rear side of the intelligent driving terminal. Therefore, the field-of-view image of the rearview mirror angle obtained in this application can be obtained based on the field-of-view images on the left and right sides, or can be obtained by fusing the field-of-view images on the left and right sides and the field-of-view image on the rear side. Compared with the traditional rearview mirror, it can cover a larger visible field-of-view range and has a larger field-of-view adjustment space.

[0011] On the one hand, this application collects images during driving through two side cameras and a rear camera, without redeploying the left and right cameras in the electronic rearview mirror system, and even without installing an optical rearview mirror, which can improve the body styling. On the other hand, this application reuses the autonomous driving controller to process the collected images to implement the function of the electronic rearview mirror, saving the overall vehicle cost. In summary, this application can greatly save costs and improve the body styling while implementing the rearview function.

[0012] In addition, combined with the high-performance perception and computing capabilities of the autonomous driving controller, it can provide users with a richer experience and improve driving safety.

[0013] In a possible implementation manner, the intelligent driving terminal further includes a screen. After obtaining the target image by processing the first image, the second image, and the third image, it further includes: sending the target image to the screen for display.

[0014] In another possible implementation manner, the screen includes a first screen and a second screen. The first screen includes a left screen and / or a right screen. The target image includes a first target image and a second target image. The first target image and the second target image are respectively used to indicate the field-of-view images of the rearview mirror angles on both sides of the intelligent driving terminal. The left screen is used to display the first target image, and the right screen is used to display the second target image.

[0015] Optionally, the first target image is obtained by processing the first image and the third image, and the second target image is obtained by processing the first image and the third image.

[0016] In yet another possible implementation, obtaining the target image according to the first image, the second image, and the third image includes: performing linear correction and image processing on the first image, the second image, and the third image to obtain the target image, where the image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, and image size transformation.

[0017] In the above implementation, since there will be actual differences between the images captured by using a traditional external mirror and the left and right side cameras (for example, the first image and the second image captured by the left and right side cameras will have a sense of distortion, or the positions of the first image and the second image captured by the left and right side cameras may be offset relative to the traditional external mirror, and the clarity and accuracy will be lower than the images captured by the traditional external mirror), the first image and the second image from the left and right side cameras can be linearly corrected, dead pixels can be removed, white balance and other image processing operations can be performed, and operations such as matte extraction, image overlay, and image size transformation can be carried out, and then sent for display in combination with the third image captured by the rear camera, so as to ensure that the quality of the images after reusing the left and right side cameras is not affected.

[0018] Optionally, the left and right side cameras are the left and right side cameras of an advanced driving assist system (ADAS). Among them, the ADAS camera is a camera used to collect the perception information required for intelligent driving, and it is usually connected to an autonomous driving controller. An ADAS algorithm can be deployed in the autonomous driving controller, and this ADAS algorithm is used to make intelligent driving decisions based on the perception information.

[0019] In yet another possible implementation, sending the target image to the screen includes: when the first screen and the system-on-chip (SoC) of the display system of the autonomous driving controller are normal, sending the target image to the first screen through the display SoC. When an abnormal situation occurs in the first screen or the display SoC of the autonomous driving controller, sending the target image to the second screen.

[0020] Among them, the first screen can be one or more screens. For example, the first screen can be two display screens located on the left and right sides of the cockpit of the intelligent driving terminal, which are called the left screen and the right screen for convenience of description. These two display screens are used to display the field of view of the rearview mirror.

[0021] In the above embodiments, the autonomous driving controller includes a display SoC, which can be connected to the first screen. After obtaining the target image, the display SoC sends the target image to the first screen for display. Exemplarily, the target image includes a first target image of the left rearview mirror view, and the first screen includes a left screen. The autonomous driving controller sends the first target image to the left screen through the display SoC.

[0022] In some solutions, if the display SoC or the first screen (such as the left screen) fails or is damaged, it may affect the driver's vision, thereby further affecting safety. When the first screen fails (for example, any one of the left and right screens fails) or the display SoC of the autonomous driving controller fails, the target image is sent for display through the second screen (such as the center console screen) to ensure driving safety. That is to say, the second screen can be regarded as a backup screen for realizing the rearview function.

[0023] It can be understood that the autonomous driving controller, as the core component of an autonomous driving vehicle, is usually connected to multiple display screens. Therefore, the sending and displaying of the target image can be realized through the autonomous driving controller.

[0024] In another possible embodiment, taking the second screen as the center console screen as an example, the autonomous driving controller further includes another SoC, which is used to send the target image to the center console screen.

[0025] Optionally, when any one of the left and right screens fails or the display SoC of the autonomous driving controller fails, the autonomous driving controller sends the target image to the center console screen through this SoC. Further optionally, the SoC for sending the image to the center console screen can adopt encoding methods such as H264 encoding and H265 encoding.

[0026] In another possible embodiment, the autonomous driving controller can process the information collected by the sensor to obtain a warning message and send the warning message to the screen.

[0027] In another possible embodiment, the target image may further include warning information, and the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0028] In the above embodiments, the driver can know the environment behind the vision through the warning information displayed on the screen, so that the driver can make timely decisions in case of emergencies, improve the braking reaction time of the driver in emergencies, and enhance the safety and convenience of driving.

[0029] In yet another possible implementation, the information collected by the sensor includes the first image, the second image, and the third image. Processing the first image, the second image, and the third image to obtain a target image includes: obtaining the target image according to the first image, the second image, the third image, and the warning information.

[0030] In the above implementation, the first image, the second image, and the third image are real picture information, while the warning information is virtual information obtained based on these real images. For example, it can be reflected on real objects through signs, texts, lines, etc. The target image includes real pictures and warning information, enabling the driver to more conveniently understand the current driving environment.

[0031] In yet another possible implementation, the method further includes: when a preset situation occurs in the intelligent driving terminal, adjusting the field of view corresponding to the target image from a first preset value to a second preset value, where the second preset value is greater than the first preset value.

[0032] Optionally, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the following vehicle, etc.

[0033] In the above implementation, the automatic driving controller can sense the external environment through two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, lidar, GPS, and other devices. For example, it is confirmed that the current intelligent driving terminal is turning on the first section of the road, and the turning amplitude is 40°. For the current situation, a decision is made to expand the field of view corresponding to the current target image. For example, the field of view angle of the target image that can be collected by the left and right cameras is usually 60°. The automatic driving controller can adjust the field of view angle that the target image can see from 60° to 80°, that is, expand the field of view of the intelligent driving terminal when turning.

[0034] In yet another possible implementation, the method further includes: after the intelligent driving terminal exits the preset situation, adjusting the field of view corresponding to the target image from the second preset value to the first preset value.

[0035] Continuing with the target situation as an example of passing through a turning scenario, after the turning is completed, the automatic driving controller can restore the target image to the normal field of view angle to provide a better driving experience for the driver.

[0036] In a second aspect, the present application provides an intelligent driving device, which includes modules or units for implementing the methods described in the first aspect or any possible implementation of the first aspect. The module can be a software module or a hardware module.

[0037] In a third aspect, the present application provides an intelligent driving terminal, which includes an autonomous driving controller, two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, and a screen. The two side cameras and the rear camera are used to collect driving scene images, the screen is used to display target images, and the autonomous driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the method described in the first aspect or any possible implementation manner of the first aspect.

[0038] In a possible implementation manner, the intelligent driving terminal is a vehicle, a drone, or a robot.

[0039] In a fourth aspect, the present application provides a chip, which includes a logic circuit and an interface, and the logic circuit and the interface are coupled; the interface is used to input and / or output information, and the logic circuit is used to execute the method described in the foregoing first aspect or any possible implementation manner of the first aspect.

[0040] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions are run on at least one processor, the method described in the foregoing first aspect or any possible implementation manner of the first aspect can be implemented.

[0041] In a sixth aspect, the present application provides a computer program product, which includes computer instructions. When the instructions are run on at least one processor, the method described in the foregoing first aspect or any possible implementation manner of the first aspect can be implemented. This computer program product can be a software installation package. In the case where the foregoing method needs to be used, this computer program product can be downloaded and executed on a computing device.

[0042] For the technical solutions provided in the second to sixth aspects of the present application, the beneficial effects can refer to the beneficial effects of the technical solution of the first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The following will briefly introduce the drawings required for the description of the embodiments.

[0044] Figure 1 is a schematic diagram of the architecture of an autonomous driving vehicle provided by an embodiment of the present application;

[0045] Figure 2 is a schematic diagram of the deployment position of a camera provided by an embodiment of the present application;

[0046] Figure 3It is a schematic diagram of a new architecture of an autonomous driving vehicle provided by an embodiment of the present application;

[0047] Figure 4 It is a schematic flowchart of an intelligent driving method provided by an embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of a scenario during driving of an intelligent driving terminal provided by an embodiment of the present application;

[0049] Figure 6 It is a schematic diagram of a target image processing process provided by an embodiment of the present application;

[0050] Figure 7 It is a schematic diagram of the display on the cockpit screen provided by an embodiment of the present application;

[0051] Figure 8 It is a schematic diagram of a target image provided by an embodiment of the present application;

[0052] Figure 9 It is a schematic diagram of the structure of an intelligent driving device 90 provided by an embodiment of the present application;

[0053] Figure 10 It is a schematic diagram of the architecture of an electronic device 100 provided by an embodiment of the present application. Detailed implementation manners

[0054] The present application provides an intelligent driving method and related devices, which can save costs and make the body shape beautiful when replacing traditional rearview mirrors with electronic rearview mirrors. The method provided by the present application can be applied to an intelligent driving terminal. The intelligent driving terminal can be a mobile phone, a personal notebook, an intelligent robot, an intelligent driving device, etc. The intelligent driving device can include land vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment, etc. For example, the intelligent driving device can be a vehicle, and the vehicle is a vehicle in a broad sense, which can be a transportation vehicle (such as a commercial vehicle, a passenger vehicle, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), an agricultural equipment (such as a lawn mower, a harvester, etc.), a recreational equipment, a toy vehicle, etc. The types of vehicles are not specifically limited in the embodiments of the present application. Again, for example, the intelligent driving terminal can be a vehicle such as an airplane or a ship.

[0055] Taking the intelligent driving terminal as an autonomous driving vehicle as an example below, the architecture of the embodiments of the present application will be described in combination with the accompanying drawings in the embodiments of the present application.

[0056] For ease of understanding, relevant terms that may be involved in the embodiments of the present application will be introduced first below.

[0057] 1. Intelligent driving

[0058] Intelligent driving is an important direction for the development of vehicle intelligence. With the development of perception technology and the improvement of chip capabilities, intelligent driving provides people with more and more rich driving functions, gradually realizing different levels of driving experiences. The Society of Automotive Engineers (SAE) provides a driving automation classification standard, including driving levels from L0 to L5. Among them, level L0 is no automation, and the vehicle is fully operated by the human driver, who can receive warnings or assistance from the driving system during driving, such as autonomous emergency braking (AEB), blind spot monitoring (BSM), or lane departure warning (LDW), etc. Level L1 is driving assistance, and the driving operation is jointly completed by the human driver and the driving system. The driving system can provide driving assistance to the steering wheel or acceleration / deceleration operations through the driving environment, and other driving operations are carried out by the human driver, such as adaptive cruise control (ACC) or lane keep assistance / support (LKA / LKS), etc.; Level L2 is partial automation, providing driving assistance to multiple items of the steering wheel and acceleration / deceleration through the driving environment, and other driving actions are carried out by the human driver, such as the following vehicle function combining adaptive cruise control (ACC) and lane keep assistance (LKA); Level L3 is conditional automation, and some driving operations can be completed by the driving system, but the human driver needs to respond to the driving system's requests at the appropriate time, that is, the human driver needs to be prepared to take over the driving system; Level L4 is highly automated, and all driving operations can be completed by the driving system, and the human driver does not necessarily need to respond to the driving system's requests. For example, under road and environmental conditions permitting (such as in a closed park, highway, urban road, or fixed driving route, etc.), the human driver can not take over the driving; Level L5 is fully automated, and all driving operations under various road and environmental conditions that the human driver can handle can be independently completed by the driving system. It can be seen that at levels L0 to L2, the driving system mainly provides support for the driver, and the driver still needs to perform driving supervision and steer, brake, or accelerate as needed to ensure safety. At levels L3 to L5, the driving system can replace the driver to complete all driving operations. At level L3, the driver needs to be prepared to take over the driving. At levels L4 and L5, the driving system can achieve full driving under partial conditions and all conditions, and the driver can choose whether to take over.

[0059] The above classification is an example. With the evolution of technology or different regulations in different countries or regions, the above classification can change. For example, the vehicle automation classification proposed by the Ministry of Industry and Information Technology of China includes 6 levels of vehicle driving automation, among which levels 0-2 are driving assistance, where the system assists humans in performing dynamic driving tasks and the driving subject is still the driver; levels 3-5 are intelligent driving, where the system replaces humans in performing dynamic driving tasks under the designed operating conditions. When the function is activated, the driving subject is the system. The names and definitions of each level are as follows: Level 0 driving automation (Emergency assistance): The system cannot continuously perform the lateral or longitudinal movement control of the vehicle in dynamic driving tasks, but has the ability to continuously detect and respond to some targets and events in dynamic driving tasks. Level 1 driving automation (Partial driver assistance): The system continuously performs the lateral or longitudinal movement control of the vehicle in dynamic driving tasks under its designed operating conditions (or called the designed operating domain ODD), and has the ability to detect and respond to some targets and events that are adapted to the lateral or longitudinal movement control of the vehicle being performed. Level 2 driving automation (Combined driver assistance): The system continuously performs the lateral and longitudinal movement control of the vehicle in dynamic driving tasks under its designed operating conditions, and has the ability to detect and respond to some targets and events that are adapted to the lateral and longitudinal movement control of the vehicle being performed. Level 3 driving automation (Conditionally automated driving): The system continuously performs all dynamic driving tasks under its designed operating conditions. Level 4 driving automation (Highly automated driving): The system continuously performs all dynamic driving tasks under its designed operating conditions and automatically executes the minimum risk strategy. Level 5 driving automation (Fully automated driving): The system continuously performs all dynamic driving tasks under any drivable conditions and automatically executes the minimum risk strategy. Among them, lateral control is mainly used for the control of vehicle steering. For example, controlling the steering wheel torque or angle to control the direction of the vehicle; longitudinal control is mainly used for the speed control of the vehicle, such as controlling the brake pedal, accelerator pedal, or gear, etc. to control the acceleration / deceleration, braking, etc. of the vehicle.

[0060] Regardless of the classification method adopted, the description of the embodiments of the present application can be applicable to the intelligent driving systems that need to participate in vehicle driving partially or wholly as above.

[0061] Please refer to Figure 1 , Figure 1It is a schematic diagram of the architecture of an autonomous vehicle provided by an embodiment of the present application. The autonomous vehicle includes sensors, a controller, and actuators. As Figure 1 shown, the sensors may include one or more of a camera, a lidar, or an ultrasonic radar, etc. The controller is used for processing and calculation. For example, it may be a domain controller (such as Figure 1 the autonomous driving controller shown), and in some solutions, the controller may run algorithms to perform processes such as data processing and logical calculation. The actuator is used to execute the control function of the vehicle or vehicle components, including executing the control decisions made by the controller, responding to the control decisions made by the user, etc. Exemplarily, the actuator may include one or more of a steering actuator, a throttle actuator, or a brake actuator, etc. Further, the autonomous vehicle also includes an electronic rearview mirror. With the emergence of more configurations, due to its practicality, the electronic rearview mirror is increasingly deployed on autonomous vehicles.

[0062] Compared with traditional optical rearview mirrors, the cost of electronic rearview mirrors is higher, and the models equipped with them may also be more high-end. And once the electronic rearview mirror is damaged, the time and money costs for repair are much greater than those of traditional optical rearview mirrors. In addition, the electronic rearview mirror depends on the normal operation of technical devices such as cameras and displays. If these devices malfunction or are damaged, it may affect the driver's vision, thereby further affecting safety. And the acquisition of the camera and the processing of information require a certain amount of time, and there may be a certain degree of delay compared with traditional optical rearview mirrors. In addition, due to the installation of the electronic rearview mirror, it will also cause the positions of the cameras on the autonomous vehicle and the cameras of the electronic rearview mirror to overlap, resulting in redundant body styling and high costs. To solve the above problems, in the embodiment of the present application, a new architecture is proposed based on the above architecture, which multiplexes the autonomous driving controller, two side cameras located on the side of the autonomous vehicle, and a rear camera located at the rear of the autonomous vehicle to collect and process images during the driving process, saving configurations such as left and right cameras and electronic rearview mirror controllers, and a new intelligent driving method is proposed based on the new architecture, which can save costs and improve the body styling while still being able to realize the function of the electronic rearview mirror.

[0063] Please refer to Figure 2 , Figure 2 which is a schematic diagram of the deployment position of the camera provided by an embodiment of the present application. As Figure 2As shown, the autonomous vehicle includes two side cameras located on the sides of the autonomous vehicle and a rear camera located at the rear of the autonomous vehicle. Among them, the two side cameras on the sides of the autonomous vehicle can be respectively deployed above the left and right front wheels of the autonomous vehicle. The field of view collected by the left side camera (e.g., denoted as FOV_2) is the field of view collected by the left camera replacing the electronic rearview mirror (e.g., denoted as FOV_L), and the field of view collected by the right side camera (e.g., denoted as FOV_1) is the field of view collected by the right camera replacing the electronic rearview mirror (e.g., denoted as

[0064] FOV_R), and the field of view collected by the rear camera can be denoted as FOV_3. The autonomous driving controller processes the images corresponding to FOV_1, FOV_2, and FOV_3 and sends them to the screen for display.

[0065] Please refer to Figure 3 , Figure 3 which is a schematic diagram of a new architecture of an autonomous vehicle provided by an embodiment of the present application. As Figure 3 shown, the autonomous vehicle includes an autonomous driving controller 301, two side cameras (left side camera 302 and right side camera 303) located on the sides of the autonomous vehicle, a rear camera 304 located at the rear of the autonomous vehicle, and a screen (left screen 305, right screen 306, and central control screen 307) and a gateway 308. Among them, the left side camera 302, the right side camera 303, and the rear camera 304 are used to collect images of the driving scene, the autonomous driving controller 301 is used to process the images of the driving scene, and the left screen 305 and the right screen 306, or the central control screen 307 are used to display the processed images of the driving scene (e.g., denoted as target images). The gateway 308 can be used for both wide area network interconnection and local area network interconnection.

[0066] The following introduces the intelligent driving method provided by the embodiment of the present application.

[0067] Please refer to Figure 4 , Figure 4 which is a schematic flowchart of an intelligent driving method provided by an embodiment of the present application. Optionally, this intelligent driving method can be applied to Figure 2 , Figure 3 the architecture shown.

[0068] The intelligent driving method includes one or more steps among steps S401 to S402 shown below. It should be understood that for the convenience of description, the steps are described in the order of steps S401 to S402 here, and it is not intended to limit that the execution must be in the above order. In the embodiments of the present application, the execution sequence, execution time, execution frequency, etc. of the above one or more steps are not limited, and other steps can be inserted between or before and after these steps as needed, where:

[0069] Step S401: The automatic driving controller acquires a first image and a second image respectively collected by two side cameras.

[0070] Exemplarily, the two side cameras are respectively cameras located on the left side and the right side of the intelligent driving terminal.

[0071] Among them, the intelligent driving terminal can be a device with computing capabilities. For example, the intelligent driving terminal can be Figure 3 the shown automatic driving vehicle. Optionally, the intelligent driving terminal can be a single device or a distributed system composed of multiple devices.

[0072] Optionally, the left side camera is an ADAS left side camera, and the right side camera is an ADAS right side camera. Among them, the ADAS camera is a camera for collecting perception information required for automatic driving, and it is usually connected to the automatic driving controller. The ADAS algorithm can be deployed in the automatic driving controller, and this ADAS algorithm is used to make intelligent driving decisions according to the perception information collected from the sensors.

[0073] As a possible implementation manner, the automatic driving controller can also acquire a third image collected by the rear camera.

[0074] Exemplarily, the rear camera is a camera located behind the intelligent driving terminal.

[0075] Taking the automatic driving controller acquiring the first image, the second image, and the third image as an example, please refer to Figure 5 , Figure 5 which is a schematic diagram of the scene of an intelligent driving terminal during driving provided by the embodiments of the present application. As shown in Figure 5As shown, the current scenario is that the intelligent driving terminal is driving on the road. The first image is data collected by the left-side camera and associated with the left-side field of view of the intelligent driving terminal in the current driving scenario. The second image is data collected by the right-side camera and associated with the right-side field of view of the intelligent driving terminal in the current driving scenario. The third image is data collected by the rear camera and associated with the rear field of view of the intelligent driving terminal in the current driving scenario. In some embodiments, the above three cameras actively send the collected first image, second image, and third image to the automatic driving controller when the intelligent driving terminal is driving. The first image, second image, and third image may carry position information (such as the current location) and time information (such as a timestamp).

[0076] In some other embodiments, the automatic driving controller may actively obtain the first image, second image, and third image collected by the above three cameras when the intelligent driving terminal is driving.

[0077] Step S402: The automatic driving controller processes the first image and the second image to obtain a target image.

[0078] Among them, the target image is used to indicate the field of view of the rearview mirror angle of the intelligent driving terminal. At this time, the driver of the intelligent driving terminal or other devices in the intelligent driving terminal (such as the automatic driving controller and other components, etc.) can judge various situations that may affect driving in the rear field of view based on the target image and make driving decisions. That is, the target image can be used for intelligent driving.

[0079] Exemplarily, the two side cameras can collect the field of view on the left and right sides of the intelligent driving terminal. The target image obtained by the automatic driving controller processing the first image and the second image is a complete rearview mirror angle field of view fused based on the field of view on the left and right sides. Among them, the rearview mirror angle field of view is all the fields of view on the side of the intelligent driving terminal that the driver can see.

[0080] As a possible implementation manner, the automatic driving controller may process the first image, the second image, and the third image to obtain a target image.

[0081] Exemplarily, the two side cameras can collect the field of view on the left and right sides of the intelligent driving terminal, and the rear camera can collect the field of view on the rear side of the intelligent driving terminal. The target image obtained by the automatic driving controller processing the first image, the second image, and the third image is a complete rearview mirror angle field of view fused based on the field of view on the left and right sides and the rear field of view. Among them, the rearview mirror angle field of view is all the fields of view on the side and rear of the intelligent driving terminal that the driver can see.

[0082] To better reflect the integrity of the solution, the following takes the first image, the second image, and the third image as examples for elaboration.

[0083] As a possible implementation, the autonomous driving controller performs linear correction and image processing on the first image, the second image, and the third image to obtain a target image. Among them, the image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, image size transformation, etc.

[0084] In a possible scenario, the target image can be a single image. For example, through a single image, the visual environments on the left side, right side, and rear side of the entire vehicle can be reflected simultaneously.

[0085] In some possible scenarios, the target image can include multiple images, and the multiple images can be respectively used to represent the visual field images of the rearview mirrors in multiple orientations.

[0086] Exemplarily, the target image includes a first target image and a second target image, and the first target image and the second target image are respectively used to indicate the visual field images of the rearview mirror perspectives on both sides of the intelligent driving terminal. At this time, the driver of the intelligent driving terminal or other devices in the intelligent driving terminal (such as the autonomous driving controller and other components, etc.) can respectively judge various possible situations that may affect driving in the rear visual field image based on the first target image and the second target image, and make driving decisions. That is, the first target image and the second target image are used for intelligent driving.

[0087] The following takes the target image including the first target image and the second target image as an example to exemplarily introduce the process of obtaining the target image. As a possible implementation, the autonomous driving controller processes the first image and the third image to obtain the first target image, and processes the first image and the third image to obtain the second target image.

[0088] Exemplarily, the first target image obtained by the autonomous driving controller processing the first image and the third image is a visual field image of the rearview mirror perspective obtained by fusing the visual field image on the left side (corresponding to the first image) and the visual field image on the rear side (corresponding to the third image), and the second target image obtained by the autonomous driving controller processing the second image and the third image is a visual field image of the rearview mirror perspective obtained by fusing the visual field image on the right side (corresponding to the second image) and the visual field image on the rear side (corresponding to the third image).

[0089] As a possible implementation, the autonomous driving controller performs linear correction and image processing on the first image and the third image to obtain the first target image, and performs linear correction and image processing on the first image and the third image to obtain the second target image.

[0090] Exemplarily, the following takes the scenario of Figure 5 as an example. For example, when the current intelligent driving terminal makes a left turn on the road, the left-side camera captures a blind spot image, the right-side camera captures a turning vision image, and the rear camera captures a vision image of the rear of the intelligent driving terminal. Please refer to Figure 6 , Figure 6 which is a schematic diagram of a target image processing process provided by an embodiment of the present application. As shown in Figure 6 , Figure 6 (a) in Figure 6 shows the first image collected by the left-side camera, the second image collected by the right-side camera, and the third image collected by the rear camera. Through the automatic driving controller, the first image, the second image, and the third image are processed to obtain the Figure 6 first target image and the second target image shown in (b). Since there are actual differences between using a traditional outside mirror and using the images collected by the left and right side cameras (for example, the first image and the second image collected by the left and right side cameras will have a sense of distortion, or the positions of the first image and the second image collected by the left and right side cameras may be offset relative to the traditional outside mirror, and the clarity and accuracy will be lower than the images collected by the traditional outside mirror), the automatic driving controller can perform image processing such as linear correction, dead pixel removal, white balance, etc. on the first image and the second image from the left and right side cameras, and combine them with the third image collected by the rear camera, and then perform operations such as image cropping, image overlaying, and image size transformation before sending them for display, so as to ensure that the quality of the images after reusing the left and right side cameras is not affected.

[0091] In some possible cases, the intelligent driving terminal further includes a screen. For example, the screen in the intelligent driving terminal includes but is not limited to one or more of a left screen, a right screen, a central control large screen, a co-pilot entertainment screen, etc. Among them, the left screen and the right screen can be respectively deployed at positions close to the windows on the left and right sides inside the intelligent driving terminal. After obtaining the target image, the automatic driving controller can send the target image for display on the screen. Specifically, the automatic driving controller sends the target image to the screen, and the screen displays the target image.

[0092] In some solutions, the intelligent driving terminal includes a first screen. Among them, the first screen can be one or more screens. For example, the first screen can be two display screens located on the left and right sides of the driving cockpit of the intelligent driving terminal, which are called the left screen and the right screen for convenience of description. These two display screens are used to display the vision images of the rearview mirror perspective. Or, the first screen can also be either the left screen or the right screen alone.

[0093] Exemplarily, in the case where the target image includes a first target image and a second target image, wherein the first target image is a field of view image from the rearview mirror perspective obtained by fusing the field of view image on the left side (corresponding to the first image) and the field of view image at the rear (corresponding to the third image), and the second target image is a field of view image from the rearview mirror perspective obtained by fusing the field of view image on the right side (corresponding to the second image) and the field of view image at the rear (corresponding to the third image). At this time, the left screen is used to display the first target image, and the right screen is used to display the second target image. Exemplarily, please refer to Figure 7 , Figure 7 which is a schematic diagram of the screen display in a driving cockpit provided by an embodiment of the present application. As Figure 7 shown, the left screen is responsible for displaying the field of view image from the rearview mirror perspective that fuses the first image and the third image (that is, the first target image is displayed), and the right screen is responsible for displaying the field of view image from the rearview mirror perspective that fuses the second image and the third image (that is, the second target image is displayed).

[0094] In some possible cases, the autonomous driving controller further includes a display SoC. Understandably, as the core component of an autonomous driving vehicle, the autonomous driving controller is usually connected to multiple display screens. For example, the display SoC can be connected to the first screen. Therefore, the delivery and display of the target image can be achieved through the autonomous driving controller.

[0095] Optionally, the two side cameras and the rear camera add the first image, the second image, and the third image to the autonomous driving controller through a serializer for deserialization, and then input them into the display SoC inside the autonomous driving controller. The delivery and display of the target image are achieved through the image signal processor (ISP) module inside the display SoC and the serialization transmission of the display serial interface (DSI).

[0096] In some solutions, when the first screen and the display SoC of the autonomous driving controller are normal, after obtaining the target image, the target image is delivered and displayed on the first screen through the display SoC.

[0097] In some possible cases, the intelligent driving terminal includes a first screen and a second screen. When the first screen fails (for example, any one of the left and right screens fails) or the display SoC of the autonomous driving controller fails, the target image is delivered and displayed through the second screen (for example, the center console screen) to ensure driving safety. That is, the second screen can be regarded as a backup screen for realizing the rearview function.

[0098] If the sending display SoC or the first screen (e.g., the left screen) fails or is damaged, it may affect the driver's vision, thereby further affecting safety. Since the autonomous driving controller can connect the first screen and the second screen, when the first screen fails, the target image can be sent to the backup screen to improve driving safety.

[0099] Taking the second screen as the center console screen and the target image including the first target image and the second target image as an example, four backup sending display situations are exemplarily introduced as follows:

[0100] In the first situation, when the left screen of the first screen fails, the first target image is sent through the center console screen, and the second target image is still displayed through the right screen of the first screen.

[0101] In the second situation, when the right screen of the first screen fails, the second target image is sent through the center console screen, and the first target image is still displayed through the left screen of the first screen.

[0102] In the third situation, when both the left screen and the right screen of the first screen fail, the first target image and the second target image are sent through the second screen.

[0103] In the fourth situation, when the sending display SoC of the autonomous driving controller fails, the first target image and the second target image are sent to the center console screen through another SoC (e.g., called the backup SoC) included in the autonomous driving controller.

[0104] Optionally, when any one of the left and right screens fails or the sending display SoC of the autonomous driving controller fails, the backup SoC for sending the target image to the center console screen can adopt encoding methods such as H264 encoding and H265 encoding. Exemplarily, H264 can transmit standard-definition digital images at a speed lower than 1 Mbps; H265 encoding can transmit ordinary high-definition audio and video with a resolution of 720P (1280*720) at a transmission speed of 1 - 2 Mbps.

[0105] In some possible situations, the autonomous driving controller can also process the information collected by the sensors to obtain warning information and send the warning information to the screen. Among them, the number of the sensors can be one or more. When the number of the sensors is multiple, the types of the multiple sensors can be different. For example, it includes but is not limited to devices such as millimeter-wave radars, cameras, lidar, and GPS. At this time, the warning information can also be included in the target image displayed on the screen.

[0106] Furthermore, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0107] Among them, the distance information includes, for example, the distance between the host vehicle and other vehicles, the distance between the host vehicle and the road edge (or lane line), the distance between the host vehicle and obstacles, etc. An obstacle refers to an entity that may retard or impede the progress of the intelligent driving terminal during its travel, such as an object, terrain, or facility. Among them, an object can include a living object or an inanimate object. In addition, the position of the obstacle can be fixed or moving. The obstacle information includes one or more of the type of the obstacle, the position of the obstacle, the distance between the obstacle and the host vehicle, etc., such as obstacles like humans, animals, bicycles, electric vehicles, road bumps, potholes, and even irregular obstacles. The lane line information can include the position of the lane line, the distance between the host vehicle and the lane line, etc. The traffic sign information can include the content, position, etc. of the traffic sign.

[0108] Optionally, the warning information can also be generated based on the first image, the second image, and the third image. Specifically, the automatic driving controller can also obtain the warning information based on the first image, the second image, and the third image, and obtain the target image based on the first image, the second image, the third image, and the warning information.

[0109] Exemplarily, the first image, the second image, and the third image are real picture information, while the warning information is virtual information obtained based on these real images, which can be reflected on real objects through, for example, signs, texts, lines, etc. The target image includes the real picture and the warning information, enabling the driver to more conveniently understand the current driving environment. Among them, the driving environment can include traffic lights, lane lines, traffic signs, animals, plants, vehicles, crosswalks, or pedestrians.

[0110] Exemplarily, please refer to Figure 8 , Figure 8 which is a schematic diagram of a target image provided by an embodiment of the present application. As Figure 8 shown, Figure 8 in (a) shows the scene where the current intelligent driving terminal is driving on the road, Figure 8 and in (b) shows the target image collected by the sensor.

[0111] Exemplarily, the automatic driving controller can process the images from the side camera and the rear camera to obtain the position of the lane line and indicate the position of the lane line in the target image. As Figure 8 shown in (b), the position of the lane line is indicated by a solid arrow in the target image.

[0112] Exemplarily, the automatic driving controller can process the images from the side camera and the rear camera, as well as the positioning data of the GPS, to obtain the distance information. Continuing as Figure 8As shown in (b) therein, the possible movement trajectory of the vehicle behind is indicated by a dashed arrow in the target image, and the driving speed of the current vehicle behind is indicated as 65 km / h to prompt the driver to pay attention to judging the vehicle distance and drive safely.

[0113] Exemplarily, the autonomous driving controller can process the images from the side cameras and the rear cameras, as well as the point cloud data of the lidar to obtain distance information. Continuing as Figure 8 shown in (b) therein, taking the distance between the host vehicle and the other vehicle as the distance information, the distance between the host vehicle and the vehicle behind obtained by the autonomous driving controller's image processing is 40 m, and this distance is indicated on the target image.

[0114] Exemplarily, the autonomous driving controller can process the images from the side cameras and the rear cameras, and the point cloud data of the lidar to obtain the type and position of the obstacle, and indicate the type and position of the obstacle on the target image. Continuing as Figure 8 shown in (b) therein, taking an example that the obstacles are a pedestrian and a puppy on the right rear side of the intelligent driving terminal, the autonomous driving controller processes the image to obtain the information that there is a pedestrian and a puppy on the right rear side of the intelligent driving terminal, and they are respectively indicated on the target image through the puppy identifier and the pedestrian identifier to prompt the driver to pay attention to the obstacles around the current environment of the intelligent driving terminal and drive carefully.

[0115] In some scenarios, the intelligent driving terminal will encounter various driving conditions during driving (such as the preset conditions described below). For example, the preset conditions include but are not limited to one or more of the intelligent driving terminal turning, changing lanes, judging the distance from the vehicle behind, etc. Generally speaking, the rear-view field of view requirements corresponding to different preset conditions are different.

[0116] As a possible implementation manner, when a preset condition occurs in the intelligent driving terminal, the autonomous driving controller adjusts the size of the target image from a first preset value to a second preset value. Further, the second preset value is greater than the first preset value. Exemplarily, the autonomous driving controller can sense the external environment through two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, lidar, GPS and other devices. For example, it is confirmed that the current intelligent driving terminal is turning on the first section of the road, and the turning amplitude is 40°. For the current situation, a decision is made to expand the field of view corresponding to the current target image. For example, usually the field of view angle of the target image that can be captured by the left and right cameras is 60°, and the autonomous driving controller can adjust the field of view angle that the target image can see from 60° to 80°, that is, expand the field of view of the intelligent driving terminal when turning.

[0117] Optionally, after the intelligent driving terminal exits the preset situation, the autonomous driving controller adjusts the size of the target image from a second preset value to a first preset value. Further, after the turning is completed, the autonomous driving controller can restore the target image to a normal viewing angle to provide a better driving experience for the driver. Optionally, the second preset value is smaller than the first preset value.

[0118] Exemplarily, the autonomous driving controller can also reduce the field of view of the target image when the intelligent driving terminal is turning according to the actual situation. For example, the field of view angle of the target image that can usually be captured by the left and right cameras is 60°. The autonomous driving controller can adjust the field of view angle that the target image can see from 60° to 40° to make the visible area clearer. After the turning is completed, it is also restored to the normal viewing angle to provide a better driving experience for the driver.

[0119] Exemplarily, taking the current intelligent driving terminal changing lanes on the second section of the road as an example, the autonomous driving controller can sense the external environment through two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, lidar, GPS and other devices, and adaptively make a decision to expand or contract the field of view corresponding to the current target image. After the lane change is completed, it is restored to the normal viewing angle to provide a better driving experience for the driver.

[0120] On the one hand, the present application collects images during the driving process through two side cameras and a rear camera, without the need to redeploy the left and right cameras in the electronic rearview mirror system, and even without installing an optical rearview mirror, which can improve the body shape. On the other hand, the present application multiplexes the autonomous driving controller to process the collected images to implement the function of the electronic rearview mirror, saving the overall vehicle cost. In summary, the present application can greatly save costs and improve the body shape while realizing the rearview function.

[0121] In addition, combined with the high-performance perception and computing capabilities of the autonomous driving controller, it can provide users with a richer experience and enhance driving safety.

[0122] The method of the embodiments of the present application is described in detail above. Next, the device of the embodiments of the present application is provided.

[0123] It can be understood that the multiple devices provided in the embodiments of the present application, such as the intelligent driving device, are for implementing the functions in the above method embodiments, and include corresponding hardware structures, software modules, or combinations of hardware structures and software structures for executing each function.

[0124] Those skilled in the art should easily realize that the units and algorithm steps of each example described in the embodiments disclosed herein can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different device implementation methods to implement the foregoing method embodiments in different usage scenarios, and different implementation methods of the device should not be considered to exceed the scope of the embodiments of this application.

[0125] The embodiments of this application can divide the device into functional modules. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of this application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0126] For example, in the case of dividing each functional module of the device in an integrated manner, this application lists several possible intelligent driving devices.

[0127] Please refer to Figure 9 , Figure 9 FIG. is a schematic structural diagram of an intelligent driving device 90 provided by the embodiments of this application. The intelligent driving device 90 can be an autonomous driving controller or a component in the autonomous driving controller, such as a chip, a software module, an integrated circuit, etc. The intelligent driving device 90 has the functions of the electronic device described in the embodiments of this application. In a possible design, the intelligent driving device 90 includes a transceiver unit 901 and a processing unit 902, where:

[0128] The transceiver unit 901 is used to obtain a first image and a second image respectively collected by two side cameras located on the side of the intelligent driving terminal.

[0129] The processing unit 902 is used to process the first image and the second image to obtain a target image, where the target image is used to indicate the field of view of the rearview mirror angle of the intelligent driving terminal, and the target image is used for intelligent driving.

[0130] In an optional implementation manner, in terms of processing the first image and the second image to obtain a target image, the processing unit 902 is specifically used to: obtain a third image collected by the rear camera. Process the first image, the second image and the third image to obtain the target image.

[0131] In another alternative embodiment, the intelligent driving terminal further includes a screen, and the transceiver unit 901 is further configured to send and display the target image on the screen.

[0132] In yet another alternative embodiment, the screen includes a first screen and a second screen. The first screen includes a left screen and / or a right screen. The target image includes a first target image and a second target image. The first target image and the second target image are respectively used to indicate the field of view of the rearview mirrors on both sides of the intelligent driving terminal. The left screen is used to display the first target image, and the right screen is used to display the second target image.

[0133] In yet another alternative embodiment, in terms of processing the first image, the second image, and the third image to obtain the target image, the processing unit 902 is specifically configured to:

[0134] Perform linear correction and image processing on the first image, the second image, and the third image to obtain the target image. The image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, image size transformation, etc.

[0135] In yet another alternative embodiment, in terms of sending and displaying the target image on the screen, the transceiver unit 901 is specifically configured to:

[0136] When the first screen and the display system-on-chip (SoC) of the automatic driving controller are normal, send and display the target image on the first screen through the display SoC. When an abnormal situation occurs in the first screen or the display SoC of the automatic driving controller, send and display the target image on the second screen.

[0137] In yet another alternative embodiment, the target image further includes warning information. The processing unit 902 is further configured to process the information collected by the sensor to obtain the warning information.

[0138] In yet another alternative embodiment, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0139] In yet another alternative embodiment, the information collected by the sensor includes the first image, the second image, and the third image. In terms of processing the first image, the second image, and the third image to obtain the target image, the processing unit 902 is specifically configured to:

[0140] Obtain the target image according to the first image, the second image, the third image, and the warning information.

[0141] In another alternative embodiment, the processing unit 902 is further configured to adjust the field of view corresponding to the target image from a first preset value to a second preset value when a preset situation occurs in the intelligent driving terminal, where the second preset value is greater than the first preset value. After the intelligent driving terminal exits the preset situation, the field of view corresponding to the target image is adjusted from the second preset value to the first preset value.

[0142] In another alternative embodiment, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the vehicle behind, etc.

[0143] The embodiments of the present application and the above-described method embodiments are based on the same concept and have the same technical effects. For the specific principle, please refer to the description of the above-described embodiments and will not be elaborated herein.

[0144] Please refer to Figure 10 , Figure 10 which is a schematic architecture diagram of an electronic device 100 provided by an embodiment of the present application. As Figure 10 shown, the electronic device 100 may be an autonomous driving controller deployed in an intelligent driving terminal, or a cloud server deployed in the cloud, or a device deployed locally. The electronic device 100 may be a single device or a distributed system composed of multiple devices. The electronic device 100 may include a processor 1001, and optionally may further include at least one memory 1002. Further optionally, the electronic device 100 may further include a communication interface 1003. More optionally, a bus 1004 may be included, where the processor 1001, the memory 1002, and the communication interface 1003 are connected through the bus 1004.

[0145] Among them, the processor 1001 is a module for performing arithmetic operations and / or logical operations, and can specifically be one or a combination of multiple processing modules such as a traditional central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a coprocessor (assisting the central processing unit to complete corresponding processing and applications), a microcontroller unit (MCU), etc.

[0146] The memory 1002 is used to provide a storage space, and data such as an operating system and computer programs can be stored in the storage space. The memory 1002 can be one or a combination of multiple types such as a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a compact disc read-only memory (CD-ROM), etc.

[0147] The communication interface 1003 can be used to provide information input or output for the at least one processor. And / or, the communication interface 1003 can be used to receive data sent externally and / or send data to the outside, and can be a wired link interface including, for example, an Ethernet cable, or a wireless link (Wi-Fi, Bluetooth, general wireless transmission, vehicle communication technology, and other wireless communication technologies, etc.) interface. Optionally, the communication interface 1003 can also include a transmitter (such as a radio frequency transmitter, an antenna, etc.) coupled to the interface, or a receiver, etc.

[0148] In a possible design, at least one processor 1001 in the electronic device 100 is used to execute the foregoing method, for example Figure 4 the intelligent driving method shown.

[0149] Optionally, the processor 1001 may be a processor dedicated to executing these methods (conveniently referred to as a dedicated processor for distinction), or a processor that executes these methods by calling a computer program, such as a general-purpose processor. Optionally, at least one processor may also include both a dedicated processor and a general-purpose processor. Optionally, the above computer program may be stored in the memory 1002.

[0150] Optionally, at least one processor 1001 in the electronic device 100 is configured to call computer instructions to perform the following operations:

[0151] Obtain a first image and a second image respectively collected by the two side cameras. Based on the first image and the second image, process to obtain a target image, where the target image is used to indicate the field of view of the rearview mirror angle of the intelligent driving terminal, and the target image is used for intelligent driving.

[0152] Optionally, the processor 1001 is further configured to:

[0153] Obtain a third image collected by the rear camera;

[0154] Based on the first image, the second image, and the third image, process to obtain the target image.

[0155] Optionally, the intelligent driving terminal further includes a screen, and the processor 1001 is further configured to:

[0156] Send the target image to the screen for display.

[0157] Optionally, the screen includes a first screen and a second screen, the first screen includes a left screen and / or a right screen, the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror angles on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

[0158] Optionally, the processor 1001 is further configured to:

[0159] Perform linear correction and image processing on the first image, the second image, and the third image to obtain the target image, and the image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, image size transformation, etc.

[0160] Optionally, the processor 1001 is further configured to:

[0161] When the first screen and the display system-on-chip (SoC) of the automatic driving controller are normal, the target image is sent to the first screen through the display SoC. When an abnormal situation occurs in the first screen or the display SoC of the automatic driving controller, the target image is sent to the second screen.

[0162] Optionally, the target image further includes a warning message. The processor 1001 is further configured to: process the information collected by the sensor to obtain the warning message.

[0163] Optionally, the warning message includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0164] Optionally, the information collected by the sensor includes the first image, the second image, and the third image. The processor 1001 is further configured to: obtain the target image according to the first image, the second image, the third image, and the warning message.

[0165] Optionally, the processor 1001 is further configured to:

[0166] When a preset situation occurs in the intelligent driving terminal, adjust the field of view corresponding to the target image from a first preset value to a second preset value, where the second preset value is greater than the first preset value. After the intelligent driving terminal exits the preset situation, adjust the field of view corresponding to the target image from the second preset value to the first preset value.

[0167] Optionally, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the vehicle behind, etc.

[0168] The embodiments of the present application and the above-described method embodiments are based on the same concept, and the technical effects brought by them are also the same. For the specific principle, please refer to the description of the above-described embodiments and will not be repeated here.

[0169] The present application further provides an intelligent driving terminal, which includes an automatic driving controller, two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, and a screen. The two side cameras and the rear camera are used to collect driving scene images, the screen is used to display a target image, and the automatic driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the foregoing intelligent driving method, for example Figure 4 The method described above.

[0170] The present application also provides a chip, which includes a logic circuit and an interface, and the logic circuit is coupled to the interface; the interface is used for inputting and / or outputting information, and the logic circuit is used for executing the foregoing intelligent driving method, for example Figure 4 the method described above.

[0171] The present application also provides a computer-readable storage medium, in which instructions are stored, and when the instructions run on at least one processor, the foregoing intelligent driving method is implemented, for example Figure 4 the method described above.

[0172] The present application also provides a computer program product, which includes computer instructions, and when executed by a computing device, the foregoing intelligent driving method is implemented, for example Figure 4 the method described above.

[0173] In the embodiments of the present application, words such as "exemplarily" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplarily" or "for example" is intended to present relevant concepts in a specific manner.

[0174] Moreover, unless otherwise stated, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first device and the second device are only for ease of description, and do not indicate differences in the structure, importance, etc. of the first device and the second device. In some embodiments, the first device and the second device may also be the same device.

[0175] In the above embodiments, depending on the context, the term "when..." can be interpreted to mean "if...", or "after...", or "in response to determining...", or "in response to detecting...". The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the concept and principles of the present application shall be included in the protection scope of the present application.

[0176] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An intelligent driving method, characterized in that, it is applied to an autonomous driving controller, the autonomous driving controller is included in an intelligent driving terminal, the intelligent driving terminal further includes two side cameras located on the side of the intelligent driving terminal and a rear camera located behind the intelligent driving terminal, and the method includes: Obtain a first image and a second image respectively collected by the two side cameras; According to the first image and the second image, process to obtain a target image, the target image is used to indicate the field of view of the rearview mirror angle of the intelligent driving terminal, and the target image is used for intelligent driving.

2. The method according to claim 1, characterized in that, the intelligent driving terminal further includes a rear camera located behind the intelligent driving terminal, and the process of obtaining the target image according to the first image and the second image includes: Obtain a third image collected by the rear camera; According to the first image, the second image and the third image, process to obtain the target image.

3. The method according to claim 1 or 2, characterized in that, the intelligent driving terminal further includes a screen; After obtaining the target image according to the first image, the second image and the third image, it further includes: Send the target image to the screen for display.

4. The method according to claim 3, characterized in that, the screen includes a first screen and a second screen, and the first screen includes a left screen and / or a right screen; the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror angles on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

5. The method according to any one of claims 2-4, characterized in that, the process of obtaining the target image according to the first image, the second image and the third image includes: Perform linear correction and image processing on the first image, the second image and the third image to obtain the target image, and the image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, image size transformation, etc.

6. The method according to claim 4, characterized in that, the process of sending the target image to the screen for display includes: When the first screen and the display system-on-chip (SoC) of the autonomous driving controller are normal, send the target image to the first screen through the display SoC; When an abnormal situation occurs in the first screen or the display SoC of the autonomous driving controller, send the target image to the second screen.

7. The method according to any one of claims 1-6, characterized in that, the target image further includes a warning message; the method further includes: Process the information collected by the sensor to obtain the warning message.

8. The method according to claim 7, characterized in that, The warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

9. The method according to claim 7 or 8, wherein, the information collected by the sensor includes the first image, the second image, and the third image; processing the first image, the second image, and the third image to obtain a target image includes: obtaining the target image according to the first image, the second image, the third image, and the warning information.

10. The method according to any one of claims 1-9, wherein, the method further includes: when a preset situation occurs in the intelligent driving terminal, adjusting the field of view corresponding to the target image from a first preset value to a second preset value, wherein the second preset value is greater than the first preset value; after the intelligent driving terminal exits the preset situation, adjusting the field of view corresponding to the target image from the second preset value to the first preset value.

11. The method according to claim 10, wherein, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the vehicle behind, etc.

12. An intelligent driving device, wherein, the intelligent driving device includes a transceiver unit and a processing unit, wherein: the transceiver unit is configured to obtain a first image and a second image respectively collected by two side cameras located on the side of the intelligent driving terminal; the processing unit is configured to process the first image and the second image to obtain a target image, the target image is used to indicate the field of view of the rearview mirror angle of the intelligent driving terminal, and the target image is used for intelligent driving.

13. The device according to claim 12, wherein, in terms of processing the first image and the second image to obtain a target image, the processing unit is specifically configured to: obtain a third image collected by the rear camera; process the first image, the second image, and the third image to obtain the target image.

14. The device according to claim 12 or 13, wherein, the intelligent driving terminal further includes a screen; the transceiver unit is further configured to send and display the target image to the screen.

15. The device according to claim 14, wherein, the screen includes a first screen and a second screen, and the first screen includes a left screen and / or a right screen; the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirrors on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

16. The device according to claim 13, wherein, in terms of processing the first image, the second image, and the third image to obtain a target image, the processing unit is specifically configured to: Perform linear correction and image processing on the first image, the second image, and the third image to obtain the target image. The image processing includes one or more of dead pixel removal, white balance, matte extraction, image overlay, image size transformation, etc.

17. The apparatus according to any one of claims 14-16, wherein, in terms of sending the target image to the screen, the transceiver unit is specifically configured to: when the first screen and the display system-on-chip (SoC) of the autonomous driving controller are normal, send the target image to the first screen through the display SoC; when an abnormal situation occurs in the first screen or the display SoC of the autonomous driving controller, send the target image to the second screen.

18. The apparatus according to any one of claims 12-17, wherein, the target image further includes warning information; the processing unit is further configured to process the information collected by the sensor to obtain the warning information.

19. The apparatus according to claim 18, wherein, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

20. The apparatus according to claim 18 or 19, wherein, the information collected by the sensor includes the first image, the second image, and the third image; in terms of processing the first image, the second image, and the third image to obtain the target image, the processing unit is specifically configured to: obtain the target image according to the first image, the second image, the third image, and the warning information.

21. The apparatus according to any one of claims 12-20, wherein, the processing unit is further configured to, when a preset situation occurs in the intelligent driving terminal, adjust the field of view corresponding to the target image from a first preset value to a second preset value, where the second preset value is greater than the first preset value; after the intelligent driving terminal exits the preset situation, adjust the field of view corresponding to the target image from the second preset value to the first preset value.

22. The apparatus according to claim 21, wherein, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the vehicle behind, etc.

23. An intelligent driving terminal, wherein, the intelligent driving terminal includes an autonomous driving controller, two side cameras located on the side of the intelligent driving terminal, a rear camera located behind the intelligent driving terminal, and a screen. The two side cameras and the rear camera are used to collect driving scene images, the screen is used to display the target image, and the autonomous driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the method according to any one of claims 1-11.

24. The intelligent driving terminal according to claim 23, wherein, the intelligent driving terminal is a vehicle, a drone, or a robot.

Images