Intelligent driving method and related device

Abstract

An intelligent driving method and a related device, applied to the field of intelligent driving. In the present application, on the one hand, images during driving are collected by means of two side cameras located on the sides of an intelligent driving terminal and a rear camera located on the rear side of the intelligent driving terminal, and there is no need to redeploy left and right cameras in an electronic rearview mirror system, so that the vehicle body shape can be improved. On the other hand, an autonomous driving controller is reused to process the collected images and send the images to a display, thereby realizing the function of an electronic rearview mirror and reducing the overall vehicle cost. Overall, the present application can significantly reduce costs and improve the vehicle body shape while a rear-view function is realized. In addition, in conjunction with a high-performance perception and computing capability of the autonomous driving controller, richer experience can be provided for a user, and the driving safety is improved.

Description

Intelligent driving method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on November 20, 2023, with application number 202311556375.1 and application name “A Intelligent Driving Method and Related Devices”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of intelligent driving, and in particular to an intelligent driving method and related devices. Background Art

[0003] Safe vehicle operation requires numerous features, such as rearview cameras, which provide visual safety. This feature provides the driver with a visual overview of their surroundings, allowing them to observe oncoming vehicles and other situations, enabling them to make timely decisions. Rearview cameras can be used, along with electronic mirrors, to monitor the physical mirror's field of view. One electronic mirror display mode uses a camera to monitor the physical mirror's field of view, transmitting it to the electronic mirror controller via a video interface. The electronic mirror controller then displays the image on a separate display screen via the video interface.

[0004] However, since there are many sensors deployed on the current smart driving terminals, and these sensors are deployed around the vehicle body, installing an electronic rearview mirror will cause the positions of the sensors on the smart driving terminal and the cameras of the electronic rearview mirror to overlap, which will lead to redundant vehicle body shape and high costs.

[0005] Therefore, how to save costs and improve the vehicle body shape while realizing the rear-view function is an urgent problem to be solved by those skilled in the art.

[0006] Summary of the Invention

[0007] The present application provides an intelligent driving method and related devices, which can save costs and improve the vehicle body shape while realizing the rear-view function.

[0008] In a first aspect, the present application provides an intelligent driving method, which is applied to an autonomous driving controller, wherein the autonomous driving controller is included in an intelligent driving terminal, and the intelligent driving terminal further includes two side cameras located on the sides of the intelligent driving terminal and a rear camera located on the rear side of the intelligent driving terminal. The method includes:

[0009] Acquire the first image and the second image respectively captured by the two side cameras, and process the first image and the second image to obtain a target image, wherein the target image is used to indicate the field of view of the rearview mirror of the intelligent driving terminal, and the target image is used for intelligent driving.

[0010] Optionally, the intelligent driving terminal further includes a rear camera located on the rear side of the intelligent driving terminal, and the processing of obtaining a target image based on the first image and the second image includes: obtaining a third image captured by the rear camera; and processing the target image based on the first image, the second image and the third image.

[0011] The autonomous driving controller can connect to multiple sensors installed in the vehicle, including but not limited to millimeter-wave radar, cameras, lidar, and the global positioning system (GPS). The controller can obtain data collected by these sensors and, through algorithmic processing, derive driving decisions and output them. The controller has high computing power and can obtain rich sensory information, so it can be used to implement rearview cameras.

[0012] The two side cameras can capture the visual field images of the left and right sides of the intelligent driving terminal, while the rear camera can capture the visual field images of the rear side of the intelligent driving terminal. Therefore, the visual field images obtained from the rearview mirror perspective in this application can be obtained based on the visual field images of the left and right sides, or based on the fusion of the visual field images of the left and right sides and the rear side. Compared with traditional rearview mirrors, it can cover a larger visual field range and has a larger field of view adjustment space.

[0013] On the one hand, this application uses two side cameras and a rear camera to capture images during driving, eliminating the need to redeploy the left and right cameras in the electronic rearview mirror system, or even install an optical rearview mirror, thereby improving the vehicle's body shape. On the other hand, this application reuses the autonomous driving controller to process the captured images, realizing the function of the electronic rearview mirror and saving the cost of the entire vehicle. In summary, this application can significantly save costs and improve the vehicle's body shape while realizing the rearview function.

[0014] In addition, combined with the high-performance perception and computing capabilities of the autonomous driving controller, it can provide users with a richer experience and improve driving safety.

[0015] In a possible implementation, the intelligent driving terminal further includes a screen, and after obtaining the target image based on the first image, the second image, and the third image, the terminal further includes: displaying the target image on the screen.

[0016] In another possible embodiment, the screen includes a first screen and a second screen, the first screen includes a left screen and / or a right screen, the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

[0017] Optionally, a first target image is obtained by processing the first image and the third image, and a second target image is obtained by processing the first image and the third image.

[0018] In another possible embodiment, the processing of the first image, the second image, and the third image to obtain the target image includes: performing linear correction and image processing on the first image, the second image, and the third image to obtain the target image, and the image processing includes one or more of bad pixel removal, white balancing, cutout, overlay, image size conversion, etc.

[0019] In the above embodiment, since there will be actual differences between the images captured by traditional exterior mirrors and those captured by left and right side cameras (for example, the first image and the second image captured by the left and right side cameras will be distorted, or the positions of the first image and the second image captured by the left and right side cameras may be offset relative to those of traditional exterior mirrors, and the clarity and accuracy will be lower than those of the images captured by traditional exterior mirrors), the automatic driving controller can be used to perform linear correction, bad pixel removal, white balance and other image processing on the first image and the second image from the left and right side cameras, in combination with the third image captured by the rear camera, as well as operations such as cropping, overlaying, and image size conversion before sending them for display, so as to ensure that the quality of the images after multiplexing the left and right side cameras is not affected.

[0020] Optionally, the left and right side cameras are Advanced Driving Assistance System (ADAS) side cameras. ADAS cameras are used to collect sensory information required for intelligent driving and are typically connected to an autonomous driving controller, which can deploy ADAS algorithms for making intelligent driving decisions based on this sensory information.

[0021] In another possible implementation, displaying the target image on the screen includes: when the first screen and a display system-on-chip (SoC) of the autonomous driving controller are normal, displaying the target image on the first screen via the display SoC. When an abnormality occurs in the first screen or the display SoC of the autonomous driving controller, displaying the target image on the second screen.

[0022] The first screen may be one or more screens. For example, the first screen may be two display screens located on the left and right sides of the cockpit of the intelligent driving terminal, which are referred to as the left screen and the right screen for ease of description. The two display screens are used to display the field of view of the rearview mirror.

[0023] In the above embodiment, the autonomous driving controller includes a display-transmitting SoC, which can be connected to the first screen. After obtaining a target image, the display-transmitting SoC transmits the target image to the first screen for display. For example, the target image includes a first target image from the perspective of the left rearview mirror, and the first screen includes the left screen. The autonomous driving controller transmits the first target image to the left screen via the display-transmitting SoC.

[0024] In some solutions, if the display SoC or the first screen (e.g., the left screen) fails or is damaged, it may affect the driver's field of view, further affecting safety. When the first screen fails (e.g., either the left or right screen fails) or the display SoC of the autonomous driving controller fails, the target image is displayed on the second screen (e.g., the central control screen) to ensure driving safety. In other words, the second screen can be regarded as a backup screen for the rearview function.

[0025] It is understandable that the autonomous driving controller, as the core component of the autonomous driving vehicle, is usually connected to multiple display screens. Therefore, the target image can be displayed through the autonomous driving controller.

[0026] In another possible implementation, taking the second screen as a central control screen as an example, the autonomous driving controller further includes another SoC, which is used to send the target image to the central control screen for display.

[0027] Optionally, if either the left or right screen fails, or the autonomous driving controller's display SoC fails, the autonomous driving controller transmits the target image to the central control screen via the SoC. Furthermore, the SoC transmitting the display to the central control screen can use encoding methods such as H264 and H265.

[0028] In another possible implementation, the autonomous driving controller may process the information collected by the sensor to obtain warning information, and then display the warning information on the screen.

[0029] In another possible implementation, the target image may further include warning information, where the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, and the like.

[0030] In the above embodiment, the driver can be aware of the environment behind the field of vision through the warning information displayed on the screen, so that the driver can make timely decisions in an emergency, improve the driver's braking reaction time in an emergency, and enhance driving safety and convenience.

[0031] In another possible implementation, the information collected by the sensor includes the first image, the second image, and the third image. Processing the first image, the second image, and the third image to obtain the target image includes: obtaining the target image based on the first image, the second image, the third image, and the warning information.

[0032] In the above embodiment, the first, second, and third images are real images, while the warning information is virtual information derived from these real images, such as symbols, text, lines, etc., displayed on real objects. The target image contains both the real image and the warning information, allowing the driver to more easily understand the current driving environment.

[0033] In another possible implementation, the method further includes: when a preset situation occurs in the intelligent driving terminal, adjusting the field of view corresponding to the target image from a first preset value to a second preset value, wherein the second preset value is greater than the first preset value.

[0034] Optionally, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the rear vehicle, etc.

[0035] In the above embodiment, the automatic driving controller can perceive the external environment through two side cameras located on the sides of the intelligent driving terminal, the rear camera located on the rear side of the intelligent driving terminal, lidar, GPS and other equipment. For example, it can confirm that the intelligent driving terminal is currently driving on the first section of the road and turning with a turning amplitude of 40°, and make a decision to expand the field of view corresponding to the current target image based on the current situation. For example, the field of view angle of the target image that can be captured by the left and right cameras is usually 60°. The automatic driving controller can adjust the field of view angle of the target image from 60° to 80°, thereby expanding the field of view of the intelligent driving terminal when turning.

[0036] In another possible implementation, the method further includes: after the intelligent driving terminal exits the preset situation, adjusting the field of view corresponding to the target image from the second preset value to the first preset value.

[0037] Continuing with the target situation of passing a turn as an example, after the turn, the autonomous driving controller can restore the target image to a normal field of view to provide the driver with a better driving experience.

[0038] In a second aspect, the present application provides an intelligent driving device, which includes a module or unit for implementing the method described in the first aspect or any possible implementation of the first aspect. The module can be a software module or a hardware module.

[0039] In the third aspect, the present application provides an intelligent driving terminal, which includes an automatic driving controller, two side cameras located on the sides of the intelligent driving terminal, a rear camera and a screen located on the rear side of the intelligent driving terminal, the two side cameras and the rear camera are used to collect driving scene images, the screen is used to display target images, and the automatic driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the method described in the first aspect or any possible implementation method of the first aspect.

[0040] In a possible implementation, the intelligent driving terminal is a vehicle, a drone, or a robot.

[0041] In a fourth aspect, the present application provides a chip, which includes a logic circuit and an interface, and the logic circuit and the interface are coupled; the interface is used to input and / or output information, and the logic circuit is used to execute the method described in the first aspect or any possible implementation method of the first aspect.

[0042] In a fifth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on at least one processor, the method described in the first aspect or any possible implementation method of the first aspect can be implemented.

[0043] In a sixth aspect, the present application provides a computer program product, comprising computer instructions that, when executed on at least one processor, can implement the method described in the first aspect or any possible implementation of the first aspect. The computer program product can be a software installation package. When the aforementioned method is required, the computer program product can be downloaded and executed on a computing device.

[0044] The beneficial effects of the technical solutions provided in the second to sixth aspects of this application can refer to the beneficial effects of the technical solution in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The following is a brief introduction to the drawings used in describing the embodiments.

[0046] FIG1 is a schematic diagram of the architecture of an autonomous driving vehicle provided in an embodiment of the present application;

[0047] FIG2 is a schematic diagram of a camera deployment position provided in an embodiment of the present application;

[0048] FIG3 is a schematic diagram of a new architecture of an autonomous driving vehicle provided by an embodiment of the present application;

[0049] FIG4 is a schematic diagram of a flow chart of an intelligent driving method provided in an embodiment of the present application;

[0050] FIG5 is a schematic diagram of a scene of an intelligent driving terminal during driving provided by an embodiment of the present application;

[0051] FIG6 is a schematic diagram of a target image processing process provided by an embodiment of the present application;

[0052] FIG7 is a schematic diagram of a cockpit screen display provided in an embodiment of the present application;

[0053] FIG8 is a schematic diagram of a target image provided in an embodiment of the present application;

[0054] FIG9 is a schematic structural diagram of an intelligent driving device 90 provided in an embodiment of the present application;

[0055] FIG10 is a schematic diagram of the architecture of an electronic device 100 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The present application provides an intelligent driving method and related devices that can save costs and improve the appearance of the vehicle body by replacing traditional rearview mirrors with electronic rearview mirrors. The method provided in this application can be applied to an intelligent driving terminal. The intelligent driving terminal can be a mobile phone, a personal notebook, an intelligent robot, an intelligent driving device, etc. The intelligent driving device can include land vehicles, water vehicles, air vehicles, industrial equipment, agricultural equipment, or entertainment equipment. For example, the intelligent driving device can be a vehicle. The vehicle is a vehicle in the broad sense, which can be a transportation vehicle (such as a commercial vehicle, a passenger car, a motorcycle, a flying car, a train, etc.), an industrial vehicle (such as a forklift, a trailer, a tractor, etc.), an engineering vehicle (such as an excavator, a bulldozer, a crane, etc.), agricultural equipment (such as a lawn mower, a harvester, etc.), amusement equipment, a toy vehicle, etc. The embodiments of this application do not specifically limit the type of vehicle. For example, the intelligent driving terminal can be a transportation vehicle such as an airplane or a ship.

[0057] The following describes the architecture of the embodiment of the present application by taking the intelligent driving terminal as an autonomous driving vehicle as an example and combining it with the drawings in the embodiment of the present application.

[0058] To facilitate understanding, the following first introduces relevant terms that may be involved in the embodiments of this application.

[0059] 1. Intelligent Driving

[0060] Intelligent driving is a key development direction for vehicle intelligence. With the advancement of perception technology and the improvement of chip capabilities, intelligent driving provides people with an increasingly rich range of driving functions, gradually realizing different levels of driving experience. The Society of Automotive Engineers (SAE) provides a driving automation grading standard, which includes driving levels L0 to L5. Level L0 represents no automation, with the human driver fully in control of the vehicle, but can receive warnings or assistance from the driving system during driving, such as autonomous emergency braking (AEB), blind spot monitoring (BSM), or lane departure warning (LDW). Level 1 is driving assistance, in which the driving operation is completed by the human driver and the driving system. The driving system can provide driving assistance for the steering wheel or acceleration and deceleration operations through the driving environment, and other driving operations are performed by the human driver, such as adaptive cruise control (ACC) or lane keep assistance / support (LKA / LKS). Level 2 is partial automation, in which the driving environment provides driving assistance for the steering wheel and multiple acceleration and deceleration operations, and other driving actions are performed by the human driver, such as a combination of adaptive cruise control (ACC) and lane keep assistance (LKA / LKS). Level 3 is conditional automation, where the driving system can perform some driving operations, but the human driver needs to respond to the driving system's requests at appropriate times, meaning the human driver needs to be prepared to take over. Level 4 is highly automated, where the driving system can perform all driving operations, but the human driver does not necessarily need to respond to the driving system's requests. For example, when road and environmental conditions permit (such as closed campuses, highways, urban roads, or fixed driving routes), the human driver may not take over. Level 5 is fully automated, where the driving system can autonomously perform all driving operations under various road and environmental conditions that a human driver can handle. As can be seen, at levels 0 to 2, the driving system primarily provides support to the driver, but the driver still needs to supervise the driving and steer, brake, or accelerate as needed to ensure safety. At levels 3 to 5, the driving system can perform all driving operations on behalf of the driver. At level 3, the driver needs to be prepared to take over. At levels 4 and 5, the driving system can achieve full driving under some or all conditions, with the driver being able to choose whether to take over.

[0061] The above classifications are examples and may change with technological evolution or due to different regulations in different countries or regions. For example, the vehicle automation classification system proposed by the Ministry of Industry and Information Technology of China includes six levels of driving automation. Levels 0-2 are driving assistance, where the system assists humans in performing dynamic driving tasks, while the driver remains the primary driver. Levels 3-5 are intelligent driving, where the system replaces humans in performing dynamic driving tasks under designed operating conditions, but when the function is activated, the system is the primary driver. The names and definitions of the levels are as follows: Level 0 driving automation (emergency assistance) means the system cannot continuously perform lateral or longitudinal vehicle motion control during dynamic driving tasks, but has the ability to continuously detect and respond to some objects and events during dynamic driving tasks. Level 1 driving automation (partial driver assistance) means the system continuously performs lateral or longitudinal vehicle motion control during dynamic driving tasks under its designed operating conditions (or design operating range (ODD)) and has the ability to detect and respond to some objects and events appropriate to the lateral or longitudinal vehicle motion control being performed. Level 2 driving automation (combined driver assistance) systems continuously perform lateral and longitudinal vehicle motion control during dynamic driving tasks under their designed operating conditions and have the ability to detect and respond to certain objects and events that are appropriate to the lateral and longitudinal vehicle motion control being performed. Level 3 driving automation (conditionally automated driving) systems continuously perform all dynamic driving tasks under their designed operating conditions. Level 4 driving automation (highly automated driving) systems continuously perform all dynamic driving tasks under their designed operating conditions and automatically implement a minimal risk strategy. Level 5 driving automation (fully automated driving) systems continuously perform all dynamic driving tasks under all drivable conditions and automatically implement a minimal risk strategy. Lateral control primarily controls vehicle steering, for example, by controlling steering wheel torque or angle to steer the vehicle. Longitudinal control primarily controls vehicle speed, for example, by controlling the brake pedal, accelerator pedal, or gear position to control vehicle acceleration / deceleration and braking.

[0062] Regardless of the classification method adopted, the description of the embodiments of the present application can be applied to the above intelligent driving systems that need to partially or fully participate in vehicle driving.

[0063] Please refer to Figure 1, which is a schematic diagram of the architecture of an autonomous driving vehicle provided in an embodiment of the present application. The autonomous driving vehicle includes sensors, controllers and actuators. As shown in Figure 1, the sensors may include one or more of cameras, lidars, or ultrasonic radars. The controller is used for processing and calculation, for example, it may be a domain controller (the autonomous driving controller shown in Figure 1). In some solutions, the controller may run algorithms to perform data processing and logical calculations. The actuator is used to perform control functions on the vehicle or its internal components, including executing control decisions made by the controller, responding to control decisions made by the user, etc. For example, the actuator may include one or more of a steering actuator, a throttle actuator, or a brake actuator. Furthermore, the autonomous driving vehicle also includes an electronic rearview mirror. With the emergence of more configurations, electronic rearview mirrors are increasingly being deployed on autonomous driving vehicles due to their practicality.

[0064] Compared to traditional optical rearview mirrors, electronic rearview mirrors are more expensive and are likely to be installed on more high-end vehicles. Once an electronic rearview mirror is damaged, repair costs significantly more time and money than with a traditional optical rearview mirror. Furthermore, electronic rearview mirrors rely on the proper functioning of technical equipment such as cameras and display screens. Failure or damage to these devices can impair the driver's field of view, further impacting safety. Camera acquisition and information processing take time, potentially resulting in a certain degree of delay compared to traditional optical rearview mirrors. Furthermore, the installation of electronic rearview mirrors can cause overlap between the camera on the autonomous vehicle and the camera on the electronic rearview mirror, leading to redundant vehicle design and high costs. To address these issues, the present invention proposes a new architecture based on the aforementioned architecture. This architecture reuses the autonomous driving controller, two side cameras located on the sides of the autonomous vehicle, and a rear camera located on the rear of the autonomous vehicle to capture and process images during driving. This eliminates the need for left and right cameras, an electronic rearview mirror controller, and other configurations. Based on this new architecture, a new intelligent driving method is proposed, which can save costs and improve vehicle design while still achieving the functionality of an electronic rearview mirror.

[0065] Please refer to Figure 2, which is a schematic diagram of a camera deployment position provided in an embodiment of the present application. As shown in Figure 2, the autonomous driving vehicle includes two side cameras located on the sides of the autonomous driving vehicle and a rear camera located on the rear side of the autonomous driving vehicle, wherein the two side cameras on the sides of the autonomous driving vehicle can be deployed above the left and right front wheels of the autonomous driving vehicle, respectively. The field of view captured by the left side camera (for example, represented as FOV_2) is the field of view captured by the left camera that replaces the electronic rearview mirror (for example, represented as FOV_L), the field of view captured by the right side camera (for example, represented as FOV_1) is the field of view captured by the right camera that replaces the electronic rearview mirror (for example, represented as FOV_R), and the field of view captured by the rear camera can be represented as FOV_3. The autonomous driving controller processes the images corresponding to FOV_1, FOV_2, and FOV_3 and sends them to the screen for display.

[0066] Please refer to Figure 3, which is a schematic diagram of a new architecture of an autonomous driving vehicle provided by an embodiment of the present application. As shown in Figure 3, the autonomous driving vehicle includes an autonomous driving controller 301, two side cameras (left side camera 302 and right side camera 303) located on the sides of the autonomous driving vehicle, a rear camera 304 and screens (left screen 305, right screen 306 and central control screen 307) located on the rear side of the autonomous driving vehicle, and a gateway 308. Among them, the left side camera 302, the right side camera 303 and the rear side camera 304 are used to collect images of the driving scene, the autonomous driving controller 301 is used to process the images of the driving scene, and the left screen 305 and the right screen 306, or the central control screen 307 is used to display the processed images of the driving scene (for example, represented as a target image). The gateway 308 can be used for both wide area network interconnection and local area network interconnection.

[0067] The following describes the intelligent driving method provided by the embodiments of the present application.

[0068] Please refer to Figure 4, which is a flow chart of an intelligent driving method provided in an embodiment of the present application. Optionally, the intelligent driving method can be applied to the architectures shown in Figures 2 and 3.

[0069] The intelligent driving method includes but is not limited to one or more steps from step S401 to step S402 as shown below. It should be understood that for the convenience of description, the order of step S401 to step S402 is described here, and it is not intended to limit the execution to the above order. The embodiment of the present application does not limit the order of execution, execution time, number of executions, etc. of the above one or more steps. Other steps can be interspersed between or before and after these steps as needed, among which:

[0070] Step S401: The autonomous driving controller obtains a first image and a second image respectively captured by two side cameras.

[0071] Exemplarily, the two side cameras are cameras located on the left and right sides of the smart driving terminal respectively.

[0072] The intelligent driving terminal may be a device with computing capabilities, for example, the intelligent driving terminal may be the autonomous driving vehicle shown in Figure 3. Optionally, the intelligent driving terminal may be a single device or a distributed system composed of multiple devices.

[0073] Optionally, the left side camera is an ADAS left side camera, and the right side camera is an ADAS right side camera. The ADAS camera is used to collect perception information required for autonomous driving. It is usually connected to an autonomous driving controller, in which an ADAS algorithm can be deployed. The ADAS algorithm is used to make intelligent driving decisions based on the perception information collected from the sensors.

[0074] As a possible implementation, the autonomous driving controller may also obtain a third image captured by the rear camera.

[0075] Exemplarily, the rear camera is a camera located behind the smart driving terminal.

[0076] Taking the automatic driving controller acquiring the first image, the second image, and the third image as an example, please refer to Figure 5. Figure 5 is a schematic diagram of a scene of an intelligent driving terminal in the driving process provided by an embodiment of the present application. As shown in Figure 5, the current scene is that the intelligent driving terminal is driving on a road. The first image is the data associated with the left side field of view of the intelligent driving terminal in the current driving scene, which is collected by the left side camera. The second image is the data associated with the right side field of view of the intelligent driving terminal in the current driving scene, which is collected by the right side camera. The third image is the data associated with the rear field of view of the intelligent driving terminal in the current driving scene, which is collected by the rear camera. In some embodiments, the above three cameras actively send the collected first image, second image, and third image to the automatic driving controller when the intelligent driving terminal is driving. The first image, second image, and third image may carry location information (for example, the current location) and time information (for example, a timestamp).

[0077] In other embodiments, the autonomous driving controller can actively obtain the first image, the second image, and the third image captured by the above three cameras when the intelligent driving terminal is driving.

[0078] Step S402: The automatic driving controller processes the first image and the second image to obtain a target image.

[0079] The target image is used to indicate the field of view of the rearview mirror of the intelligent driving terminal. At this time, the driver of the intelligent driving terminal or other devices in the intelligent driving terminal (such as the automatic driving controller and other components) can use the target image to determine various situations in the rear field of view that may affect driving and make driving decisions. In other words, the target image can be used for intelligent driving.

[0080] For example, the two side cameras can capture the fields of view to the left and right of the intelligent driving terminal. The autonomous driving controller processes the first and second images to generate a target image, which is a complete rearview mirror field of view obtained by fusing the left and right fields of view. The rearview mirror field of view represents the entire field of view to the side of the intelligent driving terminal that the driver can see.

[0081] As a possible implementation, the autonomous driving controller may process the first image, the second image, and the third image to obtain a target image.

[0082] For example, the two side cameras can capture the left and right views of the intelligent driving terminal, while the rear camera can capture the rear view of the intelligent driving terminal. The autonomous driving controller processes the first, second, and third images to generate a target image, which is a complete rearview mirror perspective image based on the fusion of the left and right views and the rear view image. The rearview mirror perspective image represents the driver's entire view to the sides and rear of the intelligent driving terminal.

[0083] To better demonstrate the completeness of the solution, the following description will be given using the first image, the second image, and the third image as examples.

[0084] As a possible implementation, the autonomous driving controller performs linear correction and image processing on the first image, the second image, and the third image to obtain a target image. The image processing includes one or more of bad pixel removal, white balancing, image cutout, image overlay, and image resizing.

[0085] In one possible scenario, the target image may be a single image. For example, a single image may simultaneously reflect the visual environment of the left side, right side, and rear side of the entire vehicle.

[0086] In some possible scenarios, the target image may include multiple images, and the multiple images may be used to represent the field of view of the rearview mirror in multiple directions.

[0087] Exemplarily, the target image includes a first target image and a second target image, and the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror on both sides of the intelligent driving terminal. At this time, the driver of the intelligent driving terminal or other devices in the intelligent driving terminal (such as the automatic driving controller and other components, etc.) can judge various situations that may affect driving in the rear field of view based on the first target image and the second target image, and make driving decisions. That is, the first target image and the second target image are used for intelligent driving.

[0088] The following describes the process of obtaining a target image, taking the example of a target image including a first target image and a second target image. As one possible implementation, the autonomous driving controller processes the first and third images to obtain the first target image, and processes the first and third images to obtain the second target image.

[0089] Exemplarily, the first target image obtained by the automatic driving controller processing the first image and the third image is a field of view image of the rearview mirror perspective obtained by the fusion of the field of view image on the left side (corresponding to the first image) and the field of view image on the rear side (corresponding to the third image), and the second target image obtained by the automatic driving controller processing the second image and the third image is a field of view image of the rearview mirror perspective obtained by the fusion of the field of view image on the right side (corresponding to the second image) and the field of view image on the rear side (corresponding to the third image).

[0090] As a possible implementation, the autonomous driving controller performs linear correction and image processing based on the first image and the third image to obtain a first target image, and performs linear correction and image processing based on the first image and the third image to obtain a second target image.

[0091] For example, the following example is given in conjunction with the scenario of Figure 5. For example, when the current intelligent driving terminal turns left on the road, the left side camera captures the blind spot, the right side camera captures the turning field of view, and the rear side camera captures the field of view of the rear end of the intelligent driving terminal. Please refer to Figure 6, which is a schematic diagram of a target image processing process provided by an embodiment of the present application. As shown in Figure 6, (a) in Figure 6 illustrates a first image captured by the left side camera, a second image captured by the right side camera, and a third image captured by the rear side camera. The first image, the second image, and the third image are processed by the automatic driving controller to obtain the first target image and the second target image illustrated in (b) in Figure 6. Since there are actual differences between the images captured by traditional exterior mirrors and those captured by left and right side cameras (for example, the first and second images captured by left and right side cameras may be distorted, or the positions of the first and second images captured by left and right side cameras may be offset relative to traditional exterior mirrors, and the clarity and accuracy will be lower than those of images captured by traditional exterior mirrors), the autonomous driving controller can perform linear correction, bad pixel removal, white balance and other image processing on the first and second images from the left and right side cameras, in combination with the third image captured by the rear camera, as well as operations such as cropping, overlaying, and image size conversion before sending them for display, so as to ensure that the quality of the images after multiplexing the left and right side cameras is not affected.

[0092] In some possible scenarios, the intelligent driving terminal also includes a screen. For example, the screen in the intelligent driving terminal may include, but is not limited to, one or more of a left screen, a right screen, a central control screen, and a passenger entertainment screen. The left and right screens may be located on the left and right sides of the intelligent driving terminal, respectively, near the vehicle windows. After obtaining the target image, the autonomous driving controller may send the target image to the screen for display. Specifically, the autonomous driving controller sends the target image to the screen, which then displays the target image.

[0093] In some embodiments, the intelligent driving terminal includes a first screen. The first screen may be one or more screens. For example, the first screen may be two display screens located on the left and right sides of the intelligent driving terminal's cockpit, referred to as the left and right screens for ease of description. These two display screens are used to display the rearview mirror's field of view. Alternatively, the first screen may be either the left or right screen alone.

[0094] Exemplarily, in the case where the target image includes a first target image and a second target image, the first target image is a field of view image from the rearview mirror perspective obtained by fusing the field of view image on the left side (corresponding to the first image) and the field of view image on the rear side (corresponding to the third image), and the second target image is a field of view image from the rearview mirror perspective obtained by fusing the field of view image on the right side (corresponding to the second image) and the field of view image on the rear side (corresponding to the third image). At this time, the left screen is used to display the first target image, and the right screen is used to display the second target image. Exemplarily, please refer to Figure 7, which is a schematic diagram of a cockpit screen display provided by an embodiment of the present application. As shown in Figure 7, the left screen is responsible for displaying the field of view image from the rearview mirror perspective that is a fusion of the first image and the third image (that is, the first target image is displayed), and the right screen is responsible for displaying the field of view image from the rearview mirror perspective that is a fusion of the second image and the third image (that is, the second target image is displayed).

[0095] In some possible scenarios, the autonomous driving controller also includes a display SoC. As a core component of an autonomous vehicle, the autonomous driving controller is typically connected to multiple display screens. For example, the display SoC can be connected to the first screen. Therefore, the target image can be displayed via the autonomous driving controller.

[0096] Optionally, the two side cameras and the rear camera serialize the first image, the second image, and the third image to the autonomous driving controller through a serializer for deserialization, and then input them into the display SoC inside the autonomous driving controller. The target image is displayed through the image signal processor (ISP) module and the display serial interface (DSI) inside the display SoC.

[0097] In some scenarios, when the first screen and the display SoC of the autonomous driving controller are functioning normally, after obtaining the target image, the target image is displayed on the first screen through the display SoC.

[0098] In some cases, the intelligent driving terminal includes a first screen and a second screen. If the first screen fails (for example, either the left or right screen fails) or the display SoC of the autonomous driving controller fails, the target image is displayed on the second screen (for example, the central control screen) to ensure driving safety. In other words, the second screen can be regarded as a backup screen for rearview function.

[0099] If the display SoC or the primary screen (e.g., the left screen) fails or is damaged, it could impair the driver's field of view, further impacting safety. However, since the autonomous driving controller can connect the primary and secondary screens, if the primary screen fails, the target image can be sent to the backup screen, improving driving safety.

[0100] The following uses the second screen as the central control screen and the target image includes the first target image and the second target image as an example to introduce four backup display situations:

[0101] In case 1, when the left screen of the first screen fails, the first target image is displayed through the central control screen, and the second target image is still displayed through the right screen of the first screen.

[0102] In the second case, when the right screen of the first screen fails, the second target image is displayed through the central control screen, and the first target image is still displayed through the left screen of the first screen.

[0103] Case three: when both the left screen and the right screen of the first screen fail, the first target image and the second target image are displayed through the second screen.

[0104] Case 4: When the display SoC of the autonomous driving controller fails, the first target image and the second target image are displayed on the central control screen through another SoC (for example, called a backup SoC) included in the autonomous driving controller.

[0105] Optionally, when either the left or right screen fails, or the display SoC of the autonomous driving controller fails, the backup SoC used to transmit the target image to the central control screen can use encoding methods such as H264 and H265. For example, H264 can transmit standard-definition digital images at speeds below 1Mbps, while H265 can transmit ordinary high-definition audio and video at 720P (resolution 1280*720) at a transmission speed of 1-2Mbps.

[0106] In some possible scenarios, the autonomous driving controller may also process the information collected by the sensors to generate warning information and display the warning information on the screen. The number of sensors may be one or more. When there are multiple sensors, the types of the multiple sensors may vary, including, but not limited to, millimeter-wave radar, cameras, lidar, GPS, and other devices. In this case, the target image displayed on the screen may also include the warning information.

[0107] Furthermore, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0108] Among them, distance information includes, for example, the distance between the vehicle and other vehicles, the distance between the vehicle and the edge of the road (or lane line), the distance between the vehicle and obstacles, etc. Obstacles refer to entities that may delay or hinder the movement of the intelligent driving terminal during the movement of the intelligent driving terminal, such as objects, terrain or facilities. Among them, objects can include living objects or inanimate objects. In addition, the position of the obstacle can be fixed or mobile. Obstacle information includes one or more of the type of obstacle, the position of the obstacle, the distance between the obstacle and the vehicle, etc., such as obstacles such as humans, animals, bicycles, electric vehicles, road bumps, potholes, and even special-shaped obstacles. Lane line information can include information such as the position of the lane line and the distance between the vehicle and the lane line. Traffic sign information can include information such as the content and position of the traffic sign.

[0109] Optionally, the warning information may be generated based on the first image, the second image, and the third image. Specifically, the autonomous driving controller may also obtain the warning information based on the first image, the second image, and the third image, and obtain the target image based on the first image, the second image, the third image, and the warning information.

[0110] For example, the first, second, and third images are real images, while the warning information is virtual information derived from these real images, such as symbols, text, lines, etc., displayed on real objects. The target image contains both the real image and the warning information, allowing the driver to more easily understand the current driving environment. The driving environment may include traffic lights, lane markings, traffic signs, animals, plants, vehicles, crosswalks, or pedestrians.

[0111] For example, please refer to Figure 8, which is a schematic diagram of a target image provided in an embodiment of the present application. As shown in Figure 8, (a) in Figure 8 illustrates the scene of the current intelligent driving terminal driving on the road, and (b) in Figure 8 illustrates the target image captured by the sensor.

[0112] For example, the autonomous driving controller can process images from the side cameras and rear cameras to determine the location of lane lines and indicate the location of lane lines in the target image. As shown in FIG8(b), the location of lane lines is indicated by solid arrows in the target image.

[0113] For example, the autonomous driving controller can process images from the side and rear cameras, as well as GPS positioning data, to obtain distance information. Continuing with Figure 8(b), the target image shows the possible trajectory of the following vehicle using a dotted arrow, as well as the current speed of the following vehicle, which is 65 km / h, to remind the driver to carefully judge the distance between vehicles and drive safely.

[0114] For example, the autonomous driving controller can process images from the side and rear cameras, as well as the LiDAR point cloud data, to obtain distance information. Continuing with Figure 8(b), the autonomous driving controller processes the images to determine the distance between the ego vehicle and the following vehicle as 40 meters, and displays this distance on the target image.

[0115] For example, the autonomous driving controller can process images from the side cameras and rear cameras, as well as the point cloud data from the LiDAR, to determine the type and location of obstacles and indicate them in the target image. Continuing with Figure 8(b), taking the obstacles as a pedestrian and a puppy to the right rear of the intelligent driving terminal, the autonomous driving controller processes the image data obtained to indicate the presence of a pedestrian and a puppy to the right rear of the intelligent driving terminal, and indicates them in the target image using a puppy identifier and a pedestrian identifier, respectively, to alert the driver to the presence of obstacles in the environment surrounding the intelligent driving terminal and to the need for caution.

[0116] In some scenarios, the intelligent driving terminal may encounter various driving conditions during driving (e.g., represented by the following preset conditions). For example, the preset conditions include, but are not limited to, one or more of the following: the intelligent driving terminal turning, changing lanes, and determining the distance to the vehicle behind. Generally speaking, different preset conditions correspond to different rearview vision requirements.

[0117] As a possible implementation method, the automatic driving controller adjusts the size of the target image from a first preset value to a second preset value when a preset situation occurs in the intelligent driving terminal. Furthermore, the second preset value is greater than the first preset value. For example, the automatic driving controller can perceive the external environment through two side cameras located on the sides of the intelligent driving terminal, a rear camera located on the rear side of the intelligent driving terminal, and laser radar, GPS and other equipment. For example, it can confirm that the current intelligent driving terminal is turning on the first section of the highway with a turning amplitude of 40°, and make a decision to expand the field of view corresponding to the current target image according to the current situation. For example, the field of view angle of the target image that can be captured by the left and right cameras is usually 60°. The automatic driving controller can adjust the field of view angle of the target image from 60° to 80°, that is, the field of view of the intelligent driving terminal when turning is expanded.

[0118] Optionally, after the intelligent driving terminal exits the preset state, the autonomous driving controller adjusts the size of the target image from the second preset value to the first preset value. Furthermore, after the turn is completed, the autonomous driving controller may restore the target image to a normal viewing angle to provide the driver with a better driving experience. Optionally, the second preset value is smaller than the first preset value.

[0119] For example, the autonomous driving controller can also narrow the field of view of the target image when the intelligent driving terminal turns, depending on the actual situation. For example, if the field of view of the target image captured by the left and right cameras is usually 60°, the autonomous driving controller can adjust the field of view of the target image from 60° to 40° to make the visible area clearer. After the turn, the field of view is restored to the normal angle to provide the driver with a better driving experience.

[0120] For example, taking the case where the current intelligent driving terminal is driving on the second section of the highway and changing lanes, the automatic driving controller can perceive the external environment through two side cameras located on the sides of the intelligent driving terminal, the rear camera located on the rear side of the intelligent driving terminal, lidar, GPS and other devices, and adaptively make decisions to expand or reduce the field of view corresponding to the current target image according to the current situation. After the lane change is completed, the field of view angle is restored to a normal level to provide the driver with a better driving experience.

[0121] On the one hand, this application uses two side cameras and a rear camera to capture images during driving, eliminating the need to redeploy the left and right cameras in the electronic rearview mirror system, or even install an optical rearview mirror, thereby improving the vehicle's body shape. On the other hand, this application reuses the autonomous driving controller to process the captured images, realizing the function of the electronic rearview mirror and saving the cost of the entire vehicle. In summary, this application can significantly save costs and improve the vehicle's body shape while realizing the rearview function.

[0122] In addition, combined with the high-performance perception and computing capabilities of the autonomous driving controller, it can provide users with a richer experience and improve driving safety.

[0123] The above describes in detail the method of the embodiment of the present application. The following provides an apparatus of the embodiment of the present application.

[0124] It can be understood that the multiple devices provided in the embodiments of the present application, such as the intelligent driving device, are intended to implement the functions in the above-mentioned method embodiments, and include hardware structures, software modules, or a combination of hardware structures and software structures corresponding to executing each function.

[0125] Those skilled in the art should easily appreciate that the units and algorithm steps of each example described in the embodiments disclosed herein can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different device implementations to implement the aforementioned method embodiments in different usage scenarios, and different implementations of the device should not be considered to exceed the scope of the embodiments of the present application.

[0126] The embodiments of the present application may divide the device into functional modules. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one functional module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. In actual implementation, other division methods may be used.

[0127] For example, in the case of dividing the various functional modules of the device in an integrated manner, this application cites several possible intelligent driving devices.

[0128] Please refer to Figure 9, which is a schematic diagram of the structure of an intelligent driving device 90 provided in an embodiment of the present application. The intelligent driving device 90 can be an autonomous driving controller or a component in the autonomous driving controller, such as a chip, software module, integrated circuit, etc. The intelligent driving device 90 has the functions of an electronic device described in the embodiment of the present application. In one possible design, the intelligent driving device 90 includes a transceiver unit 901 and a processing unit 902, wherein:

[0129] The transceiver unit 901 is used to obtain a first image and a second image respectively captured by two side cameras located on the side of the intelligent driving terminal.

[0130] The processing unit 902 is used to process the first image and the second image to obtain a target image, where the target image is used to indicate the field of view of the rearview mirror of the intelligent driving terminal, and the target image is used for intelligent driving.

[0131] In an optional embodiment, in the aspect of processing the target image based on the first image and the second image, the processing unit 902 is specifically configured to: obtain a third image captured by the rear camera, and process the target image based on the first image, the second image, and the third image.

[0132] In another optional implementation, the intelligent driving terminal further includes a screen, and the transceiver unit 901 is further configured to send the target image to the screen for display.

[0133] In another optional embodiment, the screen includes a first screen and a second screen, the first screen includes a left screen and / or a right screen, the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

[0134] In another optional implementation, in obtaining the target image based on the first image, the second image, and the third image, the processing unit 902 is specifically configured to:

[0135] Linear correction and image processing are performed on the first image, the second image, and the third image to obtain the target image, where the image processing includes one or more of bad pixel removal, white balance, cutout, overlay, image size conversion, etc.

[0136] In another optional implementation, in terms of sending the target image to the screen, the transceiver unit 901 is specifically configured to:

[0137] When the first screen and the display system-on-chip (SoC) of the autonomous driving controller are operating normally, the target image is displayed on the first screen via the display SoC. When an abnormality occurs in the first screen or the display SoC of the autonomous driving controller, the target image is displayed on the second screen.

[0138] In another optional embodiment, the target image further includes warning information. The processing unit 902 is further configured to process the information collected by the sensor to obtain the warning information.

[0139] In another optional implementation, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0140] In another optional embodiment, the information collected by the sensor includes the first image, the second image, and the third image. In terms of processing the first image, the second image, and the third image to obtain the target image, the processing unit 902 is specifically configured to:

[0141] The target image is obtained according to the first image, the second image, the third image and the warning information.

[0142] In another optional embodiment, the processing unit 902 is further configured to adjust the field of view corresponding to the target image from a first preset value to a second preset value when a preset condition occurs on the intelligent driving terminal, wherein the second preset value is greater than the first preset value. After the intelligent driving terminal exits the preset condition, the field of view corresponding to the target image is adjusted from the second preset value to the first preset value.

[0143] In another optional embodiment, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the rear vehicle, etc.

[0144] The embodiments of the present application and the above-mentioned method embodiments are based on the same concept, and the technical effects they bring are also the same. For the specific principles, please refer to the description of the above-mentioned embodiments, which will not be repeated here.

[0145] Please refer to Figure 10, which is a schematic diagram of the architecture of an electronic device 100 provided in an embodiment of the present application. As shown in Figure 10, the electronic device 100 can be an automatic driving controller deployed in an intelligent driving terminal, or a cloud server deployed in the cloud, or a device deployed locally. The electronic device 100 can be a single device or a distributed system composed of multiple devices. The electronic device 100 may include a processor 1001 and may optionally include at least one memory 1002. Further optionally, the electronic device 100 may also include a communication interface 1003. Further optionally, a bus 1004 may also be included, wherein the processor 1001, the memory 1002 and the communication interface 1003 are connected via the bus 1004.

[0146] Among them, the processor 1001 is a module that performs arithmetic operations and / or logical operations, and can specifically be a combination of one or more processing modules such as a traditional central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a coprocessor (assisting the central processing unit to complete corresponding processing and applications), and a microcontroller unit (MCU).

[0147] Memory 1002 is used to provide storage space for storing data such as an operating system and computer programs. Memory 1002 can be one or a combination of random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0148] The communication interface 1003 can be used to provide information input or output for the at least one processor. And / or, the communication interface 1003 can be used to receive data sent externally and / or send data externally. It can be a wired link interface such as an Ethernet cable, or a wireless link interface (Wi-Fi, Bluetooth, general wireless transmission, vehicle-mounted communication technology, and other wireless communication technologies). Optionally, the communication interface 1003 can also include a transmitter (such as a radio frequency transmitter, antenna, etc.) or a receiver coupled to the interface.

[0149] In one possible design, at least one processor 1001 in the electronic device 100 is used to execute the aforementioned method, such as the intelligent driving method shown in FIG. 4 .

[0150] Optionally, processor 1001 may be a processor specifically configured to execute these methods (referred to as a dedicated processor for ease of distinction), or may be a processor that executes these methods by invoking a computer program, such as a general-purpose processor. Optionally, at least one processor may include both a dedicated processor and a general-purpose processor. Optionally, the computer program may be stored in memory 1002.

[0151] Optionally, the at least one processor 1001 in the electronic device 100 is configured to call computer instructions to perform the following operations:

[0152] Acquire a first image and a second image captured by the two side cameras, respectively. Process the first image and the second image to obtain a target image, wherein the target image is used to indicate a field of view of a rearview mirror of the intelligent driving terminal, and the target image is used for intelligent driving.

[0153] Optionally, the processor 1001 is further configured to:

[0154] Acquire a third image captured by the rear camera;

[0155] The target image is obtained by processing the first image, the second image and the third image.

[0156] Optionally, the intelligent driving terminal further includes a screen, and the processor 1001 is further configured to:

[0157] The target image is sent to the screen for display.

[0158] Optionally, the screen includes a first screen and a second screen, the first screen includes a left screen and / or a right screen, the target image includes a first target image and a second target image, the first target image and the second target image are respectively used to indicate the field of view of the rearview mirror on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

[0159] Optionally, the processor 1001 is further configured to:

[0160] Linear correction and image processing are performed on the first image, the second image, and the third image to obtain the target image, where the image processing includes one or more of bad pixel removal, white balance, cutout, overlay, image size conversion, etc.

[0161] Optionally, the processor 1001 is further configured to:

[0162] When the first screen and the display system-on-chip (SoC) of the autonomous driving controller are operating normally, the target image is displayed on the first screen via the display SoC. When an abnormality occurs in the first screen or the display SoC of the autonomous driving controller, the target image is displayed on the second screen.

[0163] Optionally, the target image further includes warning information. The processor 1001 is further configured to process the information collected by the sensor to obtain the warning information.

[0164] Optionally, the warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

[0165] Optionally, the information collected by the sensor includes the first image, the second image, and the third image. The processor 1001 is further configured to obtain the target image according to the first image, the second image, the third image, and the warning information.

[0166] Optionally, the processor 1001 is further configured to:

[0167] When a preset situation occurs on the intelligent driving terminal, the field of view corresponding to the target image is adjusted from a first preset value to a second preset value, wherein the second preset value is greater than the first preset value. After the intelligent driving terminal exits the preset situation, the field of view corresponding to the target image is adjusted from the second preset value to the first preset value.

[0168] Optionally, the preset situation includes one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the rear vehicle, etc.

[0169] The embodiments of the present application and the above-mentioned method embodiments are based on the same concept, and the technical effects they bring are also the same. For the specific principles, please refer to the description of the above-mentioned embodiments, which will not be repeated here.

[0170] The present application also provides an intelligent driving terminal, which includes an automatic driving controller, two side cameras located on the sides of the intelligent driving terminal, a rear camera and a screen located on the rear side of the intelligent driving terminal, the two side cameras and the rear camera are used to collect driving scene images, the screen is used to display target images, and the automatic driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the aforementioned intelligent driving method, such as the method described in Figure 4.

[0171] The present application also provides a chip, which includes a logic circuit and an interface, and the logic circuit and the interface are coupled; the interface is used to input and / or output information, and the logic circuit is used to execute the aforementioned intelligent driving method, such as the method described in Figure 4.

[0172] The present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on at least one processor, the aforementioned intelligent driving method, such as the method described in FIG. 4 , is implemented.

[0173] The present application also provides a computer program product, which includes computer instructions and, when executed by a computing device, implements the aforementioned intelligent driving method, such as the method described in FIG4 .

[0174] In the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0175] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish multiple objects and are not used to limit the order, timing, priority, or importance of multiple objects. For example, the first device and the second device are only for ease of description and do not indicate differences in structure, importance, etc. between the first and second devices. In some embodiments, the first device and the second device can also be the same device.

[0176] In the above embodiments, the term "when" can be interpreted to mean "if...", "after...", "in response to determining...", or "in response to detecting...", depending on the context. The above are merely optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the concepts and principles of the present application shall be included in the scope of protection of the present application.

[0177] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An intelligent driving method, characterized in that: Applied to an automatic driving controller, the automatic driving controller is included in an intelligent driving terminal, the intelligent driving terminal further includes two side cameras located on the sides of the intelligent driving terminal and a rear camera located on the rear side of the intelligent driving terminal, the method includes: Acquire a first image and a second image respectively captured by the two side cameras; According to the first image and the second image, a target image is processed to obtain the target image, which is used to indicate the field of view of the rearview mirror of the intelligent driving terminal, and the target image is used for intelligent driving.

2. The method according to claim 1, characterized in that: The intelligent driving terminal further includes a rear camera located at the rear side of the intelligent driving terminal, and the processing to obtain a target image according to the first image and the second image includes: Acquire a third image captured by the rear camera; The target image is obtained by processing the first image, the second image and the third image.

3. The method according to claim 1 or 2, characterized in that: The intelligent driving terminal also includes a screen; After obtaining the target image according to the first image, the second image and the third image, the method further includes: The target image is sent to the screen for display.

4. The method according to claim 3, characterized in that The screen includes a first screen and a second screen, and the first screen includes a left screen and / or a right screen; The target image includes a first target image and a second target image, wherein the first target image and the second target image are respectively used to indicate the field of view of the rearview mirrors on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

5. The method according to any one of claims 2 to 4, characterized in that: The step of obtaining a target image according to the first image, the second image and the third image includes: Linear correction and image processing are performed on the first image, the second image and the third image to obtain the target image, wherein the image processing includes one or more of bad pixel removal, white balance, cutout, overlay, image size conversion, etc.

6. The method according to claim 4, characterized in that The sending and displaying the target image to the screen comprises: When the first screen and the display system-on-chip SoC of the automatic driving controller are normal, the target image is displayed on the first screen through the display SoC; When an abnormal situation occurs in the first screen or the display SoC of the automatic driving controller, the target image is displayed on the second screen.

7. The method according to any one of claims 1 to 6, characterized in that: The target image also includes warning information; The method further comprises: The information collected by the sensor is processed to obtain the warning information.

8. The method according to claim 7, characterized in that The warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

9. The method according to claim 7 or 8, characterized in that: The information collected by the sensor includes the first image, the second image and the third image; The step of obtaining a target image by processing the first image, the second image, and the third image includes: The target image is obtained according to the first image, the second image, the third image and the warning information.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: When a preset situation occurs in the intelligent driving terminal, the field of view corresponding to the target image is adjusted from a first preset value to a second preset value, wherein the second preset value is greater than the first preset value; After the intelligent driving terminal exits the preset situation, the field of view corresponding to the target image is adjusted from the second preset value to the first preset value.

11. The method according to claim 10, characterized in that The preset conditions include one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the rear vehicle, etc.

12. An intelligent driving device, characterized in that: The intelligent driving device comprises a transceiver unit and a processing unit, wherein: The transceiver unit is used to obtain a first image and a second image respectively captured by two side cameras located on the side of the intelligent driving terminal; The processing unit is used to process the first image and the second image to obtain a target image, wherein the target image is used to indicate the field of view of the rearview mirror of the intelligent driving terminal, and the target image is used for intelligent driving.

13. The device according to claim 12, characterized in that In the aspect of obtaining the target image by processing the first image and the second image, the processing unit is specifically used for: Acquire a third image captured by the rear camera; The target image is obtained by processing the first image, the second image and the third image.

14. The device according to claim 12 or 13, characterized in that The intelligent driving terminal also includes a screen; The transceiver unit is also used to send the target image to the screen for display.

15. The device according to claim 14, characterized in that The screen includes a first screen and a second screen, and the first screen includes a left screen and / or a right screen; The target image includes a first target image and a second target image, wherein the first target image and the second target image are respectively used to indicate the field of view of the rearview mirrors on both sides of the intelligent driving terminal, the left screen is used to display the first target image, and the right screen is used to display the second target image.

16. The device according to claim 13, characterized in that In the aspect of obtaining the target image according to the first image, the second image and the third image, the processing unit is specifically used for: Linear correction and image processing are performed on the first image, the second image and the third image to obtain the target image, wherein the image processing includes one or more of bad pixel removal, white balance, cutout, overlay, image size conversion, etc.

17. The device according to any one of claims 14 to 16, characterized in that: In the aspect of sending the target image to the screen, the transceiver unit is specifically used for: When the first screen and the display system-on-chip SoC of the automatic driving controller are normal, the target image is displayed on the first screen through the display SoC; When an abnormal situation occurs in the first screen or the display SoC of the automatic driving controller, the target image is displayed on the second screen.

18. The device according to any one of claims 12 to 17, characterized in that: The target image also includes warning information; The processing unit is also used to process the information collected by the sensor to obtain the warning information.

19. The device according to claim 18, characterized in that The warning information includes one or more of distance information, obstacle information, lane line information, traffic sign information, etc.

20. The device according to claim 18 or 19, characterized in that The information collected by the sensor includes the first image, the second image and the third image; In the aspect of obtaining the target image by processing the first image, the second image and the third image, the processing unit is specifically used for: The target image is obtained according to the first image, the second image, the third image and the warning information.

21. The device according to any one of claims 12 to 20, characterized in that: The processing unit is also used to adjust the field of view corresponding to the target image from a first preset value to a second preset value when a preset situation occurs in the intelligent driving terminal, wherein the second preset value is greater than the first preset value; and after the intelligent driving terminal exits the preset situation, adjust the field of view corresponding to the target image from the second preset value to the first preset value.

22. The device according to claim 21, characterized in that The preset conditions include one or more of the intelligent driving terminal turning, changing lanes, judging the distance to the rear vehicle, etc.

23. An intelligent driving terminal, characterized in that: The intelligent driving terminal includes an automatic driving controller, two side cameras located on the sides of the intelligent driving terminal, a rear camera and a screen located on the rear side of the intelligent driving terminal, the two side cameras and the rear camera are used to capture driving scene images, the screen is used to display target images, and the automatic driving controller is used to process the driving scene images to control the intelligent driving terminal to implement the method described in any one of claims 1-11.

24. The intelligent driving terminal according to claim 23, characterized in that: The intelligent driving terminal is a vehicle, a drone or a robot.

Images