Vehicle control method, vehicle control device and vehicle

Abstract

The invention provides a vehicle control method, a vehicle control device and a vehicle, the method relates to the technical field of driving assistance, the method comprises the following steps: obtaining the gazing information of a driver, the gazing information comprising the gazing position and the gazing duration of the driver; determining a target area in a driving environment of the vehicle based on the gaze information; determining a shooting angle of a first camera in the vehicle based on the target area; and based on the shooting angle, controlling the first camera to shoot at the shooting angle, and controlling a display screen in the vehicle to display a picture shot by the first camera. According to the method, driving environment information can be provided for a driver, and driving safety is ensured.

Description

Technical Field

[0001] The present application relates to the field of driving assistance technology, and more particularly, to a vehicle control method, a vehicle control device, and a vehicle in the field of driving assistance technology. Background Art

[0002] With the development of vehicle control technology, vehicles can be equipped with cameras and display screens, which can be used to display vehicle driving images captured by the cameras. In the existing technology, the shooting angle of the vehicle camera is fixed and cannot adapt to the user's visual attention needs in some dynamic and complex scenes.

[0003] Therefore, how to provide drivers with driving environment information and ensure driving safety is an urgent problem that needs to be solved. Summary of the Invention

[0004] The present application provides a vehicle control method, a vehicle control device and a vehicle, which can provide the driver with driving environment information to ensure driving safety.

[0005] In a first aspect, a vehicle control method is provided, the method comprising: Obtaining the driver's gaze information, including the driver's gaze location and gaze duration; Determine a target area in the vehicle's driving environment based on the gaze information; Determining a shooting angle of a first camera in the vehicle based on the target area; Based on the shooting angle, the first camera is controlled to shoot at the shooting angle, and the display screen in the vehicle is controlled to display the image shot by the first camera.

[0006] In an embodiment of the present application, the driver's gaze information is obtained. Since the gaze information includes the driver's gaze position and gaze duration, the target area, that is, the driver's visual attention area in the current driving environment, can be determined; based on the target area, the shooting angle of the configured camera is further determined, the camera is controlled to adjust the angle according to the target area of ​​the driver's attention, and the captured image is displayed on a display screen in the vehicle; compared with the fixed-angle shooting camera configured in the vehicle in the prior art, the method in the prior art can only capture images of a fixed driving environment, and cannot provide the driver with complete driving environment information in complex driving scenarios, so that there are hidden dangers to driving safety; this solution determines the driver's visual attention area in the current driving environment, and controls the camera to adjust the angle and display the image according to the target area of ​​the driver's attention, so as to provide the driver with more accurate and comprehensive visual information, thereby improving driving safety.

[0007] In conjunction with the first aspect, in some possible implementations, the method further includes: Obtain the vehicle's driving scene; Based on the gaze information, the target area in the vehicle's driving environment is determined, including: Determine the target area based on gaze information and driving scene.

[0008] In an embodiment of the present application, by obtaining the driving scene of the vehicle, the target area of ​​the driver's attention is determined according to the gaze information and the driving scene. Therefore, it is possible to determine the areas that the driver may pay more attention to in different environments according to different driving scenes, so that the determined target area is more in line with the driver's needs in the specific driving scene; combining the driver's gaze information with the vehicle's driving scene can improve the accuracy and rationality of the target area, so that the camera can capture images suitable for the current driving scene of the vehicle, thereby improving driving safety.

[0009] In combination with the first aspect and the above implementations, in some possible implementations, determining the target area based on gaze information and the driving scene includes: Determining, based on the driving scene, a weight of a visual field area corresponding to the driving scene, where the visual field area includes at least one sub-area; The target area is determined based on the weight of the gaze information and the visual field area.

[0010] In an embodiment of the present application, the weight of the field of view area corresponding to the driving scene is determined based on the driving scene, so that the importance of each area in the field of view area can be obtained for the current driving scene; the target area is determined based on the gaze information and the weight of the field of view area, so that the area with high driver attention and importance in the current scene can be determined as the target area, so that the images captured by the camera and displayed on the display screen are more in line with the actual driving needs in the current driving scene, thereby improving driving safety.

[0011] In combination with the first aspect and the above implementations, in some possible implementations, determining the target area based on the weight of the gaze information and the visual field area includes: Based on the gaze position, determining a target sub-region corresponding to the gaze position within the visual field; If the fixation duration exceeds a preset duration and the weight of the target sub-region is greater than a preset weight threshold, the target sub-region is determined as the target region.

[0012] In an embodiment of the present application, the target sub-area corresponding to the gaze position in the field of view is determined based on the gaze position, and the specific area of ​​the driver's gaze position in the preset field of view can be determined; if the gaze duration exceeds the preset duration and the weight of the target sub-area is greater than the preset weight threshold, the sub-area is determined as the target area; since the gaze duration and the weight of the target sub-area are taken into account, it is equivalent to taking into account the driver's subjective behavior and objective environmental factors when determining the target area, so this solution can accurately determine the area where the driver has high attention and is of critical significance to the current driving scene, thereby improving driving safety.

[0013] In combination with the first aspect and the above implementations, in some possible implementations, controlling a display screen in a vehicle to display a picture captured by the first camera includes: Determine the screen display ratio based on the gaze duration; The control display screen displays the image captured by the first camera in an image display ratio.

[0014] In an embodiment of the present application, the screen display ratio is determined based on the gaze duration, and the proportion of the area in the display screen can be dynamically adjusted according to the time the driver continuously pays attention to a certain area; because a longer gaze time usually represents a higher degree of attention paid by the driver to the area, this solution can display the area with high driver attention at a larger proportion on the display screen by adjusting the screen display ratio, thereby improving the visibility of the driver's attention area and helping the driver to observe the driving environment of the target area more clearly.

[0015] In combination with the first aspect and the above implementations, in some possible implementations, the present invention further includes: Obtain the driver's eye socket position and corneal reflection information; The obtaining of the driver's gaze information includes: Obtain gaze position based on orbital position and corneal reflection information.

[0016] In an embodiment of the present application, the method of obtaining the driver's eye socket position and corneal reflection information and determining the gaze position based on the information can effectively reduce the impact of factors such as changes in ambient light and changes in the driver's head posture on the recognition accuracy, compared to the method of determining the gaze position based only on corneal reflection information, and can achieve accurate recognition of the driver's gaze position, thereby providing precise data for subsequent adjustment of the camera angle to improve driving safety.

[0017] In combination with the first aspect and the above implementations, in some possible implementations, the present invention further includes: Get the current driving mode of the vehicle; Obtain driver's gaze information, including: When the current driving mode is the off-road mode, gaze information is obtained.

[0018] In an embodiment of the present application, when it is detected that the current driving mode is the off-road mode, the driver's gaze information is obtained, and the control of the camera and the display of the display screen are determined based on the gaze information; since the driving environment in the off-road mode is complex, the driver's attention to the surrounding environment of the vehicle body is higher, so this solution obtains the gaze information in the off-road mode and performs subsequent processing based on the gaze information, which can improve the targetedness and control efficiency of the vehicle's control logic and improve the safety of the vehicle in off-road mode.

[0019] In combination with the first aspect and the above implementations, in some possible implementations, the vehicle is configured with a second camera, further comprising: When it is detected that the target area includes the preset area, the second camera is turned on to capture the lateral environment of the vehicle; The control display shows the image captured by the second camera.

[0020] In an embodiment of the present application, when it is detected that the target area includes a preset area, the second camera configured in the vehicle for photographing the lateral environment of the vehicle is turned on, and the display screen is controlled to display the image captured by the second camera. When the area of ​​concern to the driver involves the lateral environment of the vehicle, supplementary information of the vehicle's lateral environment perspective can be provided in a timely manner, thereby enhancing the driver's perception of the vehicle's surrounding environment, thereby improving the driving assistance effect and driving safety under complex road conditions.

[0021] In combination with the first aspect and the above implementation, in some possible implementations, controlling the display screen to display the image captured by the second camera includes: The image captured by the second camera is stitched together with the image captured by the first camera to obtain a target image; The Control Display shows the target screen.

[0022] In an embodiment of the present application, when the target area includes a preset area, the target image is obtained by splicing the image captured by the second camera with the image captured by the first camera, and the display screen is controlled to display the target image. The perspective information captured by multiple cameras can be integrated and spliced ​​to improve the integrity of the image information; therefore, this solution can enhance the driver's perception of the driving environment and improve the accuracy and safety of driving assistance.

[0023] In a second aspect, a vehicle control device is provided, the device comprising: An acquisition module is used to acquire the driver's gaze information, which includes the driver's gaze position and gaze duration; The processing module is used to determine a target area in the vehicle's driving environment based on the gaze information; determine the shooting angle of a first camera in the vehicle based on the target area; control the first camera to shoot at the shooting angle based on the shooting angle, and control the display screen in the vehicle to display the image captured by the first camera.

[0024] It should be understood that the expansion, limitation, explanation and description of the relevant content in the above-mentioned first aspect also apply to the same content in the second aspect.

[0025] In a third aspect, a vehicle is provided, comprising a memory and a processor; the memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0026] In a fourth aspect, a computer program product is provided, which includes: computer program code, which, when running on a computer, enables the computer to execute the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0027] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the vehicle control method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a vehicle system architecture provided by an embodiment of the present application; Figure 2 is a schematic flow chart of a vehicle control method provided in an embodiment of the present application; Figure 3 is a schematic diagram of a field of view provided in an embodiment of the present application; Figure 4 is a schematic flow chart of another vehicle control method provided in an embodiment of the present application; Figure 5 is a structural schematic diagram of a vehicle control device provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will clearly and thoroughly describe the technical solutions in this application in conjunction with the accompanying drawings. In the description of the embodiments of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more than two.

[0030] In the following, the terms "first" and "second" are used for descriptive purposes only and should not be understood to imply or suggest relative importance or implicitly indicate the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0031] In the prior art, a vehicle may be equipped with a camera and a display screen. The camera may be used to capture the vehicle's driving environment, such as the road conditions in front of and on both sides of the vehicle, the distribution of obstacles, slope changes, and the surrounding environment. The display screen may display the images captured by the camera. However, the vehicle's camera has a fixed shooting angle. When driving on complex road conditions, the driver usually needs to frequently observe and judge the driving environment around the vehicle to obtain key information related to the current driving scene. For example, when a vehicle is climbing a steep slope, the driver needs to focus on the obstacles in front and the vehicle's climbing posture; when the vehicle passes through a flooded section, the driver needs to focus on the water level and the water flow around the vehicle; when the vehicle is stuck in mud and needs to be rescued, the driver needs to focus on the contact between the tires and the ground. When faced with the above situations, the prior art has a fixed camera angle, which easily leads to missing key information that the driver needs to focus on, thus posing a driving hazard.

[0032] In view of this, the present application provides a vehicle control method, a vehicle control device and a vehicle, which first obtains the driver's gaze information. Since the gaze information includes the driver's gaze position and gaze duration, the target area, that is, the driver's visual focus area in the current driving environment, can be determined based on the gaze information; based on the target area, the shooting angle of the camera is further determined, and the camera is controlled to adjust the angle according to the target area of ​​the driver's focus, and the captured picture is displayed on a display screen in the vehicle; compared with the fixed-angle shooting camera configured in the vehicle in the prior art, which can only capture pictures of a fixed driving environment, the method in the prior art cannot provide the driver with complete driving environment information in complex driving scenarios, so that there are hidden dangers to driving safety; this solution determines the driver's visual focus area in the current driving environment, and controls the camera to adjust the angle and display the picture according to the target area of ​​the driver's focus, so as to provide the driver with more accurate and comprehensive visual information, thereby improving driving safety.

[0033] For example, Figure 1 This is a schematic diagram of a vehicle system architecture provided by an embodiment of the present application. Figure 1 As shown, the vehicle 100 may include a gyroscope 101 , a three-dimensional acceleration sensor 102 , an in-vehicle infrared camera 103 , a multi-DOF pan / tilt controller 104 , a roof camera 105 , a side camera 106 , a HUT 116 and a display screen 117 .

[0034] The gyroscope 101 can be used to detect the vehicle's posture information, including pitch angle, roll angle and heading angle, to assist in determining the vehicle's current posture state and provide reference data for driving scene recognition and camera angle adjustment.

[0035] The three-dimensional acceleration sensor 102 can be used to detect the acceleration information of the vehicle in three directions to determine the dynamic state and driving conditions of the vehicle, and combined with the gyroscope data for driving scene analysis and driving status monitoring.

[0036] The in-car infrared camera 103 can be used to collect image information of the driver's eye area in order to identify the eye movement trajectory and calculate the gaze point position, thereby obtaining the driver's gaze information. For example, the in-car infrared camera 103 can be equipped with a camera with a wide-angle lens and a low-light enhancement sensor. The wide-angle lens can expand the shooting range to ensure that the driver's entire facial image can be captured; the low-light enhancement sensor can still clearly capture the driver's eye image in a dimly lit environment. In order to resist the bumpy interference of the vehicle during off-road driving, the camera adopts a special shock-absorbing mounting structure to ensure the stability of image acquisition under complex road conditions.

[0037] The multi-degree-of-freedom pan-tilt controller 104 can be used to drive the roof camera to adjust in multiple directions according to the shooting angle determined by the target area to ensure that the camera can be aimed at the target area for shooting.

[0038] The roof camera 105 can be used to capture images of the driving environment around the vehicle to assist the driver in observing complex terrain.

[0039] The side cameras 106 can be used to capture images of the vehicle's lateral environment and are activated when it detects that the target area includes a preset lateral area, assisting the driver in observing obstacles or road conditions on both sides of the vehicle.

[0040] The display screen 117 can be used to display images captured by the roof camera and the side cameras, or to display the target images after splicing, to provide the driver with complete visual information.

[0041] Among them, HUT 116 is the vehicle's onboard host, which can also be called a vehicle computer, host, onboard host system, cockpit domain controller, onboard information processing unit, multimedia host, cockpit host, terminal information display unit, etc. It is the core control unit of the vehicle's electronic system, responsible for receiving data from various sensors and cameras, performing driving scene analysis, gaze information parsing, camera control, image processing and display control functions. HUT 116 may include an off-road mode activation control module 107, a hybrid positioning model 108, a multi-degree-of-freedom pan-tilt control module 109, a roof camera processing module 110, a side camera processing module 111, a gaze point dynamic analysis unit module 118 and a display data output driver 115.

[0042] The off-road mode activation control module 107 can be used to detect whether the current driving mode is the off-road mode, and start the corresponding driving environment recognition, camera control and image display logic when the off-road mode is activated.

[0043] The hybrid positioning model 108 can be used to identify the driver's gaze position and locate it. Exemplarily, the hybrid positioning model 108 can use a hybrid positioning model based on corneal reflection and eye socket contour to determine the driver's eye position. Traditional pupil positioning algorithms may be affected by factors such as lighting and eye movement speed, resulting in inaccurate positioning. The hybrid positioning model of the present application combines information on corneal reflection and eye socket contour to more accurately locate the position of the eye. Specifically, corneal reflection refers to the reflection spots generated on the corneal surface when infrared light shines on the eye. By detecting the position of these reflection spots, the approximate direction of the eye can be obtained. At the same time, the information on the eye socket contour can serve as an auxiliary to further improve the accuracy of positioning. In actual applications, the system first extracts the feature points of the corneal reflection spots and the eye socket contour through an image processing algorithm, and then calculates the center position and gaze direction of the eye based on the positional relationship of these feature points.

[0044] The multi-degree-of-freedom pan / tilt control module 109 receives the target area's location information and sends control commands to the multi-degree-of-freedom pan / tilt controller 104 to adjust the roof camera's shooting angle. For example, the multi-degree-of-freedom pan / tilt control module 109 can be equipped with a multi-degree-of-freedom pan / tilt with dual-axis motors, enabling flexible vertical and horizontal camera rotation. The pan / tilt response time is less than 200ms, enabling rapid adjustment of the camera's shooting angle based on the driver's gaze.

[0045] Optionally, the multi-degree-of-freedom (MDF) pan / tilt control module 109 can be equipped with a shake compensation function to ensure stable camera recording even when the vehicle shakes during off-road driving. This module integrates terrain sensors (such as a gyroscope). The gyroscope in the MDF pan / tilt control module 109 can detect changes in the vehicle's posture in real time. Based on the gyroscope's feedback, the pan / tilt movement is compensated, eliminating the impact of vehicle shake on camera recording and ensuring clear and stable images.

[0046] The roof camera processing module 110 can be used to receive images captured by the roof camera, perform distortion correction, color correction, cropping or splicing on the images, generate image data for display and transmit the image data.

[0047] The side camera processing module 111 can be used to receive images captured by the side camera, perform distortion correction, color correction, cropping or splicing on the images, generate image data for display and transmit the image data.

[0048] The display data output driver 115 can be used to transmit the processed target image to the display screen 117 and control the resolution, display ratio and display area of ​​the image.

[0049] The gaze point dynamic analysis module 118 can be used to analyze and interpret the driver's gaze behavior in real time, thereby extracting key areas of focus related to the driving environment from the driver's line of sight. For example, the gaze point dynamic analysis module 118 can include an eye movement trajectory recognition module 112, a gaze grid coordinate calculation module 113, and an off-road scene attention weight calculation module 114.

[0050] The eye movement trajectory recognition module 112 can be used to analyze the driver's eye images collected by the in-vehicle infrared camera 103, extract the eye movement trajectory information, and determine the gaze direction and change trend.

[0051] The gaze grid coordinate calculation unit 113 may be configured to map the eye gaze direction to a preset visual field grid and calculate the grid coordinate position of the gaze point.

[0052] The off-road scene attention weight calculation unit 114 can be used to combine the driving scene and the preset weight of the sub-area where the gaze point is located to calculate the comprehensive attention weight value of the target area to assist in determining whether to set the area as the shooting target of the camera.

[0053] The following combination Figure 2 A vehicle control method provided in an embodiment of the present application is described in detail.

[0054] Figure 2 This is a schematic flow chart of a vehicle control method provided in an embodiment of the present application. Figure 2 As shown, the method 200 includes S210 to S240, and S210 to S240 are described in detail below.

[0055] For example, Figure 2 The method 200 shown may be executed by a vehicle; or, executed by a processor in a vehicle; or, executed by a chip mounted in a processor in a vehicle; or, executed by a software platform integrated in an electronic device.

[0056] S210: Obtain the driver's gaze information.

[0057] Among them, the gaze information includes the driver's gaze position and gaze duration; the gaze position can be used to indicate the driver's current gaze position, and can also be used to indicate at least one position that the driver has gazed at within a preset time; when the gaze position is used to indicate the driver's current gaze position, the gaze duration can be used to indicate the duration for which the driver has gazed at the gaze position; when the gaze position is used to indicate at least one position that the driver has gazed at within a preset time, the gaze duration can be used to indicate the cumulative duration for which the driver has gazed at at least one position within the preset time.

[0058] In an embodiment of the present application, the vehicle may be equipped with a driver monitoring system, which may be used to track the driver's eye movement trajectory to determine the gaze position and record the driver's gaze time at a certain gaze position.

[0059] Among them, the driver monitoring system may include an eye movement trajectory recognition unit, which can process the driver's eye images collected by the camera in the vehicle cabin through a deep learning model to identify the driver's movement trajectory.

[0060] Optionally, a lightweight YOLOv5s model can be used to process the collected eye images. YOLOv5s is a lightweight object detection model with high detection speed and accuracy, capable of detecting and tracking the eyeballs in the image in real time. In the embodiment of the present application, the collected image is input into a trained YOLOv5s model, which outputs information such as the position and movement direction of the eyeballs.

[0061] Exemplarily, the coordinates of the driver's gaze point are obtained (eg, X=-1.2m, Y=1.5m), the coordinates are determined to be located in the left front wheel contact area, and the gaze duration at this location is continuously monitored for 2.3 seconds.

[0062] Optionally, it is possible to detect whether the change in gaze position exceeds a preset threshold (eg, 2°), and continuously record the time when the change in the driver's gaze position is continuously smaller than the preset threshold.

[0063] In one implementation, the method further includes: Obtain the driver's eye socket position and corneal reflection information; The obtaining of the driver's gaze information includes: Obtain gaze position based on orbital position and corneal reflection information.

[0064] The eye socket position may be used to represent the driver's eye socket or the contour area of ​​the eye socket; and the corneal reflection information may include the corneal reflection spot position.

[0065] In an embodiment of the present application, a facial image of the driver is captured by an in-car camera, and the orbital position and corneal reflection information are extracted, and the driver's gaze position is calculated based on the orbital position and corneal reflection information.

[0066] For example, the driver's facial image is captured by the in-car camera, and the driver's eye socket position, that is, the outline position of the driver's eye socket, is identified based on the facial image to determine a stable reference frame of the eyes on the face; the position of the reflected light spot formed by infrared light on the corneal surface is detected, and the offset of the reflected light spot position relative to the center of the eye socket is calculated to determine the direction and angle of eyeball rotation, thereby determining the driver's gaze position.

[0067] In the above implementation method, the method of obtaining the driver's eye socket position and corneal reflection information and determining the gaze position based on this information can effectively reduce the impact of factors such as changes in ambient light and changes in the driver's head posture on the recognition accuracy, compared to the method of determining the gaze position based only on corneal reflection information. It can achieve accurate recognition of the driver's gaze position, thereby providing precise data for subsequent adjustment of the camera angle to improve driving safety.

[0068] In one implementation, the method further includes: Get the current driving mode of the vehicle; Obtain driver's gaze information, including: When the current driving mode is the off-road mode, gaze information is obtained.

[0069] Among them, the off-road mode is used for the vehicle to travel on unpaved roads or complex terrain. In off-road mode, the vehicle's power system, suspension system, traction system, etc. will be adjusted, such as increasing the low-speed torque output of the engine / motor to facilitate the vehicle to start and climb on low-adhesion roads.

[0070] In an embodiment of the present application, a sensor configured in the vehicle for detecting the vehicle status is used to determine whether the vehicle enters the off-road mode. When it is detected that the vehicle enters the off-road mode, the driver's gaze information is automatically obtained and S210 to S240 are executed.

[0071] Exemplarily, when the sensor detects that the current slope of the vehicle is greater than a preset slope (eg, 15°), or detects that the driving mode of the vehicle is switched to the four-wheel drive mode, it is determined that the vehicle enters the off-road mode.

[0072] For example, the user may issue an instruction to enter the off-road mode, and when it is detected that the user has issued an instruction to enter the off-road mode, it is determined that the vehicle has entered the off-road mode.

[0073] Optionally, the instruction may be detected through voice recognition, gesture recognition, text input, or key detection.

[0074] Optionally, when the vehicle is in sports mode, snow mode, wading mode or climbing mode, the driver's gaze information is obtained, the target area is determined based on the gaze information, the shooting angle of the camera is determined, and the shooting is performed and displayed on the screen.

[0075] In the above implementation, when it is detected that the current driving mode is off-road mode, the driver's gaze information is obtained, and the control of the camera and the display of the display screen are determined based on the gaze information; since the driving environment in off-road mode is complex, the driver's attention to the surrounding environment of the vehicle body is higher, so this solution obtains gaze information in off-road mode and performs subsequent processing based on the gaze information, which can improve the targetedness and control efficiency of the vehicle's control logic and improve the safety of the vehicle in off-road mode.

[0076] S220: Determine a target area in the driving environment of the vehicle based on the gaze information.

[0077] The target area is used to represent the area in the vehicle driving environment that the driver pays close attention to, such as the left front wheel contact area, the right front wheel contact area, the left area, the right area, and the front wheel contact area.

[0078] In an embodiment of the present application, a target area where the driver has a high degree of attention is determined based on the driver's gaze position and gaze duration.

[0079] In one implementation, the method further includes: Obtain the vehicle's driving scene; Based on the gaze information, the target area in the vehicle's driving environment is determined, including: Determine the target area based on gaze information and driving scene.

[0080] In an embodiment of the present application, the current driving scene of the vehicle is obtained, and the target area that needs to be focused on is determined based on the driver's gaze position, gaze duration and the current driving scene of the vehicle.

[0081] For example, the current driving data of the vehicle is obtained through sensors configured on the vehicle to determine that the vehicle is currently in a muddy scene. Since it is usually necessary to pay attention to the left front wheel contact area and the right front wheel contact area in a muddy scene, the target area that needs to be focused on is determined in combination with the muddy scene and the driver's gaze position and gaze duration.

[0082] In the above implementation method, by obtaining the driving scene of the vehicle, the target area of ​​the driver's attention is determined according to the gaze information and the driving scene. Therefore, according to different driving scenes, the areas that the driver may pay more attention to in different environments can be determined, so that the determined target area is more in line with the driver's needs in the specific driving scene; combining the driver's gaze information with the vehicle's driving scene can improve the accuracy and rationality of the target area, so that the camera can capture pictures suitable for the current driving scene of the vehicle, thereby improving driving safety.

[0083] In one implementation, the method includes: Based on the driving scene, determining the weight of the field of view area corresponding to the driving scene; The target area is determined based on the weight of the gaze information and the visual field area.

[0084] Among them, the field of view area includes at least one sub-area, which is used to represent the area of ​​the driving environment around the vehicle that can be captured by the camera and used for display. The field of view area can be divided into multiple sub-areas according to the structure of the vehicle, such as the left front wheel contact area, the right front wheel contact area, the left area, the right area and the front wheel contact area, etc.

[0085] It should be noted that in order to accurately determine the driver's gaze area, the field of view area can be divided into several sub-areas, and the grid coordinates of the driver's gaze can be calculated based on the eye movement trajectory recognition unit. For example, when it is detected that the driver's eyes are looking at a sub-area, the coordinate information of the grid is automatically recorded.

[0086] For example, the field of view area can be used to represent the key field of view around the vehicle that the driver needs to pay attention to in some driving scenarios (for example, off-road mode). The field of view range may include the areas in front and on both sides of the vehicle that are closely related to the movement of the wheels, such as Figure 3 As shown, the field of view area 300 can be divided into 301, 302, 303, 304 and 305, where 301 is the left front wheel contact area, 302 is the right front wheel contact area, 303 is the front wheel contact area, 304 is the left area, and 305 is the right area.

[0087] For example, if the current vehicle driving scene is detected as a muddy scene, the weights corresponding to the muddy scene in the attention weight library are obtained, such as the weight of the left front wheel contact area is 0.6, the weight of the right front wheel contact area is 0.6, the weight of the left area is 0.1, the weight of the right area is 0.1, and the weight of the front wheel contact area is 0.3. The target area is determined based on the weight of the driver's gaze position in the field of view and the gaze duration.

[0088] It should be understood that the weights corresponding to the sub-areas in the field of view can be used to represent the attention priority of each area in the scene. In an embodiment of the present application, the attention weight library corresponding to each scene can be obtained based on the gaze information collected in advance during the safe driving process.

[0089] For example, when the current vehicle driving scene is a muddy scene, it is detected that the driver's gaze position is located in the left front wheel contact area, and the gaze duration is 2 seconds. Based on the gaze duration and the weight of the left front wheel contact area corresponding to the muddy scene, the attention value is calculated, and the attention value is compared with the preset attention value. If it is greater than the preset attention value, the left front wheel contact area is determined as the target area.

[0090] For example, when the current vehicle driving scene is a muddy scene, it is detected that the driver's gaze time on the left front wheel contact area is 20 seconds, the gaze time on the right front wheel contact area is 20 seconds, and the gaze time on the front wheel contact area is 20 seconds within the past preset time (for example, one minute). Based on the gaze time of each area and the weight of each area in the muddy scene, the attention value is calculated, and the area with an attention value greater than the preset attention value is determined as the target area.

[0091] It should be understood that the above-mentioned weights corresponding to muddy scenes are examples, and the embodiments of the present application do not limit the driving scenes and the weight values ​​in different scenes.

[0092] In the above implementation, the weight of the field of view area corresponding to the driving scene is determined based on the driving scene, so that the importance of each area in the field of view area can be obtained for the current driving scene; the target area is determined based on the gaze information and the weight of the field of view area, so that the area with high driver attention and importance in the current scene can be determined as the target area, so that the images captured by the camera and displayed on the display screen are more in line with the actual driving needs in the current driving scene, thereby improving driving safety.

[0093] In one implementation, the method includes: Based on the gaze position, determining a target sub-region corresponding to the gaze position within the visual field; If the fixation duration exceeds a preset duration and the weight of the target sub-region is greater than a preset weight threshold, the target sub-region is determined as the target region.

[0094] In an embodiment of the present application, based on the driver's gaze position, a target sub-area corresponding to the gaze position within a preset field of view is determined. If the weight corresponding to the target sub-area in the current scene is greater than a preset weight threshold, and the driver's gaze time at the position exceeds a preset time, the area is determined to be a target area that requires special attention.

[0095] For example, when the current vehicle driving scenario is a wading scenario, it is detected that the driver's contact time with the front wheel is 3 seconds, which exceeds the preset time (for example, 2 seconds), and the weight of the front wheel contact area in the wading scenario is 0.6, which is greater than the preset weight threshold (for example, 0.5), and the front wheel contact area is determined to be the target area.

[0096] For example, when the current driving scene of the vehicle is a wading scene, it is detected that the driver's gaze time on the left front wheel contact area in the past preset time (for example, one minute) is 10 seconds, which is less than the preset time (for example, 15 seconds); it is detected that the driver's gaze time on the right front wheel contact area in the past preset time is 30 seconds, and the weight of the right front wheel contact area in the wading scene is 0.4, which is less than the preset weight threshold (for example, 0.5); it is detected that the gaze time on the front wheel contact area in the past preset time is 20 seconds, which exceeds the preset time, and the weight of the right front wheel contact area in the wading scene is 0.6, which is greater than the preset weight threshold (for example, 0.5), so the front wheel contact area is determined as the target area.

[0097] Optionally, if it is detected that the driver has been gazing at a sub-region for most of the time during the duration period, and the sub-region has a higher weight corresponding to the current driving scenario, then the sub-region is determined to be the current target region.

[0098] For example, if it is detected that the driver has stared at a certain area for more than a preset percentage (for example, 50%) within a preset time (for example, one minute), for example, it is detected that the driver has stared at the front wheel contact area for more than 30 seconds in the past minute, and the weight of the front wheel contact area in the current driving scenario is greater than the preset weight, the front wheel contact area is determined to be the target area.

[0099] In the above implementation, the corresponding target sub-area in the field of view is determined based on the gaze position, so that the specific area of ​​the driver's gaze position in the preset field of view can be determined; if the gaze duration exceeds the preset duration and the weight of the target sub-area is greater than the preset weight threshold, the sub-area is determined as the target area; since the gaze duration and the weight of the target sub-area are taken into account, it is equivalent to taking into account the driver's subjective behavior and objective environmental factors when determining the target area. Therefore, this solution can accurately determine the area where the driver pays high attention and is of critical significance to the current driving scene, thereby improving driving safety.

[0100] S230: Determine a shooting angle of a first camera in the vehicle based on the target area.

[0101] Among them, the first camera is used to shoot the driving environment around the vehicle and can be configured on the roof.

[0102] In an embodiment of the present application, after the target area is determined, the shooting angle corresponding to the target area is determined according to the relative position of the area and the vehicle.

[0103] For example, when a target sub-area is detected, the corresponding shooting angle of the target sub-area can be determined based on a predetermined mapping relationship between each sub-area in the field of view and each shooting angle of the first camera. For example, when the target area is detected as the front wheel contact area, the shooting angle is tilted downward by 15 degrees.

[0104] S240: Based on the shooting angle, control the first camera to shoot at the shooting angle, and control a display screen in the vehicle to display the image shot by the first camera.

[0105] In an embodiment of the present application, after determining the shooting angle of the first camera, the first camera is controlled to rotate to shoot at a shooting angle corresponding to the target area, and the captured image is transmitted to the vehicle's display screen in real time, so that the driver and passengers can intuitively see the driving environment of the target area.

[0106] Optionally, after controlling the first camera to shoot at a shooting angle, the macro mode can be turned on to clearly capture the contact between the front wheel and the ground.

[0107] In one implementation, the method includes: Determine the screen display ratio based on the gaze duration; The control display screen displays the image captured by the first camera in an image display ratio.

[0108] In an embodiment of the present application, the screen display ratio is determined based on the length of time the driver gazes at the target area, and the screen captured by the camera is displayed on the display screen according to the determined screen display ratio.

[0109] For example, it is detected that the driver stares at the front wheel contact area for 30 seconds within a preset time (for example, one minute), which accounts for 50% of the preset time, and the screen display ratio corresponding to the front wheel contact area is determined to be 50%. When the display screen displays the captured image, the image of the front wheel contact area is displayed at 50% of the entire image showing the vehicle's driving environment.

[0110] Exemplarily, it is detected that the target areas are the front wheel contact area and the left front wheel contact area, and it is detected that the driver's gaze time at the front wheel contact area within a preset time (for example, one minute) is 30 seconds, accounting for 50% of the preset time, and the screen display ratio corresponding to the front wheel contact area is determined to be 50%; it is detected that the gaze time at the left front wheel contact area is 15 seconds, accounting for 25% of the preset time, and the screen display ratio corresponding to the left front wheel contact area is determined to be 25%; the screen of the front wheel contact area is displayed at 50% of the entire screen displaying the vehicle driving environment, and the screen of the left front wheel contact area is displayed at 25% of the entire screen displaying the vehicle driving environment.

[0111] Optionally, the gaze duration may be positively correlated with the screen display ratio. The longer the gaze duration, the larger the corresponding screen display ratio, thereby highlighting the key areas.

[0112] In one embodiment, the display can be performed in a progressive manner. For example, when it is detected that the driver continues to stare at the left front wheel contact area of ​​the vehicle for 3 seconds, the image of the left front wheel contact area in the image captured by the first camera is increased from the original 30% to 50%.

[0113] In the above implementation, the screen display ratio is determined based on the gaze duration, and the proportion of the area in the display screen can be dynamically adjusted according to the time the driver continuously pays attention to a certain area; because the longer the gaze time usually means that the driver pays more attention to the area, this solution can display the area with high driver attention at a larger proportion on the display screen by adjusting the screen display ratio, thereby improving the visibility of the driver's attention area and helping the driver to observe the driving environment of the target area more clearly.

[0114] In one implementation, the method further includes: When it is detected that the target area includes the preset area, turning on the second camera; The control display shows the image captured by the second camera.

[0115] The second camera can be used to capture the lateral environment of the vehicle.

[0116] In an embodiment of the present application, the preset area can be a lateral area of ​​the vehicle, such as the left area or the right area. When it is detected that the target area includes the left area or the right area, a camera for capturing the lateral environment of the vehicle is activated, and the image captured by the camera is transmitted to a display screen, which displays the lateral driving environment of the vehicle.

[0117] It should be noted that the second camera can be installed below or above the rearview mirror of the vehicle, or installed at the left and right tires of the vehicle, or installed on the suspension of the vehicle.

[0118] In the above implementation, when it is detected that the target area includes a preset area, the second camera configured in the vehicle for photographing the lateral environment of the vehicle is turned on, and the display screen is controlled to display the image captured by the second camera. When the area of ​​concern to the driver involves the lateral environment of the vehicle, supplementary information of the vehicle's lateral environment perspective can be provided in a timely manner, thereby enhancing the driver's perception of the vehicle's surrounding environment, thereby improving the driving assistance effect and driving safety under complex road conditions.

[0119] In one implementation, the method includes: The image captured by the second camera is stitched together with the image captured by the first camera to obtain a target image; The Control Display shows the target screen.

[0120] In an embodiment of the present application, when the target area includes the left area or the right area of ​​the vehicle, the second camera for photographing the lateral environment of the vehicle is turned on to photograph the lateral driving environment of the vehicle, and the picture taken by the second camera is spliced ​​with the picture taken by the first camera, and the display screen displays the spliced ​​picture.

[0121] For example, when the target area only includes the left area, the camera that shoots the lateral environment of the vehicle is turned on, and the picture taken by the camera that shoots the lateral environment of the vehicle is spliced ​​with the picture taken by the roof camera to obtain the target picture, which is displayed on the display screen.

[0122] Optionally, when it is detected that the target area includes a preset area, the screen display ratio is determined based on the length of time the driver gazes at the preset area, and the picture captured by the camera that captures the lateral environment of the vehicle is displayed on the display screen based on the picture display ratio.

[0123] Optionally, the image captured by the first camera and the image captured by the second camera may be spliced ​​into a wide-angle target image through an image fusion algorithm.

[0124] Optionally, when it is detected that the target area includes a preset area, an outline annotation is superimposed on the display screen to help the driver intuitively understand the position and shape of the side obstacle.

[0125] In the above implementation, when the target area includes a preset area, the target image is obtained by splicing the image taken by the second camera with the image taken by the first camera, and the display screen is controlled to display the target image. The perspective information captured by multiple cameras can be integrated and spliced ​​to improve the integrity of the image information; therefore, this solution can enhance the driver's perception of the driving environment and improve the accuracy and safety of driving assistance.

[0126] In the above embodiment, the driver's gaze information is obtained. Since the gaze information includes the driver's gaze position and gaze duration, the target area, that is, the driver's visual attention area in the current driving environment, can be determined based on the gaze information; based on the target area, the shooting angle of the camera is further determined, the camera is controlled to adjust the angle according to the target area of ​​the driver's attention, and the shooting picture is displayed on the display screen in the vehicle; compared with the fixed-angle shooting camera configured in the vehicle in the prior art, which can only capture the picture of a fixed driving environment, the method in the prior art cannot provide the driver with complete driving environment information in a complex driving scene, so that there is a driving safety hazard; this solution determines the driver's visual attention area in the current driving environment, and controls the camera to adjust the angle and display the picture according to the target area of ​​the driver's attention, so as to ensure that the camera captures the driver's visual attention focus, provide the driver with more accurate and comprehensive visual information, and thus improve driving safety.

[0127] The following combination Figure 4 Another vehicle control method provided in an embodiment of the present application is described in detail.

[0128] Figure 4 This is a schematic flow chart of another vehicle control method provided in an embodiment of the present application. Figure 4 As shown, the method 400 includes S401 to S408, and S401 to S408 are described in detail below.

[0129] For example, Figure 4 The illustrated method 400 may be executed by a vehicle; or, executed by a processor in a vehicle; or, executed by a chip mounted in a processor in a vehicle; or, executed by a software platform integrated in an electronic device.

[0130] S401. Determine whether the vehicle enters an off-road mode based on a sensor of the vehicle, and obtain the current slope or driving mode of the vehicle when a slope sensor of the vehicle is detected.

[0131] Among them, the off-road mode is used for the vehicle to travel on unpaved roads or complex terrain. In off-road mode, the vehicle's power system, suspension system, traction system, etc. will be adjusted, such as increasing the low-speed torque output of the engine / motor to facilitate the vehicle to start and climb on low-adhesion roads.

[0132] Optionally, the implementation of S401 can refer to Figure 2 The relevant description in S210 is not repeated here.

[0133] S402: Determine whether the current slope is greater than a preset slope threshold or whether the driving mode is the four-wheel drive mode; if so, execute S403; if not, execute S401.

[0134] In an embodiment of the present application, it is determined whether the current slope is greater than a preset slope threshold or whether the driving mode is a four-wheel drive mode; if the current slope is greater than the preset slope threshold, or the driving mode is a four-wheel drive mode, S403 is executed; otherwise, S401 is executed.

[0135] Exemplarily, when the sensor detects that the current slope of the vehicle is greater than a preset slope (eg, 15°), or detects that the driving mode of the vehicle is switched to the four-wheel drive mode, it is determined that the vehicle enters the off-road mode.

[0136] Optionally, the implementation of S402 can refer to Figure 2 The relevant description in S210 is not repeated here.

[0137] S403: Acquire the driver's image through the infrared camera inside the vehicle, and determine the driver's gaze position based on corneal reflection and eye socket contour.

[0138] For example, an image of the driver, including an image of the driver's eyes, is captured using an in-vehicle infrared camera. The coordinates of the driver's gaze point (e.g., X=-1.2m, Y=1.5m) are obtained from the driver's eye image and determined to be located within the left front wheel contact area.

[0139] For example, the driver's facial image is captured by the in-car camera, and the driver's eye socket position, that is, the outline position of the driver's eye socket, is identified based on the facial image to determine the stable reference frame of the eye on the face; the position of the reflected light spot formed by infrared light on the corneal surface is detected, and the offset of the reflected light spot position relative to the center of the eye socket is calculated to determine the direction and angle of eyeball rotation, thereby determining the driver's gaze position. Optionally, the implementation method of S403 can be found in Figure 2 The relevant description in S210 is not repeated here.

[0140] S404: Determine the gaze area corresponding to the driver's gaze position, and count the gaze time of each gaze area.

[0141] For example, an image of the driver, including an image of the driver's eyes, is captured using an in-vehicle infrared camera. The coordinates of the driver's gaze point (e.g., X = -1.2 m, Y = 1.5 m) are obtained from the driver's eye image. This coordinate is determined to be located in the left front wheel contact area, and the driver's gaze duration at this location is continuously monitored for 2.3 seconds.

[0142] Optionally, it is possible to detect whether the change in gaze position exceeds a preset threshold (eg, 2°), and continuously record the time when the change in the driver's gaze position is continuously smaller than the preset threshold.

[0143] Optionally, the implementation of S404 can refer to Figure 2 The relevant description in S210 is not repeated here.

[0144] S405: Obtain the weight corresponding to the gaze area in the current driving scene.

[0145] For example, it is detected that the current driving scene of the vehicle is a mud scene, and the weights corresponding to the mud scene in the attention weight library are obtained, for example, the weight of the left front wheel contact area is 0.6, the weight of the right front wheel contact area is 0.6, the weight of the left area is 0.1, the weight of the right area is 0.1, and the weight of the front wheel contact area is 0.3.

[0146] Optionally, the implementation of S405 can refer to Figure 2 The relevant description in S220 will not be repeated here.

[0147] S406: Determine whether the weight corresponding to the gaze area is greater than a preset weight and the gaze duration exceeds a preset duration. If so, execute S407; if not, execute S403.

[0148] In an embodiment of the present application, it is determined whether the weight corresponding to the gaze area is greater than the preset weight and the gaze duration exceeds the preset duration. If the weight corresponding to the gaze area is greater than the preset weight and the gaze duration exceeds the preset duration, S407 is executed; if the weight corresponding to the gaze area is less than or equal to the preset weight, or the gaze duration does not exceed the preset duration, S403 is executed.

[0149] Optionally, the implementation of S406 can refer to Figure 2 The relevant description in S220 will not be repeated here.

[0150] S407: Determine the gaze area with a weight greater than a preset weight and a gaze duration longer than a preset duration as a target area.

[0151] In an embodiment of the present application, based on the driver's gaze position, a target sub-area corresponding to the gaze position within a preset field of view is determined. If the weight corresponding to the target sub-area in the current scene is greater than a preset weight threshold, and the driver's gaze time at the position exceeds a preset time, the area is determined to be a target area that requires special attention.

[0152] For example, when the current vehicle driving scenario is a wading scenario, it is detected that the driver's contact time with the front wheel is 3 seconds, which exceeds the preset time (for example, 2 seconds), and the weight of the front wheel contact area in the wading scenario is 0.6, which is greater than the preset weight threshold (for example, 0.5), and the front wheel contact area is determined to be the target area.

[0153] Optionally, the implementation of S407 can be found in Figure 2The relevant description in S220 will not be repeated here.

[0154] S408 : Control the multi-degree-of-freedom pan / tilt to adjust the camera's shooting angle according to the target area, and display the camera's shooting image on a display screen.

[0155] For example, when it is detected that the target area is the front wheel contact area, the shooting angle can be determined to be 15° downward based on the mapping relationship between each area in the preset field of view and the various shooting angles of the first camera, and the multi-degree-of-freedom gimbal can be controlled to adjust the shooting angle of the camera, control it to rotate downward 15°, and display the picture taken by the camera on the display screen.

[0156] For example, it is detected that the driver stares at the front wheel contact area for 30 seconds within a preset time (for example, one minute), which accounts for 50% of the preset time, and the screen display ratio corresponding to the front wheel contact area is determined to be 50%. When the display screen displays the captured image, the image of the front wheel contact area is displayed at 50% of the entire image showing the vehicle's driving environment.

[0157] Optionally, the gaze duration may be positively correlated with the screen display ratio. The longer the gaze duration, the larger the corresponding screen display ratio, thereby highlighting the key areas.

[0158] Optionally, when it is detected that the target area includes the left area or the right area, the camera for photographing the lateral environment of the vehicle is turned on, the image captured by the camera is transmitted to a display screen, and the lateral driving environment of the vehicle is displayed on the display screen.

[0159] Optionally, the implementation of S408 can refer to Figure 2 The relevant description in S240 will not be repeated here.

[0160] In the above embodiment, the driver's gaze information is obtained. Since the gaze information includes the driver's gaze position and gaze duration, the target area, that is, the driver's visual attention area in the current driving environment, can be determined based on the gaze information; based on the target area, the shooting angle of the camera is further determined, the camera is controlled to adjust the angle according to the target area of ​​the driver's attention, and the shooting picture is displayed on the display screen in the vehicle; compared with the fixed-angle shooting camera configured in the vehicle in the prior art, which can only capture the picture of a fixed driving environment, the method in the prior art cannot provide the driver with complete driving environment information in a complex driving scene, so that there is a driving safety hazard; this solution determines the driver's visual attention area in the current driving environment, and controls the camera to adjust the angle and display the picture according to the target area of ​​the driver's attention, so as to ensure that the camera captures the driver's visual attention focus, provide the driver with more accurate and comprehensive visual information, and thus improve driving safety.

[0161] Combined with the above Figures 1 to 4 A vehicle control method provided by an embodiment of the present application is described in detail; Figure 5 and Figure 6 The device embodiments of the present application are described in detail. It should be understood that the devices in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0162] Figure 5 FIG. 5 is a schematic diagram of a vehicle control device according to an embodiment of the present invention, wherein the vehicle control device 500 includes an acquisition module 510 and a processing module 520 .

[0163] An acquisition module is used to acquire the driver's gaze information, which includes the driver's gaze position and gaze duration; The processing module is used to determine a target area in the vehicle's driving environment based on the gaze information; determine the shooting angle of a first camera in the vehicle based on the target area; control the first camera to shoot at the shooting angle based on the shooting angle, and control the display screen in the vehicle to display the image captured by the first camera.

[0164] Optionally, as an embodiment, the processing module 520 is further specifically used to: obtain the driving scene of the vehicle; and determine the target area in the driving environment of the vehicle based on the gaze information, including: determining the target area based on the gaze information and the driving scene.

[0165] Optionally, as an embodiment, the processing module 520 is specifically used to: determine the weight of the field of view corresponding to the driving scene based on the driving scene, the field of view area including at least one sub-area; determine the target area based on the gaze information and the weight of the field of view area.

[0166] Optionally, as an embodiment, the processing module 520 is specifically used to: determine the target sub-area corresponding to the gaze position in the field of view based on the gaze position; if the gaze duration exceeds a preset duration and the weight of the target sub-area is greater than a preset weight threshold, determine the target sub-area as the target area.

[0167] Optionally, as an embodiment, the processing module 520 is specifically configured to: determine a picture display ratio based on the gaze duration; and control the display screen to display the picture captured by the first camera at the picture display ratio.

[0168] Optionally, as an embodiment, the processing module 520 is further configured to: obtain the driver's eye socket position and corneal reflection information; obtaining the driver's gaze information includes: obtaining the gaze position based on the eye socket position and corneal reflection information.

[0169] Optionally, as an embodiment, the processing module 520 is further specifically used to: obtain the current driving mode of the vehicle; obtain the driver's gaze information, including: obtaining the gaze information when the current driving mode is the off-road mode.

[0170] Optionally, as an embodiment, the processing module 520 is further specifically used to: when it is detected that the target area includes a preset area, turn on the second camera, the second camera is used to shoot the lateral environment of the vehicle; and control the display screen to display the picture shot by the second camera.

[0171] Optionally, as an embodiment, the processing module 520 is specifically configured to: splice the picture taken by the second camera and the picture taken by the first camera to obtain a target picture; and control the display screen to display the target picture.

[0172] It should be noted that the vehicle control device 500 is implemented in the form of a functional unit. The term "module" herein can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0173] For example, a "module" may be a software program, hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application-specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group of processors) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0174] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] Figure 6 It is a structural schematic diagram of a vehicle provided in an embodiment of the present application.

[0176] Exemplarily, vehicle 600 includes a processor 610 , a memory 620 , and executable program code 630 .

[0177] Exemplarily, vehicle 600 includes one or more processors 610, which can support vehicle 600 in implementing the vehicle control method of the method embodiment. Processor 610 can be a general-purpose processor or a special-purpose processor. For example, processor 610 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit, a field programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

[0178] For example, the processor 610 can be used to control the vehicle 600, execute software programs, and process data of the software programs. The vehicle 600 can also include a communication unit to implement signal input (reception) and output (transmission).

[0179] Exemplarily, the vehicle 600 may include one or more memories 620 storing executable program code 630. The executable program code 630 may be executed by the processor 610 to generate instructions, causing the processor 610 to execute the vehicle control method described in the above method embodiment according to the instructions. For example, the processor 610 executes the following according to the instructions: obtaining the driver's gaze information, which includes the driver's gaze location and gaze duration; determining a target area in the vehicle's driving environment based on the gaze information; determining a shooting angle of a first camera in the vehicle based on the target area; controlling the first camera to capture images at the shooting angle based on the shooting angle; and controlling a display screen in the vehicle to display the image captured by the first camera.

[0180] Optionally, data may be stored in the memory 620. Optionally, the processor 610 may read data stored in the memory 620. The data may be stored at the same storage address as the executable program code 630, or may be stored at a different storage address from the executable program code 630.

[0181] Exemplarily, the processor 610 and the memory 620 may be provided separately or integrated together, for example, integrated on a system on chip (SOC) of the terminal device.

[0182] Exemplarily, the memory 620 can be used to store relevant programs of the vehicle control method provided in the embodiment of the present application, and the processor 610 can be used to call the executable program code 630 stored in the memory 620 when controlling the vehicle to execute the vehicle control method of the embodiment of the present application. The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the vehicle control method of any of the aforementioned embodiments.

[0183] Among them, computer-readable storage media may include, but are not limited to, any type of disk, including floppy disks, optical disks, Digital Video Discs (DVDs), Compact Disc Read-Only Memory (CD-ROMs), microdrives and magneto-optical disks, Read-Only Memory (ROMs), Random Access Memory (RAMs), Erasable Programmable Read-Only Memory (EPROMs), Electrically Erasable Programmable Read-Only Memory (EEPROMs), Dynamic Random Access Memory (DRAMs), Video Random Access Memory (VRAMs), flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0184] The present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the vehicle control method in the above-mentioned embodiment.

[0185] In addition, the electronic device provided in the embodiments of the present application can specifically be a chip, component or module, and the electronic device may include a connected processor and memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor can call and execute the instructions so that the chip executes the vehicle control method in the above embodiment.

[0186] Among them, the vehicle, computer-readable storage medium, computer program product or chip provided in this application are all used to execute the corresponding vehicle control method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding vehicle control method provided above, and will not be repeated here.

[0187] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0188] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0189] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A vehicle control method, characterized in that: The control method includes: Acquiring the driver's gaze information, wherein the gaze information includes the driver's gaze position and gaze duration; determining a target area in a driving environment of the vehicle based on the gaze information; Determining a shooting angle of a first camera in the vehicle based on the target area; Based on the shooting angle, the first camera is controlled to shoot at the shooting angle, and the display screen in the vehicle is controlled to display the image shot by the first camera.

2. The control method according to claim 1, characterized in that: Also includes: Acquiring a driving scene of the vehicle; The determining of a target area in the driving environment of the vehicle based on the gaze information includes: The target area is determined based on the gaze information and the driving scene.

3. The control method according to claim 2, characterized in that: The determining the target area based on the gaze information and the driving scene includes: Determining, based on the driving scene, a weight of a visual field area corresponding to the driving scene, the visual field area including at least one sub-area; The target area is determined based on the gaze information and the weight of the visual field area.

4. The control method according to claim 3, characterized in that: The determining the target area based on the gaze information and the weight of the visual field area includes: Based on the gaze position, determining a target sub-region corresponding to the gaze position within the field of view; If the gaze duration exceeds a preset duration and the weight of the target sub-region is greater than a preset weight threshold, the target sub-region is determined as the target region.

5. The control method according to claim 1, characterized in that: The controlling a display screen in the vehicle to display the image captured by the first camera includes: Determining a screen display ratio based on the gaze duration; Control the display screen to display the image captured by the first camera at the image display ratio.

6. The control method according to claim 1, characterized in that: Also includes: Obtaining the driver's eye socket position and corneal reflection information; The obtaining of the driver's gaze information includes: The gaze position is acquired based on the orbital position and the corneal reflection information.

7. The control method according to any one of claims 1 to 6, characterized in that: The vehicle is equipped with a second camera and further includes: When it is detected that the target area includes a preset area, turning on the second camera, where the second camera is used to photograph the lateral environment of the vehicle; Control the display screen to display the image captured by the second camera.

8. The control method according to claim 7, characterized in that: The controlling the display screen to display the image captured by the second camera includes: splicing the image captured by the second camera with the image captured by the first camera to obtain a target image; Control the display screen to display the target picture.

9. A vehicle control device, characterized in that: The device comprises: An acquisition module, configured to acquire the driver's gaze information, wherein the gaze information includes the driver's gaze position and gaze duration; A processing module is used to determine a target area in the vehicle's driving environment based on the gaze information; determine a shooting angle of a first camera in the vehicle based on the target area; control the first camera to shoot at the shooting angle based on the shooting angle, and control a display screen in the vehicle to display the image captured by the first camera.

10. A vehicle, characterized in that: The vehicle comprises: a memory for storing executable program code; A processor is configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 8.

Images