Image display method and device, vehicle-mounted terminal and vehicle

Abstract

The embodiment of the invention is suitable for the technical field of automobiles, and provides an image display method and device, a vehicle-mounted terminal and a vehicle, and the method comprises the steps: inputting an environment image of an environment where the vehicle is located into a preset image processing algorithm, and determining the wading information of the vehicle; generating a corresponding wading schematic diagram of the vehicle in the environment based on the wading information and the vehicle type information corresponding to the vehicle; and displaying the wading schematic diagram through a preset display device. Through the method provided by the embodiment of the invention, a driver can intuitively determine the water accumulation condition of the road surface through the wading schematic diagram, so that the driving safety of the vehicle on the water accumulation road is improved.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of automobile technology, and in particular, relate to an image display method, device, vehicle-mounted terminal and vehicle. Background Art

[0002] In seasons with frequent rainstorms, water often accumulates on low-lying roads. In the prior art, the vehicle-mounted terminal can inform the driver of the water accumulation on the road by displaying the water level value at the water accumulation location or by displaying the first water wading warning information when the water level value is higher than a threshold. However, only using numerical values ​​or the first warning information cannot intuitively and comprehensively present the water accumulation on the road to the driver. For example, the driver cannot intuitively know where the water on the road will reach the vehicle based on the water level value. It can be seen that the prior art cannot provide the driver with a single water wading information, which reduces the safety of vehicles driving on flooded roads. Summary of the invention

[0003] In view of this, the embodiments of the present application provide an image display method, device, vehicle-mounted terminal and vehicle to solve the technical problem of the single water-related information provided by the vehicle-mounted terminal, thereby improving the safety of vehicles driving on flooded roads.

[0004] A first aspect of an embodiment of the present application provides an image display method, comprising: Inputting an environmental image of the environment where the vehicle is located into a preset image processing algorithm to determine the wading information of the vehicle; Based on the wading information and the vehicle model information corresponding to the vehicle, generating a wading schematic diagram corresponding to the vehicle in the environment; The wading schematic diagram is displayed through a preset display device.

[0005] The embodiment of the present application has the following beneficial effects: In the embodiment of the present application, the vehicle-mounted terminal can input the environmental image of the environment in which the vehicle is located into the image processing algorithm pre-set by the R&D personnel, so as to determine the wading information of the vehicle from the environmental image through the image processing algorithm; then, the vehicle-mounted terminal can generate a wading schematic diagram of the vehicle in the current environment based on the wading information and the vehicle model information corresponding to the vehicle, and display the generated wading schematic diagram through the display device. Since the wading schematic diagram generated by the vehicle-mounted terminal can display the wading situation of the vehicle in the current environment, compared with the prior art that only displays the water level value, the method provided by this embodiment allows the driver to intuitively obtain the wading situation of the vehicle through the wading schematic diagram, thereby improving the driver's safety when wading.

[0006] In a possible implementation manner of the first aspect, the method includes: Inputting the sensor data into a preset obstacle detection algorithm to generate road surface information; the road surface information includes obstacle information and / or road surface depression information; The generating, based on the water wading information and the vehicle type information corresponding to the vehicle, a water wading schematic diagram corresponding to the vehicle in the environment includes: The wading schematic diagram is generated based on the road surface information, the wading information and the vehicle type information.

[0007] The embodiments of the present application have the following beneficial effects: since the vehicle-mounted terminal can generate a wading schematic diagram in combination with road surface information, that is, the wading schematic diagram generated by the vehicle-mounted terminal can display the obstacles and / or road depressions on the underwater road surface, the driver can directly know the obstacles and / or road depressions under the water surface through the wading schematic diagram, thereby further improving the driver's safety when wading.

[0008] In a possible implementation manner of the first aspect, generating the wading schematic diagram based on the road surface information, the wading information, and the vehicle type information includes: If the road surface information satisfies the preset obstacle avoidance condition, determining whether there is an obstacle avoidance route according to the first position information of the obstacle and / or the second position information of the road surface depression in the road surface information; If the obstacle avoidance route exists, the wading schematic diagram is generated based on the obstacle avoidance route, the obstacle information, the wading information and the vehicle type information.

[0009] The embodiments of the present application have the following beneficial effects: since the wading schematic diagram generated by the vehicle-mounted terminal can include an obstacle avoidance route, the method provided in this embodiment enables the vehicle-mounted terminal to provide a wading schematic diagram to provide the driver with an obstacle avoidance route, thereby preventing the vehicle from colliding with larger obstacles or falling into road depressions when wading, thereby further improving the safety of the vehicle's wading driving.

[0010] In a possible implementation manner of the first aspect, the method further includes: If the obstacle avoidance route does not exist, rescue information is generated according to the current third position information of the vehicle, and the rescue information is sent to the server.

[0011] The embodiments of the present application have the following beneficial effects: since the vehicle-mounted terminal can send rescue information to the server, the method provided by the embodiments can increase the speed of seeking help when the vehicle is in a dangerous situation, thereby reducing the time the vehicle stays in a dangerous environment.

[0012] In a possible implementation manner of the first aspect, the water-related information includes a water level value and a flow rate value; The generating, based on the water wading information and the vehicle type information corresponding to the vehicle, a water wading schematic diagram corresponding to the vehicle in the environment includes: Inputting the environment image into a preset three-dimensional model to determine a three-dimensional scene of the environment where the vehicle is located; Inputting the water level value and the flow velocity value into a preset fluid dynamics model to determine water flow texture data corresponding to the water body; The water flow texture data, the water level scale determined based on the water level value, and the three-dimensional vehicle model corresponding to the vehicle model information are added to the three-dimensional scene to generate the wading schematic diagram.

[0013] The embodiments of the present application have the following beneficial effects: since the vehicle-mounted terminal can generate a three-dimensional wading schematic diagram based on the three-dimensional vehicle model and the three-dimensional scene, and further, since the three-dimensional wading schematic diagram can improve the driver's perception of the external environment, the method provided in this embodiment enables the driver to identify dangers and avoid risks in the first place, thereby improving the driver's emergency response capabilities.

[0014] In a possible implementation manner of the first aspect, the method further includes: If the difference between the water level value and the height value of any first component on the vehicle is less than or equal to the alarm threshold, the area where any first component is located is displayed on the three-dimensional vehicle model by a preset alarm color; the first component is the component in the vehicle that has water wading alarm requirements.

[0015] The embodiments of the present application have the following beneficial effects: since the vehicle-mounted terminal can perform alarm operations on components that have water wading alarm requirements, the method provided by the embodiment enables the driver to accurately know the risk of water ingress to the vehicle.

[0016] In a possible implementation manner of the first aspect, the water-related information includes a water level value and a flow rate value, and further includes: If the water level value satisfies the preset first speed limit condition and / or the flow rate value satisfies the preset second speed limit condition, the vehicle speed threshold is determined according to the water level value and the flow rate value, and a first alarm message is generated according to the vehicle speed threshold; the first alarm message is used to prompt the driver to maintain the vehicle's driving speed less than or equal to the vehicle speed threshold.

[0017] The embodiment of the present application has the following beneficial effects: since the vehicle-mounted terminal can prompt the driver to control the vehicle speed to be less than the vehicle speed threshold through the first warning information, the method provided by the embodiment can reduce the problem of the first component of the vehicle being flooded due to splashing of accumulated water caused by excessive speed when the vehicle is wading. It can be seen that the method provided by the embodiment can reduce the probability of water flooding the first component of the vehicle when the vehicle is wading, thereby extending the service life of the first component.

[0018] A second aspect of an embodiment of the present application provides an image display device, including: An information determination module, used for inputting an environmental image of an environment where a vehicle is located into a preset image processing algorithm to determine wading information of the vehicle; An image generation module, configured to generate a corresponding wading schematic diagram of the vehicle in the environment based on the wading information and vehicle type information corresponding to the vehicle; A display module is used to display the wading schematic diagram through a preset display device.

[0019] A third aspect of an embodiment of the present application provides a vehicle-mounted terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image display method as described in the first aspect above is implemented.

[0020] A fourth aspect of the embodiments of the present application provides a vehicle, including a vehicle-mounted terminal and a vehicle-mounted camera; the vehicle-mounted terminal is communicatively connected to the vehicle-mounted camera; The vehicle-mounted camera is used to collect environmental images of the environment where the vehicle is located; The vehicle-mounted terminal is used to execute the method for displaying wading images as described in the first aspect above according to the collected environmental images.

[0021] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the image display method as described in the first aspect above is implemented.

[0022] A sixth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the image display method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 is a schematic diagram of a vehicle provided in an embodiment of the present application; Figure 2 is a schematic diagram of a method for displaying a wading image provided by an embodiment of the present application; Figure 3 It is a wading schematic diagram provided in an embodiment of the present application; Figure 4 is a schematic diagram of another method for displaying a wading image provided by an embodiment of the present application; Figure 5 is a schematic diagram of generating an obstacle avoidance route provided in an embodiment of the present application; Figure 6 is a schematic diagram of a display device for wading images provided in an embodiment of the present application; Figure 7 It is a schematic diagram of a vehicle-mounted terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are proposed for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from hindering the description of the present application.

[0026] During the rainy season, roads with low terrain often have water accumulation on the road surface. Therefore, in order to reduce the probability of damage to vehicle property and personal property, more and more vehicles are equipped with wading warning functions. When driving in the rain, because the driver's line of sight will be blocked by rain, the driver often cannot accurately judge the depth of water accumulation on the road surface and the road conditions under the water surface (for example, when the water is turbid, the driver may not be able to see whether the manhole cover on the road surface is opened or whether there are sharp objects on the road surface that can easily puncture the tire), which causes great inconvenience to the driver's wading. In view of this, an embodiment of the present application proposes an image display method. In this scheme, the on-board terminal can generate and display the vehicle's wading schematic diagram using panoramic technology based on the environmental image captured by the vehicle's surround view camera. Thereby improving the driver's environmental perception, so that the driver can better identify dangers and avoid risks at the first time.

[0027] The technical solution of the present application is described below through specific embodiments.

[0028] See also Figure 1 , showing a schematic diagram of a vehicle provided by an embodiment of the present application, such as Figure 1 As shown, the vehicle 1 may include a vehicle-mounted camera 11 and a vehicle-mounted terminal 12. Among them, the signal output end of the vehicle-mounted camera 11 can be connected to the signal input end of the vehicle-mounted terminal 12. Specifically, the vehicle-mounted camera 11 can be any one or more cameras that can be used to collect video, such as a surround view camera, a front view camera, a rear view camera, a panoramic camera, etc. on the vehicle. Among them, the vehicle-mounted camera 11 should have characteristics such as wide angle and low distortion to ensure image quality. Exemplarily, the vehicle-mounted camera 11 installed on the vehicle 1 can be a panoramic camera. The panoramic camera can be used to collect a 360-degree or larger angle (such as 720 degrees) of vision around the vehicle. When the vehicle is started, the vehicle-mounted camera 11 can continuously collect environmental images of the environment in which the vehicle is located, and transmit the collected environmental images to the vehicle-mounted terminal 12 through the signal output port. The vehicle-mounted terminal 12 can receive the environmental image transmitted by the vehicle-mounted camera 11 through the signal input port, and generate a wading schematic diagram of the vehicle according to the received environmental image. Among them, the specific method for the vehicle-mounted terminal 12 to generate a wading schematic diagram can be found in the following method embodiment.

[0029] Reference Figure 2 , showing a schematic diagram of a method for displaying a wading image provided by an embodiment of the present application. The method can be applied to a vehicle-mounted terminal. Among them, the above-mentioned vehicle-mounted terminal can be an electronic control unit (Electronic Control Unit, ECU), a microcontroller unit (Microcontroller Unit, MCU), a central processing unit (Central Processing Unit, CPU), an automobile engine control module (Automobile Engine Control Module, ECM) and other devices. The above-mentioned method for displaying a wading image can specifically include the following steps: S201, inputting an environmental image of the environment where the vehicle is located into a preset image processing algorithm to determine the wading information of the vehicle.

[0030] In this embodiment, when the vehicle is in the startup state, the vehicle-mounted camera can collect video information of the environment in which the vehicle is located in real time. After receiving the video information collected by any vehicle-mounted camera, the vehicle-mounted terminal can extract image frames from the video information. After extracting the image frames, the vehicle-mounted terminal can perform preprocessing operations on the image frames through a preprocessing algorithm preset by the R&D personnel. Among them, the preprocessing operation may include but is not limited to operations such as denoising and color correction. After preprocessing the image frames, the vehicle-mounted terminal can input multiple image frames collected by all vehicle-mounted cameras at the same time into a splicing algorithm preset by the R&D personnel, so as to splice the image frames collected by multiple vehicle-mounted cameras through the splicing algorithm to generate a panoramic environmental image. Among them, the splicing algorithm can splice the image frames according to the positional relationship and shooting angle between each vehicle-mounted camera to generate an accurate and natural panoramic environmental image.

[0031] After acquiring a panoramic environmental image of the vehicle's environment, the vehicle-mounted terminal can input the environmental image into an image processing algorithm pre-set by the R&D personnel to generate the vehicle's current wading information through the image processing algorithm. Among them, the image processing algorithm can be used to perform image segmentation, image recognition, water level value detection, flow rate value detection and other operations on the environmental image to generate wading information. It should be noted that the image processing algorithm in this embodiment can be composed of any one or more algorithms such as a deep learning model, a template-based matching algorithm, an image segmentation algorithm, etc., and the embodiment of this application is not used to specifically limit the image processing algorithm.

[0032] In one possible implementation, the wading information generated by the image processing algorithm may include but is not limited to the water level value and flow rate value of the vehicle's current wading. After generating the vehicle's wading information, the on-board terminal may determine whether the water level value in the current wading information is greater than or equal to the first water level threshold value preset by the R&D personnel. Exemplarily, the first water level threshold value may be 3 centimeters. If the on-board terminal determines that the water level value in the wading information is less than the first water level threshold value, the on-board terminal may determine that the vehicle is not currently wading, and the on-board terminal may not generate a wading schematic diagram. When the on-board terminal determines that the water level value in the wading information is greater than or equal to the first water level threshold value, the on-board terminal may determine that the vehicle is currently wading. At this time, the on-board terminal may execute the following S202~S203 to provide the driver with a wading schematic diagram of the vehicle.

[0033] In a possible implementation, the water-related information generated by the vehicle-mounted terminal may include but is not limited to the water level value and flow rate value of the water body where the vehicle is currently located. After acquiring the current water-related information of the vehicle, the vehicle-mounted terminal may also generate corresponding warning information based on the current water-related information of the vehicle. Specifically, the vehicle-mounted terminal may determine whether the speed limit condition is met based on the water-related information, and generate a first warning message when the water-related information meets the speed limit condition. The speed limit condition may include a first speed limit condition and a second speed limit condition. The first warning message generated by the vehicle-mounted terminal may be used to prompt the driver to maintain the vehicle's driving speed less than or equal to a speed threshold.

[0034] Specifically, the vehicle-mounted terminal can determine whether the water-related information meets the first speed limit condition by judging whether the water level value in the water-related information is greater than or equal to the second water level threshold. The second water level threshold may be greater than the first water level threshold. If the vehicle-mounted terminal determines that the water level value is greater than or equal to the second water level threshold, the vehicle-mounted terminal can determine that the water level value in the water-related information meets the first speed limit condition. If the vehicle-mounted terminal determines that the water level value is less than the second water level threshold, the vehicle-mounted terminal can determine that the water level value in the water-related information does not meet the first speed limit condition.

[0035] The vehicle terminal can determine whether the water-wading information meets the second speed limit condition by judging whether the flow rate value in the water-wading information is greater than or equal to the first flow rate threshold. If the vehicle terminal determines that the flow rate value is greater than or equal to the first flow rate threshold, the vehicle terminal can determine that the flow rate value in the water-wading information meets the second speed limit condition. If the vehicle terminal determines that the flow rate value is less than the first flow rate threshold, the vehicle terminal can determine that the flow rate value in the water-wading information does not meet the second speed limit condition.

[0036] If the vehicle-mounted terminal determines that the water level value in the water-wading information does not meet the first speed limit condition and the flow rate value does not meet the second speed limit condition, the vehicle-mounted terminal may not generate an alarm message. If the vehicle-mounted terminal determines that the water level value in the water-wading information meets the first speed limit condition and / or the flow rate value meets the second speed limit condition, the vehicle-mounted terminal may determine the current speed threshold of the vehicle based on the water level value and the flow rate value. Specifically, the vehicle-mounted terminal may query the threshold mapping relationship pre-set by the R&D personnel based on the water level value and the flow rate value to determine the current speed threshold of the vehicle based on the water level value and the flow rate value. After determining the speed threshold, the vehicle-mounted terminal may generate a first alarm message based on the speed threshold, and display the generated first alarm message through a display device, and / or play the first alarm message through the vehicle audio.

[0037] Through the method provided in this embodiment, when the vehicle is wading, the vehicle terminal can determine the vehicle speed threshold according to the water level and flow rate of the water body, and prompt the driver to control the vehicle speed to be less than the vehicle speed threshold through the first warning information. According to the method provided in this embodiment, since the driver can control the vehicle's driving speed according to the first warning information and keep the vehicle driving at a low speed on the flooded road surface, the method provided in this embodiment can avoid the problem of the first component of the vehicle being flooded due to splashing of accumulated water due to excessive speed when the vehicle is wading. It can be seen that the method provided in this embodiment can reduce the probability of water entering the first component of the vehicle when the vehicle is wading, thereby extending the service life of the first component of the vehicle.

[0038] In a possible implementation, when the vehicle-mounted terminal determines that the water level value in the wading information is greater than or equal to the third water level threshold, the vehicle-mounted terminal can generate a second warning message. The third water level threshold may be greater than the second water level threshold. Exemplarily, the third water level threshold may be half of the wheel height value of the vehicle. The second warning message generated by the vehicle-mounted terminal can be used to advise the driver not to take risks, as taking risks may cause the vehicle to stall, damage the electronic system, or more serious mechanical failures. At this point, the vehicle-mounted terminal can also generate other navigation routes based on the destination information entered by the user and the third current position information of the vehicle, and display the new navigation route through the display device.

[0039] S202: Based on the wading information and the vehicle model information corresponding to the vehicle, generate a wading schematic diagram corresponding to the vehicle in the environment.

[0040] In this embodiment, after obtaining the vehicle's wading information, the on-board terminal can query the three-dimensional vehicle model corresponding to the vehicle model information from the database according to the vehicle's current vehicle model information. Then, the on-board terminal can generate a wading schematic diagram of the vehicle in the current environment based on the three-dimensional vehicle model corresponding to the vehicle model information and the vehicle's wading information. Exemplarily, the on-board terminal can generate a wading schematic diagram through environmental simulation display technology (Surrounding Reality, SR). Specifically, environmental perception technology refers to obtaining environmental information around the vehicle through sensors and / or on-board cameras, and rendering the schematic diagram displayed on the display device in real time based on the real-time collected environmental information, thereby providing the driver with a schematic diagram that can reproduce the environment in which the vehicle is located.

[0041] In a possible implementation, the wading information obtained by the vehicle terminal through the image processing algorithm may include but is not limited to the water level value and the flow rate value. When the vehicle terminal determines that the water level value in the wading information is greater than or equal to the first water level threshold, that is, when the vehicle terminal determines that the vehicle is currently wading, the vehicle terminal can generate a wading schematic diagram of the vehicle according to the environment perception algorithm pre-set by the R&D personnel. Specifically, the vehicle terminal can input the environmental image into the image detection algorithm pre-set by the R&D personnel to generate environmental information on the road through the image detection algorithm. For example, environmental information such as lane lines, front vehicles, rear vehicles, and railings on the road. Specifically, the image detection algorithm can be composed of a target detection model, a semantic segmentation model, and a feature extraction model. Among them, the target detection model can be used to generate target object information corresponding to the environmental image. The semantic segmentation model can be used to perform semantic segmentation on the environmental image to generate a semantic segmentation map corresponding to the environmental image. The feature extraction model can be used to generate environmental feature information corresponding to the environmental image.

[0042] Specifically, the target information can be used to indicate the categories of each target detected by the target detection model in the environment image (such as pedestrians, vehicles, obstacles, etc.), the location information of the target, the confidence corresponding to the target, the orientation information of the target, and the relative size of the target. Exemplarily, the target detection model can indicate the location of the target by adding a bounding box to the target. The confidence corresponding to the target can be used to indicate the certainty of the target detection model on the target information. Specifically, when the vehicle terminal generates a wading schematic diagram, it can generate a three-dimensional target model corresponding to each target on the road according to the category of the target information. The vehicle terminal can also determine whether it is necessary to add the three-dimensional target model corresponding to the target in the three-dimensional scene according to the confidence corresponding to the target. Then, the vehicle terminal can also add the three-dimensional target model corresponding to each target to the three-dimensional scene according to the location of the target. Specifically, when adding the three-dimensional target model, the vehicle terminal can adjust the posture of the three-dimensional target model according to the orientation information of the target, and adjust the size of the three-dimensional target model according to the relative size of the target. Finally, the vehicle-mounted terminal can place the adjusted three-dimensional target model into the three-dimensional object scene according to the position information of the target object, so that the scene in the wading schematic diagram is consistent with the real environment.

[0043] The vehicle terminal can also input the environment image into the semantic segmentation model, and the semantic segmentation model can assign a category label to each pixel in the environment image, and generate a semantic segmentation map corresponding to the environment image according to the category labels corresponding to all pixels in the environment image. Among them, the semantic segmentation map can include multiple different areas and category labels corresponding to each area, such as road labels, sidewalk labels, grass labels, building labels, etc. Since the semantic segmentation map generated by the semantic segmentation model can provide the vehicle terminal with accurate classification information of each area in the environment image, the semantic segmentation map can provide a location reference for the precise superposition of various three-dimensional models when generating a wading schematic diagram. Exemplarily, the semantic segmentation map can include the road surface type of the road surface where the vehicle is currently located, such as grass type, cement road surface type, asphalt road surface type, mud type, etc. When the vehicle terminal generates a three-dimensional scene, it can adjust the rendering effect of the road surface in the three-dimensional scene according to the road surface type in the semantic segmentation map, so that the generated three-dimensional scene is more in line with the actual environment.

[0044] The vehicle-mounted terminal can also input the environmental image into the feature extraction model to generate environmental feature information corresponding to the environmental image through the feature extraction model. Among them, the environmental feature information generated by the feature extraction model may include but is not limited to information such as lighting information and weather type of the vehicle's environment. When drawing a three-dimensional wading schematic diagram, the vehicle-mounted terminal can add a three-dimensional environmental model that conforms to the actual situation of the vehicle to the three-dimensional scene of the wading schematic diagram based on the environmental feature information to make the generated three-dimensional scene more realistic. Exemplarily, when generating a three-dimensional scene, the vehicle-mounted terminal can adjust the brightness, shadow and other attributes of each three-dimensional model in the wading schematic diagram according to the lighting information in the environmental feature information to ensure that the driver can clearly see important information.

[0045] The vehicle terminal can also obtain map data corresponding to the vehicle's current location through the vehicle navigation system. The map data obtained by the vehicle terminal may include but is not limited to the vehicle's current third location information, route information between the current location and the destination, basic map information queried from the geographic information database, etc. Then, the vehicle terminal can input the environmental information and image data into the data fusion algorithm to generate fusion information that can represent the actual situation of the vehicle's current environment. After the vehicle terminal generates the fusion information, it can input the fusion information into the three-dimensional model to construct a three-dimensional scene of the vehicle's environment through the three-dimensional model.

[0046] At the same time, the vehicle terminal can also input the water level value and flow rate value in the wading information into the fluid dynamics model pre-set by the R&D personnel to generate the water flow texture data of the water body where the vehicle is currently located through the fluid dynamics model. The vehicle terminal can also generate a water level scale based on the water level value in the wading information. Finally, the vehicle terminal can add the generated water flow texture data, water level scale and three-dimensional vehicle model corresponding to the vehicle model information to the corresponding area of ​​the three-dimensional scene to generate a three-dimensional wading schematic diagram.

[0047] When generating a three-dimensional water-wading schematic diagram, the vehicle terminal can also determine whether the water level values ​​corresponding to each area in the water-wading schematic diagram are greater than or equal to the fourth water level threshold value based on the water-wading information. If the water level value corresponding to a certain area in the water-wading schematic diagram is greater than or equal to the fourth water level threshold value, the vehicle terminal can display the area in a warning color pre-set by the R&D personnel.

[0048] Through the method provided in this embodiment, the vehicle terminal can construct a three-dimensional wading schematic diagram that can reflect the real situation of the vehicle's environment based on the environmental image collected by the vehicle camera and the sensor data collected by the sensor. Since the three-dimensional wading schematic diagram can improve the driver's perception of the external environment, the method provided in this embodiment can enable the driver to identify dangers and avoid risks in the first place, thereby improving the driver's safety when wading.

[0049] After generating the wading diagram, the vehicle terminal can also dynamically adjust the display parameters of the wading diagram according to the environmental characteristic information to adapt to different driving scenarios and user needs. For example, if the environmental characteristic information indicates that the vehicle is currently at night or in bad weather conditions, the vehicle terminal can automatically increase the brightness and contrast of the wading diagram to ensure that the driver can clearly see important information.

[0050] In a possible implementation, before adding the three-dimensional vehicle model to the three-dimensional scene, the vehicle terminal can also determine whether it is necessary to perform an alarm operation on the first component with a wading alarm requirement on the vehicle according to the water level value in the wading information. Specifically, the vehicle terminal can determine whether the difference between the water level value and the height value of any first component on the vehicle is less than or equal to the alarm threshold preset by the R&D personnel. Among them, the first component can be a component with a wading alarm requirement in the vehicle, such as an engine, electric power, power battery, chassis and other components. If the vehicle terminal determines that the difference between the water level value and the height value of any first component on the vehicle is less than or equal to the alarm threshold, the vehicle terminal can display the area where the first component is located in a preset alarm color on the three-dimensional vehicle model of the wading schematic diagram. If the vehicle terminal determines that the difference between the water level value and the height values ​​of all first components on the vehicle is greater than the alarm threshold, the vehicle terminal can directly add the three-dimensional vehicle model corresponding to the vehicle model information to the three-dimensional scene, and continue to determine whether the difference between the height value of any first component and the water level value is less than or equal to the alarm threshold.

[0051] See also Figure 3 , shows a schematic diagram of a wading method provided in an embodiment of the present application. Figure 3 It can be a side view of a three-dimensional wading schematic diagram. Figure 3 As shown, the wading schematic diagram may include a water flow generated according to the water flow texture data and a water level scale for indicating a water level value of 5 cm. Figure 3 In the wading schematic diagram shown, since the difference between the water level value and the chassis of the vehicle is less than the alarm threshold at this time, the area where the chassis is located can be displayed in black in the wading schematic diagram at this time.

[0052] Through the method provided in this embodiment, when the accumulated water is close to a first component, the vehicle terminal can display the area where the first component is located in a warning color on the wading diagram to inform the driver that there is a risk of water ingress in this area. Therefore, the method provided in this embodiment allows the driver to accurately know the risk of water ingress in the vehicle, so that the driver can make preparations in advance, thereby improving the driver's driving experience.

[0053] S203: Displaying the wading schematic diagram through a preset display device.

[0054] In this embodiment, after generating the wading schematic diagram, the vehicle-mounted terminal can send the generated wading schematic diagram to a display device preset by the R&D personnel, so as to display the current wading schematic diagram of the vehicle through the display device. The display device can be a central control display screen, a head-up display device, etc. on the vehicle, and this embodiment of the application is not limited to this.

[0055] In this embodiment, since the vehicle terminal can determine the wading information of the vehicle based on the environmental image of the vehicle's environment, and further draw a wading schematic diagram of the vehicle based on the wading information, through the method provided by this embodiment, the driver can intuitively and comprehensively determine which part of the vehicle the road surface water has reached based on the wading schematic diagram displayed by the display device, thereby helping the driver to intuitively and comprehensively understand the water accumulation situation on the road surface and providing the driver with safety when driving on flooded roads.

[0056] In a possible implementation, the wading schematic diagram generated by the vehicle-mounted terminal is a three-dimensional image. When the driver needs to switch the display viewing angle of the wading schematic diagram or change the display field of view of the wading schematic diagram, the driver can change the display viewing angle and / or display field of view of the wading schematic diagram through interactive actions. The vehicle-mounted terminal can determine the viewing angle conversion value and / or field of view transformation value of the wading schematic diagram based on the user's interactive actions, and display the new wading schematic diagram on the display device based on the viewing angle conversion value and / or field of view transformation value. Exemplarily, the user can scale, rotate, mark, and perform other operations on the wading schematic diagram by making gestures on the touch screen or voice control.

[0057] On the basis of the first embodiment, in order to further enrich the amount of information in the water wading schematic diagram, the second embodiment of the present application provides a method for generating a water wading schematic diagram in combination with road surface information. Figure 4 The specific implementation flow chart of a method S202 for displaying a wading image provided by the second embodiment of the present application is shown. Figure 3 , compared to Figure 2 In the embodiment, a method for displaying a wading image provided in the embodiment includes before S202: S401, and S202 includes: S402, which is specifically described as follows: S401, inputting sensor data into a preset obstacle detection algorithm to generate road surface information; the road surface information includes obstacle information and / or road surface depression information.

[0058] In this embodiment, when the vehicle is in the startup state, the sensors on the vehicle can continuously collect sensor data and transmit the collected sensor data to the vehicle terminal. Exemplarily, the sensor data received by the vehicle terminal may include but is not limited to laser data fed back by a lidar sensor, millimeter wave data fed back by a millimeter wave radar, ultrasonic data fed back by an ultrasonic sensor, infrared data fed back by an infrared sensor, and other sensor data. When the vehicle terminal determines that the vehicle is wading, the vehicle terminal can also input the currently received sensor data into the obstacle detection algorithm pre-set by the R&D personnel to generate the road surface information of the road surface where the vehicle is currently located through the obstacle detection algorithm. Among them, the road surface information generated by the obstacle detection algorithm may include but is not limited to obstacle information on the road surface and / or road depression information.

[0059] S402: Generate a wading schematic diagram based on road surface information, wading information and vehicle type information.

[0060] In this embodiment, after determining the road surface information, the vehicle-mounted terminal can generate a wading schematic diagram according to the road surface information, wading information and vehicle model information corresponding to the environment. Specifically, after determining the road surface information, the vehicle-mounted terminal can determine whether there is an obstacle on the underwater road surface according to the obstacle information in the road surface information, and determine whether there is a road depression on the underwater road surface according to the road surface depression information. When the vehicle-mounted terminal determines that there is an obstacle, the vehicle-mounted terminal can draw a three-dimensional obstacle model according to the obstacle information in the road surface information. When the vehicle-mounted terminal determines that there is a road depression, the vehicle-mounted terminal can draw a three-dimensional depression model according to the road surface depression information in the road surface information. Then, the vehicle-mounted terminal can add the obstacle model and / or the depression model to the three-dimensional scene generated by the vehicle-mounted terminal according to the video information, and generate a wading schematic diagram of the vehicle in the current state in combination with the water flow texture data corresponding to the wading information, the water level gauge corresponding to the wading information and the three-dimensional vehicle model corresponding to the vehicle model information. Among them, the specific method for the vehicle-mounted terminal to generate a wading schematic diagram in combination with the video information, the wading information and the vehicle model information is shown in the content of the first embodiment of this application, which will not be repeated here.

[0061] Through the method provided by this embodiment, since the vehicle-mounted terminal can generate a wading schematic diagram of the vehicle based on obstacle information and road depression information, when there is an obstacle on the underwater road surface, the wading schematic diagram displayed by the display device can include the obstacle; when there is a road depression on the underwater road surface, the wading schematic diagram displayed by the display device can include the road depression. Therefore, the method provided by this embodiment allows the driver to intuitively determine the specific conditions of the underwater road surface through the display device, thereby avoiding driving accidents caused by the driver's unclear water quality and inability to see the specific conditions of the road surface when wading, thereby improving the driver's safety when wading.

[0062] In one possible implementation, see Figure 5 , shows a schematic diagram of generating an obstacle avoidance route provided by an embodiment of the present application. Figure 5As shown, after determining the road surface information, the vehicle-mounted terminal can also determine whether the road surface information of the vehicle's current environment meets the obstacle avoidance conditions pre-set by the R&D personnel. Specifically, the vehicle-mounted terminal can determine whether the obstacle information in the road surface information meets the obstacle avoidance conditions by judging whether the obstacle volume in the obstacle information is greater than or equal to the volume threshold pre-set by the R&D personnel. The vehicle-mounted terminal can also determine whether the road surface depression information in the road surface information meets the obstacle avoidance conditions by judging whether the depression degree in the road surface depression information is greater than or equal to the depression threshold pre-set by the R&D personnel. When the vehicle-mounted terminal determines that the obstacle volume is less than the volume threshold, and / or the depression degree is less than the depression threshold, the vehicle-mounted terminal can determine that the current road surface information does not meet the obstacle avoidance conditions. At this time, the vehicle-mounted terminal may not generate an obstacle avoidance route, and the vehicle-mounted terminal may generate a wading schematic diagram based on the road surface information, wading information and vehicle model information.

[0063] When the vehicle-mounted terminal determines that the volume of the obstacle is greater than or equal to the volume threshold, and / or the degree of depression is greater than or equal to the depression threshold, the vehicle-mounted terminal can determine that the current road surface information meets the obstacle avoidance conditions. At this time, the vehicle-mounted terminal can determine whether there is an obstacle avoidance route based on the first position information in the obstacle information whose volume is greater than or equal to the volume threshold, the second position information in the road depression information whose degree of depression is greater than or equal to the depression threshold, and the map data obtained by the vehicle-mounted terminal through the vehicle navigation system. If the vehicle-mounted terminal determines that there is an obstacle avoidance route, the vehicle-mounted terminal can generate an obstacle avoidance route based on the first position information, the second position information and the map data. After generating the obstacle avoidance route, the vehicle-mounted terminal can generate a wading schematic diagram based on the obstacle avoidance route, obstacle information, wading information and vehicle model information.

[0064] Through the method provided in this embodiment, when the obstacle on the road is too large or the road depression is too large, the vehicle terminal can display the obstacle avoidance route through the wading schematic diagram to guide the driver to bypass the obstacle or road depression. Therefore, the method provided in this embodiment can provide the driver with an obstacle avoidance route when the vehicle is wading, thereby preventing the vehicle from hitting a large obstacle or falling into a road depression when wading, thereby further improving the safety of the vehicle wading.

[0065] If the vehicle terminal determines that there is no obstacle avoidance route when the road surface information meets the obstacle avoidance condition, and / or the vehicle terminal detects that any of the first components has water ingress, the vehicle terminal can obtain the current third position information of the vehicle through the vehicle navigation system, and generate rescue information based on the third position information, vehicle model information, and other information. The vehicle terminal can send rescue information containing the third position information to the server where the vehicle management platform is located. The vehicle management platform can determine the rescue center closest to the vehicle based on the third position information in the rescue information, and send rescue information to the rescue center to ask for help from the rescue center.

[0066] Through the method provided in this embodiment, since the vehicle-mounted terminal can automatically send rescue information based on the vehicle's third position information, the method provided in this embodiment can increase the speed of seeking help when the vehicle is in a dangerous situation, thereby reducing the vehicle's stay time in a dangerous environment, improving the rescue speed and user favorability.

[0067] It should be noted that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0068] Reference Figure 6 , shows a schematic diagram of a display device for wading images provided in an embodiment of the present application, which may specifically include an information determination module 601, an image generation module 602 and a display module 603, wherein: An information determination module 601 is used to input an environmental image of the environment where the vehicle is located into a preset image processing algorithm to determine the wading information of the vehicle; An image generation module 602 is used to generate a corresponding wading schematic diagram of the vehicle in the environment based on the wading information and the vehicle model information corresponding to the vehicle; The display module 603 is used to display the wading schematic diagram through a preset display device.

[0069] The information determination module may also be used to input the sensor data into a preset obstacle detection algorithm to generate road surface information; the road surface information includes obstacle information and / or road surface depression information; The image generation module can also be used to generate the wading schematic diagram based on the road surface information, the wading information and the vehicle type information.

[0070] The image generation module can also be used to determine whether there is an obstacle avoidance route based on the first position information of the obstacle and / or the second position information of the road surface depression in the road surface information if the road surface information meets the preset obstacle avoidance conditions; if the obstacle avoidance route exists, the wading schematic diagram is generated based on the obstacle avoidance route, the obstacle information, the wading information and the vehicle model information.

[0071] The image generation module can also be used to generate rescue information based on the current third position information of the vehicle if the obstacle avoidance route does not exist, and send the rescue information to the server.

[0072] The image generation module can also be used to input the environmental image into a preset three-dimensional model to determine the three-dimensional scene of the vehicle's environment; input the water level value and the flow rate value into a preset fluid dynamics model to determine the water flow texture data corresponding to the water body; add the water flow texture data, the water level scale determined based on the water level value, and the three-dimensional vehicle model corresponding to the vehicle model information to the three-dimensional scene to generate the wading schematic diagram.

[0073] The image generation module can also be used to display the area where any first component is located on the three-dimensional vehicle model with a preset alarm color if the difference between the water level value and the height value of any first component on the vehicle is less than or equal to the alarm threshold; the first component is a component in the vehicle that has water wading alarm requirements.

[0074] The information determination module can also be used to determine the vehicle speed threshold according to the water level value and the flow rate value if the water level value meets the preset first speed limit condition and / or the flow rate value meets the preset second speed limit condition, and generate a first warning message according to the vehicle speed threshold; the first warning message is used to prompt the driver to maintain the vehicle's driving speed less than or equal to the vehicle speed threshold.

[0075] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0076] Reference Figure 7 , shows a schematic diagram of a vehicle-mounted terminal provided by an embodiment of the present application. Figure 7 As shown, the vehicle-mounted terminal 700 in the embodiment of the present application includes: a processor 710, a memory 720, and a computer program 721 stored in the memory 720 and executable on the processor 710. When the processor 710 executes the computer program 721, the steps in each embodiment of the above-mentioned method for displaying a wading image are implemented, for example Figure 1 Alternatively, when the processor 710 executes the computer program 721, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 6 Functions of modules 601 to 603 are shown.

[0077] Exemplarily, the computer program 721 may be divided into one or more modules / units, which are stored in the memory 720 and executed by the processor 710 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which may be used to describe the execution process of the computer program 721 in the vehicle-mounted terminal 700. For example, the computer program 721 may be divided into an information determination module, an image generation module, and a display module, and the specific functions of each module are as follows: An information determination module, used for inputting an environmental image of an environment where a vehicle is located into a preset image processing algorithm to determine wading information of the vehicle; An image generation module, configured to generate a corresponding wading schematic diagram of the vehicle in the environment based on the wading information and vehicle type information corresponding to the vehicle; A display module is used to display the wading schematic diagram through a preset display device.

[0078] The vehicle terminal 700 may include, but is not limited to, a processor 710 and a memory 720. Those skilled in the art will appreciate that Figure 7 It is only an example of the vehicle-mounted terminal 700 and does not constitute a limitation of the vehicle-mounted terminal 700. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the vehicle-mounted terminal 700 may also include input and output devices, network access devices, buses, etc.

[0079] The processor 710 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0080] The memory 720 may be an internal storage unit of the vehicle-mounted terminal 700, such as a hard disk or memory of the vehicle-mounted terminal 700. The memory 720 may also be an external storage device of the vehicle-mounted terminal 700, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the vehicle-mounted terminal 700. Further, the memory 720 may also include both an internal storage unit and an external storage device of the vehicle-mounted terminal 700. The memory 720 is used to store the computer program 721 and other programs and data required by the vehicle-mounted terminal 700. The memory 720 may also be used to temporarily store data that has been output or is to be output.

[0081] An embodiment of the present application also discloses a vehicle-mounted terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for displaying wading images as described in the aforementioned embodiments is implemented.

[0082] The embodiments of the present application further disclose a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for displaying a wading image as described in the aforementioned embodiments is implemented.

[0083] The embodiments of the present application further disclose a computer program product. When the computer program product is run on a computer, the computer is enabled to execute the method for displaying wading images described in the aforementioned embodiments.

[0084] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application is described in detail with reference to the above-mentioned embodiments, a person skilled in the art should understand that the technical solutions described in the above-mentioned embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An image display method, characterized in that: include: Inputting an environmental image of the environment where the vehicle is located into a preset image processing algorithm to determine the wading information of the vehicle; Based on the wading information and the vehicle model information corresponding to the vehicle, generating a wading schematic diagram corresponding to the vehicle in the environment; The wading schematic diagram is displayed through a preset display device.

2. The method according to claim 1, characterized in that include: Inputting the sensor data into a preset obstacle detection algorithm to generate road surface information; the road surface information includes obstacle information and / or road depression information; The generating, based on the water wading information and the vehicle type information corresponding to the vehicle, a water wading schematic diagram corresponding to the vehicle in the environment, comprises: The wading schematic diagram is generated based on the road surface information, the wading information and the vehicle type information.

3. The method according to claim 2, characterized in that The generating the wading schematic diagram based on the road surface information, the wading information and the vehicle type information includes: If the road surface information satisfies the preset obstacle avoidance condition, judging whether there is an obstacle avoidance route according to the first position information of the obstacle and / or the second position information of the road surface depression in the road surface information; If the obstacle avoidance route exists, the wading schematic diagram is generated based on the obstacle avoidance route, the obstacle information, the wading information and the vehicle type information.

4. The method according to claim 3, characterized in that: Also includes: If the obstacle avoidance route does not exist, rescue information is generated according to the current third position information of the vehicle, and the rescue information is sent to the server.

5. The method according to claim 1, characterized in that The water-related information includes water level value and flow rate value; The generating, based on the water wading information and the vehicle type information corresponding to the vehicle, a water wading schematic diagram corresponding to the vehicle in the environment includes: Inputting the environment image into a preset three-dimensional model to determine a three-dimensional scene of the environment where the vehicle is located; Inputting the water level value and the flow velocity value into a preset fluid dynamics model to determine water flow texture data corresponding to the water body; The water flow texture data, the water level scale determined based on the water level value, and the three-dimensional vehicle model corresponding to the vehicle model information are added to the three-dimensional scene to generate the wading schematic diagram.

6. The method according to claim 5, characterized in that Also includes: If the difference between the water level value and the height value of any first component on the vehicle is less than or equal to the alarm threshold, the area where any first component is located is displayed on the three-dimensional vehicle model by a preset alarm color; the first component is a component in the vehicle that has water wading alarm requirements.

7. The method according to claim 1, characterized in that The water-related information includes water level value and flow rate value. Also includes: If the water level value satisfies the preset first speed limit condition and / or the flow rate value satisfies the preset second speed limit condition, the vehicle speed threshold is determined according to the water level value and the flow rate value, and a first alarm message is generated according to the vehicle speed threshold; the first alarm message is used to prompt the driver to maintain the vehicle's driving speed less than or equal to the vehicle speed threshold.

8. An image display device, characterized in that: include: An information determination module, used for inputting an environmental image of an environment where a vehicle is located into a preset image processing algorithm to determine wading information of the vehicle; An image generation module, configured to generate a corresponding wading schematic diagram of the vehicle in the environment based on the wading information and vehicle type information corresponding to the vehicle; A display module is used to display the wading schematic diagram through a preset display device.

9. A vehicle-mounted terminal, characterized in that: The vehicle-mounted terminal comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the vehicle-mounted terminal implements the image display method as described in any one of claims 1 to 7.

10. A vehicle, characterized in that: It includes a vehicle-mounted terminal and a vehicle-mounted camera; the vehicle-mounted terminal is communicatively connected with the vehicle-mounted camera; The vehicle-mounted camera is used to collect environmental images of the environment where the vehicle is located; The vehicle-mounted terminal is used to execute the image display method according to any one of claims 1 to 7 according to the collected environmental image.

Images