A computer vision-based ar / mr anti-terrorism high method and system

Abstract

The present application provides a method and system for anti-terrorism high method for automobile, wherein the method comprises: acquiring real-time image of the environment around the automobile and current altitude; determining whether there is a high-fall area in the driver's field of view based on the acquired real-time image and the current altitude; in response to determining that there is a high-fall area in the driver's field of view, detecting the eye position of the driver; and based on the detected eye position, displaying an overlay image in the computer vision system to cover the high-fall area in the driver's field of view.

Description

Technical Field

[0001] This invention relates to the field of automotive electronics, and more specifically, to an AR / MR anti-terrorism method and system based on computer vision. Background Technology

[0002] In recent years, as AR and MR technologies have become the next big trend in automotive technology development, they have been increasingly applied to human-computer interaction in smart cars. These applications include augmented reality information display, augmented reality driving assistance systems, and augmented reality motion-sensing interaction systems.

[0003] For drivers with acrophobia, driving on mountain roads or roads at high altitudes without high guardrails can cause panic and pose a risk of dangerous driving. Although AR / MR technology is already widely used in automobiles, there is no system that uses AR / MR technology to prevent acrophobia.

[0004] Therefore, it is desirable to provide a computer vision-based AR / MR anti-fear system to prevent drivers from panicking while driving, thereby improving the driver's driving experience and achieving safer driving. Summary of the Invention

[0005] This summary is provided to introduce, in a simplified form, some concepts that will be further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0006] To address the above problems, according to one aspect of the present invention, a method for counter-terrorism in automobiles is provided, the method comprising:

[0007] Acquire real-time images of the vehicle's surroundings and its current altitude;

[0008] Based on the acquired real-time images and the current altitude, determine whether there are areas of high elevation difference in the driver's field of vision;

[0009] In response to determining the presence of a high-elevation area in the driver's field of vision, the driver's eye position is detected; and

[0010] Based on the detected eye position, an overlay image is displayed in the computer vision system to cover the high-altitude area in the driver's field of vision.

[0011] According to one embodiment of the present invention, determining whether a high-altitude region exists in the driver's field of vision based on the acquired real-time image and the current altitude further includes:

[0012] The acquired real-time image is input into the high-drop region calculation model to calculate the region in the real-time image corresponding to the high-drop region.

[0013] According to a further embodiment of the present invention, determining whether a high-altitude region exists in the driver's field of vision based on the acquired real-time image and the current altitude further includes:

[0014] If a region corresponding to the high elevation difference exists in the acquired real-time image and the current altitude exceeds a threshold, it is determined that a high elevation difference exists in the driver's field of vision.

[0015] According to a further embodiment of the present invention, displaying the overlay image in the computer vision system to cover the high-drop area in the driver's field of vision further includes:

[0016] In the computer vision system, the area corresponding to the high-altitude region in the acquired real-time image is rendered as grass or flat land to cover the high-altitude region in the driver's field of vision.

[0017] According to a further embodiment of the present invention, the rendering further includes:

[0018] Based on the detected eye position, the driver's gaze point region is determined; and

[0019] Rendering is performed at a reduced resolution outside the gaze point area.

[0020] According to a further embodiment of the present invention, displaying the overlay image in the computer vision system further includes:

[0021] The overlay image is displayed by projecting it onto the windshield of the car via a head-up display (HUD) or by using augmented reality (AR) glasses or mixed reality (MR) glasses.

[0022] According to a further embodiment of the present invention, the high-altitude area includes one or more of the following: cliffs, valleys, canyons, overpasses or bridges without high guardrails.

[0023] According to another aspect of the present invention, a vehicle with anti-terrorism capabilities is provided, the vehicle comprising:

[0024] A sensor system configured to capture real-time images of the vehicle's surroundings and acquire the current altitude;

[0025] An image processing system, the image processing system being configured to:

[0026] The sensor system acquires real-time images of the vehicle's surroundings and its current altitude.

[0027] Based on the acquired real-time images and the current altitude, determine whether there are areas of high elevation difference in the driver's field of vision;

[0028] An eye-tracking system configured to detect the driver's eye position in response to determining the presence of a high-drop region in the driver's field of vision; and

[0029] A computer vision system configured to display an overlay image based on the detected eye position to cover the high-drop area in the driver's field of vision.

[0030] According to one embodiment of the present invention, the sensor system includes one or more of an altitude sensor, GPS, and a visual sensor.

[0031] According to a further embodiment of the present invention, determining whether a high-altitude region exists in the driver's field of vision based on the acquired real-time image and the current altitude further includes:

[0032] The acquired real-time image is input into the high-drop region calculation model to calculate the region in the real-time image corresponding to the high-drop region.

[0033] According to a further embodiment of the present invention, determining whether a high-altitude region exists in the driver's field of vision based on the acquired real-time image and the current altitude further includes:

[0034] If a region corresponding to the high elevation difference exists in the acquired real-time image and the current altitude exceeds a threshold, it is determined that a high elevation difference exists in the driver's field of vision.

[0035] According to a further embodiment of the present invention, the display overlay image to cover the high-drop area in the driver's field of vision further includes:

[0036] The area in the acquired real-time image corresponding to the high-altitude region is rendered as grass or flat land to cover the high-altitude region in the driver's field of vision.

[0037] According to a further embodiment of the present invention, the rendering further includes:

[0038] Based on the detected eye position, the driver's gaze point region is determined; and

[0039] Rendering is performed at a reduced resolution outside the gaze point area.

[0040] According to a further embodiment of the present invention, the computer vision system includes one or more of a head-up display (HUD), augmented reality (AR) glasses, or mixed reality (MR) glasses.

[0041] According to a further embodiment of the present invention, the high-altitude area includes one or more of the following: cliffs, valleys, canyons, overpasses or bridges without high guardrails.

[0042] To address the problems existing in the prior art, this invention provides an AR / MR anti-terrorism height system based on computer vision, which has at least the following advantages:

[0043] 1. By utilizing image processing to render high-altitude areas within the driver's field of vision, it prevents drivers from panicking due to acrophobia, thus improving the driving experience and achieving safer driving; and

[0044] 2. Utilize AR / MR and eye-tracking technologies to enhance the human-vehicle interaction experience, enabling a better fusion of virtual and real images.

[0045] These and other features and advantages will become apparent from the following detailed description and with reference to the accompanying drawings. It should be understood that the foregoing general description and the following detailed description are illustrative only and do not limit the scope of the claims. Attached Figure Description

[0046] To gain a more detailed understanding of the manner in which the features of the present invention are described above, reference can be made to various embodiments to provide a more specific description of the above-briefly summarized aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of the invention and should not be considered as limiting its scope, as this description may allow for other equivalent and effective aspects.

[0047] Figure 1 This is a structural schematic diagram of a car with anti-terrorism high-altitude function according to an embodiment of the present invention.

[0048] Figure 2 This is a schematic flowchart of an AR / MR anti-terrorism height method according to an embodiment of the present invention.

[0049] Figure 3a and 3b Examples of scenarios with and without activated anti-fear of heights, according to an embodiment of the present invention, are shown respectively. Detailed Implementation

[0050] The present invention will now be described in detail with reference to the accompanying drawings, and its features will become further apparent in the following detailed description.

[0051] Figure 1 This is a schematic diagram of the structure of a car 100 with anti-terrorism capabilities according to an embodiment of the present invention. Figure 1As shown, the vehicle 100 may include at least a sensor system 101, an image processing system 102, an eye-tracking system 103, and a computer vision system 104. The sensor system 101 may include, for example, a variety of sensors mounted on the vehicle, including but not limited to vision sensors (e.g., onboard cameras), altitude sensors, millimeter-wave radar, lidar, etc.

[0052] Image processing system 102 can acquire information collected from sensor system 101 (e.g., real-time images acquired from the car's front-facing camera or the camera of MR mixed reality glasses, current altitude acquired from an altitude sensor or GPS, etc.), and perform image processing on the acquired real-time images to determine whether a high-altitude region exists in the driver's field of vision. Further, image processing system 102 can input the captured real-time image into a high-altitude region calculation model to calculate the region in the real-time image corresponding to the high-altitude region, wherein the high-altitude region calculation model can be trained through machine learning (e.g., linear regression, support vector machine, incremental training, etc.). A high-altitude region can refer to an area with a large height difference, such as a cliff without high guardrails, valley, canyon, overpass, or bridge. Further, if a region corresponding to a high-altitude region exists in the acquired real-time image and the current altitude exceeds a threshold, image processing system 102 can determine that a high-altitude region exists in the driver's field of vision, thereby activating the anti-altitude-fear function to render images of the region corresponding to the high-altitude region in the acquired real-time image.

[0053] The eye-tracking system 103 can detect the driver's eye position in response to the presence of a high-drop area in the driver's field of vision. The eye-tracking system 103 can identify the driver's focal point of gaze inside the vehicle and correct the display position of the overlay image by determining the position of the eye's gaze point. Specifically, it can perform high-definition rendering only on the gaze point area and enhance the display effect of the gaze point area based on the existing rendering effect, while gradually reducing the rendering resolution of the surrounding areas. This display effect is consistent with the imaging pattern when the human eye views an object, further improving the human-vehicle interaction experience.

[0054] The computer vision system 104 can display an overlay image based on the detected eye position to cover high-drop areas in the driver's field of vision. The computer vision system 104 may include, but is not limited to, a head-up display (HUD), augmented reality (AR) glasses, or mixed reality (MR) glasses. After the image processing system 102 calculates the area corresponding to the high-drop area in the captured real-time image, the corresponding high-drop area can be rendered in the computer vision system 104 to cover that area in the driver's field of vision. As an example, after the image processing system 102 calculates the area corresponding to a cliff without a high guardrail in the captured real-time image, the area can be rendered as grass or flat ground in the HUD display, and the rendered overlay image can be projected onto the windshield to obscure the cliff area in the driver's field of vision, so that a driver with a fear of heights will not experience panic upon seeing the cliff.

[0055] Figure 2 This is a schematic flowchart of an AR / MR anti-terrorism altitude method 200 performed by a vehicle according to an embodiment of the present invention. Method 200 begins at step 201, where an image processing system 102 acquires real-time images of the vehicle's surroundings and the current altitude. For example, the vehicle's camera system can be used to capture real-time images of the vehicle's surroundings, such as in… Figure 3a The image shows real-time images captured by a front-facing camera as the car travels on a winding mountain road. Additionally, the car's altitude sensor can be used to determine its current altitude.

[0056] In step 202, the image processing system 102 determines whether a high-altitude region exists in the driver's field of vision based on the acquired real-time image and the current altitude. For example, the acquired real-time image can be input into a high-altitude region calculation model to calculate the region in the acquired real-time image corresponding to the high-altitude region. The high-altitude region calculation model can be established by pre-training using machine learning algorithms. Incremental learning can also be used to update the high-altitude region calculation model in real time to make the model more accurate. Figure 3a In the scenario shown, real-time images captured by the front-facing camera while the car is driving on a mountain road are input into a high-altitude region calculation model to obtain the region corresponding to the cliff without tree obstruction. If a region corresponding to a high-altitude region (e.g., a cliff without a high guardrail) exists in the acquired real-time images and the current altitude exceeds a threshold (e.g., 1000m), it is determined that a high-altitude region exists in the driver's field of vision.

[0057] In step 203, the eye-tracking system 103 detects the driver's eye position in response to determining that a high-drop area exists in the driver's field of vision. Specifically, upon determining that a high-drop area exists in the driver's field of vision, an anti-altitude-fear function is activated because the probability of the driver currently being in a state of acrophobia is high. This anti-altitude-fear function is achieved by rendering the area corresponding to the high-drop area in the captured real-time image to obtain an overlay image and detecting the eye position to dynamically correct the display position of the overlay image in the computer vision system.

[0058] In step 204, the computer vision system 104 displays an overlay image based on the detected eye position to cover the high-drop area within the driver's field of vision. For example, in Figure 3b In the scene shown, when it is determined that there is a high-altitude area (a cliff without high guardrails) in the driver's field of vision, "grassland" is used for rendering. Figure 3a The area corresponding to the high drop is captured in the real-time image and projected onto the car's windshield via a HUD display so that the driver sees grass instead of a cliff when looking out of the car, thus preventing drivers with a fear of heights from panicking.

[0059] The foregoing description includes examples of various aspects of the claimed subject matter. It is certainly impossible to describe every conceivable combination of components or methods for the purpose of depicting the claimed subject matter, but those skilled in the art will recognize that many further combinations and arrangements of the claimed subject matter are possible. Thus, the disclosed subject matter is intended to cover all such changes, modifications, and variations that fall within the spirit and scope of the appended claims.

Claims

1. A method for counter-terrorism in automobiles, the method comprising: Acquire real-time images of the vehicle's surroundings and its current altitude; Based on the acquired real-time images and the current altitude, determine whether there are areas of high elevation difference in the driver's field of vision; In response to the determination of a high-elevation area in the driver's field of vision, the driver's eye position is detected; as well as Based on the detected eye position, an overlay image is displayed in the computer vision system to cover the high-altitude area in the driver's field of vision.

2. The method as described in claim 1, characterized in that, The step of determining whether there is a high elevation difference area in the driver's field of vision based on the acquired real-time images and the current altitude further includes: The acquired real-time image is input into the high-drop region calculation model to calculate the region in the real-time image corresponding to the high-drop region.

3. The method as described in claim 2, characterized in that, The step of determining whether there is a high elevation difference area in the driver's field of vision based on the acquired real-time images and the current altitude further includes: If a region corresponding to the high elevation difference exists in the acquired real-time image and the current altitude exceeds a threshold, it is determined that a high elevation difference exists in the driver's field of vision.

4. The method as described in claim 2, characterized in that, The step of displaying the overlay image in the computer vision system to cover the high-elevation area in the driver's field of vision further includes: In the computer vision system, the area corresponding to the high-altitude region in the acquired real-time image is rendered as grass or flat land to cover the high-altitude region in the driver's field of vision.

5. The method as described in claim 4, characterized in that, The rendering further includes: Based on the detected eye position, the driver's gaze point region is determined; and Rendering is performed at a reduced resolution outside the gaze point area.

6. The method as described in claim 1, characterized in that, The display of the overlay image in the computer vision system further includes: The overlay image is displayed by projecting it onto the windshield of the car via a head-up display (HUD) or by using augmented reality (AR) glasses or mixed reality (MR) glasses.

7. The method as described in claim 1, characterized in that, The high-altitude areas include one or more of cliffs, viaducts, or bridges without high guardrails.

8. An automobile, said automobile comprising: A sensor system configured to capture real-time images of the vehicle's surroundings and acquire the current altitude; An image processing system, the image processing system being configured to: The sensor system acquires real-time images of the vehicle's surroundings and its current altitude. Based on the acquired real-time images and the current altitude, determine whether there are areas of high elevation difference in the driver's field of vision; An eye-tracking system configured to detect the driver's eye position in response to determining that a high-drop region exists in the driver's field of vision; as well as A computer vision system configured to display an overlay image based on the detected eye position to cover the high-drop area in the driver's field of vision.

9. The automobile as described in claim 8, characterized in that, The sensor system includes one or more of an altitude sensor, GPS, and a visual sensor.

10. The automobile as described in claim 8, characterized in that, The step of determining whether there is a high elevation difference area in the driver's field of vision based on the acquired real-time images and the current altitude further includes: The acquired real-time image is input into the high-drop region calculation model to calculate the region in the real-time image corresponding to the high-drop region.

11. The automobile as claimed in claim 10, characterized in that, The step of determining whether there is a high elevation difference area in the driver's field of vision based on the acquired real-time images and the current altitude further includes: If a region corresponding to the high elevation difference exists in the acquired real-time image and the current altitude exceeds a threshold, it is determined that a high elevation difference exists in the driver's field of vision.

12. The automobile as described in claim 10, characterized in that, The display overlay image to cover the high elevation difference area in the driver's field of vision further includes: The area in the acquired real-time image corresponding to the high-altitude region is rendered as grass or flat land to cover the high-altitude region in the driver's field of vision.

13. The automobile as described in claim 12, characterized in that, The rendering further includes: Based on the detected eye position, the driver's gaze point region is determined; and Rendering is performed at a reduced resolution outside the gaze point area.

14. The automobile as described in claim 8, characterized in that, The computer vision system includes one or more of a head-up display (HUD), augmented reality (AR) glasses, or mixed reality (MR) glasses.

15. The automobile as described in claim 8, characterized in that, The high-altitude areas include one or more of cliffs, viaducts, or bridges without high guardrails.

Images