Driving environment monitoring system and driving environment monitoring method thereof

Abstract

The invention provides a driving environment monitoring system which is configured on a mobile carrier. The driving environment monitoring system comprises a sight analysis module, an environment analysis module and a processor. The sight analysis module is used for collecting eye images of a driver; and analyzing the eye image to obtain a plurality of sight probabilities that the sight of the eye image is located in the plurality of partitions. The environment analysis module is used for collecting environment images around the mobile carrier; and analyzing the environment image to obtain the probability that at least one object of the environment image is located in a plurality of subareas. The processor is configured to: obtain an integration probability of a line-of-sight probability and an object probability of each partition; obtaining the advancing direction of the moving carrier; and sending a reminding signal on the basis that the integration probability of the partition where the advancing direction is located is greater than a preset probability.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present invention claims priority to U.S. Patent Application No. 63 / 523,400, filed on June 27, 2023, which is incorporated herein by reference for all purposes as if fully set forth herein. Technical Field

[0003] The present invention relates to a driving environment monitoring system and a driving environment monitoring method, and in particular to a driving environment monitoring system and a driving environment monitoring method capable of estimating a driver's line of sight. Background Art

[0004] As the number and types of transportation vehicles increase, the probability of traffic accidents increases year by year. Obviously, in addition to the continuous progress in the power part of transportation technology, the improvement of safety during driving is another topic that needs to be focused on. For example, there are many objects that the driver needs to pay attention to on the driving route of the car, such as traffic signs, pedestrians, obstacles, etc. The driver needs to concentrate on observing the road conditions as a basis for executing the next driving action.

[0005] In today's society, traffic accidents often occur when drivers are not paying attention to the road conditions. Drivers may not notice traffic signs or obstacles on the road due to fatigue or distraction, resulting in failure to perform correct driving actions according to traffic signs or obstacles in front of them. Summary of the invention

[0006] Therefore, the present invention provides a vehicle environment monitoring system and a vehicle environment monitoring method thereof, which can improve the problems existing in the above-mentioned prior art.

[0007] One embodiment of the present invention proposes a driving environment monitoring system. The driving environment monitoring system is configured on a mobile vehicle. The driving environment monitoring system includes a line of sight analysis module, an environment analysis module and a processor. The line of sight analysis module is used to: capture an eye image of a driver; and, analyze the eye image to obtain several line of sight probabilities that a line of sight of the eye image is located in several partitions. The environment analysis module is used to: capture an environment image around the mobile vehicle; and, analyze the environment image to obtain several object probabilities that at least one object in the environment image is located in these partitions. The processor is used to: obtain an integrated probability of the line of sight probability and the object probability of each partition; obtain a forward direction of the mobile vehicle; and, based on the integrated probability that the partition where the forward direction is located is greater than a preset probability, issue a reminder signal.

[0008] Another embodiment of the present invention provides a driving environment monitoring method. The driving environment monitoring method includes the following steps: a sight line analysis module collects an eye image of a driver; the sight line analysis module analyzes the eye image to obtain a plurality of sight line probabilities that a sight line of the eye image is located in a plurality of partitions; an environment analysis module collects an environment image around a mobile vehicle; the environment analysis module analyzes the environment image to obtain a plurality of object probabilities that at least one object in the environment image is located in these partitions; a processor obtains an integrated probability of the sight line probability and the object probability of each partition; the processor obtains a forward direction of the mobile vehicle; and the processor sends a reminder signal based on the integrated probability that the forward direction is located in the partition being greater than a preset probability.

[0009] In order to better understand the above and other aspects of the present invention, embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A functional block diagram of a vehicle environment monitoring system according to an embodiment of the present invention is shown;

[0011] Figure 2A Draw Figure 1 Schematic diagram of eye image;

[0012] Figure 2B Draw Figure 1 Schematic diagram of the line of sight;

[0013] Figure 3 A distribution diagram of the sight line probability of several subareas according to an embodiment of the present invention is shown;

[0014] Figure 4 A graph showing the probability distribution of objects in several partitions according to an embodiment of the present invention is shown;

[0015] Figure 5 A diagram showing the integrated probability distribution of several partitions according to an embodiment of the present invention;

[0016] Fig. 6A Draw Figure 1 A schematic diagram of an environmental camera collecting environmental images around a mobile vehicle;

[0017] Figure 6B Draw Fig. 6A Schematic diagram of object detection based on an environmental image;

[0018] Figure 6C Draw Figure 6B A schematic diagram showing objects of an environmental image being mapped to a plurality of partitions;

[0019] Fig.6D Draw Figure 6C Dynamic diagram of objects;

[0020] Figure 7 Draw Figure 1 Flow chart of the environment monitoring method of the driving environment monitoring system.

[0021] Wherein, the reference numerals are:

[0022] 100: Driving environment monitoring system

[0023] 10: Mobile Vehicle

[0024] 110: Line of sight analysis module

[0025] 111: Line of sight camera

[0026] 112: Line of Sight Analyzer

[0027] 120: Environmental Analysis Module

[0028] 121A~121D: Environmental Camera

[0029] 122: Object Analyzer

[0030] 130: Processor

[0031] M1: Eye Image

[0032] V1: Line of sight

[0033] R1~R12: Partition

[0034] M21~M24: Environmental images

[0035] A1~A5, M21_A1~M21_A5, M22_A1~M22_A2, M24_A3~M24_A5: Object

[0036] M21_F1~M21_F5、M22_F1~M22_F2、M24_F3~M24_F5: Object frame

[0037] M21_C1~M21_C5, M22_C1~M22_C2, M24_C3~M24_C5: Feature coordinates

[0038] D10: Direction of Advance

[0039] S1: Reminder signal

[0040] EB: Eyeball

[0041] EP:Pupil

[0042] O1, O2: Center

[0043] P1: Line of sight judgment model

[0044] P2: Object Judgment Model

[0045] S110A~S150: steps. DETAILED DESCRIPTION

[0046] Please refer to Figure 1 to Figure 5 , Figure 1 FIG. 1 is a functional block diagram of a vehicle environment monitoring system 100 according to an embodiment of the present invention. Figure 2A Draw Figure 1 Schematic diagram of eye image M1, Figure 2B Draw Figure 1 Schematic diagram of the line of sight V1, Figure 3 FIG. 1 shows a distribution diagram of the sight line probability of several regions R1 to R12 according to an embodiment of the present invention. Figure 4 FIG. 1 shows a probability distribution diagram of objects in several partitions R1 to R12 according to an embodiment of the present invention. Figure 5 FIG. 4 is a diagram showing the integrated probability distribution of several partitions R1 - R12 according to an embodiment of the present invention.

[0047] like Figure 1 As shown, the driving environment monitoring system 100 is configured on the mobile vehicle 10, and includes a sight line analysis module 110, an environment analysis module 120 and a processor 130. The sight line analysis module 110 is used to collect the driver's eye image M1, and analyze the eye image M1 to obtain several sight line probabilities that the sight line V1 of the eye image M1 is located in several partitions R1~R12. The environment analysis module 120 is used to: collect the environment images M21~M24 around the mobile vehicle 10, and analyze the environment images M21~M24 to obtain several object probabilities that several objects A1~A5 of the environment images M21~M24 are located in these partitions R1~R12. The processor 130 is used to: obtain the integrated probability of the sight line probability and the object probability of each partition; obtain the forward direction D10 of the mobile vehicle 10; and, based on the integrated probability that the forward direction D10 is located in the partition is greater than the preset probability, send a reminder signal S1. In detail, when the integrated probability of the partition where the moving direction D10 of the mobile vehicle 10 is located is greater than the preset probability, it means that there may be an object in the partition, but the line of sight V1 is not focused on the partition. Therefore, the driving environment monitoring system 100 sends a reminder signal S1 to remind the driver to reduce the accident rate.

[0048] The reminder signal S1 sent by the processor 130 can, for example, control an indicator (not shown) to send an indication signal (not shown). The indicator is, for example, a display, a speaker, a vibrator, etc., and the indication signal is, for example, a corresponding image, sound, vibration, etc. The indicator can be configured at any position in the mobile vehicle where the driver can receive the indication signal, such as a windshield, a dashboard, a steering wheel, or other components.

[0049] In addition, the number of environmental images depends on the actual situation and is not limited in the embodiment of the present invention. Although the number of partitions in the embodiment of the present invention is 12 as an example, this is not used to limit the embodiment of the present invention. In other embodiments, the number of partitions may be less than or more than 12.

[0050] In one embodiment, the sight analysis module 110 is, for example, at least a part of a driver monitoring system (DMS). The driver monitoring system, for example, includes a camera module disposed near the driving position in the mobile vehicle 10 to capture images toward the driver. The driver monitoring system, for example, further includes a sound receiving device (such as a microphone), a radar, and at least one sensor, wherein the sound receiving device can perform voice recognition, and the sensor is, for example, a physiological information sensor that can monitor heart rhythm, body temperature, heartbeat, etc. The environment analysis module 120 is, for example, at least a part of an around view monitoring system (AVM), which, for example, includes a plurality of fisheye cameras disposed outside the mobile vehicle 10 to collect surrounding environment images, or, for example, includes a detector for external object sensing, wherein the detector is, for example, an ultrasonic sensor, a lidar or a radar sensor, etc. The aforementioned processor 130 is, for example, an integrated circuit that can be formed by at least one semiconductor process, such as a semiconductor chip, a semiconductor package, etc.

[0051] For example, if Figure 3 As shown in the figure, the sight probability of the zones R1 to R12 is 60%, 90%, 60%, 10%, 0%, 0%, 0%, 0%, 0%, 0%, 0%, 0% and 10% respectively. For example, the sight probability of the zone R2 is 90%, which means that the driver's sight V1 has a 90% chance of paying attention to the zone R2. Figure 4 As shown, the object probabilities of partitions R1 to R12 are 10% and 90% respectively. 10%, 50%, 80%, 95%, 10%, 10%, 5%, 5%, 90% and 20%. For example, the object probability of partition R2 is 90%, which means that the probability of an object appearing in partition R2 is 90%. The integrated probability is, for example, the result of an operation on the line of sight probability and the object probability. In one embodiment, the operation is, for example, a difference operation. For example, the processor 130 obtains the difference between the line of sight probability and the object probability of each partition, and uses the difference as the integrated probability. In this way, Figure 5 As shown, the integration probabilities of the partitions R1 to R12 are -50%, 0%, -50%, 40%, 80%, 95%, 10%, 10%, 5%, 5%, 90% and 10% respectively. In addition, the aforementioned operations may also be other mathematical operations.

[0052] The greater the probability of an object in a partition, the greater the probability of an object appearing in the partition; the greater the probability of sight in the partition, the greater the attention paid to the partition. The aforementioned integration probability is, for example, the result obtained by using the calculation method of "offsetting or balancing the sight probability and the object probability". In this way, for partition R2, although the probability of an object appearing is as high as 90%, the sight probability is also as high as 90%, and the integration probability after the two are offset or balanced is 0%. For partition R1, although the probability of an object appearing is low (10%), because the sight probability is 60%, the integration probability after the two are offset or balanced is less than 0%. For partition R5, the probability of an object appearing is as high as 80%, but because the sight probability is not high (40%), the integration probability after the two are offset or balanced is still greater than 0% (40%). The greater the integration probability of a partition, the greater the probability of an accident occurring in the partition (if the moving direction D10 of the mobile vehicle 10 is also located in the same partition).

[0053] Figure 3 to Figure 5 The sight line probability and object probability of each partition depends on the actual sight line and the actual object distribution, and is not limited by the embodiment of the present invention. Figure 3 to Figure 5 The partitions of the distribution graph of are determined by the same definition. For example, Figure 3 Partitions R1 to R12, Figure 4 Partitions R1 to R12 and Figure 5 The partitions R1 to R12 are the same in number, central angle and position. Figure 3 to Figure 5 The boundaries of the several sub-areas are defined by circles, but the actual monitoring range is not limited by the circles shown in the figure. In this embodiment, in the same distribution diagram, each sub-area has a central angle, and the sum of these central angles is 360 degrees. The vertex (center) of the central angle is, for example, the center point of the eyeball ( Figure 2B The origin of the reference coordinate system XYZ). In one embodiment, the center angles of each partition are equal, but may be different. The size of the center angle of the partition can be determined as needed, and the center angle can also be minimized to make the partition close to continuous distribution. Figure 2B The XZ plane is, for example, a top-view plane of the mobile vehicle 10 (parallel to the ground), and the Y axis is perpendicular to the XZ plane (ie, perpendicular to the ground). Figure 2B The angle is also called the yaw angle (yaw), which is defined in the XZ plane, and the angle is also called the pitch angle (pitch), which is the angle of the line of sight V1 relative to the XZ plane. In addition, the number of objects depends on the actual situation and is not limited in the embodiment of the present invention. Figure 2B As shown, the line of sight V1 herein is, for example, the direction from the center O1 of the eyeball EB to the center O2 of the pupil EP.

[0054] like Figure 5As shown, if the moving direction D10 of the mobile vehicle 10 is located in the partition R2, since the integration probability of the partition R2 is greater than 0% (preset probability), it means that the sight is focused on the object, so the driving environment monitoring system 100 does not send a reminder signal. If the moving direction D10 of the mobile vehicle 10 is located in the partition R4, since the integration probability of the partition R4 is 40%, which is greater than 0% (preset probability), it means that the sight may not be focused on the object, and the driving environment monitoring system 100 sends a reminder signal to remind the driver. The aforementioned preset probability is not limited to 0%, it can be other values, such as 5%, higher or lower.

[0055] like Figure 1 and 2A As shown, the sight line analysis module 110 can analyze the eyeball EV and pupil EP of the driver's eye image M1 according to the sight line judgment model P1 to obtain the sight line probability of the driver's sight line V1 being located in each partition. The sight line judgment model P1 can be obtained by training a plurality of known eye images using machine learning technology (e.g., deep learning technology). In addition, the embodiment of the present invention does not limit the technology for obtaining the sight line judgment model P1. In one embodiment, the eye image M1 is input into the sight line judgment model P1, and after being analyzed by the sight line judgment model P1, the sight line probability of each partition is output.

[0056] like Figure 1 and 3 As shown, the sight line analysis module 110 includes a sight line camera 111 and a sight line analyzer 112. The sight line camera 111 can be configured in the mobile vehicle 10 at a position facing the driver's face or eyes (for example, on the dashboard) to collect the driver's eye image M1. The sight line analyzer 112 can be, for example, an integrated circuit formed by at least one semiconductor process, such as a semiconductor chip, a semiconductor package, etc. The sight line analyzer 112 is electrically connected to the sight line camera 111, and is used to: analyze the eye image M1 to obtain several sight line probabilities that the sight line V1 of the eye image M1 is located in several partitions R1~R12.

[0057] like Figure 1 and 4As shown, the environment analysis module 120 includes several environment cameras 121A~121D and an object analyzer 122. The environment camera is, for example, a wide-angle camera, and its field of view (FoV) is, for example, more than 140 degrees. These environment cameras 121A~121D are arranged on several outer surfaces of the mobile vehicle 10, wherein each environment camera 121 is used to collect environment images M21~M24, and each environment image includes at least one corresponding object. The environment images M21~M24 can constitute a 360-degree panoramic image. The object analyzer 122 can be, for example, an integrated circuit formed by at least one semiconductor process, such as a semiconductor chip, a semiconductor package, etc. The object analyzer 122 is used to analyze at least one object in each environment image and determine whether at least one object in each environment image is the same object. The object analyzer 122 can use the object judgment model P2 to determine whether there is an object in each environment image. Objects are, for example, specific objects in a moving or fixed state, such as people, animals, vehicles, obstacles, and traffic signs. The object judgment model P2 can be obtained by training multiple known object images using machine learning technology (for example, deep learning technology). In addition, the embodiment of the present invention does not limit the technology for obtaining the object judgment model P2. If there is an object in the environmental image, the object analyzer 122 can obtain the object frame of the object in the environmental image and obtain the characteristic coordinates of the object frame for subsequent dynamic tracking of the object. The following is a further example.

[0058] Please refer to Figure 6A to Figure 6D , Fig. 6A Draw Figure 1 The environmental cameras 121A-121D capture the environmental images M21-M24 around the mobile vehicle 10, Figure 6B Draw Fig. 6A Schematic diagram of object detection in environmental images M21 to M24. Figure 6C Draw Figure 6B The objects of the environment image M21 to M24 are mapped to a plurality of partitions. Fig.6D Draw Figure 6C Dynamic diagram of the object.

[0059] like Fig. 6A As shown, multiple environmental cameras 121A-121D are disposed on several outer surfaces of the mobile vehicle 10. For example, multiple environmental cameras 121A-121D are respectively disposed on the front, right, rear and left sides of the mobile vehicle 10 to respectively capture environmental images M21-M24. Depending on the actual situation, the environmental image may have at least one object or no object.

[0060] The two environmental images M21-M24 have a partial image overlap region, and in the image overlap region, the two objects of the two environmental images may be the same object. For example, the environmental image M21 has five objects M21_A1-M21_A5, and the environmental image M22 has two objects M22_A1-M22_A2, wherein the object M21_A1 of the environmental image M21 and the object M22_A1 of the environmental image M22 are the same object, and the object M21_A2 of the environmental image M21 and the object M22_A2 of the environmental image M22 are the same object. For another example, the environment image M24 has three objects M24_A3 to M24_A5, wherein object M24_A3 of the environment image M24 and object M21_A3 of the environment image M21 are the same object, object M24_A4 of the environment image M24 and object M21_A3 of the environment image M21 are the same object, and object M24_A5 of the environment image M24 and object M21_A5 of the environment image M21 are the same object.

[0061] like Figure 6B As shown, the object analyzer 122 can use the object judgment model P2 to determine whether there is an object in the environment image. If there is, the object analyzer 122 can use the object judgment model P2 to obtain the object probability of each partition. In addition, the object probability can be accumulated over time. In detail, if at least one object appears in a partition at two time points, the object probability of the partition can be accumulated over time. In addition, the more objects there are in a partition, the higher its object probability is.

[0062] like Figure 6B As shown, the environment analysis module 120 is further used to: analyze the environment image to obtain the object frame of at least one object in the environment image; and obtain the characteristic coordinates of the object frame. Figure 6BAs shown, taking the environment image M21 as an example, the object analyzer 122 can analyze the environment image M21 to obtain the object frames M21_F1-M21_F5 of the objects M21_A1-M21_A5 of the environment image M21; and obtain the characteristic coordinates M21_C1-M21_C5 of the object frames M21_F1-M21_F5. Taking the environment image M22 as an example, the object analyzer 122 can analyze the environment image M22 to obtain the object frames M22_F1-M22_F2 of the objects M22_A1-M22_A2 of the environment image M22; and obtain the characteristic coordinates M22_C1-M22_C2 of the object frames M22_F1-M22_F2. Taking the environment image M24 as an example, the object analyzer 122 can analyze the environment image M24 to obtain the object frames M24_F3 to M24_F5 of the objects M24_A3 to M24_A5 of the environment image M24; and obtain the characteristic coordinates M24_C3 to M24_C5 of the object frames M24_F3 to M24_F5. In one embodiment, the object frame has a bottom edge, and the characteristic coordinates are, for example, the coordinates of the middle point of the bottom edge. In other embodiments, the characteristic coordinates can be other position points of the object frame, such as a vertex, any point of a side length (e.g., a midpoint), the intersection of two diagonals, a point defined according to at least one side length of the object frame, a point inside the object frame, etc. The characteristic coordinates of each object in each environment image are determined by the same definition, for example.

[0063] like Figure 6C As shown, the object analyzer 122 of the environment analysis module 120 is further used to determine the partition where the feature coordinates are located according to the feature coordinates.

[0064] For example, the object M21_A1 of the environment image M21 and the object M22_A1 of the environment image M22 belong to the same object ( Figure 6C denoted by the same symbol A1), which are located in the same partition R2. For example, the object M21_A2 of the environment image M21 and the object M22_A2 of the environment image M22, the object M21_A2 and the object M22_A2 belong to the same object ( Figure 6C denoted by the same symbol A2), which are located in the same partition R2. For example, the object M21_A3 of the environment image M21 and the object M22_A3 of the environment image M22, the object M21_A3 and the object M22_A3 belong to the same object ( Figure 6C denoted by the same symbol A3), which are located in the same partition R1. For example, the object M21_A4 of the environment image M21 and the object M24_A4 of the environment image M24, the object M21_A4 and the object M24_A4 belong to the same object ( Figure 6Cdenoted by the same symbol A4), which are located in the same partition R1. For example, the object M21_A5 of the environment image M21 and the object M24_A5 of the environment image M24, the object M21_A5 and the object M24_A5 belong to the same object ( Figure 6C Indicated by the same symbol A5), which is located in the same partition R12.

[0065] Each of the graphic elements of each of the environmental images M21 to M24 can be referenced to the reference coordinate system XYZ, that is, each of the graphic elements of each of the environmental images M21 to M24 has coordinates referenced to the reference coordinate system XYZ. After the environmental camera is installed on the mobile vehicle 10, the coordinate value of each of the graphic elements of each of the environmental images M21 to M24 relative to the reference coordinate system XYZ can be determined through methods such as actual shooting tests. Therefore, for the same object, its two characteristic coordinates located in the two environmental images roughly overlap in the same partition. For example, Figure 6B The object frame M21_F1 and the object frame M22_F1 belong to the same object A1, so their feature coordinates M21_C1 and feature coordinates M22_C1 are approximately Figure 6C The partition R2 of the object overlaps with point A1 (hereinafter referred to as object A1). The environment analysis module 120 can analyze Figure 6C The dynamics of objects, such as position changes, speed, acceleration, etc.

[0066] like Fig.6D As shown, the environment analysis module 120 is further used to: track the dynamic changes of feature coordinates. Taking object A1 as an example, for example, the object analyzer 122 is used to: analyze the distance difference between object M21_A1 of the environment image M21 at two points in time, and obtain the speed of object M21_A1 based on the distance difference and the time difference between the two points in time (for example, along the Z axis or along the X axis or both axes); the object analyzer 122 can analyze the distance difference between object M22_A1 of the environment image M22 at two points in time, and obtain the speed of object M22_A1 based on the distance difference and the time difference between the two points in time (for example, along the Z axis or along the X axis or both axes); and, based on the speed of object M21_A1 and the speed of object M22_A1, determine the composite speed of object A1 (for example, Fig.6D The length of arrow DA1) and the synthetic direction (for example, Fig.6D The object analyzer 122 is further used to obtain the synthesis speed and synthesis direction of other objects A2-A5 in the same manner.

[0067] Please refer to Figure 7 , which shows Figure 1 Flow chart of the environment monitoring method of the driving environment monitoring system 100.

[0068] In step S110A, if Figure 1 and Figure 2A As shown, the sight line analysis module 110 collects the driver's eye image M1. For example, the sight line camera 111 of the sight line analysis module 110 collects the driver's eye image M1, wherein the eye image M1 includes the eyeball EV and the pupil EP.

[0069] In step S110B, the sight line analyzer 112 of the sight line analysis module 110 analyzes the eye image M1 to obtain a plurality of sight line probabilities that the sight line V1 of the eye image M1 is located in a plurality of subareas R1-R12. For example, the sight line analyzer 112 analyzes the eyeball EV and the pupil EP of the driver's eye image M1 according to the sight line judgment model P1 to obtain the sight line probability that the driver's sight line V1 is located in each subarea, such as Figure 3 The line of sight probability distribution is shown.

[0070] In step S120A, the multiple environmental cameras 121A- 121D of the environmental analysis module 120 collect environmental images M21 - M24 around the mobile vehicle 10 .

[0071] In step S120B, the object analyzer 122 of the environment analysis module 120 analyzes the environment images M21-M24 to obtain several object probabilities that at least one object in the environment images M21-M24 is located in the regions R1-R12, such as Figure 4 Object probability distribution diagram.

[0072] In step S130, the processor 130 obtains the integrated probability of the object probability and the line of sight probability of each partition, such as Figure 5 The integrated probability distribution diagram is, for example, a result obtained by using an operation method of "offsetting or balancing the object probability and the line of sight probability". The above operation is, for example, a difference operation.

[0073] In step S140, if Figure 6C As shown, the processor 130 obtains the forward direction D10 of the mobile vehicle 10. In one embodiment, the processor 130 can obtain it from the vehicle control computer (not shown) of the mobile vehicle 10, or according to the rotation angle of the steering wheel (not shown) of the mobile vehicle 10. For example, when the mobile vehicle 10 is moving straight (not turning, roughly parallel to the -Z axis of the reference coordinate system XYZ), the angle of the steering wheel is defined as 0 degrees. The processor 130 can obtain the partition where the forward direction D10 of the mobile vehicle 10 is located based on the rotation direction (e.g., clockwise or counterclockwise) and the rotation angle (relative to the -Z axis) of the steering wheel.

[0074] In step S150, the processor 130 issues a reminder signal S1 (the reminder signal S1 is shown in FIG. 150 ) based on the fact that the integrated probability of the partition in which the forward direction D10 is located is greater than the preset probability. Figure 1). For example, taking the preset probability as 0%, when the integration probability of a partition is equal to or less than 0%, it means that there may be an object in the partition and the line of sight V1 pays attention to the partition, or there is no object in the partition and the line of sight V1 pays attention to or does not pay attention to the partition, both of which belong to relatively safe partitions. If the forward direction D10 of the mobile vehicle 10 is also located in the partition, the processor 130 does not issue a reminder signal. On the contrary, when the integration probability of a partition is greater than 0%, it means that there may be an object in the partition but the line of sight V1 does not pay attention to it or the attention is low, which belongs to a relatively dangerous partition. If the forward direction D10 of the mobile vehicle 10 is also located in the partition, the processor 130 sends a reminder signal S1 to remind the driver to pay attention to safety.

[0075] The remaining steps of the environment monitoring method of the embodiment of the present invention have been described above and will not be repeated here.

[0076] In summary, the embodiment of the present invention provides a driving environment monitoring system and a driving environment monitoring method thereof, which can at least detect the object probability of at least one object in an environmental image being distributed in multiple partitions and the sight probability of the driver's sight being distributed in multiple partitions, and perform an operation on the object probability and the sight probability to obtain a balanced integrated probability. In one embodiment, the driving environment monitoring system can issue a reminder signal based on the integrated probability being equal to or greater than a preset probability; and the driving environment monitoring system can not issue a reminder signal based on the integrated probability being less than the preset probability.

[0077] In summary, although the present application has been disclosed as above by way of embodiments, it is not intended to limit the present application. A person skilled in the art of the present application may make various modifications and improvements without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be subject to the scope defined by the patent application.

Claims

1. A vehicle environment monitoring system, configured on a mobile vehicle, characterized in that: The driving environment monitoring system includes: A line of sight analysis module, used to: capturing an image of one eye of a driver; and Analyzing the eye image to obtain a plurality of sight line probabilities that a sight line of the eye image is located in a plurality of subareas; An environmental analysis module, used to: Capturing an environmental image around the mobile vehicle; and Analyzing the environment image to obtain a plurality of object probabilities that at least one object in the environment image is located in the plurality of partitions; and A processor for: Obtaining an integrated probability of the sight probability and the object probability of each of the partitions; obtaining a moving direction of the mobile vehicle; and Based on the fact that the integrated probability that the forward direction is located in the partition is greater than a preset probability, a reminder signal is issued.

2. The vehicle environment monitoring system according to claim 1, characterized in that: The processor is further configured to: A difference between the object probability and the sight probability of each partition is obtained, and the difference is used as the integrated probability.

3. The vehicle environment monitoring system according to claim 1, characterized in that: The environmental analysis module is further used to: Analyzing the environment image to obtain an object frame of each of the at least one object in the environment image; Obtaining a feature coordinate of the object frame; and The partition where the feature coordinates are located is determined according to the feature coordinates.

4. The vehicle environment monitoring system according to claim 3, characterized in that: The object frame has a bottom edge, and the characteristic coordinates are coordinates of a middle point of the bottom edge.

5. The vehicle environment monitoring system according to claim 1, characterized in that: The environmental analysis module includes: A plurality of environmental cameras are disposed on a plurality of outer surfaces of the mobile vehicle, wherein each of the environmental cameras is used to capture the environmental image, and each of the environmental images includes the corresponding at least one object; An object analyzer, for: The at least one object in each of the environmental images is analyzed to determine that the at least one object in each of the environmental images is the same object.

6. The vehicle environment monitoring system according to claim 1, characterized in that: Each of the partitions has a central angle, and the sum of the central angles is 360 degrees.

7. The vehicle environment monitoring system according to claim 1, characterized in that: Each of the partitions has a central angle, and each of the central angles is equal.

8. A method for monitoring a vehicle driving environment, characterized in that: The driving environment monitoring method comprises: A sight line analysis module collects an eye image of a driver; The sight line analysis module analyzes the eye image to obtain a plurality of sight line probabilities that a sight line of the eye image is located in a plurality of subareas; An environment analysis module collects an environment image around a mobile vehicle; The environment analysis module analyzes the environment image to obtain a plurality of object probabilities that at least one object in the environment image is located in the plurality of partitions; A processor obtains an integrated probability of the sight probability and the object probability of each partition; The processor obtains a forward direction of the mobile vehicle; and The processor sends a reminder signal based on the fact that the integrated probability that the forward direction is located in the partition is greater than a preset probability.

9. The vehicle driving environment monitoring method according to claim 8, characterized in that: The driving environment monitoring method also includes: The processor obtains a difference between the sight probability and the object probability of each partition, and uses the difference as the integrated probability.

10. The vehicle driving environment monitoring method according to claim 8, characterized in that: The driving environment monitoring method also includes: The processor analyzes the environmental image to obtain an object frame of each of the at least one object in the environmental image; The processor obtains a feature coordinate of the object frame; and The processor determines the partition where the feature coordinate is located according to the feature coordinate.

11. The vehicle driving environment monitoring method according to claim 10, characterized in that: The object frame has a bottom edge, and the characteristic coordinates are coordinates of a middle point of the bottom edge.

12. The vehicle driving environment monitoring method according to claim 8, characterized in that: The driving environment monitoring method comprises: The environment analysis module collects a plurality of the environment images, wherein each of the environment images includes the corresponding at least one object; The environment analysis module analyzes the at least one object in each of the environment images to determine that the at least one object in each of the environment images is the same object.

13. The vehicle driving environment monitoring method according to claim 8, characterized in that: Each of the partitions has a central angle, and the sum of the central angles is 360 degrees.

14. The vehicle driving environment monitoring method according to claim 8, characterized in that: Each of the partitions has a central angle, and each of the central angles is equal.

Images