Intelligent vehicle-mounted safety auxiliary system based on image fusion

Abstract

The invention relates to the technical field of safety assistance, in particular to an intelligent vehicle-mounted safety assistance system based on image fusion, and the system comprises the steps: firstly inputting a face image in advance, obtaining a visible light image and an infrared light image, then carrying out image fusion processing, and extracting portrait features; comparing the portrait features with portrait features in an input face image, analyzing whether the extracted portrait features are qualified or not according to the matching degree of the portrait features, and when the matching degree is between a first preset matching degree and a second preset matching degree, judging whether the portrait features are qualified or not; secondary judgment is performed on whether the extracted portrait features are qualified or not according to the coincidence degree of the main feature edge and the preset feature edge, so that the control precision of the collected image is improved, the facial expression and the sight direction of the driver are captured, whether the driver has distraction or not in the driving process is judged, and the driving experience is improved. And the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of safety assistance technology, and in particular to an intelligent vehicle-mounted safety assistance system based on image fusion. Background Art

[0002] Image fusion technology achieves information redundancy and complementarity by integrating image information from multiple sensors. When a sensor fails or is interfered with, the information from other sensors can still ensure the normal operation of the system, thereby improving the reliability and stability of the entire intelligent vehicle system and providing strong support for the large-scale application of autonomous driving. Existing technologies use multiple sensors to obtain information about the external environment, allowing drivers to control the vehicle's environment at any time, thereby effectively avoiding obstacles around the vehicle and improving driving safety. However, the driver's own state is not analyzed. The driver's driving state is affected by many factors. If the driver's attention is distracted and no timely reminder is given, it is very easy to cause danger.

[0003] Chinese patent application number: CN202010653921.3 discloses a vehicle-mounted safety assistance system, including a vehicle-mounted terminal device and a user handheld device, wherein the vehicle-mounted terminal device includes a processor unit, and an identity authentication unit, a vehicle status information acquisition unit, an image acquisition unit, a display unit, a voice prompt unit, a parking security unit, an infrared sensor unit, a data storage unit, a wireless transmission unit, an expansion interface unit, and a power supply unit, which are respectively connected to the processor unit. Through this application, the driver can control the environment of the vehicle at any time and effectively avoid obstacles around the vehicle. Especially in crowded situations, it can achieve blind spot observation and achieve the purpose of safe driving.

[0004] However, the prior art still has the following problems: The driver's own state has not been analyzed. The driver's driving state is affected by many factors. If the driver is not reminded in time when his attention is distracted, it is very easy to cause danger. Summary of the invention

[0005] To this end, the present invention provides an intelligent vehicle-mounted safety assistance system based on image fusion to overcome the problem in the prior art that the driver's own state is not analyzed. The driver's driving state is affected by many factors. If the driver's attention is distracted without timely reminder, it is very easy to cause danger.

[0006] To achieve the above objectives, the present invention provides an intelligent vehicle safety assistance system based on image fusion. It includes: An image acquisition unit, comprising a first image acquisition module for acquiring visible light images and a second image acquisition module for acquiring infrared light images; An image fusion unit, which is connected to the image acquisition unit and is used to fuse the visible light image and the infrared light image to obtain a fused image; A feature extraction unit, which is connected to the image fusion unit and is used to extract portrait features in the fused image. The portrait features include main features and secondary features. Among them, the main feature is the eye feature, and the secondary feature is the face contour feature, and the secondary feature takes the nose feature as the central feature; A coordinate construction unit, which is respectively connected to the image fusion unit and the feature extraction unit, and is used to construct a coordinate system based on the extracted portrait features and transfer the portrait features into the coordinate system; An analysis unit, which is respectively connected to the image acquisition unit, the image fusion unit, the feature extraction unit and the coordinate construction unit, and is used to compare the extracted portrait features with preset portrait features to calculate the matching degree, analyze whether the extracted portrait features are qualified according to the matching degree. When it is initially determined that the extracted portrait features are unqualified, based on the coincidence degree between the main feature edge and the preset main feature edge, make a secondary determination on whether the extracted portrait features are qualified, or analyze the reasons for the unqualified extracted portrait features; An output unit, which is connected to the analysis unit and is used to generate a corresponding signal instruction based on the determined reasons for the unqualified extracted portrait features.

[0007] Further, the analysis unit is used to analyze whether the extracted portrait features are qualified according to the matching degree, including: Compare the extracted portrait features with the preset portrait features, Determine the area of the overlapping part between the portrait features and the preset portrait features to obtain the feature overlapping area, Calculate the ratio of the feature overlapping area to the total area of the preset portrait features to obtain the matching degree, If the matching degree is greater than or equal to the first preset matching degree, the analysis unit determines that the extracted portrait features are qualified; If the matching degree is less than the first preset matching degree and greater than or equal to the second preset matching degree, the analysis unit initially determines that the extracted portrait features are unqualified, and makes a secondary determination on whether the extracted portrait features are qualified based on the coincidence degree between the main feature edge and the preset main feature edge; If the matching degree is less than the second preset matching degree, the analysis unit determines that the extracted portrait features are unqualified.

[0008] Further, the analysis unit is used to make a secondary determination on whether the extracted portrait features are qualified based on the coincidence degree between the main feature edge and the preset main feature edge, including: Determine the length of the overlapping part between the main feature edge and the preset main feature edge, Calculate the ratio of the length of the overlapping part to the total length of the preset main feature edge to get the overlap degree. If the overlap is less than or equal to a preset overlap, the analysis unit determines that the main feature position is offset; If the overlap degree is greater than the preset overlap degree, the analysis unit determines that the extracted portrait features are unqualified and analyzes the reasons for the unqualifiedness.

[0009] Further, the analysis unit is used to analyze whether to correct the determination criterion based on the relative position relationship between each main feature and the secondary feature under the condition that the main feature position is determined to be offset, including: Calculate the absolute value of the difference between each main feature and the central feature, If the absolute value is less than or equal to the preset absolute value, the analyzing unit determines whether to adjust the reference coordinates of each preset main feature based on the ordinate of the central feature; If the absolute value is greater than the preset absolute value, the analysis unit determines that the face is not facing the collection device and continues monitoring.

[0010] Further, the analysis unit is used to determine whether to adjust the reference coordinates of each preset main feature based on the ordinate of the central feature, including: Determine the absolute distance between the center feature and the preset center feature based on the ordinate, If the absolute distance is less than or equal to the preset absolute distance, the analysis unit determines to correct the reference coordinates of each preset main feature; If the absolute distance is greater than the preset absolute distance, the analysis unit determines that the face is not facing the collection device and continues monitoring.

[0011] Furthermore, the analysis unit corrects the reference coordinates of each preset main feature, including: Calculating the difference between the matching degree and the second preset matching degree to obtain a matching degree difference, The correction amplitude of the horizontal coordinate of the preset main feature is positively correlated with the matching degree difference.

[0012] Furthermore, the analysis unit is further configured to monitor a duration during which the absolute distance is less than or equal to a preset absolute distance. If the duration is greater than or equal to the preset duration, the analysis unit determines that there is a problem with the driver's driving habits.

[0013] Further, the analysis unit calculates the area of ​​the overlapping portion of the secondary feature and the preset secondary feature under the condition that the extracted portrait feature is determined to be unqualified, and the analysis unit determines the overlapping area ratio according to the ratio of the overlapping area to the total area of ​​the preset secondary feature, so as to analyze the reason why the extracted portrait feature is unqualified according to the overlapping area ratio, including: If the overlap area ratio is less than or equal to the preset overlap area ratio, the analysis unit determines the reason why the extracted portrait features based on the overlap situation of the secondary feature edges are unqualified; If the overlap area ratio is greater than the preset overlap area ratio, the analysis unit determines that the reason why the extracted portrait features are unqualified is that the driver's status does not meet the standards, and issues a driver status unqualified signal.

[0014] Furthermore, the analysis unit is used to analyze the reasons why the extracted portrait features are unqualified based on the overlap of the secondary feature edges, including: Determine the length of the overlap between the secondary feature edge and the preset secondary feature edge. Calculate the ratio of the length of the overlapping part to the total length of the preset secondary feature edge to obtain the secondary feature overlap degree. If the secondary feature overlap is less than or equal to the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image fusion process is unqualified; If the secondary feature overlap is greater than the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image acquisition process is unqualified, and sends an image acquisition unqualified signal.

[0015] Furthermore, the analysis unit adjusts the sharpening intensity in the image fusion process based on the secondary feature overlap when determining that the image fusion process is unqualified, wherein the increase in the sharpening intensity is negatively correlated with the secondary feature overlap.

[0016] Compared with the prior art, the beneficial effect of the present invention lies in that, in the present invention, a facial image is first pre-recorded, a visible light image and an infrared light image are obtained, and then image fusion processing is performed to extract portrait features, and the portrait features are compared with the portrait features in the recorded facial image. Whether the extracted portrait features are qualified is analyzed according to the matching degree of the portrait features. When the matching degree is between a first preset matching degree and a second preset matching degree, a secondary judgment is made on whether the extracted portrait features are qualified according to the overlap degree of the main feature edge with the preset feature edge, thereby improving the control accuracy of the collected image, and then capturing the driver's facial expression and line of sight direction, so as to determine whether the driver is distracted during driving, thereby improving driving safety.

[0017] Furthermore, in the present invention, the matching degree of the portrait features is first determined according to the overlapping area of ​​the portrait features in the acquired fusion image, and a preliminary analysis is performed on whether the extracted portrait features are qualified according to the matching degree. When the matching degree is between the first matching degree and the second preset matching degree, a secondary judgment is made on whether the extracted portrait features are qualified according to the edge contour of the portrait features, thereby improving the control precision of the acquired image and the analysis accuracy of the image.

[0018] Furthermore, the present invention calculates the distance between each main feature and the central feature, and solves the difference of each distance. The symmetry of the collected facial image is analyzed based on the distance difference, and then the driving state of the driver is analyzed based on the symmetry of the image. When the absolute value of the difference is large, it is determined that the driver's face is not facing the collection device, and continuous monitoring is carried out to facilitate subsequent reminders based on the length of time the driver's face is not facing the collection device, thereby improving driving safety.

[0019] Furthermore, the present invention first calculates the distance between the central feature of the acquired portrait feature and the vertical coordinate of the preset central feature, and analyzes whether the driver is looking up or looking down based on the distance. Taking into account the driver's driving habits, the coordinates of the preset central feature are adjusted according to the analysis results, thereby improving the matching degree between the judgment benchmark and the actual situation and improving the analysis accuracy of the facial image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a structural block diagram of an intelligent vehicle safety assistance system based on image fusion; Figure 2 A flow chart for analyzing whether the extracted portrait features are qualified; Figure 3 A flow chart for determining whether the extracted portrait features are qualified for secondary determination; Figure 4 This is a decision flow chart for analyzing whether to modify the decision criteria. DETAILED DESCRIPTION

[0021] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0022] It should be pointed out that the data in this embodiment are obtained by comprehensive analysis and evaluation of the historical data of the system of the present invention in the six months before this determination and the corresponding historical determination results. It can be understood by those skilled in the art that the determination method of the system of the present invention for a single parameter mentioned above can be to select the value with the highest proportion as the preset standard parameter according to the data distribution, use weighted summation to use the obtained value as the preset standard parameter, substitute each historical data into a specific formula and use the value obtained by the formula as the preset standard parameter or other selection methods, as long as the system of the present invention can clearly define different specific situations in the single determination process through the obtained values.

[0023] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0024] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0025] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] See also Figure 1 As shown, it is a structural block diagram of an intelligent vehicle safety assistance system based on image fusion.

[0027] The intelligent vehicle safety assistance system based on image fusion provided in this embodiment includes: An image acquisition unit, comprising a first image acquisition module for acquiring visible light images and a second image acquisition module for acquiring infrared light images; an image fusion unit, connected to the image acquisition unit, for fusing the visible light image with the infrared light image to obtain a fused image; A feature extraction unit, connected to the image fusion unit, for extracting portrait features from the fused image, wherein the portrait features include primary features and secondary features; A coordinate construction unit, which is respectively connected to the image fusion unit and the feature extraction unit, and is used to construct a coordinate system based on the extracted portrait features and transfer the portrait features into the coordinate system; An analysis unit, which is respectively connected to the image acquisition unit, the image fusion unit, the feature extraction unit and the coordinate construction unit, and is used to compare the extracted portrait features with preset portrait features to calculate the matching degree, analyze whether the extracted portrait features are qualified according to the matching degree, and when it is initially determined that the extracted portrait features are unqualified, perform a secondary determination on whether the extracted portrait features are qualified based on the coincidence degree between the main feature edge and the preset main feature edge, or analyze the reason why the extracted portrait features are unqualified; An output unit, which is connected to the analysis unit and is used to generate a corresponding signal instruction based on the determined reason for the unqualified extracted portrait features.

[0028] Specifically, in this embodiment, the main feature is the eye feature, and the secondary feature is the face contour feature, wherein the secondary feature takes the nose feature as the central feature.

[0029] Specifically, in this embodiment, the preset main feature and the preset secondary feature are both portrait features in the pre-collected face images of the driver.

[0030] In the present invention, the face image is pre-recorded first to obtain a visible light image and an infrared light image, then image fusion processing is performed, portrait features are extracted, the portrait features are compared with the portrait features in the recorded face image, and it is analyzed whether the extracted portrait features are qualified according to the matching degree of the portrait features. When the matching degree is between the first preset matching degree and the second preset matching degree, a secondary determination is made on whether the extracted portrait features are qualified according to the coincidence degree between the main feature edge and the preset feature edge, so as to improve the control accuracy of the collected images, and then capture the facial expressions and line-of-sight directions of the driver, so as to judge whether there is a distraction during driving and improve driving safety.

[0031] Please refer to Figure 2 as shown, which is a determination flowchart for analyzing whether the extracted portrait features are qualified.

[0032] Specifically, the analysis unit is used to analyze whether the extracted portrait features are qualified according to the matching degree, including: Compare the extracted portrait features with the preset portrait features, Determine the area of the overlapping part between the portrait features and the preset portrait features to obtain the feature overlapping area, Calculate the ratio of the feature overlapping area to the total area of the preset portrait features to obtain the matching degree, If the matching degree is greater than or equal to the first preset matching degree, the analysis unit determines that the extracted portrait features are qualified; If the matching degree is less than the first preset matching degree and greater than or equal to the second preset matching degree, the analysis unit preliminarily determines that the extracted portrait feature is unqualified, and performs a secondary determination on whether the extracted portrait feature is qualified based on the overlap between the main feature edge and the preset main feature edge; If the matching degree is less than the second preset matching degree, the analyzing unit determines that the extracted portrait features are unqualified.

[0033] Specifically, in this embodiment, the first preset matching degree is selected in the interval [0.92, 0.95], and the second preset matching degree is selected in the interval [0.8, 0.85].

[0034] See also Figure 3 As shown, it is a flow chart for secondary determination of whether the extracted portrait features are qualified.

[0035] Specifically, the analysis unit is used to perform a secondary determination on whether the extracted portrait features are qualified based on the overlap between the main feature edge and the preset main feature edge, including: Determine the length of the overlap between the edge of the main feature and the preset edge of the main feature, Calculate the ratio of the length of the overlapping part to the total length of the preset main feature edge to get the overlap degree. If the overlap is less than or equal to a preset overlap, the analysis unit determines that the main feature position is offset; If the overlap degree is greater than the preset overlap degree, the analysis unit determines that the extracted portrait features are unqualified and analyzes the reasons for the unqualifiedness.

[0036] Specifically, in this embodiment, the preset overlap degree is selected between the interval [0.72, 0.78].

[0037] In the present invention, the matching degree of the portrait features is first determined according to the overlapping area of ​​the portrait features in the acquired fusion image, and a preliminary analysis is performed on whether the extracted portrait features are qualified according to the matching degree. When the matching degree is between a first matching degree and a second preset matching degree, a secondary judgment is made on whether the extracted portrait features are qualified according to the edge contour of the portrait features, thereby improving the control accuracy of the acquired image and the analysis accuracy of the image.

[0038] See also Figure 4 As shown in FIG. 1 , it is a determination flowchart for analyzing whether to correct the determination criterion.

[0039] Specifically, the analysis unit is used to analyze whether to correct the determination criterion based on the relative position relationship between each main feature and the secondary feature under the condition that the main feature position is determined to be offset, including: Calculate the absolute value of the difference between each main feature and the central feature, If the absolute value is less than or equal to the preset absolute value, the analyzing unit determines whether to adjust the reference coordinates of each preset main feature based on the ordinate of the central feature; If the absolute value is greater than the preset absolute value, the analysis unit determines that the face is not facing the collection device and continues monitoring.

[0040] Specifically, in this embodiment, the preset absolute value is obtained by pre-measurement, obtaining several qualified fused images, extracting the portrait features in each fused image, calculating the absolute value of the difference between the distance of each main feature and the central feature, solving the mean of the absolute values, and obtaining the preset absolute value.

[0041] In the present invention, the distance between each main feature and the central feature is calculated, and the difference of each distance is solved. The symmetry of the collected facial image is analyzed according to the distance difference, and then the driving state of the driver is analyzed according to the symmetry of the image. When the absolute value of the difference is large, it is determined that the driver's face is not facing the collection device, and continuous monitoring is carried out to facilitate subsequent reminders based on the length of time the driver's face is not facing the collection device, thereby improving driving safety.

[0042] Specifically, the analysis unit is used to determine whether to adjust the reference coordinates of each preset main feature based on the vertical coordinate of the central feature, including: Determine the absolute distance between the center feature and the preset center feature based on the ordinate, If the absolute distance is less than or equal to the preset absolute distance, the analysis unit determines to correct the reference coordinates of each preset main feature; If the absolute distance is greater than the preset absolute distance, the analysis unit determines that the face is not facing the collection device and continues monitoring.

[0043] Specifically, in this embodiment, the preset central feature is a central feature extracted based on a pre-collected target facial image input.

[0044] Specifically, in this embodiment, the preset absolute distance is obtained by pre-measurement, and several qualified fused images are obtained. The absolute value of the difference between the vertical coordinates of the central feature in each fused image and the preset central feature is calculated respectively to obtain the absolute distance, and the mean of the absolute distance is solved to obtain the preset absolute distance.

[0045] In the present invention, the distance between the central feature of the acquired portrait feature and the vertical coordinate of the preset central feature is first calculated, and based on the distance, it is analyzed whether the driver is looking up or looking down. Taking into account the driving habits of the driver, the coordinates of the preset central feature are adjusted according to the analysis results, thereby improving the matching degree between the judgment benchmark and the actual situation and improving the analysis accuracy of the face image.

[0046] Specifically, the analysis unit corrects the reference coordinates of each preset main feature, including: Calculating the difference between the matching degree and the second preset matching degree to obtain a matching degree difference, The correction amplitude of the horizontal coordinate of the preset main feature is positively correlated with the matching degree difference.

[0047] In this embodiment, optionally, The matching difference is compared with the first preset matching difference and the second preset matching difference. If the matching degree difference is less than or equal to the first preset matching degree difference, the horizontal coordinate of the preset main feature is moved along the direction of the corresponding current main feature, and the moving distance is 0.4 times the initial distance, where the initial distance is the distance between the main feature before the movement and the preset main feature; If the matching degree difference is greater than the first preset matching degree difference and less than or equal to the second preset matching degree difference, the horizontal coordinate of the preset main feature is moved along the direction of the corresponding current main feature, and the moving distance is 0.6 times the initial distance; If the matching degree difference is greater than the second preset matching degree difference, the horizontal coordinate of the preset main feature is moved along the direction of the corresponding current main feature, and the moving distance is 0.7 times the initial distance; The first preset matching degree difference is 0.15 to 0.2 times the second preset matching degree difference, and the second preset matching degree difference is 0.22 to 0.25 times the second preset matching degree difference.

[0048] Specifically, the analysis unit is further used to monitor the duration of time during which the absolute distance is less than or equal to the preset absolute distance. If the duration is greater than or equal to the preset duration, the analysis unit determines that there is a problem with the driver's driving habits.

[0049] Specifically, the analysis unit calculates the area of ​​the overlapping portion of the secondary feature and the preset secondary feature under the condition that the extracted portrait feature is determined to be unqualified, and the analysis unit determines the overlapping area ratio according to the ratio of the overlapping area to the total area of ​​the preset secondary feature, so as to analyze the reason why the extracted portrait feature is unqualified according to the overlapping area ratio, including: If the overlap area ratio is less than or equal to the preset overlap area ratio, the analysis unit determines the reason why the extracted portrait features based on the overlap situation of the secondary feature edges are unqualified; If the overlap area ratio is greater than the preset overlap area ratio, the analysis unit determines that the reason why the extracted portrait features are unqualified is that the driver's status does not meet the standards, and issues a driver status unqualified signal.

[0050] Specifically, in this embodiment, the preset overlap area ratio is selected between the interval [0.8, 0.85].

[0051] Specifically, the analysis unit is used to analyze the reasons why the extracted portrait features are unqualified based on the overlap of the secondary feature edges, including: Determine the length of the overlap between the secondary feature edge and the preset secondary feature edge. Calculate the ratio of the length of the overlapping part to the total length of the preset secondary feature edge to obtain the secondary feature overlap degree. If the secondary feature overlap is less than or equal to the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image fusion process is unqualified; If the secondary feature overlap is greater than the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image acquisition process is unqualified, and sends an image acquisition unqualified signal.

[0052] Specifically, in this embodiment, the preset secondary feature overlap is selected in the interval [0.85, 0.9].

[0053] Specifically, the analysis unit adjusts the sharpening intensity in the image fusion process based on the secondary feature overlap when determining that the image fusion process is unqualified, wherein the increase in the sharpening intensity is negatively correlated with the secondary feature overlap.

[0054] In this embodiment, optionally, The secondary feature coincidence degree is compared with the first preset secondary coincidence degree and the second preset secondary coincidence degree, If the secondary feature overlap is less than or equal to the first preset secondary overlap, the sharpening intensity is increased to 1.05 times the initial sharpening intensity; If the secondary feature overlap is greater than the first preset secondary overlap and less than or equal to the second preset secondary overlap, increasing the sharpening intensity to 1.1 times the initial sharpening intensity; If the secondary feature overlap is greater than the second preset secondary overlap, increasing the sharpening intensity to 1.15 times the initial sharpening intensity; Among them, the first preset secondary overlap is 0.75 times the preset secondary feature overlap, and the second preset secondary overlap is 0.85 times the preset secondary feature overlap.

[0055] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An intelligent vehicle safety assistance system based on image fusion, characterized in that: include: An image acquisition unit, comprising a first image acquisition module for acquiring visible light images and a second image acquisition module for acquiring infrared light images; an image fusion unit, connected to the image acquisition unit, for fusing the visible light image with the infrared light image to obtain a fused image; A feature extraction unit, connected to the image fusion unit, for extracting portrait features from the fused image, wherein the portrait features include primary features and secondary features; wherein the primary features are eye features, the secondary features are face contour features, and the secondary features use nose features as central features; A coordinate construction unit, which is connected to the image fusion unit and the feature extraction unit respectively, and is used to construct a coordinate system based on the extracted portrait features and transfer the portrait features to the coordinate system; an analysis unit, which is respectively connected to the image acquisition unit, the image fusion unit, the feature extraction unit and the coordinate construction unit, and is used to compare the extracted portrait features with the preset portrait features to calculate the matching degree, analyze whether the extracted portrait features are qualified according to the matching degree, and when the extracted portrait features are initially determined to be unqualified, perform a secondary determination on whether the extracted portrait features are qualified based on the overlap between the main feature edge and the preset main feature edge, or analyze the reason why the extracted portrait features are unqualified; An output unit is connected to the analysis unit and is used to generate a corresponding signal instruction based on the determined reason why the extracted portrait features are unqualified.

2. The intelligent vehicle safety assistance system based on image fusion according to claim 1 is characterized in that: The analysis unit is used to analyze whether the extracted portrait features are qualified according to the matching degree, including: Compare the extracted portrait features with the preset portrait features, Determine the area of ​​the overlapping part of the portrait feature and the preset portrait feature to obtain the feature overlapping area, Calculate the ratio of the feature overlap area to the total area of ​​the preset portrait features to get the matching degree. If the matching degree is greater than or equal to the first preset matching degree, the analysis unit determines that the extracted portrait features are qualified; If the matching degree is less than the first preset matching degree and greater than or equal to the second preset matching degree, the analysis unit preliminarily determines that the extracted portrait feature is unqualified, and performs a secondary determination on whether the extracted portrait feature is qualified based on the overlap between the main feature edge and the preset main feature edge; If the matching degree is less than the second preset matching degree, the analyzing unit determines that the extracted portrait features are unqualified.

3. The intelligent vehicle safety assistance system based on image fusion according to claim 2 is characterized in that: The analysis unit is used to perform a secondary determination on whether the extracted portrait features are qualified based on the overlap between the main feature edge and the preset main feature edge, including: Determine the length of the overlap between the edge of the main feature and the preset edge of the main feature, Calculate the ratio of the length of the overlapping part to the total length of the preset main feature edge to get the overlap degree. If the overlap is less than or equal to a preset overlap, the analysis unit determines that the main feature position is offset; If the overlap degree is greater than the preset overlap degree, the analysis unit determines that the extracted portrait features are unqualified and analyzes the reasons for the unqualifiedness.

4. The intelligent vehicle safety assistance system based on image fusion according to claim 3 is characterized in that: The analysis unit is used to analyze whether to correct the determination criterion based on the relative position relationship between each main feature and the secondary feature under the condition that the main feature position is determined to be offset, including: Calculate the absolute value of the difference between each main feature and the central feature, If the absolute value is less than or equal to the preset absolute value, the analysis unit determines whether to adjust the reference coordinates of each preset main feature based on the ordinate of the central feature; If the absolute value is greater than the preset absolute value, the analysis unit determines that the face is not facing the collection device and continues monitoring.

5. The intelligent vehicle safety assistance system based on image fusion according to claim 4 is characterized in that: The analyzing unit is used to determine whether to adjust the reference coordinates of each preset main feature based on the vertical coordinate of the central feature, including: Determine the absolute distance between the center feature and the preset center feature based on the ordinate, If the absolute distance is less than or equal to the preset absolute distance, the analysis unit determines to correct the reference coordinates of each preset main feature; If the absolute distance is greater than the preset absolute distance, the analysis unit determines that the face is not facing the collection device and continues monitoring.

6. The intelligent vehicle safety assistance system based on image fusion according to claim 5 is characterized in that: The analysis unit corrects the reference coordinates of each preset main feature, including: Calculating the difference between the matching degree and the second preset matching degree to obtain a matching degree difference, The correction amplitude of the horizontal coordinate of the preset main feature is positively correlated with the matching degree difference.

7. The intelligent vehicle safety assistance system based on image fusion according to claim 5, characterized in that: The analysis unit is further configured to monitor the duration of time during which the absolute distance is less than or equal to a preset absolute distance. If the duration is greater than or equal to the preset duration, the analysis unit determines that there is a problem with the driver's driving habits.

8. The intelligent vehicle safety assistance system based on image fusion according to claim 2 is characterized in that: The analysis unit calculates the area of ​​the overlapping portion of the secondary feature and the preset secondary feature under the condition that the extracted portrait feature is determined to be unqualified, and determines the overlapping area ratio according to the ratio of the overlapping area to the total area of ​​the preset secondary feature, so as to analyze the reason why the extracted portrait feature is unqualified according to the overlapping area ratio, including: If the overlap area ratio is less than or equal to the preset overlap area ratio, the analysis unit determines the reason why the extracted portrait features based on the overlap situation of the secondary feature edges are unqualified; If the overlap area ratio is greater than the preset overlap area ratio, the analysis unit determines that the reason why the extracted portrait features are unqualified is that the driver's status does not meet the standards, and issues a driver status unqualified signal.

9. The intelligent vehicle safety assistance system based on image fusion according to claim 8, characterized in that: The analysis unit is used to analyze the reasons why the extracted portrait features are unqualified based on the overlap of the secondary feature edges, including: Determine the length of the overlap between the secondary feature edge and the preset secondary feature edge. Calculate the ratio of the length of the overlapping part to the total length of the preset secondary feature edge to obtain the secondary feature overlap degree. If the secondary feature overlap is less than or equal to the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image fusion process is unqualified; If the secondary feature overlap is greater than the preset secondary feature overlap, the analysis unit determines that the reason why the extracted portrait feature is unqualified is that the image acquisition process is unqualified, and sends an image acquisition unqualified signal.

10. The intelligent vehicle safety assistance system based on image fusion according to claim 9, characterized in that: The analysis unit adjusts the sharpening intensity in the image fusion process based on the secondary feature overlap when determining that the image fusion process is unqualified, wherein the increase in the sharpening intensity is negatively correlated with the secondary feature overlap.

Images