Vehicle-mounted state display and evaluation method, system and device integrated with OMS camera shooting and storage medium

Abstract

The invention discloses a vehicle-mounted state display and evaluation method, system and device integrated with OMS camera shooting and a storage medium. The method comprises the following steps that a vehicle-mounted displayer is installed at the front end of a vehicle cab, and a camera is connected to the upper portion of the middle of the vehicle-mounted displayer; the camera obtains image data of passengers and a driver in the vehicle; the vehicle-mounted display displays a shot image, and the OMS system and the DMS system respectively obtain image data captured by the camera; the DMS system generates first state information according to the image data, and the OMS system generates second state information according to the image data; the DMS system monitors the fatigue state of the driver according to the first state information and generates driving state information, and the OMS system monitors the riding state of the passenger according to the second state information and generates riding state information of the passenger; the states of the driver and the passengers are evaluated according to the driving state information and the passenger taking state information, and the vehicle-mounted displayer displays the states of the driver and the passengers and evaluation results.

Description

Technical Field

[0001] The present invention relates to the field of vehicle safety technology, and in particular to a vehicle status display and evaluation method, system, device and storage medium integrated with an OMS camera. Background Art

[0002] The in-vehicle passenger monitoring system (OMS) relies on the camera below the rearview mirror to realize real-time video monitoring of the passenger status. The information collected by the OMS camera will be uploaded to the vehicle's central control system. After being processed by the central control system, it will be displayed through the on-board display to let the driver or other people in the car know the status of the passengers.

[0003] The driver monitoring system (DMS) relies on the camera at the front roof light to capture the driver's facial features and head posture and upload the image information to the vehicle's central control system to determine whether the driver is driving fatigued, inattentive, etc.

[0004] The vehicle display is mainly a terminal device for information display and interactive operation. It is like a visual "window" that can present information from the vehicle's central control system in the form of intuitive pictures, icons, text, etc., making it convenient for people in the vehicle to view and operate. Its function focuses on presenting various types of information and receiving operation instructions, and it does not have the camera capability of image acquisition.

[0005] The in-vehicle passenger monitoring system (OMS) and the driver monitoring system (DMS) are two different monitoring systems, so different monitoring cameras need to be set up. At the same time, in the prior art, the OMS camera is set below the interior rearview mirror. When the angle of the rearview mirror is adjusted, the angle of the camera is affected. When the DMS camera is located at the front ceiling light position of the vehicle, it is easily affected by the reflection of the camera's infrared light from the roof canopy. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a vehicle-mounted status display method, system, device and storage medium integrated with OMS camera to solve the problem in the prior art that the camera of the vehicle OMS and DMS system is easily affected and thus affects system monitoring.

[0007] The present invention first provides a vehicle-mounted status display and evaluation method integrated with OMS camera, comprising the following steps:

[0008] An on-board display is installed at the front end of the vehicle cab, and a camera is connected to the upper middle portion of the on-board display;

[0009] The camera obtains image data of the occupants and the driver in the vehicle;

[0010] The on-board display visualizes the image data and displays the captured image, and the OMS system and the DMS system respectively obtain the image data captured by the camera;

[0011] The DMS system generates first status information according to the image data, and the OMS system generates second status information according to the image data;

[0012] The DMS system monitors the driver's fatigue state according to the first state information and generates driving state information, and the OMS system monitors the passenger's riding state according to the second state information and generates the passenger's riding state information;

[0013] The states of the driver and the passengers are evaluated according to the driving state information and the passenger riding state information to obtain an evaluation result, and the vehicle-mounted display displays the states of the driver and the passengers and the evaluation result.

[0014] Preferably, the vehicle-mounted display performs visual processing on the image data and displays the captured image, and the OMS system and the DMS system respectively obtain the image data captured by the camera, including the following steps:

[0015] The camera obtains images of the vehicle occupants and the driver’s status;

[0016] The image sensor forms a raw image signal based on the image of the occupant and the driver's status;

[0017] The image signal processor optimizes and processes the original image signal to form image data;

[0018] The serializer converts the image data into serial image data output;

[0019] The OMS system and the DMS system acquire serial image data respectively;

[0020] The deserializer obtains serial image data;

[0021] The deserializer converts the serial image data into parallel image data;

[0022] The MCU analyzes and processes the parallel image data to generate corresponding driving waveform signals;

[0023] The FOG display module adjusts the arrangement of liquid crystal molecules in the liquid crystal glass according to the set pattern according to the driving waveform signal to display the image captured by the camera.

[0024] Preferably, the DMS system monitors the driver's fatigue state according to the first state information and generates driving state information including the following steps:

[0025] The DMS system classifies and marks the image data, and the DMS system obtains the image data corresponding to the driver from the classified and marked image data;

[0026] The DMS system analyzes image data and extracts the driver's head posture and eye status features;

[0027] The DMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the driver's fatigue status classification result, and generates the driver's driving status information based on the driver's fatigue status classification result.

[0028] Preferably, the OMS system monitors the passenger's riding status according to the second status information and generates the passenger's riding status information, including the following steps:

[0029] The OMS system classifies and marks the image data, and the OMS system obtains image data corresponding to the occupant from the classified and marked image data;

[0030] The OMS system analyzes image data to extract the occupant's body posture angle characteristics and the degree of deviation of the occupant's head position from the center;

[0031] The OMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the passenger's riding status classification results, and generates the passenger's riding status information based on the passenger's riding status classification results.

[0032] Preferably, the step of evaluating the driver's state according to the driving state information to obtain an evaluation result, and displaying the driver's state and the evaluation result on the vehicle-mounted display comprises the following steps:

[0033] Setting a target reference value of the driver's driving state information, and setting a normal driving state information curve graph according to the target reference value of the driving state information;

[0034] Setting a driving state information judgment fuzzy set according to a normal driving state information curve diagram;

[0035] Acquire the driving state information of the driver during driving, and draw a driving state information curve chart according to the driving state information;

[0036] The driving state information curve graph and the normal driving state information curve graph are judged according to the driving state information judgment fuzzy set;

[0037] Obtaining the driver's driving state judgment result;

[0038] The on-board display shows the driver's driving status judgment results, changes the color of the display area according to the driver's driving status and issues a warning sound.

[0039] Preferably, the step of evaluating the passenger's state according to the passenger's riding state information to obtain an evaluation result, and displaying the passenger's state and the evaluation result on the vehicle-mounted display comprises the following steps:

[0040] Setting a target reference value of the riding status information of the passengers, and setting a normal riding status information curve graph according to the target reference value of the riding status information;

[0041] According to the normal riding state information curve diagram, a riding state information judgment fuzzy set is set;

[0042] Acquire the riding status information of passengers during the riding process, and draw a riding status information curve chart according to the riding status information;

[0043] The riding state information curve graph and the normal riding state information curve graph are judged according to the riding state information judgment fuzzy set;

[0044] Obtaining the passenger's riding status judgment result;

[0045] The on-board display shows the result of judging the passenger's riding status, changes the color of the display area according to the passenger's riding status, and issues an alarm sound.

[0046] The present invention also provides a vehicle-mounted status display system integrated with OMS camera, comprising:

[0047] A first processing unit, configured to generate first state information and second state information according to the image data;

[0048] a second processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger;

[0049] a third processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger;

[0050] The fourth processing unit is used to evaluate the status of the driver and the passenger according to the driving status information and the passenger riding status information to obtain an evaluation result.

[0051] The present invention also provides a vehicle-mounted status display device with integrated OMS camera, the vehicle-mounted display device includes a vehicle-mounted display, the vehicle-mounted display is connected to a camera, an infrared emitting diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module and an MCU single-chip computer, the camera is arranged at a position directly above the middle of the vehicle-mounted display, the image sensor is connected to the image signal processor, the image signal processor is connected to the serializer, the FOG display module is connected to the deserializer and the MCU single-chip computer, and the deserializer is connected to the MCU single-chip computer.

[0052] The present invention also provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the steps of the above-mentioned vehicle-mounted status display and evaluation method integrated with OMS camera.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. Connect a camera to the vehicle display. The camera can monitor the status of the passengers and the driver in the vehicle. The OMS system and the DMS system use the images captured by the same set of cameras to monitor the passengers and the driver in the vehicle, without the need to install more cameras.

[0055] 2. The information captured by the camera can be directly transmitted to the on-board display. The on-board display directly displays the image captured by the camera without uploading it to the vehicle's central control system. The information is then transmitted from the vehicle's central control system to the on-board display for display. This simplifies the image data transmission link and reduces transmission delays.

[0056] 3. Connect a camera in the upper middle part of the vehicle display. The camera can monitor the driver and passengers at the same time. The camera is installed in a position that can prevent the OMS system camera from being affected by the adjustment of the rearview mirror, and prevent the DMS system camera from being affected by the reflection of the camera's infrared light from the roof, thereby reducing the impact of the camera stage on the monitoring effects of the DMS and OMS systems.

[0057] 4. The vehicle-mounted status display device, the serializer and the deserializer realize the accuracy and high speed of image data transmission, reduce the wiring complexity, and enhance the long-distance transmission capability of image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0059] Figure 1 A schematic diagram of an in-vehicle display method integrating OMS camera provided by the present invention;

[0060] Figure 2 A schematic structural diagram of a vehicle-mounted display device integrated with an OMS camera provided by the present invention. DETAILED DESCRIPTION

[0061] The technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0063] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself. If the terms "first", "second", "third" are described, they are only used for the purpose of description and distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0064] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0065] Example 1

[0066] Figure 1 A schematic diagram of an in-vehicle display method integrated with OMS camera provided by the present invention, see Figure 1 , a vehicle-mounted status display and evaluation method integrated with OMS camera, comprising the following steps:

[0067] An on-board display is installed at the front end of the vehicle cab, and a camera is connected to the upper middle portion of the on-board display.

[0068] The camera acquires image data of the vehicle occupants and the driver.

[0069] The on-board display visualizes the image data and displays the captured image, and the OMS system and DMS system respectively obtain the image data captured by the camera.

[0070] The DMS system generates first status information according to the image data, and the OMS system generates second status information according to the image data.

[0071] The DMS system monitors the driver's fatigue state according to the first state information and generates driving state information, and the OMS system monitors the passenger's riding state according to the second state information and generates the passenger's riding state information.

[0072] The states of the driver and the passengers are evaluated according to the driving state information and the passenger riding state information to obtain an evaluation result, and the vehicle-mounted display displays the states of the driver and the passengers and the evaluation result.

[0073] The image data captured by the camera can be transmitted to the on-board display, and the on-board display can display the image captured by the camera in real time. In addition, the image data captured by the camera is output by a serializer, and the on-board display receives the image data through a deserializer, achieving high-speed data transmission and ensuring image resolution and frame rate.

[0074] Among them, a vehicle display is installed at the front end of the vehicle cab, and a camera is connected to the upper middle part of the vehicle display. The camera there can capture the passengers and driver in the car. The OMS system and the DMS system can obtain the information captured by the camera, so that the OMS system and the DMS system can use the same set of camera images to monitor the passengers and drivers in the car respectively, without the need for each system to use a separate camera module. It can prevent the existing OMS system camera from being affected by the adjustment of the rearview mirror, and prevent the DMS system camera from being affected by the reflection of the camera infrared light from the roof.

[0075] The DMS system generates first status information based on the image data, monitors the driver's fatigue status based on the first status information and generates driving status information. The driver's status is evaluated based on the driving status information, thereby monitoring the driver. In addition to displaying the driver's status, the on-board display can also display the evaluation results. The OMS system generates second status information based on the image data, monitors the passenger's riding status based on the second status information and generates passenger riding status information, evaluates the passenger's status based on the passenger riding status information, thereby monitoring the passenger. In addition to displaying the passenger's status, the on-board display can also display the evaluation results.

[0076] The vehicle can monitor the driver and passengers in real time to improve the safety performance of the vehicle during use.

[0077] Example 2

[0078] The vehicle-mounted display performs visual processing on the image data and displays the captured image, and the OMS system and the DMS system respectively obtain the image data captured by the camera, including the following steps:

[0079] The camera captures images of the vehicle occupants and the driver's status.

[0080] The image sensor forms a raw image signal based on the image of the occupant and the driver's status.

[0081] The image signal processor optimizes and processes the original image signal to form image data.

[0082] The serializer converts the image data into serial image data output.

[0083] The OMS system and the DMS system acquire serial image data respectively.

[0084] The deserializer obtains the serial image data.

[0085] The deserializer converts the serial image data into parallel image data.

[0086] The MCU analyzes and processes the parallel image data to generate corresponding driving waveform signals.

[0087] The FOG display module adjusts the arrangement of liquid crystal molecules in the liquid crystal glass according to the set pattern according to the driving waveform signal to display the image captured by the camera.

[0088] Example 3

[0089] The DMS system monitors the driver's fatigue state according to the first state information and generates driving state information, including the following steps:

[0090] The DMS system classifies and labels the image data, and obtains the image data corresponding to the driver from the classified and labeled image data; the DMS system analyzes the image data and extracts the driver's head posture features and eye state features; the DMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the driver's fatigue status classification result, and generates the driver's driving status information based on the driver's fatigue status classification result.

[0091] When fuzzy rules are used for fuzzy reasoning, the image feature variables are the driver's head posture features and eye state features, and the output driver's fatigue state is classified into sober, mild fatigue and severe fatigue. The driver's driving state information generated according to the driver's fatigue state classification is, for example, severe fatigue driving, mild fatigue driving and sober driving.

[0092] Example 4

[0093] The OMS system monitors the passenger's riding status according to the second status information and generates passenger riding status information, including the following steps:

[0094] The OMS system classifies and labels the image data, and obtains the image data corresponding to the occupant from the classified and labeled image data; the OMS system analyzes the image data to extract the occupant's body posture angle characteristics and the occupant's head position deviation degree characteristics; the OMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted characteristics to obtain the occupant's riding status classification results, and generates the occupant's riding status information based on the occupant's riding status classification results.

[0095] Among them, when using fuzzy rules for fuzzy reasoning, the image feature variables are the occupant's body posture angle and the degree of deviation of the occupant's head position from the center, and the output occupant's riding status is classified into normal status and abnormal status. When formulating rules, if the occupant's body posture angle is normal and the degree of deviation of the occupant's head position from the center is slight or moderate, then the occupant's status is classified as normal; if the occupant's body posture angle is severely deviated or the degree of deviation of the occupant's head position from the center is severe, then the occupant's status is classified as abnormal.

[0096] Example 5

[0097] The step of evaluating the driver's state according to the driving state information to obtain an evaluation result, and displaying the driver's state and the evaluation result on the vehicle-mounted display comprises the following steps:

[0098] Setting a target reference value of the driver's driving state information, and setting a normal driving state information curve graph according to the target reference value of the driving state information;

[0099] Setting a driving state information judgment fuzzy set according to a normal driving state information curve diagram;

[0100] Acquire the driving state information of the driver during driving, and draw a driving state information curve chart according to the driving state information;

[0101] The driving state information curve graph and the normal driving state information curve graph are judged according to the driving state information judgment fuzzy set;

[0102] Obtaining the driver's driving state judgment result;

[0103] The on-board display shows the driver's driving status judgment results, changes the color of the display area according to the driver's driving status and issues a warning sound.

[0104] According to the driving state information of the driver during the driving process, the target reference value of the driving state information of the driver during the driving process is set. This driving state information target reference value reflects the normal driving state of the driver during the driving process. According to this driving state information target reference value, a normal driving state information curve chart is set, and the normal driving state information curve chart reflects the state of the driver during normal driving. The system obtains the driving state information of the driver during the driving process. This driving state information is the driving state information of the driver when driving. According to this driving state information, the system can draw a driving state information curve chart, and the driving state information curve chart reflects the state change of the driver during the driving process.

[0105] The system judges the driving state information curve and the normal driving state information curve according to the driving state information judgment fuzzy set. By comparing the fuzzy intervals of the two curves in the same time period, it can be determined whether the state information of the driver in the current driving process deviates from the normal state. When the system makes fuzzy judgments, it calculates the instantaneous change rate of the driving state information curve and the normal driving state information curve respectively; the positive and negative values ​​of the instantaneous change rate of the normal driving state information curve are used as the judgment conditions of the fuzzy set. When the positive and negative values ​​of the instantaneous change rate of the driving state information curve are different from the positive and negative values ​​of the instantaneous change rate of the normal driving state information curve in a certain time period, such as the instantaneous change rate of the normal driving state information curve is positive and the instantaneous change rate of the driving state information curve is negative, it is judged that the driving state of the driver in the driving process is abnormal. The system sends the judgment result that the driver's driving state is abnormal to the on-board display, and the on-board display issues a voice of "Riding is dangerous" according to the judgment result, and the edge red light flashes in the display area of ​​the on-board display.

[0106] Among them, when the driving state information curve and the normal driving state information curve are judged according to the driving state information judgment fuzzy set, the deviation rate of the driving state information curve and the normal driving state information curve is calculated as the judgment condition of the fuzzy set. For example, when the deviation rate is less than 5%, it is judged that the driver is awake and in a normal driving state during driving, which indicates that the driving state change of the driver in the actual driving process is very close to the expected target, and the driver's state is stable and normal; when the deviation rate is greater than 5% and less than 15%, it is judged as abnormal driving and the driver is slightly fatigued. At this time, the system closely monitors the driver and links with the in-car fragrance system to release low-concentration refreshing fragrance into the car; when the deviation rate is greater than 15%, it is judged as fatigue driving, and links with the in-car fragrance system to release high-concentration refreshing fragrance into the car to enhance the driver's sensory experience and relieve fatigue.

[0107] The deviation rate = deviation value of driving state information / target reference value of driving state information; the target reference value of driving state information is the value on the normal driving state information curve, and the deviation value of driving state information is the difference between the driving state information value obtained during driving and the target reference value of driving state information at the same time.

[0108] Example 6

[0109] The process of evaluating the state of the occupant according to the occupant riding state information to obtain an evaluation result, and displaying the state of the occupant and the evaluation result on the vehicle-mounted display comprises the following steps:

[0110] Setting a target reference value of the riding status information of the passengers, and setting a normal riding status information curve graph according to the target reference value of the riding status information;

[0111] According to the normal riding state information curve diagram, a riding state information judgment fuzzy set is set;

[0112] Acquire the riding status information of passengers during the riding process, and draw a riding status information curve chart according to the riding status information;

[0113] The riding state information curve graph and the normal riding state information curve graph are judged according to the riding state information judgment fuzzy set;

[0114] Obtaining the passenger's riding status judgment result;

[0115] The on-board display shows the result of judging the passenger's riding status, changes the color of the display area according to the passenger's riding status, and issues an alarm sound.

[0116] According to the riding status information of the passengers during the riding process, the target reference value of the riding status information during the riding process is set. This riding status information target reference value reflects the normal riding status of the passengers during the riding process. According to this riding status information target reference value, a normal riding status information curve chart is set, and the normal riding status information curve chart reflects the state of the ride when it is in a normal riding state. The system obtains the riding status information of the passengers during the riding process. This riding status information is the riding status information of the passengers when they are riding. According to this riding status information, the system can draw a riding status information curve chart, and the riding status information curve chart reflects the state changes during the riding process.

[0117] The system judges the riding state information curve and the normal riding state information curve according to the riding state information judgment fuzzy set. By comparing the fuzzy intervals of the two curves in the same time period, it can be determined whether the state information of the passengers in the current riding process deviates from the normal state. When the system makes fuzzy judgments, it calculates the instantaneous change rate of the riding state information curve and the normal riding state information curve respectively; the positive and negative values ​​of the instantaneous change rate of the normal riding state information curve are used as the judgment conditions of the fuzzy set. When the positive and negative values ​​of the instantaneous change rate of the riding state information curve are different from the positive and negative values ​​of the instantaneous change rate of the normal riding state information curve in a certain time period, such as the instantaneous change rate of the normal riding state information curve is positive and the instantaneous change rate of the riding state information curve is negative, it is judged that the driving state of the passengers in the riding process is abnormal. The system sends the judgment result that the riding state of the passengers is abnormal to the on-board display, and the on-board display issues a voice of "abnormal passengers in the car" according to the judgment result, and the edge yellow light flashes in the display area of ​​the on-board display.

[0118] Among them, when judging the riding state information curve and the normal riding state information curve according to the riding state information judgment fuzzy set, the deviation rate of the riding state information curve and the normal riding state information curve is calculated as the judgment condition of the fuzzy set. For example, when the deviation rate is less than 15%, it is judged that the passenger is in a normal state during the riding process, which indicates that the change of the passenger's riding state during the actual riding process is very close to the expected target, and the passenger's state is normal; when the deviation rate is greater than 15%, it is judged that the passenger is abnormal, and at this time the on-board display issues a voice of "abnormal passenger in the car" according to the judgment result.

[0119] The deviation rate = deviation value of riding status information / target reference value of riding status information; the target reference value of riding status information is the value on the normal riding status information curve, and the deviation value of riding status information is the difference between the riding status information value obtained during the riding process and the target reference value of riding status information at the same time.

[0120] Example 7

[0121] An in-vehicle status display system integrated with an OMS camera, comprising:

[0122] A first processing unit, configured to generate first state information and second state information according to the image data;

[0123] a second processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger;

[0124] a third processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger;

[0125] The fourth processing unit is used to evaluate the status of the driver and the passenger according to the driving status information and the passenger riding status information to obtain an evaluation result.

[0126] Example 8

[0127] Figure 2 This is a structural diagram of a vehicle-mounted display device integrated with an OMS camera provided by the present invention, see Figure 2 , a vehicle-mounted status display device with integrated OMS camera, the vehicle-mounted display device includes a vehicle-mounted display, the vehicle-mounted display is connected with a camera, an infrared emitting diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module and an MCU single-chip computer, the camera is arranged at a position directly above the middle of the vehicle-mounted display, the image sensor is connected to the image signal processor, the image signal processor is connected to the serializer, the FOG display module is connected to the deserializer and the MCU single-chip computer, and the deserializer is connected to the MCU single-chip computer.

[0128] Among them, infrared emitting diodes (IR LEDs) can fill in the light of the camera (LENS) shooting area. With the help of infrared rays, the camera can capture clearer images, effectively improving the monitoring capabilities of vehicles when driving or parking at night. Infrared rays are invisible light. Using infrared emitting diodes can make it difficult for people in the vehicle to notice that the camera is shooting and monitoring through infrared rays, thus achieving covert monitoring.

[0129] Among them, the image sensor can convert the optical image into an electrical signal. The image sensor is connected to the camera to obtain the optical image captured by the camera. The image signal processor processes the raw image signal output by the image sensor, is responsible for extracting and optimizing the image information from the raw data output by the sensor, and finally outputs image data for display, storage or further processing.

[0130] The serializer is connected to the image signal processor. The serializer can convert the image data output by the image signal processor into high-speed serial data for transmission, thereby increasing the transmission rate and meeting the image transmission scenario for real-time monitoring inside the vehicle. The deserializer accurately restores the high-speed serial data into parallel image data to ensure that the image can be fully presented.

[0131] Among them, after the serializer serializes the data, only one or a few signal lines are needed to complete the transmission, reducing the complexity of wiring. In-vehicle camera acquisition has strict space requirements, so the serializer simplifies the hardware circuit design and reduces the size of the device.

[0132] The vehicle-mounted display adopts a FOG display module, which has the advantages of high integration and lightness and thinness. Its excellent display performance enables the vehicle-mounted display to provide a clearer and more stable image display effect and improve the clarity of the picture.

[0133] As a control unit, the MCU processes the data transmitted from the deserializer and outputs control signals to the FOG display module. It can also receive user input and adjust the display output accordingly.

[0134] Example 9

[0135] A computer-readable storage medium is used to store program codes, and the program codes are used to execute the steps of the above-mentioned vehicle-mounted status display and evaluation method integrated with OMS camera.

[0136] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A vehicle status display and evaluation method integrated with OMS camera, characterized in that: The steps include: An on-board display is installed at the front end of the vehicle cab, and a camera is connected to the upper middle portion of the on-board display; The camera obtains image data of the occupants and the driver in the vehicle; The on-board display visualizes the image data and displays the captured image, and the OMS system and the DMS system respectively obtain the image data captured by the camera; The DMS system generates first status information according to the image data, and the OMS system generates second status information according to the image data; The DMS system monitors the driver's fatigue state according to the first state information and generates driving state information, and the OMS system monitors the passenger's riding state according to the second state information and generates the passenger's riding state information; The states of the driver and the passengers are evaluated according to the driving state information and the passenger riding state information to obtain an evaluation result, and the vehicle-mounted display displays the states of the driver and the passengers and the evaluation result.

2. The vehicle-mounted status display and evaluation method integrated with OMS camera according to claim 1 is characterized in that: The vehicle-mounted display performs visual processing on the image data and displays the captured image, and the OMS system and the DMS system respectively obtain the image data captured by the camera, including the following steps: The camera obtains images of the vehicle occupants and the driver’s status; The image sensor forms a raw image signal based on the image of the occupant and the driver's status; The image signal processor optimizes and processes the original image signal to form image data; The serializer converts the image data into serial image data output; The OMS system and the DMS system acquire serial image data respectively; The deserializer obtains serial image data; The deserializer converts the serial image data into parallel image data; The MCU analyzes and processes the parallel image data to generate corresponding driving waveform signals; The FOG display module adjusts the arrangement of liquid crystal molecules in the liquid crystal glass according to the set pattern according to the driving waveform signal to display the image captured by the camera.

3. The vehicle-mounted status display and evaluation method integrated with OMS camera according to claim 1 is characterized in that: The DMS system monitors the driver's fatigue state according to the first state information and generates driving state information, including the following steps: The DMS system classifies and marks the image data, and the DMS system obtains the image data corresponding to the driver from the classified and marked image data; The DMS system analyzes image data and extracts the driver's head posture and eye status features; The DMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the driver's fatigue status classification result, and generates the driver's driving status information based on the driver's fatigue status classification result.

4. The vehicle-mounted status display and evaluation method integrated with OMS camera according to claim 1 is characterized in that: The OMS system monitors the passenger's riding status according to the second status information and generates passenger riding status information, including the following steps: The OMS system classifies and marks the image data, and the OMS system obtains image data corresponding to the occupant from the classified and marked image data; The OMS system analyzes image data to extract the occupant's body posture angle characteristics and the degree of deviation of the occupant's head position from the center; The OMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the passenger's riding status classification results, and generates the passenger's riding status information based on the passenger's riding status classification results.

5. The vehicle-mounted status display and evaluation method integrated with OMS camera according to claim 1 is characterized in that: The step of evaluating the driver's state according to the driving state information to obtain an evaluation result, and displaying the driver's state and the evaluation result on the vehicle-mounted display comprises the following steps: Setting a target reference value of the driver's driving state information, and setting a normal driving state information curve graph according to the target reference value of the driving state information; Setting a driving state information judgment fuzzy set according to a normal driving state information curve diagram; Acquire the driving state information of the driver during driving, and draw a driving state information curve chart according to the driving state information; The driving state information curve graph and the normal driving state information curve graph are judged according to the driving state information judgment fuzzy set; Obtaining the driver's driving state judgment result; The on-board display shows the driver's driving status judgment results, changes the color of the display area according to the driver's driving status and issues a warning sound.

6. The vehicle-mounted status display and evaluation method integrated with OMS camera according to claim 1 is characterized in that: The process of evaluating the state of the occupant according to the occupant riding state information to obtain an evaluation result, and displaying the state of the occupant and the evaluation result on the vehicle-mounted display comprises the following steps: Setting a target reference value of the riding status information of the passengers, and setting a normal riding status information curve graph according to the target reference value of the riding status information; According to the normal riding state information curve diagram, a riding state information judgment fuzzy set is set; Acquire the riding status information of passengers during the riding process, and draw a riding status information curve chart according to the riding status information; The riding state information curve graph and the normal riding state information curve graph are judged according to the riding state information judgment fuzzy set; Obtaining the passenger's riding status judgment result; The on-board display shows the result of judging the passenger's riding status, changes the color of the display area according to the passenger's riding status, and issues an alarm sound.

7. A vehicle status display system integrated with OMS camera, characterized in that: include A first processing unit, configured to generate first state information and second state information according to the image data; a second processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger; a third processing unit, configured to monitor the fatigue state of the driver according to the first state information and generate driving state information, and to monitor the riding state of the passenger according to the second state information and generate the riding state information of the passenger; The fourth processing unit is used to evaluate the status of the driver and the passenger according to the driving status information and the passenger riding status information to obtain an evaluation result.

8. A vehicle-mounted status display device integrated with OMS camera, characterized in that: The vehicle-mounted display device includes a vehicle-mounted display, to which a camera, an infrared emitting diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module and an MCU single-chip computer are connected. The camera is arranged at a position directly above the middle of the vehicle-mounted display, the image sensor is connected to the image signal processor, the image signal processor is connected to the serializer, the FOG display module is connected to the deserializer and the MCU single-chip computer, and the deserializer is connected to the MCU single-chip computer.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the steps of the vehicle-mounted status display and evaluation method of integrated OMS camera as described in claims 1-6.

Images