A method, system, device, and storage medium for vehicle status display and evaluation integrating OMS camera.

Abstract

This invention discloses a method, system, device, and storage medium for in-vehicle status display and evaluation integrating an OMS camera. The method includes the following steps: installing an in-vehicle display at the front of the vehicle's driver's cab, and connecting a camera to the upper center of the in-vehicle display; the camera acquiring image data of the occupants and driver; the in-vehicle display displaying the captured images, and the OMS system and DMS system acquiring the image data captured by the camera respectively; the DMS system generating first status information based on the image data, and the OMS system generating second status information based on the image data; the DMS system monitoring the driver's fatigue state based on the first status information and generating driving status information, and the OMS system monitoring the occupant's riding status based on the second status information and generating occupant riding status information; evaluating the status of the driver and occupants based on the driving status information and occupant riding status information, and displaying the status of the driver and occupants and the evaluation results on the in-vehicle display.

Description

Technical Field

[0001] This invention relates to the field of vehicle safety technology, and in particular to a method, system, device, and storage medium for vehicle status display and evaluation integrating an OMS camera. Background Technology

[0002] The in-vehicle passenger monitoring system (OMS) uses a camera located below the rearview mirror to monitor the status of passengers in real time. The information collected by the OMS camera is uploaded to the vehicle's central control system. After being processed by the central control system, it is displayed on the in-vehicle display so that the driver or other people in the vehicle are aware of the passengers' status.

[0003] The Driver Monitoring System (DMS) uses a camera located at the front dome light to capture the driver's facial features and head posture, which is then uploaded to the vehicle's central control system to determine whether the driver is fatigued or inattentive.

[0004] An in-vehicle display is primarily a terminal device for information display and interactive operation. Like a visual "window," it presents information from the vehicle's central control system in an intuitive format, using images, icons, and text, making it convenient for occupants to view and operate. Its main function is to present various types of information and receive operational commands; it does not possess image capture capabilities.

[0005] The in-vehicle passenger monitoring system (OMS) and the driver monitoring system (DMS) are two different monitoring systems, so different monitoring cameras need to be installed. In addition, in the existing technology, the OMS camera is located below the rearview mirror, and the camera angle is affected when the rearview mirror is adjusted. When the DMS camera is located at the position of the front dome light, it is easily affected by the infrared light of the camera reflected from the roof light. Summary of the Invention

[0006] In view of the shortcomings of the prior art, the present invention provides a vehicle status display method, system, device and storage medium integrating OMS camera, so as to solve the problem that the cameras of the vehicle OMS and DMS systems in the prior art are easily affected, thus affecting the system monitoring.

[0007] This invention first provides a method for displaying and evaluating vehicle status using an integrated OMS camera, comprising the following steps:

[0008] Install an in-vehicle display at the front of the vehicle's cab, and connect a camera to the upper center of the in-vehicle display;

[0009] The camera acquires image data of the vehicle occupants and driver;

[0010] The in-vehicle display visualizes the image data and displays the captured images; the OMS system and DMS system respectively acquire the image data captured by the camera.

[0011] The DMS system generates first status information based on image data, and the OMS system generates second status information based on image data.

[0012] The DMS system monitors the driver's fatigue state based on the first state information and generates driving state information, while the OMS system monitors the passenger's riding status based on the second state information and generates passenger riding status information.

[0013] The status of the driver and passengers is evaluated based on driving status information and passenger riding status information, and the evaluation results are displayed on the vehicle display screen.

[0014] Preferably, the in-vehicle display performs visualization processing on the image data and displays the captured images. The OMS system and DMS system respectively acquire the image data captured by the camera, including the following steps:

[0015] The camera captures images of the occupants and driver inside the vehicle;

[0016] The image sensor generates raw image signals based on images of the occupants' and driver's states;

[0017] Image signal processors optimize and process raw image signals to form image data;

[0018] The serializer converts image data into serial image data for output.

[0019] The OMS system and the DMS system acquire serial image data respectively;

[0020] The deserializer acquires serial image data;

[0021] The deserializer converts serial image data into parallel image data;

[0022] The MCU microcontroller analyzes and processes parallel image data to generate corresponding drive waveform signals;

[0023] The FOG display module adjusts the arrangement of liquid crystal molecules within the liquid crystal glass according to a set pattern based on the driving waveform signal, thereby displaying the image captured by the camera.

[0024] Preferably, the DMS system monitors the driver's fatigue state and generates driving status information based on the first state information, including the following steps:

[0025] The DMS system classifies and labels image data, and then retrieves the image data corresponding to the driver from the classified and labeled image data.

[0026] The DMS system analyzes image data and extracts driver head posture features and eye state features;

[0027] The DMS system uses fuzzy rules to perform fuzzy inference based on the extracted features to obtain the driver's fatigue state classification results, and generates the driver's driving state information based on the driver's fatigue state classification results.

[0028] Preferably, the OMS system monitors the passenger's riding status based on the second status information and generates passenger riding status information by including the following steps:

[0029] The OMS system classifies and labels image data, and then retrieves the image data corresponding to the occupants from the classified and labeled image data.

[0030] The OMS system analyzes image data to extract occupant body posture angle features and occupant head position deviation from the center features;

[0031] The OMS system uses fuzzy rules to perform fuzzy reasoning based on the extracted features to obtain the passenger riding status classification results, and generates passenger riding status information based on the passenger riding status classification results.

[0032] Preferably, the process of evaluating the driver's state based on driving state information, obtaining an evaluation result, and displaying the driver's state and evaluation result on the in-vehicle display includes the following steps:

[0033] Set the target reference value for the driver's driving status information, and set the normal driving status information curve based on the target reference value;

[0034] Set a fuzzy set for judging driving status information based on the normal driving status information curve;

[0035] Acquire the driver's driving status information during the driving process, and draw a driving status information curve based on the driving status information;

[0036] The driving status information curve and the normal driving status information curve are judged based on the fuzzy set of driving status information.

[0037] Obtain the driver's driving status assessment results;

[0038] The in-vehicle display shows the driver's driving status assessment results, changes the color of the display area according to the driver's driving status, and issues warning sounds.

[0039] Preferably, the process of evaluating the occupant's status based on occupant riding status information, obtaining an evaluation result, and displaying the occupant's status and evaluation result on the vehicle-mounted display includes the following steps:

[0040] Set the target reference value for the passenger's riding status information, and set the normal riding status information curve based on the target reference value;

[0041] Set a fuzzy set for judging ride status information based on the normal ride status information curve;

[0042] Acquire passenger status information during the ride and plot a ride status information curve based on the ride status information;

[0043] The ride status information curve and the normal ride status information curve are judged based on the fuzzy set of the ride status information.

[0044] Obtain the passenger's riding status assessment results;

[0045] The in-vehicle display shows the passenger's riding status assessment results, changes the color of the display area according to the passenger's riding status, and issues an alarm sound.

[0046] This invention also provides an in-vehicle status display system integrating an OMS camera, including...

[0047] The first processing unit is used to generate first state information and second state information based on image data;

[0048] The second processing unit is used to monitor the driver's fatigue state based on the first state information and generate driving state information, and to monitor the passenger's riding status based on the second state information and generate passenger riding status information.

[0049] The third processing unit is used to monitor the driver's fatigue state based on the first state information and generate driving state information, and to monitor the passenger's riding status based on the second state information and generate passenger riding status information.

[0050] The fourth processing unit is used to evaluate the status of the driver and passengers based on driving status information and passenger riding status information, and obtain evaluation results.

[0051] The present invention also provides an in-vehicle status display device integrating an OMS camera. The in-vehicle display device includes an in-vehicle display, on which a camera, an infrared emitting diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module, and an MCU microcontroller are connected. The camera is positioned at the top center of the in-vehicle display. The image sensor is connected to the image signal processor, which is connected to the serializer. The FOG display module is connected to the deserializer and the MCU microcontroller, and the deserializer is connected to the MCU microcontroller.

[0052] The present invention also provides a computer-readable storage medium for storing program code for executing the steps of the above-described method for displaying and evaluating vehicle status using an integrated OMS camera.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. Connect a camera to the in-vehicle display. The camera can monitor the status of the occupants and driver. The OMS system and DMS system use the same set of cameras to monitor the passengers and driver, eliminating the need to install additional cameras.

[0055] 2. The information captured by the camera can be directly transmitted to the in-vehicle display, which then displays the image captured by the camera directly. There is no need to upload it to the vehicle's central control system and then transmit it to the in-vehicle display for display. This simplifies the image data transmission link and reduces transmission latency.

[0056] 3. Connect a camera to the upper center of the vehicle display. The camera can monitor both the driver and passengers simultaneously. The camera's installation position avoids the OMS system camera being affected by the rearview mirror adjustment and the DMS system camera being affected by the infrared light reflected from the roof, thereby reducing the impact of the camera's operation on the monitoring effect of the DMS and OMS systems.

[0057] 4. The aforementioned vehicle status display device, serializer and deserializer, achieves accuracy and high speed in image data transmission, reduces wiring complexity, and enhances the long-distance transmission capability of image data. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0059] Figure 1 A schematic diagram of an in-vehicle display method integrating an OMS camera provided by the present invention;

[0060] Figure 2 This is a schematic diagram of the structure of an in-vehicle display device integrating an OMS camera, provided by the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0062] In the description of this invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0063] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. Where the terms "first," "second," and "third" are used for descriptive purposes and to distinguish technical features, they should not be construed as indicating or implying relative importance, or implicitly specifying the number of indicated technical features or the sequential relationship between indicated technical features.

[0064] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Furthermore, the technical features involved in the different embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0065] Example 1

[0066] Figure 1 This is a schematic diagram of an in-vehicle display method integrating an OMS camera provided by the present invention, see reference. Figure 1 A method for displaying and evaluating vehicle status using an integrated OMS camera, comprising the following steps:

[0067] An in-vehicle display is installed at the front of the vehicle's cab, and a camera is connected to the upper center of the in-vehicle display.

[0068] The camera captures image data of the occupants and driver inside the vehicle.

[0069] The in-vehicle display visualizes the image data and displays the captured images, while the OMS system and DMS system acquire the image data captured by the camera, respectively.

[0070] The DMS system generates first status information based on the image data, and the OMS system generates second status information based on the image data.

[0071] The DMS system monitors the driver's fatigue state based on the first state information and generates driving status information, while the OMS system monitors the passenger's riding status based on the second state information and generates passenger riding status information.

[0072] The status of the driver and passengers is evaluated based on driving status information and passenger riding status information, and the evaluation results are displayed on the vehicle display screen.

[0073] The camera captures image data, which is then transmitted to an in-vehicle display that shows the images in real time. The camera outputs the image data using a serializer, and the in-vehicle display receives the data using a deserializer, achieving high-speed data transmission and ensuring image resolution and frame rate.

[0074] The system includes an in-vehicle display installed at the front of the driver's cab, with a camera connected to the upper center of the display. This camera captures images of the occupants and driver. The OMS (Operating Management System) and DMS (Driving Management System) can then utilize the information captured by the camera, allowing both systems to monitor passengers and the driver separately using the same set of cameras, eliminating the need for separate camera modules for each system. This approach avoids the interference of rearview mirror adjustments on the OMS system's cameras and the reflection of infrared light from the roof awning on the DMS system's cameras.

[0075] The DMS (Driver Monitoring System) generates first status information based on image data, and monitors the driver's fatigue state based on this first status information to generate driving status information. The driving status information is then used to evaluate the driver's condition, thus achieving driver monitoring. The in-vehicle display, in addition to showing the driver's status, can also display the evaluation results. The OMS (Occupant Monitoring System) generates second status information based on image data, and monitors the occupant's riding status based on this second status information to generate occupant riding status information. The occupant's riding status is then used to evaluate the occupant's condition, thus achieving occupant monitoring. The in-vehicle display, in addition to showing the occupant's status, can also display the evaluation results.

[0076] This enables real-time monitoring of the driver and passengers inside the vehicle, improving vehicle safety during use.

[0077] Example 2

[0078] The in-vehicle display visualizes the image data and displays the captured images. The OMS system and DMS system acquire the image data captured by the camera by the following steps:

[0079] The camera captures images of the occupants and driver inside the vehicle.

[0080] The image sensor generates raw image signals based on images of the occupants' and driver's states.

[0081] Image signal processors optimize and process raw image signals to form image data.

[0082] The serializer converts image data into serial image data output.

[0083] The OMS system and DMS system acquire serial image data respectively.

[0084] The deserializer acquires serial image data.

[0085] The deserializer converts serial image data into parallel image data.

[0086] The MCU microcontroller analyzes and processes parallel image data to generate corresponding drive waveform signals.

[0087] The FOG display module adjusts the arrangement of liquid crystal molecules within the liquid crystal glass according to a set pattern based on the driving waveform signal, thereby displaying the image captured by the camera.

[0088] Example 3

[0089] The DMS system monitors the driver's fatigue state based on the first state information and generates driving status information, including the following steps:

[0090] The DMS system classifies and labels image data, then retrieves the driver's corresponding image data from the classified and labeled image data. The DMS system analyzes the image data and extracts the driver's head posture features and eye state features. Based on the extracted features, the DMS system uses fuzzy rules to perform fuzzy inference to obtain the driver's fatigue state classification result, and generates the driver's driving state information based on the driver's fatigue state classification result.

[0091] In fuzzy inference using fuzzy rules, the image feature variables are the driver's head posture and eye state features. The output driver fatigue state is classified as alert, mildly fatigued, and severely fatigued. Based on this fatigue state classification, the driver's driving status information is generated, such as severely fatigued driving, mildly fatigued driving, and alert driving.

[0092] Example 4

[0093] The OMS system monitors the passenger's riding status based on the second status information and generates passenger riding status information, including the following steps:

[0094] The OMS system classifies and labels image data, and then extracts the image data corresponding to the occupants from the classified and labeled image data. The OMS system analyzes the image data and extracts the occupants' body posture angle features and the degree of deviation of the occupants' head positions from the center. Based on the extracted features, the OMS system uses fuzzy rules to perform fuzzy inference to obtain the occupants' riding status classification results, and generates occupants' riding status information based on the occupants' riding status classification results.

[0095] In fuzzy inference using fuzzy rules, the image feature variables are the occupant's body posture angle and the degree of head deviation from the center. The output occupant riding status is classified as normal or abnormal. When formulating the rules, if the occupant's body posture angle is normal and the degree of head deviation from the center is slight or moderate, then the occupant status is classified as normal; if the occupant's body posture angle is severely deviated or the degree of head deviation from the center is severe, then the occupant status is classified as abnormal.

[0096] Example 5

[0097] The process of evaluating the driver's condition based on driving status information, obtaining an evaluation result, and displaying the driver's condition and evaluation result on the in-vehicle display includes the following steps:

[0098] Set the target reference value for the driver's driving status information, and set the normal driving status information curve based on the target reference value;

[0099] Set a fuzzy set for judging driving status information based on the normal driving status information curve;

[0100] Acquire the driver's driving status information during the driving process, and draw a driving status information curve based on the driving status information;

[0101] The driving status information curve and the normal driving status information curve are judged based on the fuzzy set of driving status information.

[0102] Obtain the driver's driving status assessment results;

[0103] The in-vehicle display shows the driver's driving status assessment results, changes the color of the display area according to the driver's driving status, and issues warning sounds.

[0104] Based on the driver's driving state information during the driving process, a target reference value for the driver's driving state information is set. This target reference value reflects the driver's normal driving state during driving. A normal driving state information curve is then created based on this target reference value, reflecting the driver's state when driving normally. The system acquires the driver's driving state information during the driving process; this driving state information is the driver's state while actively driving. Based on this driving state information, the system can draw a driving state information curve, which reflects the changes in the driver's state during driving.

[0105] The system uses a fuzzy set to judge the driving status information curve and the normal driving status information curve. By comparing the fuzzy intervals of the two curves within the same time period, it can determine whether the driver's status information deviates from the normal state during the current driving process. When making fuzzy judgments, the system calculates the instantaneous rate of change of both the driving status information curve and the normal driving status information curve; the sign of the instantaneous rate of change of the normal driving status information curve is used as the judgment condition for the fuzzy set. When the sign of the instantaneous rate of change of the driving status information curve differs from that of the normal driving status information curve within a certain time period (e.g., the instantaneous rate of change of the normal driving status information curve is positive, while the instantaneous rate of change of the driving status information curve is negative), the system judges the driver's driving state to be abnormal. The system sends the judgment result of the abnormal driving state to the in-vehicle display, which then issues a "Danger" voice warning and simultaneously flashes a red light at the edge of the display area.

[0106] Specifically, when judging the driving state information curve and the normal driving state information curve based on the fuzzy set of driving state information judgment, the deviation rate between the driving state information curve and the normal driving state information curve is calculated as the judgment condition of the fuzzy set. For example, when the deviation rate is less than 5%, it is judged that the driver is awake and in a normal driving state during the driving process. This indicates that the changes in the driver's driving state during the actual driving process are very close to the expected target, and the driver's state is stable and normal. When the deviation rate is greater than 5% but less than 15%, it is judged as abnormal driving, and the driver is mildly fatigued. At this time, the system closely monitors the driver and links with the in-vehicle fragrance system to release a low concentration of refreshing fragrance into the vehicle. When the deviation rate is greater than 15%, it is judged as fatigued driving, and links with the in-vehicle fragrance system to release a high concentration of refreshing fragrance into the vehicle to enhance the driver's sensory experience and relieve fatigue.

[0107] Wherein, deviation rate = deviation value of driving status information / target reference value of driving status information; the target reference value of driving status information is the value on the normal driving status information curve, and the deviation value of driving status information is the difference between the driving status information value obtained during driving and the target reference value of driving status information at the same moment.

[0108] Example 6

[0109] The process of evaluating the occupant's status based on their riding status information, obtaining an evaluation result, and displaying the occupant's status and evaluation result on the in-vehicle display includes the following steps:

[0110] Set the target reference value for the passenger's riding status information, and set the normal riding status information curve based on the target reference value;

[0111] Set a fuzzy set for judging ride status information based on the normal ride status information curve;

[0112] Acquire passenger status information during the ride and plot a ride status information curve based on the ride status information;

[0113] The ride status information curve and the normal ride status information curve are judged based on the fuzzy set of the ride status information.

[0114] Obtain the passenger's riding status assessment results;

[0115] The in-vehicle display shows the passenger's riding status assessment results, changes the color of the display area according to the passenger's riding status, and issues an alarm sound.

[0116] Based on the passenger's riding status information during the ride, a target reference value for the passenger's riding status information is set. This target reference value reflects the passenger's normal riding status during the ride. A normal riding status information curve is then created based on this target reference value, reflecting the state of the ride during normal riding. The system acquires the passenger's riding status information during the ride; this information represents the passenger's riding status while they are actually riding. Based on this riding status information, the system can draw a riding status information curve, which reflects the changes in the riding status during the ride.

[0117] The system uses a fuzzy set to judge the passenger status information curve and the normal passenger status information curve. By comparing the fuzzy intervals of the two curves within the same time period, it can determine whether the passenger's status information deviates from the normal state during the current ride. When making fuzzy judgments, the system calculates the instantaneous rate of change of both the passenger status information curve and the normal passenger status information curve; the sign of the instantaneous rate of change of the normal passenger status information curve is used as the judgment condition for the fuzzy set. When the sign of the instantaneous rate of change of the passenger status information curve differs from that of the normal passenger status information curve within a certain time period (e.g., the instantaneous rate of change of the normal passenger status information curve is positive, while the instantaneous rate of change of the passenger status information curve is negative), the passenger's driving status is judged to be abnormal. The system sends the judgment result of abnormal passenger status to the in-vehicle display, which then issues a voice message stating "Passenger abnormality," and simultaneously, a yellow light flashes at the edge of the display area.

[0118] Specifically, when judging the passenger status information curve and the normal passenger status information curve based on the fuzzy set of passenger status information, the deviation rate between the passenger status information curve and the normal passenger status information curve is calculated as the judgment condition of the fuzzy set. For example, when the deviation rate is less than 15%, it is determined that the passenger is in a normal state during the ride, which indicates that the actual passenger status change during the ride is very close to the expected target, and the passenger's state is normal; when the deviation rate is greater than 15%, it is determined that the passenger is abnormal, and at this time the vehicle display will issue a voice message "Passenger abnormality in the vehicle" based on the judgment result.

[0119] Wherein, deviation rate = deviation value of ride status information / target reference value of ride status information; the target reference value of ride status information is the value on the normal ride status information curve, and the deviation value of ride status information is the difference between the ride status information value obtained during the ride and the target reference value of ride status information at the same time.

[0120] Example 7

[0121] An in-vehicle status display system integrating OMS camera, including

[0122] The first processing unit is used to generate first state information and second state information based on image data;

[0123] The second processing unit is used to monitor the driver's fatigue state based on the first state information and generate driving state information, and to monitor the passenger's riding status based on the second state information and generate passenger riding status information.

[0124] The third processing unit is used to monitor the driver's fatigue state based on the first state information and generate driving state information, and to monitor the passenger's riding status based on the second state information and generate passenger riding status information.

[0125] The fourth processing unit is used to evaluate the status of the driver and passengers based on driving status information and passenger riding status information, and obtain evaluation results.

[0126] Example 8

[0127] Figure 2 This is a schematic diagram of the structure of an in-vehicle display device integrating an OMS camera provided by the present invention, see reference. Figure 2 An in-vehicle status display device integrating an OMS camera is disclosed. The in-vehicle display device includes an in-vehicle display, on which a camera, an infrared emitting diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module, and an MCU microcontroller are connected. The camera is positioned at the top center of the in-vehicle display. The image sensor is connected to the image signal processor, which is connected to the serializer. The FOG display module is connected to the deserializer and the MCU microcontroller, and the deserializer is connected to the MCU microcontroller.

[0128] Infrared emitting diodes (IR LEDs) provide supplemental lighting to the area captured by the camera (LENS), enabling the camera to capture clearer images and effectively improving the vehicle's monitoring capabilities when driving or parked at night. Furthermore, infrared light is invisible, making it difficult for occupants to detect the camera's infrared activity, thus achieving covert surveillance.

[0129] The image sensor converts optical images into electrical signals. Connected to a camera, it acquires the optical images captured by the camera. The image signal processor processes the raw image signal output from the image sensor, extracting and optimizing image information from the raw data, ultimately outputting image data that can be displayed, stored, or further processed.

[0130] The serializer connects to the image signal processor (ISP). It converts the image data output from the ISP into high-speed serial data for transmission, increasing the transmission rate and meeting the needs of real-time in-vehicle monitoring. The deserializer accurately restores the high-speed serial data into parallel image data, ensuring complete image presentation.

[0131] The serializer serializes the data, requiring only one or a few signal lines to complete the transmission, reducing the complexity of wiring. In-vehicle camera acquisition has strict space requirements, so the serializer simplifies the hardware circuit design and reduces the size of the device.

[0132] The in-vehicle display uses a FOG display module, which has the advantages of high integration and thinness. Its excellent display performance enables the in-vehicle display to provide a clearer and more stable image display effect, improving the clarity of the picture.

[0133] As a control unit, the MCU processes the data transmitted from the deserializer and outputs control signals to the FOG display module. It can also receive user input and adjust the display output accordingly.

[0134] Example 9

[0135] A computer-readable storage medium for storing program code for executing the steps of the above-described method for displaying and evaluating vehicle status using an integrated OMS camera.

[0136] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A vehicle-mounted state display and evaluation method integrated with OMS imaging, characterized in that, It comprises the following steps: A vehicle-mounted display is installed at the front end of the cab of a vehicle, and a camera is connected to the upper middle part of the vehicle-mounted display; The camera acquires image data of the passengers and the driver in the vehicle; The vehicle-mounted display visualizes and displays the captured image based on the image data, and the OMS system and the DMS system acquire the image data captured by the camera; The DMS system generates first state information based on the image data, and the OMS system generates second state information based on the image data; The DMS system monitors the fatigue state of the driver based on the first state information and generates driving state information, and the OMS system monitors the riding state of the passengers based on the second state information and generates passenger riding state information; The state of the driver and the passengers is evaluated based on the driving state information and the passenger riding state information, and an evaluation result is obtained, and the vehicle-mounted display displays the state of the driver and the passengers and the evaluation result; The vehicle-mounted display visualizes and displays the captured image based on the image data, and the OMS system and the DMS system acquire the image data captured by the camera, which comprises the following steps: the camera acquires image data of the passengers and the driver in the vehicle; the image sensor forms an original image signal based on the image data of the passengers and the driver; the image signal processor optimizes and processes the original image signal to form image data; the serializer converts the image data into serial image data and outputs; the OMS system and the DMS system acquire the serial image data; the deserializer acquires the serial image data; the deserializer converts the serial image data into parallel image data; the MCU single-chip microcomputer analyzes and processes the parallel image data to generate corresponding driving waveform signals; and the FOG display module adjusts the arrangement of liquid crystal molecules in the liquid crystal glass according to the driving waveform signals to display the captured image of the camera; The evaluation result is obtained by evaluating the state of the driver based on the driving state information, and the vehicle-mounted display displays the state of the driver and the evaluation result, which comprises the following steps: setting a target reference value of the driving state information of the driver, setting a normal driving state information curve based on the target reference value of the driving state information, setting a driving state information judgment fuzzy set based on the normal driving state information curve, acquiring the driving state information of the driver during driving, drawing a driving state information curve based on the driving state information, judging the driving state information curve and the normal driving state information curve based on the driving state information judgment fuzzy set, acquiring the driving state judgment result of the driver, and displaying the driving state judgment result of the driver on the vehicle-mounted display, changing the color of the display area according to the driving state of the driver, and issuing an alarm prompt sound. The evaluation of the state of the occupant according to the occupant riding state information, the evaluation result, the vehicle-mounted display displays the state of the occupant and the evaluation result include the following steps: setting the target reference value of the occupant riding state information, setting the normal riding state information curve according to the target reference value of the riding state information; setting the riding state information judgment fuzzy set according to the normal riding state information curve; obtaining the riding state information of the occupant during the riding process, drawing the riding state information curve according to the riding state information; judging the riding state information curve and the normal riding state information curve according to the riding state information judgment fuzzy set; obtaining the riding state judgment result of the occupant; the vehicle-mounted display displays the riding state judgment result of the occupant, changes the color of the display area according to the riding state of the occupant and sends an alarm prompt sound.

2. The integrated OMS camera-based in-vehicle status display and evaluation method according to claim 1, characterized in that, The DMS system monitors the fatigue state of the driver according to the first state information and generates driving state information, which includes the following steps: The DMS system classifies and labels the image data, and the DMS system obtains the image data corresponding to the driver from the classified and labeled image data; The DMS system analyzes the image data and extracts the head posture feature and eye state feature of the driver; The DMS system uses fuzzy rules for fuzzy reasoning according to the extracted features to obtain the fatigue state classification result of the driver, and generates the driving state information of the driver according to the fatigue state classification result of the driver.

3. The integrated OMS camera-based in-vehicle status display and evaluation method of claim 1, wherein, The OMS system monitors the riding state of the occupant according to the second state information and generates occupant riding state information, which includes the following steps: The OMS system classifies and labels the image data, and the OMS system obtains the image data corresponding to the occupant from the classified and labeled image data; The OMS system analyzes the image data and extracts the body posture angle feature and the head position deviation center degree feature of the occupant; The OMS system uses fuzzy rules for fuzzy reasoning according to the extracted features to obtain the riding state classification result of the occupant, and generates the occupant riding state information according to the riding state classification result of the occupant.

4. An integrated OMS camera-based vehicle status display system, characterized by, It includes The first processing unit is used for generating the first state information and the second state information according to the image data; The second processing unit is used for monitoring the fatigue state of the driver according to the first state information and generating driving state information, and monitoring the riding state of the occupant according to the second state information and generating occupant riding state information; The third processing unit is used for monitoring the fatigue state of the driver according to the first state information and generating driving state information, and monitoring the riding state of the occupant according to the second state information and generating occupant riding state information; The fourth processing unit is used for evaluating the state of the driver and the occupant according to the driving state information and the occupant riding state information, and obtaining the evaluation result; The vehicle-mounted display is installed at the front end of the vehicle cab, and the camera is connected above the middle part of the vehicle-mounted display; the camera acquires image data of the passengers and the driver in the vehicle; the vehicle-mounted display performs visual processing on the image data and displays the photographed image; the OMS system and the DMS system acquire the image data captured by the camera respectively; the DMS system generates first state information according to the image data, and the OMS system generates second state information according to the image data; The vehicle-mounted display performs visual processing on the image data and displays the photographed image, and the OMS system and the DMS system acquire the image data captured by the camera respectively, including the following steps: The camera acquires image data of the passengers and the driver in the vehicle; the image sensor forms original image signals according to the image data of the passengers and the driver; the image signal processor optimizes and processes the original image signals to form image data; the serializer converts the image data into serial image data and outputs; the OMS system and the DMS system acquire the serial image data respectively; the deserializer acquires the serial image data; the deserializer converts the serial image data into parallel image data; the MCU single-chip microcomputer analyzes and processes the parallel image data to generate corresponding driving waveform signals; and the FOG display module adjusts the arrangement of liquid crystal molecules in the liquid crystal glass according to the driving waveform signals to display the photographed image of the camera. The driving state information of the driver is acquired during driving, and a driving state information curve is drawn according to the driving state information; the driving state information curve and the normal driving state information curve are judged according to the driving state information judgment fuzzy set; the driving state judgment result of the driver is acquired; the vehicle-mounted display displays the driving state judgment result of the driver, changes the color of the display area according to the driving state of the driver, and emits an alarm prompt sound. The passenger state information is acquired during the ride, and a passenger state information curve is drawn according to the passenger state information; the passenger state information curve and the normal passenger state information curve are judged according to the passenger state information judgment fuzzy set; the passenger state judgment result is acquired; the vehicle-mounted display displays the passenger state judgment result, changes the color of the display area according to the passenger state, and emits an alarm prompt sound.

5. A vehicle-mounted state display device integrated with an OMS camera, characterized by comprising: The integrated OMS camera vehicle-mounted state display device includes a vehicle-mounted display, a camera, an infrared emission diode, an image sensor, an image signal processor, a serializer, a deserializer, a FOG display module and an MCU single-chip microcomputer are connected to the vehicle-mounted display, the camera is arranged at a position directly above the middle of the vehicle-mounted display, the image sensor is connected to the image signal processor, the image signal processor is connected to the serializer, the FOG display module is connected to the deserializer and the MCU single-chip microcomputer, and the deserializer is connected to the MCU single-chip microcomputer. The vehicle-mounted display performs visual processing on image data and displays a photographed image, the OMS system and the DMS system respectively acquire image data captured by the camera, including the following steps: the camera acquires images of the states of passengers and a driver in the vehicle; the image sensor forms original image signals according to the images of the states of the passengers and the driver; the image signal processor optimizes and processes the original image signals to form image data; the serializer converts the image data into serial image data and outputs the serial image data; the OMS system and the DMS system respectively acquire the serial image data; the deserializer acquires the serial image data; the deserializer converts the serial image data into parallel image data; the MCU single-chip microcomputer analyzes and processes the parallel image data to generate corresponding drive waveform signals; and the FOG display module adjusts liquid crystal molecules in liquid crystal glass to change the arrangement according to the drive waveform signals to display the photographed image of the camera. The state of the driver is evaluated according to the driving state information to obtain an evaluation result, and the vehicle-mounted display displays the state of the driver and the evaluation result, including the following steps: setting a target reference value of driving state information of the driver, setting a normal driving state information curve according to the target reference value of the driving state information, setting a driving state information judgment fuzzy set according to the normal driving state information curve, acquiring driving state information of the driver during driving, drawing a driving state information curve according to the driving state information, judging the driving state information curve and the normal driving state information curve according to the driving state information judgment fuzzy set, acquiring a driving state judgment result of the driver, and the vehicle-mounted display displays the driving state judgment result of the driver, changes the color of a display area according to the driving state of the driver and emits an alarm prompt sound. The method comprises the following steps: setting a target reference value of the passenger's riding state information, setting a normal riding state information curve according to the target reference value of the riding state information; setting a riding state information judgment fuzzy set according to the normal riding state information curve; obtaining the riding state information of the passenger in the riding process, and drawing a riding state information curve according to the riding state information; judging the riding state information curve and the normal riding state information curve according to the riding state information judgment fuzzy set; obtaining a riding state judgment result of the passenger; and displaying the riding state judgment result of the passenger on the vehicle-mounted display, changing the color of a display area according to the riding state of the passenger, and emitting an alarm prompt sound.

6. A computer-readable storage medium, characterized in that, The computer readable storage medium is used for storing program codes, and the program codes are used for executing the steps of the integrated OMS camera-based vehicle-mounted state display and evaluation method in any one of claims 1-3.

Images