Method for carrying out safety detection and evaluation on cabin environment

Abstract

The invention discloses a method for carrying out safety detection and evaluation on a cabin environment, and relates to the technical field of automotive electronics and safety assistance, and the method comprises the steps that an object detection module multiplexes a cabin monitoring camera and is combined with an air bag sensor group to collect images and physical pressure quality data; the vehicle cloud processing module performs difference calculation with the reference image to extract a non-original added object area, and identifies object types and material attributes; receiving a detection rule set constructed based on historical accident data, and calculating an object risk coefficient in combination with real-time data fed back by a physical sensor; and calculating a cabin environment safety score according to the risk coefficient, and outputting a detection result through a display module. According to the invention, through multi-modal fusion of visual perception and physical sensing, comprehensive evaluation of the material hardness and the physical load of the added article is realized, the accuracy of cabin safety detection is improved, and dynamic risk identification and visual rectification guidance can be carried out on newly appearing in-vehicle ornaments.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automotive electronics and safety assistance, and in particular to a method for safety detection and evaluation of a cabin environment. BACKGROUND

[0002] With the gradual evolution of the automotive cabin into the user's third living space, it has become increasingly common for drivers and passengers to place fragrances, ornaments, electronic device holders, and various decorative items in the vehicle.

[0003] However, if these non-factory-installed items are placed inappropriately, they can easily become high-speed projectiles in the event of a collision, causing secondary injuries, or they can cover critical locations such as the steering wheel and instrument panel, hindering the normal deployment of airbags, thereby seriously threatening the personal safety of the occupants.

[0004] Current cabin monitoring technologies mainly focus on monitoring personnel, such as driver fatigue monitoring systems (DMS) and passenger monitoring systems (OMS), which are centered on identifying facial orientation, eye state, and body movements. Although some visual algorithms have begun to attempt to detect items left in the vehicle, existing single visual detection solutions have obvious limitations.

[0005] On the one hand, simple image analysis cannot obtain physical property information of the object, and cannot effectively distinguish the material hardness and actual mass of the object. For example, a visual sensor cannot distinguish a light plush toy from a hard decorative item that is similar in appearance but internally filled with heavy objects, nor can it perceive whether an object covering the airbag cover exerts sufficient physical pressure to affect airbag deployment. If a dedicated array of gravity or pressure sensors is laid out comprehensively in order to obtain these physical data, it will increase the hardware cost and manufacturing complexity of the vehicle.

[0006] On the other hand, the detection logic of existing vehicle systems is usually based on a static database and fixed algorithms pre-installed at the time of vehicle shipment. In the face of a wide variety of new automotive products on the market, the recognition model on the local end often cannot accurately classify due to the lack of feature data. At the same time, existing risk assessment standards are mostly based on theoretical assumptions and lack of correlation analysis with real traffic accident injury data, making it difficult for the system to objectively and quantitatively evaluate the actual harm risk of a specific object in a specific location. This problem of data update lag and single evaluation dimension makes it difficult for existing cabin safety detection technologies to meet the growing demand for in-vehicle environment safety. SUMMARY

[0007] The purpose of the present application is to provide a method for safety detection and evaluation of a cabin environment, which at least solves the technical problem of being unable to accurately identify non-factory-installed items in the cabin and evaluate their potential safety risks in the prior art.

[0008] The present application provides the following solutions:

[0009] According to one aspect of the present application, a method for safety detection and evaluation of a cabin environment is provided, which is run in an electronic control unit or a vehicle-mounted computing platform of a vehicle, the vehicle comprising an object detection module, a vehicle cloud processing module and a display module; the method comprising:

[0010] Step S1, initial state calibration and real-time data acquisition: the object detection module records the initial arrangement sequence of each region of the cabin and generates a reference image, and when performing a detection function, controls a camera to take a picture again to generate a to-be-detected image, and reads real-time pressure values and mass values of a physical sensor group;

[0011] Step S2, differential image processing and region extraction: the vehicle cloud processing module performs registration and difference calculation on the reference image and the to-be-detected image, obtains a change set of an object detection frame, and segments an image region of a non-factory-installed object;

[0012] Step S3, multi-label object recognition and attribute analysis: the vehicle cloud processing module extracts features of the segmented image region, recognizes the type of the object in the vehicle and the material attribute of the object;

[0013] Step S4, risk evaluation based on a rule set: the vehicle cloud processing module receives a detection rule set, matches the position of the object, the type of the object and the material attribute with the detection rule set, and combines the real-time pressure values and mass values of the physical sensor group to calculate a risk coefficient of the object;

[0014] Step S5, comprehensive scoring and result feedback: the vehicle cloud processing module calculates an overall environmental safety score of the cabin region according to the risk coefficient, and outputs a detection result and a rectification suggestion by the display module.

[0015] Further, in step S1, the object detection module reuses the driver monitoring system DMS camera and the passenger monitoring system OMS camera of the vehicle to take pictures; meanwhile, data is read through a connected physical sensor group, the physical sensor group comprising a steering wheel airbag sensor, a seat airbag sensor and a side window airbag sensor; the object detection module reads static mass values and pressure values of the regions where the above sensors are located, and monitors the physical load state on the airbag ejection path.

[0016] Further, in step S2, the specific process of differential image processing and region extraction comprises:

[0017] The feature point matching algorithm is used to perform geometric correction on the to-be-detected image and the reference image; the absolute difference value of each pixel point of the current image and the reference image is calculated to generate a difference gray image; a gray threshold is set to perform a binary operation on the difference gray image to generate a binary mask image, and morphological opening and closing operations are performed on the binary mask image; an independent connected region in the image is searched by using a connected domain analysis algorithm, a minimum circumscribed rectangle is calculated, and a to-be-classified image slice set is cropped out according to the minimum circumscribed rectangle.

[0018] Further, in step S3, the vehicle cloud processing module constructs and loads a multi-task convolutional neural network model, which includes a shared feature extraction backbone network and a category classification branch and a material attribute branch; the category classification branch is used to identify the category of the object in the vehicle, and the category includes a steering wheel cover, a pendant, an ornament, a mobile phone support, a foot mat, an interior door handle decoration, and a modified lamp strip;

[0019] The material attribute branch is used to detect the material hardness coefficient of the object, and a preset material hardness mapping table is used to distinguish hard objects and soft objects. For the mobile phone support category, the vehicle cloud processing module detects whether there is a reflection feature or a black screen rectangular feature specific to the mobile phone screen in the image, and outputs a load state flag bit; when the mobile phone is detected to be in place, the quality estimation value and the hardness coefficient of the object are automatically adjusted.

[0020] Further, in step S4, the detection rule set is generated by the cloud server based on historical accident data of the same type of vehicle and by using a machine learning algorithm; the construction process of the detection rule set includes: mining the accident data and associating the features, extracting feature vectors such as object category, relative position coordinates, and injury severity level; a mapping model between object features and risk levels is established, a basic risk weight is calculated for a specific category and position combination, and a rule library containing trigger conditions and risk attributes is generated according to a sensitive area set in the cabin.

[0021] Further, in step S4, the specific process of calculating the risk coefficient of the object includes: determining the spatial relationship between the object position and the dangerous area defined in the detection rule set to determine the position danger coefficient; combining the rule weight issued by the cloud and the physical attribute detected locally to calculate a basic risk value, which is obtained by multiplying the position danger coefficient after weighted summation of the material hardness coefficient, the normalized mass factor, and the historical accident weight;

[0022] The base risk value is corrected by using the data of the physical sensor group monitored in step S1: when the position of the article corresponds to the coverage range of a sensor in the physical sensor group, the pressure change amount of the sensor is detected; if the pressure change amount exceeds the maximum safe additional pressure threshold allowed in the area, a sensor correction coefficient greater than 1 is generated according to a penalty factor; and the base risk value is multiplied by the sensor correction coefficient to obtain the comprehensive risk score of the article.

[0023] Further, in step S5, the specific calculation process of the comprehensive score is: aggregating the comprehensive risk scores of all detected articles in the cabin; setting an initial full score of the system, calculating the difference between the initial full score and the sum of the comprehensive risk scores of all detected non-original articles, and taking the maximum value between the difference and 0 to obtain the whole vehicle environment safety score; and dividing the current cabin environment into three levels of safe state, warning state or dangerous state according to the whole vehicle environment safety score.

[0024] The result feedback process includes: the display module generates an augmented reality frame at the position corresponding to the article on the center control screen using coordinate mapping technology, and renders it using different colors or flashing frequencies according to the risk level of the article; when the detected risk is caused by the material properties or structural design defects of the article itself, the vehicle cloud processing module retrieves the corresponding inventory unit in the original equipment database, and if there is an original equipment that has passed the safety test, the whole vehicle safety monitoring application is controlled to display the comparative advantage information and recommendation entry of the original equipment.

[0025] Through the above scheme, the following beneficial technical effects are obtained:

[0026] The present application realizes the monitoring of the cabin environment by multiplexing the original DMS and OMS cameras of the vehicle and the physical sensor group of the airbag system without increasing the cost of additional hardware. Compared with a single visual detection scheme, this method introduces reading and analysis of the physical sensor pressure and mass value, which can effectively identify hidden dangers that are difficult to judge by visual algorithms, such as small in appearance but heavy in mass, or heavy objects hidden above the airbag cover, thereby ensuring accurate monitoring of the physical load state of critical areas such as the airbag ejection path.

[0027] The present application constructs and dynamically updates the detection rule set based on historical accident data by using the collaborative mechanism of the vehicle cloud processing module and the cloud server. This mechanism enables the vehicle-mounted system to obtain the features and risk data of new automotive products in a timely manner, overcoming the limitation that local fixed algorithms cannot identify new accessories; by converting macro accident statistics data into micro execution rules, it ensures that the risk assessment logic can be continuously optimized as real traffic safety data accumulates, improving the objectivity and timeliness of the detection results.

[0028] The application adopts a risk assessment strategy of fusing visual features and physical attributes, analyzes the material hardness while identifying the object category, and corrects the mass estimation value combined with sensor feedback. This strategy solves the technical problems of distinguishing the hardness of object material and estimating the actual mass of object by relying on image recognition alone, avoids misreporting soft low-risk objects as dangerous sources, ensures accurate early warning of hard or heavy high-risk objects, and provides users with practical guidance for rectification suggestions through quantitative comprehensive scoring and grading feedback. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a system function framework schematic diagram provided by one or more embodiments of the application.

[0030] Figure 2 is a method flowchart provided by one or more embodiments of the application.

[0031] Figure 3 is a detection flowchart provided by one specific embodiment of the application.

[0032] 1, object detection module; 2, vehicle cloud processing module; 3, display module. DETAILED DESCRIPTION

[0033] The technical solutions of the application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0034] Figure 1 The embodiment of the application provides a method for safety detection and evaluation of cabin environment, which is run in an electronic control unit or a vehicle-mounted computing platform of a vehicle, and includes the following steps:

[0035] The object detection module 1, the vehicle cloud processing module 2 and the display module 3 are connected through a vehicle-mounted bus or an internal communication protocol.

[0036] The object detection module 1 is configured to acquire environmental data in the cabin. The object detection module 1 is connected to and reuses the original driver monitoring system (DMS) camera and the passenger monitoring system (OMS) camera of the vehicle. The DMS camera covers the driver area, and the OMS camera covers the co-driver and rear passenger area.

[0037] The object detection module 1 is also connected to a physical sensor group distributed on the vehicle body, and the physical sensor group includes a steering wheel airbag sensor, a seat airbag sensor, and a side window airbag sensor. In order to reuse existing components without increasing additional hardware costs, the object detection module 1 adds a signal monitoring definition in the physical sensor group, which is configured to monitor the mass and pressure values allowed to be applied in a safe state, and is specifically used to read the static mass and pressure values of the above-mentioned areas to monitor the physical load state on the airbag ejection path.

[0038] The vehicle cloud processing module 2 is configured to receive the image data and sensor data collected by the object detection module 1, and the vehicle cloud processing module 2 includes an image processing unit and a logic operation unit, which are used to perform binaryzation processing, contour extraction, multi-label classification, and risk rule matching of images. The deployment mode of the vehicle cloud processing module 2 includes local deployment or cloud model deployment.

[0039] In an embodiment, the vehicle cloud processing module 2 communicates with a cloud server to obtain an updated detection rule set.

[0040] The display module 3 is configured to output the detection result. The display module 3 is connected to a display screen of a central control vehicle machine, and feeds back a cabin safety score, a hidden danger position, and a rectification suggestion to a user through a whole-vehicle safety monitoring application program (APP).

[0041] Figure 2 The embodiment of the present application provides a method for safety detection and evaluation of a cabin environment, and the method comprises the following steps:

[0042] In step S1, an initial state calibration and real-time data acquisition are performed. In a factory or initialization stage of a vehicle, an initial arrangement sequence of each area of a cabin is recorded, an original factory part state is photographed to generate a reference image, and when a detection function is executed, a DMS camera and an OMS camera are controlled to photograph cabin images again to generate to-be-detected images. Meanwhile, real-time pressure values and mass values of a steering wheel airbag sensor, a seat airbag sensor, and a side window airbag sensor are read.

[0043] In step S2, differential image processing and region extraction are performed. The vehicle cloud processing module 2 performs registration on the reference image and the to-be-detected image, performs binaryzation processing and difference calculation on the two images, performs contour detection to obtain a change set of an object detection frame, and performs segmentation on the image to separate an image area of a non-original factory added object from a background.

[0044] Step S3, multi-label object recognition and attribute analysis, using mainstream multi-label image classification detection algorithm, feature extraction is performed on the segmented image region, the type of in-vehicle object is recognized, the types include steering wheel cover, pendant, ornament, mobile phone support, foot mat, interior door handle decoration and modified lamp strip, and the material attribute of the object is detected to distinguish hard objects from soft objects.

[0045] Step S4, risk assessment based on rule set, the vehicle cloud processing module 2 receives the detection rule set issued by the cloud, the detection rule set is constructed based on the historical accident data of the same type of vehicle, assisted by machine learning related algorithm, including dangerous position calibration and dangerous object type, the object position, object type and material attribute recognized are matched with the detection rule set, combined with the pressure value and mass value of the physical sensor, the risk coefficient of the object is calculated.

[0046] Step S5, comprehensive score and result feedback, according to the risk coefficient, the overall environmental safety score of the cabin area is calculated, the output result is displayed through the display module 3, and the driver is informed of the position of the hidden danger in the vehicle, when the third-party accessories are detected to be inconsistent with the safety rules, specific placement suggestions are provided, and the information of the original factory accessories meeting the safety standards is displayed in the application.

[0047] Figure 3 In order to accurately identify the non-original factory installed objects in the cabin, an effective environment benchmark needs to be established and the difference features are extracted. The specific implementation process of steps S1 and S2 is as follows.

[0048] In step S1, the system performs initial state calibration, when the vehicle is detected at the factory offline or the user first initializes the system, in the empty and no litter state, the DMS camera and OMS camera are controlled to take multiple angle shots of the interior environment of the cabin, the vehicle cloud processing module 2 receives the photographed image data and stores it as benchmark image data, the benchmark image covers the driver seat, co-pilot seat, rear seat and center console area, for the image collected by each camera, it is defined as wherein represents the pixel coordinates of the image, represents the pixel gray value at the coordinates.

[0049] The object detection module 1 also records the initial state value of the physical sensor group, the physical sensor group is installed under the steering wheel airbag cover plate, in the airbag pop-up position of the seat side wing and the inner side of the side curtain covering area of the interior panel, the initial reading of the above position sensor under the action of external force is recorded, denoted as wherein represents the number of the th sensor.

[0050] When the user triggers the detection instruction or the vehicle starts the self-checking program, the system collects the current cabin image data, denoted as , reads the real-time readings of the first sensor at the same time , and calculates the change amount of the sensor value . The change amount calculation formula is as follows:

[0051] ;

[0052] If the change amount exceeds the preset sensor sensitivity threshold, it is marked that the physical area corresponding to the sensor has potential coverings or external force compression.

[0053] In step S2, the vehicle cloud processing module 2 performs differential processing and region extraction on the collected image.

[0054] The vehicle cloud processing module 2 first uses a feature point matching algorithm to perform geometric correction on and to eliminate artifacts caused by vehicle vibration or camera displacement. Fixed feature points in the image are extracted, including A-pillar edges, instrument panel contours, and air outlet edges. A transformation matrix is constructed to map to the same coordinate system as . For specific algorithm implementation of image feature extraction and geometric transformation, those skilled in the art can use the ORB (Oriented FAST and Rotated BRIEF) feature extraction or SIFT (Scale-Invariant Feature Transform) algorithm, which is a known technology in the art and will not be described here.

[0055] After registration is completed, the pixel difference between the current image and the reference image is calculated to generate a difference gray image , and the calculation formula is as follows:

[0056] ;

[0057] This formula calculates the absolute difference value of each pixel point to reflect the changes in illumination and object addition or subtraction in the cabin environment.

[0058] In order to separate the target region from the difference gray image, a threshold value is set for binary operation. The gray threshold is set to , a binary mask image is generated, and the calculation logic is as follows:

[0059] ;

[0060] Among them, represents when the condition is met, takes value 1; and represents in other cases where the above condition is not met, takes value 0, and the value 1 represents a foreground region, i.e. an existing non- original item region; and the value represents a background region, The value of the value can be adaptively adjusted according to the intensity of the in-vehicle ambient light. When the light is sufficient, a higher value is taken to reduce shadow interference; when the light is insufficient, a lower value is taken to ensure detection sensitivity.

[0061] Due to uneven light or sensor noise, isolated noise points or broken regions exist in the binary mask image The vehicle cloud processing module 2 performs morphological opening operation and closing operation on The opening operation is used to eliminate small background noise, and the closing operation is used to fill the holes in the target and connect the broken edges.

[0062] After morphological processing, all independent connected regions in the image are retrieved using a connected component analysis algorithm. For each connected region, the minimum bounding rectangle is calculated to obtain the object detection frame set , wherein represents the detection frame coordinate information of the potential item .

[0063] Based on the object detection frame set , the region of interest (ROI) is cropped on the original current image , and the image region corresponding to each detection frame is segmented to form an image slice set to be classified. This set will be used as the input data of the subsequent multi-label object recognition model. Through the above differential extraction step, the system can effectively filter out the background information such as original seat and instrument table, and focus only on the third-party items added by the user, thereby reducing the computational load of the subsequent algorithm and improving the recognition accuracy.

[0064] In step S3, the vehicle cloud processing module 2 performs deep feature analysis on the classified image slice set segmented in step S2 to determine the specific category and physical properties of the new item.

[0065] The vehicle cloud processing module 2 constructs and loads a multi-task convolutional neural network model. As for the specific object detection algorithm, the present application can adopt an algorithm with higher accuracy in detecting small accessory objects among mainstream algorithms such as CNN (Convolutional Neural Network), RCNN (Region-based Convolutional Neural Network), SVM (Support Vector Machine), or YOLO (You Only Look Once). The model adopts a Multi-Task Learning architecture, containing a shared feature extraction backbone network and two independent output branches: a category classification branch and a material attribute branch. The feature extraction backbone network is used to extract a high-dimensional feature vector from the input image slice A person skilled in the art can use ResNet (Residual Network), VGG, or MobileNet as the basic architecture of the backbone network to capture the texture, edge, and shape features of the image using convolutional layers and pooling layers. The construction and training of this part of the network structure are known in the art and will not be described here.

[0066] Next, the vehicle cloud processing module 2 identifies the category of the object using the category classification branch, which receives the high-dimensional feature vector and maps it to a pre-defined category space through a fully connected layer. The category space contains specific detection objects defined in the specification: steering wheel cover, pendant, ornament, mobile phone holder, foot mat, interior door handle decoration, and modified light strip. For the first input image slice , the model calculates the confidence score of the th category , converts the score to a probability distribution using the Softmax function, and the calculation formula is as follows:

[0067] ;

[0068] where: is the summation index variable, used to iterate through all categories; is the base of the natural logarithm.

[0069] After obtaining the probability distribution, the vehicle cloud processing module 2 selects the category with the highest probability as the identification result of the object:

[0070] ;

[0071] where, represents the final determined object category name; ​represents the maximum value parameter operation, that is, finding the category that makes the maximum value . If the maximum probability value is lower than the preset confidence threshold (for example, 0.6), the system marks the item as an unknown foreign object.

[0072] At the same time, the vehicle cloud processing module 2 analyzes the hardness characteristics of the item by using the material attribute branch. In order to accurately evaluate the secondary damage caused by the object in the collision accident, it is necessary to distinguish between hard objects and soft objects. The physical materials include metal, hard plastic, ceramic, glass, fabric, leather and plush. The material hardness coefficient is defined , the value range is [0, 1], and the value closer to 1 indicates that the material is harder (such as metal, glass), and the value closer to 0 indicates that the material is softer (such as plush, sponge). After the model outputs the material classification result, the hardness coefficient of the item is obtained by looking up the preset material hardness mapping table .

[0073] In addition, for specific items such as mobile phone holders, the model is configured to detect whether it is in a loaded state (that is, whether a mobile phone is placed on the holder). By detecting whether there is a reflection feature or a black screen rectangular feature unique to the mobile phone screen in the image, a load state flag bit is output. When the mobile phone is detected to be in place, the system will automatically adjust the mass estimation value and the hardness coefficient of the item.

[0074] Finally, the vehicle cloud processing module 2 performs mass estimation and physical parameter fusion. It is difficult to accurately obtain the mass of the object only by visual image. The system combines the category priori knowledge and the physical sensor data for comprehensive estimation. First, according to the identified object category name (such as a perfume bottle), the average density and the standard volume range of the item of this category are obtained from the database. Second, the apparent volume of the object is calculated by using the pixel area of the image slice in the original image, combining the camera intrinsic matrix and the distance estimation from the object to the camera . Combined with the physical sensor mass change amount obtained in step S1 (if the object is located in the sensor sensing area), the estimated mass of the object is calculated , and the calculation logic is as follows:

[0075] ;

[0076] When the physical sensor detects that the mass change amount is greater than the effective detection threshold , the sensor measured data is used as the mass of the object ; when the sensor is not triggered or the data is invalid (that is ), the mass value estimated based on vision is used; an average density of the category of items represented in the database; a volume of the object estimated based on the visual image.

[0077] Through the above processing, the system generates a feature vector containing the position coordinates, object category, material hardness, and estimated mass, providing an accurate data basis for subsequent risk assessment.

[0078] In step S4, the vehicle cloud processing module 2 performs risk assessment according to the detection rule set obtained from the cloud server. The construction process of the detection rule set is not a static preset input, but a dynamic evolution process based on big data. This process is periodically executed in the cloud server, and the specific implementation process is as follows.

[0079] The cloud server performs accident data mining and feature association. The system periodically collects data from the databases of cooperating insurance agencies, traffic accident identification centers, and desensitized accident logs uploaded by vehicles of the same brand. The vehicle cloud processing module 2 cleans the above data and selects specific samples of secondary injuries caused by objects in the vehicle.

[0080] For each sample, the key feature vector is extracted , where represents the type of object involved in the accident (e.g., crystal perfume holder); represents the relative position coordinates or region number of the object in the vehicle; represents the injury severity level caused by the accident, with a value range of [0, 10], and a larger value indicating more severe injury.

[0081] To convert discrete accident data into executable detection rules, the cloud server uses machine learning algorithms (such as association rule mining algorithms, statistical regression analysis, etc.) to establish a mapping model between object features and risk levels. The system defines a risk weight calculation formula to quantify the danger level of a specific object in a specific location. For a specific category and position , the basic risk weight is calculated as:

[0082] ;

[0083] where represents the comprehensive risk weight when an object of category is located at position ; represents the occurrence frequency of injury accidents caused by the object at that position in the historical database; represents the total number of associated samples; This item represents the posterior probability of accident occurrence; represents the average injury severity of the combination in the historical data; with is a normalized weight coefficient, and satisfies .

[0084] After calculating the base risk weight, the cloud server constructs a dangerous rule library containing spatial constraints. Each rule in the rule library contains two parts: trigger conditions and risk attributes.

[0085] The trigger conditions define a set of sensitive areas in the cabin , which clearly defines the deployment trajectory of the airbag and the operation interference area of the driver. Specifically, it includes:

[0086] The surface of the steering wheel airbag cover and its normal ejection path area;

[0087] The airbag burst port coverage area of the co-pilot instrument desk;

[0088] The airbag ejection path of the front seat side wing;

[0089] The operation space of the driver's foot brake pedal and accelerator pedal. The system maps the spatial coordinate range of the above areas to a three-dimensional bounding box (3D Bounding Box) as the basis for position matching.

[0090] The cloud server generates a structured detection rule set file (such as JSON or XML format) based on the calculated , in which file, for each sensitive area , a list of prohibited or restricted item categories and corresponding risk thresholds are listed. For example, for the steering wheel airbag area, the rule is set as:

[0091] If the item material hardness coefficient H>0.5H>0.5 (i.e. hard object), the risk level is determined as extremely high;

[0092] If the item is a mobile phone holder, the risk level is determined as high.

[0093] The vehicle cloud processing module 2 establishes an encrypted connection with the cloud server through the T-Box (Telematics Box) communication unit of the vehicle. When the vehicle is started or in a dormant wake-up state, it queries the version number of the cloud rule set. If the cloud version is higher than the local version, it downloads the latest rule set file and replaces the local cache. This mechanism ensures that the vehicle has the ability to identify and risk warning for new automotive products (such as new types of car heaters, irregular ornaments), without the need to change the local firmware code to update the safety logic.

[0094] Through the above cloud collaboration mechanism, the macroscopic accident big data is converted into microcosmic vehicle-machine execution logic, so that the vehicle cloud processing module 2 can make judgments according to objective data with statistical significance when performing subsequent risk assessment, instead of relying only on fixed preset parameters.

[0095] In steps S4 and S5, the vehicle cloud processing module 2 performs quantitative risk calculation based on the object feature vector obtained in the previous steps and the cloud rule set, and the process aims to convert the cabin state of the physical world into a measurable digital safety index. The specific calculation and evaluation logic includes the following sub-steps:

[0096] The vehicle cloud processing module 2 traverses each non-original item identified in step S3 . Extract the feature vector of the item , where is the position coordinate, is the item category, is the material hardness coefficient, is the estimated mass.

[0097] The system first determines the spatial relationship between the item position and the dangerous area defined in the detection rule set . By calculating the intersection-over-union (loU) or inclusion relationship between the item's three-dimensional bounding box and the dangerous area's bounding box, the position danger coefficient is determined.

[0098] If the item is completely located in the airbag ejection path or pedal operation area, set ;

[0099] If the item is located in the edge buffer area of the above area (for example, within 5 cm of the airbag cover plate edge), set ;

[0100] If the item is located in a non-sensitive area (such as the interior of the car door storage compartment), set .

[0101] The vehicle cloud processing module 2 combines the rule weight issued by the cloud and the physical properties detected locally to calculate the basic risk value of the th item . The calculation formula is defined as follows:

[0102] ;

[0103] Where: represents the basic risk score of the th item; represents the position danger coefficient determined above; denotes the normalized hardness coefficient of the material, which is obtained in step S3 and ranges from 0 to 1; denotes the normalized quality factor, which is estimated by the system and mapped to the interval [0, 1] using a variant of the Sigmoid function to ensure that the factor approaches 1 when the quality exceeds a threshold value (e.g., 200 g) that causes human injury; denotes the historical accident weight of the category of the object at the location, which is queried from the cloud rule set; , , are the weighting coefficients for each sub-item, and satisfy In a preferred embodiment, considering that the hardness has the greatest impact on secondary injuries (e.g., ejection), is set to 1. and .

[0104] In the case of occlusion or recognition error relying solely on the visual algorithm, the system corrects the base risk value using the physical sensor data monitored in step S1 to calculate the sensor correction coefficient When the position of the first object corresponds to the coverage range of the first physical sensor (e.g., steering wheel airbag sensor), the pressure change amount of the sensor is detected. The correction logic is as follows:

[0105] ;

[0106] wherein: denotes the correction coefficient; denotes the penalty factor, which is a constant greater than 0 (e.g., 0.5) used to increase the risk value when the physical sensor detects an abnormally heavy pressure; is an indicator function that takes the value 1 when the inequality holds, and 0 otherwise; is the maximum safe additional pressure threshold allowed in this area.

[0107] Finally, the comprehensive risk score of the first object is calculated as:

[0108] ;

[0109] Through this correction step, even if the visual algorithm mistakenly identifies a heavy decoration as a light object, the abnormal reading of the physical sensor can force the risk score to be increased, thereby ensuring the robustness of the system.

[0110] The vehicle cloud processing module 2 aggregates the comprehensive risk scores of all detected items in the cabin to generate a full-vehicle environment safety score . The initial full score of the system is set to (typically 100 points), and the calculation formula is as follows:

[0111] ;

[0112] wherein, is the total number of detected non-original items, and the formula adopts a deduction system logic, with higher risk and lower score. The function is used to ensure that the final score is not negative.

[0113] The system divides the current cabin environment into three safety levels according to the calculated :

[0114] When (e.g. 85 points), it is determined to be a safe state (Green), indicating that there are no miscellaneous items or the miscellaneous items are in a low-risk position;

[0115] When (e.g. 60 to 85 points), it is determined to be a warning state (Yellow), indicating that there are potential risk items, and the user is advised to adjust;

[0116] When (e.g. 60 points), it is determined to be a dangerous state (Red), indicating that there are extremely high-risk items (such as hard diamonds pasted on the steering wheel), which can cause serious secondary injuries in accidents.

[0117] The classification result will be transmitted as an output signal to the display module 3 to trigger different levels of user interaction strategies. Those skilled in the art can understand that the specific values of the above-mentioned thresholds and can be adjusted according to the safety design standards of the vehicle or user preferences. Such parameter adjustment is a reasonable extension of the technical solution.

[0118] After completing the risk assessment in step S5, the system executes the result output and interaction logic through the display module 3. This process is not just a simple display of data, but a technical closed loop from risk perception to behavior correction. The specific implementation steps are as follows:

[0119] The display module 3 receives the full-vehicle environment safety score output by the vehicle cloud processing module 2, as well as the coordinate information and risk level of each detected item. The display module 3 renders a virtual digital model of the cabin or displays a real-time cabin monitoring image on the center screen.

[0120] The detected item position coordinates in the physical world are converted into pixel regions in the screen display coordinate system using coordinate mapping technology, and an augmented reality border or highlight mask is generated in the screen image corresponding to the position of the item. The display module 3 generates an augmented reality border or highlight mask in the screen image corresponding to the position of the item according to the comprehensive risk score of the item . Decide the rendering color: if the item is determined to be in a dangerous state, use a red highlight border and prompt at a flashing frequency (e.g. 1 Hz);

[0121] If the item is determined to be in a warning state, use a yellow constant border;

[0122] If the item is determined to be in a safe state, no border is displayed or a green thin line frame is used for identification.

[0123] When the user clicks on the highlighted risk area in the screen interface, the display module 3 calls the risk explanation subroutine, which reads the specific factors that cause the risk score of the item to increase and generates a text prompt box on the interface.

[0124] The content displayed in the text prompt box includes: the identified item category (e.g. steering wheel counterweight ring); the specific risk rule triggered (e.g. abnormal steering wheel gravity sensor data or the item located in the main driver airbag ejection path); the description of the resulting injury consequences (e.g. causing the airbag to fail to deploy normally or the object flying at high speed to cause injury during a collision).

[0125] Through this interactive mode, the system converts abstract algorithm logic into user-understandable safety knowledge.

[0126] For each item marked as a risk, the vehicle cloud processing module 2 queries the safety placement strategy table in the cloud rule library, which defines the allowed placement area set for each type of item in the vehicle .

[0127] The system calculates the spatial distance between the current position and the allowed placement area set , and filters out the recommended area that is closest and not currently occupied .

[0128] The display module 3 indicates to the user to move the item from the current risk position to the recommended area in the form of a guide arrow on the interface . For example, if a hard tissue box is detected to be placed at the rear windshield (easy to fly out during sudden braking), the system suggests moving it to the door storage slot or the glove compartment.

[0129] When the detected risk is caused by the material properties of the item itself (such as When the system detects a potential safety hazard (e.g. a loose pedal) that is not simply a positional error (e.g. a pedal that is too high or a pedal that is stuck due to a design flaw such as a universal mat), the system executes the accessory matching logic. The vehicle cloud processing module 2 extracts the category of the item , and searches the pre-stored OEM database for the corresponding SKU.

[0130] If there is a corresponding OEM accessory that has passed the vehicle regulatory safety tests, the recommendation portal is generated by the display module 3 below the hazard detail page. The portal displays the comparative advantage information of the OEM accessory (e.g. the OEM mat has a non-slip buckle design), and provides a corresponding information link or QR code. This step solves the safety hazard problem caused by third-party inferior accessories through technical means.

[0131] After the user completes the item removal, position adjustment or accessory replacement operation according to the system suggestion, the user can trigger a re-inspection instruction through the application.

[0132] In response to the instruction, the system reactivates the detection process of steps S1 to S5. The object detection module 1 collects images and sensor data again to verify whether the physical sensor readings have returned to the baseline range (i.e. ) and whether the risk item in the visual image has disappeared or moved to a safe area.

[0133] If the re-inspection is passed, the display module 3 updates the full-vehicle safety score to a full score state and removes the warning signal; if the re-inspection is not passed, the warning state is maintained and the specific reason for not meeting the standard is prompted.

[0134] For the convenience of description, the above system is described as various units and modules. Of course, the functions of the units and modules can be implemented in the same or multiple software and / or hardware in the implementation of the present application.

[0135] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood as having meanings consistent with those in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0136] For the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the embodiments of the present application are not limited by the order of the described actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present application.

[0137] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary universal hardware platforms. Based on such an understanding, the technical solutions of the present application, in essence or in the part that makes a contribution to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, or the like, and includes a plurality of instructions to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of the present application.

[0138] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for safety detection and evaluation of a cockpit environment, characterized in that, Running in the electronic control unit or vehicle-mounted computing platform of the vehicle, the vehicle includes an object detection module, a vehicle cloud processing module, and a display module; comprising the following steps: Step S1, initial state calibration and real-time data acquisition: the object detection module records the initial arrangement sequence of each region of the cabin and generates a reference image, and when the detection function is executed, the camera is controlled to shoot again to generate a to-be-detected image, and the real-time pressure value and mass value of the physical sensor group are read; Step S2, differential image processing and region extraction: the vehicle cloud processing module performs registration and difference calculation on the reference image and the to-be-detected image, obtains a change set of object detection boxes, and segments out the image region of the non-factory-installed object; Step S3, multi-label object recognition and attribute analysis: the vehicle cloud processing module extracts features from the segmented image region, identifies the type of the object in the vehicle and the material attribute of the object; Step S4, risk assessment based on rule set: the vehicle cloud processing module receives a detection rule set, matches the identified object position, object type and material attribute with the detection rule set, and combines the real-time pressure value and mass value of the physical sensor group to calculate the risk coefficient of the object; Step S5, comprehensive scoring and result feedback: the vehicle cloud processing module calculates the overall environmental safety score of the cabin region according to the risk coefficient, and outputs the detection result and rectification suggestion by the display module.

2. The method for safety detection and evaluation of cabin environment according to claim 1, characterized in that, In step S1, the process of the object detection module acquiring data includes: Reuse the driver monitoring system DMS camera and passenger monitoring system OMS camera of the original vehicle to shoot images; And read data through the connected physical sensor group, which includes a steering wheel airbag sensor, a seat airbag sensor, and a side window airbag sensor; The object detection module reads the static mass value and pressure value of the area where the steering wheel airbag sensor, seat airbag sensor, and side window airbag sensor are located, and monitors the physical load state on the airbag ejection path.

3. The method for safety detection and evaluation of cabin environment according to claim 1, characterized in that, In step S2, the process of differential image processing and region extraction includes: Using a feature point matching algorithm to perform geometric correction on the to-be-detected image and the reference image; Calculate the absolute difference value of each pixel point of the current image and the reference image to generate a difference gray image; Set a gray threshold to perform a binary operation on the difference gray image to generate a binary mask image, and perform morphological opening and closing operations on the binary mask image; Use the connected component analysis algorithm to search for independent connected regions in the image, calculate the minimum bounding rectangle, and cut out the image slice set to be classified accordingly.

4. The method for safety detection and evaluation of cabin environment according to claim 1, characterized in that, In step S3, the vehicle cloud processing module constructs and loads a multi-task convolutional neural network model, which includes a shared feature extraction backbone network and a class classification branch and a material attribute branch; Use the class classification branch to identify the type of the object in the vehicle, which includes a steering wheel cover, a pendant, an ornament, a mobile phone support, a foot mat, an interior door handle decoration, and a modified lamp strip; The material attribute branch is used to detect the material hardness coefficient of the object, and a preset material hardness mapping table is looked up to distinguish a hard object from a soft object.

5. The method for safety detection and evaluation of cabin environment according to claim 4, characterized in that, In step S3, for the mobile phone support category, the vehicle cloud processing module detects whether the image has a mobile phone screen-specific reflection feature or a black screen rectangular feature, and outputs a load state flag bit; when the mobile phone is detected to be in place, the vehicle cloud processing module adjusts the mass estimation value and the hardness coefficient of the object.

6. The method for safety detection and evaluation of cabin environment according to claim 1, characterized in that, In step S4, the detection rule set is generated by the cloud server based on historical accident data of similar vehicle models and using a machine learning algorithm; The construction process of the detection rule set includes: mining and feature association of accident data, extracting feature vectors such as object category, relative position coordinates, and injury severity level; A mapping model between object features and risk levels is established, the basic risk weight is calculated for the combination of object categories and positions, and a rule library containing trigger conditions and risk attributes is generated according to the sensitive area set in the cabin.

7. The method for safety detection and evaluation of cabin environment according to claim 1, characterized in that, In step S4, the process of calculating the risk coefficient of the object includes: Determine the spatial relationship between the object position and the dangerous area defined in the detection rule set to determine the position danger coefficient; Combine the rule weight issued by the cloud and the physical properties detected locally to calculate the basic risk value, which is obtained by weighting and summing the material hardness coefficient, the normalized mass factor, and the historical accident weight, and then multiplying the position danger coefficient; The data of the physical sensor group monitored in step S1 is used to correct the basic risk value to obtain a comprehensive risk score.

8. The method for safety detection and evaluation of cabin environment according to claim 7, characterized in that, The process of correcting the basic risk value using the data of the physical sensor group monitored in step S1 includes: When the position of the object corresponds to the coverage range of a sensor in the physical sensor group, detect the pressure change amount of the sensor; If the pressure change amount exceeds the maximum safe additional pressure threshold allowed by the area, a sensor correction coefficient greater than 1 is generated according to a penalty factor; Multiply the basic risk value by the sensor correction coefficient to obtain the comprehensive risk score of the object.

9. The method for safety inspection and evaluation of cabin environment according to claim 1, characterized in that, In step S5, the calculation process of the comprehensive score is: Aggregate the comprehensive risk scores of all detected objects in the cabin; Set the initial full score of the system, calculate the difference between the initial full score and the sum of the comprehensive risk scores of all detected non-factory objects, and take the maximum value between the difference and 0 to obtain the overall vehicle environment safety score; According to the overall vehicle environment safety score, the current cabin environment is divided into three levels of safe state, warning state, and dangerous state.

10. The method for safety inspection and evaluation of cabin environment according to claim 1, characterized in that, In step S5, the process of result feedback includes: The display module uses coordinate mapping technology to generate an augmented reality frame at the position of the corresponding object on the center screen, and renders it using different colors or flashing frequencies according to the risk level of the object. When the detected risk is originated from material properties or structural design defects of the article itself, the vehicle cloud processing module retrieves corresponding inventory units in the original equipment database, and if there is original equipment that has passed safety testing, the vehicle safety monitoring application program is controlled to display comparative advantage information and a recommendation entry of the original equipment.

Images