A car occupant state classification method and system based on multi-source information fusion

Abstract

The application provides a car occupant state classification method and system based on multi-source information fusion, when a vehicle appears an emergency braking or collision warning information, seat images and seat parameters in the target vehicle are collected, global context features and local block appearance difference features are extracted according to the seat images and seat parameters in the target vehicle, prior probability, seat and each category likelihood are calculated, the prior probability and the likelihood at the current moment, the posterior probability at the last moment are combined, the posterior probability at the current moment is comprehensively calculated, and finally the category of the target seat is determined, so as to adapt to the change of the external environment, provide accurate category information for subsequent intelligent safety, and improve the safety of the vehicle.

Description

Technical Field

[0001] This application relates to the field of automotive passive safety system technology, specifically to a method and system for classifying the state of automotive occupants based on multi-source information fusion. Background Technology

[0002] Vehicle airbag systems are critical passive safety devices, but their explosive deployment can cause injury to evacuated occupants, children, and others. Therefore, it is necessary to identify and differentiate occupant status categories (whether there are occupants, and the occupant category and status). Currently, the main methods for identifying these categories include the following three:

[0003] Systems based on weight sensors estimate occupant weight using pressure sensors within the seat cushion to determine the presence and type of passenger. However, this method struggles to distinguish between children and heavy objects placed on the seat, and it cannot detect occupant posture or position. Furthermore, its accuracy is affected by vehicle posture and acceleration.

[0004] Ultrasonic-based systems detect occupants by emitting and receiving ultrasonic waves. However, this method has low sensitivity to stationary targets, is easily attenuated by absorption from soft surfaces inside the vehicle (such as clothing), and cannot provide detailed shape information, resulting in limited classification capabilities.

[0005] Vision-based systems are susceptible to changes in lighting, the diversity of occupant postures, and adjustments to seat structure. For example, extreme and dynamic changes in in-vehicle lighting conditions can cause drastic fluctuations in image features. Occupants of the same type (such as adults) may appear very different due to differences in clothing, body shape, and posture (sitting upright, sitting sideways, or leaning forward). Furthermore, the forward and backward movement of the seat and the adjustment of the backrest angle can completely alter the appearance and position of the occupant in the image.

[0006] Existing solutions show a significant decrease in accuracy and robustness in practical applications with dynamically changing environments and conditions, making it difficult to meet the high reliability requirements of automotive-grade applications. Therefore, there is an urgent need for a method that can accurately identify occupant status categories. Summary of the Invention

[0007] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method and system for classifying vehicle occupant states based on multi-source information fusion.

[0008] According to one aspect of this application, a method for classifying the state of vehicle occupants based on multi-source information fusion is provided, comprising: acquiring warning information from an inertial measurement unit; responding to the warning information by acquiring images of seats and seat parameters inside a target vehicle; extracting global context features and appearance difference features of local blocks of the target seat based on the seat images and seat parameters; calculating the prior probability of the target seat based on the global context features and the seat parameters; calculating the likelihood of the target seat with each category based on the appearance difference features of the local blocks; calculating the posterior probability of the target seat at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time; and determining the category of the target seat based on the posterior probability at the current time.

[0009] In one embodiment, extracting global context features and appearance difference features of local blocks of the target seat based on the seat image and the seat parameters includes: performing scale normalization processing on the seat image; using a filter bank to scan the scale-normalized seat image with a sliding window to extract the global context features; calculating the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image based on the seat image and the reflectance reference image corresponding to the seat parameters; and determining the appearance difference features of the target local block based on the negative normalized correlation coefficient.

[0010] In one embodiment, the method for determining the reflectance reference image corresponding to the seat parameter includes: acquiring experimental images under different lighting conditions for each seat parameter; and calculating the reflectance reference image corresponding to the seat parameter based on the lighting conditions and the experimental images.

[0011] In one embodiment, calculating the reflectance reference image corresponding to the seat parameters based on the illumination conditions and the experimental images includes: calculating the difference between the product of all the experimental images and the reflectance reference image and the corresponding illumination conditions; and solving the difference minimization problem to obtain the reflectance reference image.

[0012] In one embodiment, calculating the prior probability of the target seat based on the global context features and the seat parameters includes: inputting the global context features and the seat parameters into a context-aware prior model to obtain the prior probability of the target seat.

[0013] In one embodiment, calculating the likelihood of the target seat with each category based on the appearance difference features of the local blocks includes: for each category, calculating the likelihood of the target seat with the corresponding category based on the appearance difference features of the local blocks and their Gaussian distribution.

[0014] In one embodiment, calculating the posterior probability at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time includes: updating the posterior probability of the target seat at the current time using Bayesian filtering based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time.

[0015] In one embodiment, determining the category of the target seat based on the posterior probability at the current time includes: selecting the category corresponding to the maximum value of the posterior probability at the current time as the category of the target seat.

[0016] In one embodiment, the acquisition of seat images and seat parameters inside the target vehicle includes: acquiring seat images of the target seat, its front-to-back position on the guide rail, and the angle between the seat back and the vertical plane.

[0017] According to another aspect of this application, a vehicle occupant state classification system based on multi-source information fusion is provided, comprising: a warning information acquisition module for acquiring warning information from an inertial measurement unit; a seat data acquisition module for acquiring seat images and seat parameters inside a target vehicle in response to the warning information; a seat feature extraction module for extracting global context features and appearance difference features of local blocks of a target seat based on the seat images and the seat parameters; a prior probability calculation module for calculating the prior probability of the target seat based on the global context features and the seat parameters; a likelihood calculation module for calculating the likelihood of the target seat with each category based on the appearance difference features of the local blocks; a posterior probability calculation module for calculating the posterior probability of the target seat at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time; and a category determination module for determining the category of the target seat based on the posterior probability at the current time.

[0018] This application provides a vehicle occupant state classification method and system based on multi-source information fusion. The method involves: acquiring warning information from an inertial measurement unit; responding to the warning information by collecting seat images and parameters within the target vehicle; extracting global context features and local block appearance difference features of the target seat based on the seat images and parameters; calculating the prior probability of the target seat based on the global context features and seat parameters; calculating the likelihood of the target seat with each category based on the local block appearance difference features; and calculating the posterior probability of the target seat based on its prior probability and likelihood at the current time step and the posterior probability at the previous time step. The system calculates the posterior probability up to the current time step; based on the posterior probability at the current time step, it determines the category of the target seat; when the vehicle experiences warning information such as emergency braking or collision, it extracts global context features and appearance difference features of local blocks based on the seat image and seat parameters inside the target vehicle, and calculates the prior probability, the likelihood of the seat and each category. Combining the prior probability and likelihood at the current time step with the posterior probability at the previous time step, it comprehensively calculates the posterior probability at the current time step, and finally determines the category of the target seat. This adapts to changes in the external environment, provides accurate category information for subsequent intelligent safety, and thus improves vehicle safety. Attached Figure Description

[0019] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 This is a flowchart illustrating a vehicle occupant status classification method based on multi-source information fusion provided in an exemplary embodiment of this application.

[0021] Figure 2 This is a schematic diagram of the structure of a vehicle occupant status classification system based on multi-source information fusion provided in an exemplary embodiment of this application. Detailed Implementation

[0022] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0023] Figure 1 This is a flowchart illustrating a vehicle occupant state classification method based on multi-source information fusion provided in an exemplary embodiment of this application. Figure 1 As shown, the vehicle occupant status classification method based on multi-source information fusion includes the following steps:

[0024] Step 110: Obtain early warning information from the inertial measurement unit.

[0025] This application sets up an onboard inertial measurement unit (IMU) to provide three-axis acceleration and angular velocity information of the vehicle, which is used to sense the vehicle's emergency braking, collision and other conditions. When the vehicle experiences emergency braking, collision or other conditions, the inertial measurement unit will issue a warning message.

[0026] Step 120: In response to the warning information, collect images and parameters of the seats inside the target vehicle.

[0027] This application employs a monocular vision sensor and a near-infrared (NIR) illumination module. The monocular vision sensor is preferably a global shutter, high dynamic range (HDR) automotive-grade grayscale camera, deployed near the rearview mirror in the vehicle's roof, covering the entire front passenger area. The NIR illumination module consists of an LED array with a wavelength of 780-1100nm (preferably 850nm or 940nm), triggered synchronously with the camera to provide active and uniform illumination under low-light conditions, eliminating the effects of sudden changes in ambient light. Upon receiving a warning from the inertial measurement unit, the monocular vision sensor and NIR illumination module acquire images of the seats inside the target vehicle, and sensors mounted on the seats collect seat parameters.

[0028] Step 130: Based on the seat image and seat parameters, extract the global context features and local block appearance difference features of the target seat.

[0029] Based on the acquired seat images and seat parameters, this application extracts the global context features and local block appearance difference features of the target seat to obtain the occupant status of the target seat from both global and local dimensions.

[0030] Step 140: Calculate the prior probability of the target seat based on global context features and seat parameters.

[0031] This application combines global context features and seat parameters to calculate the prior probability of the target seat, that is, to estimate the occupant state category probability of the target seat under the current seat parameters based on global context features.

[0032] Step 150: Calculate the likelihood of the target seat with each category based on the appearance difference features of the local blocks.

[0033] This application calculates the likelihood of the target seat with each category based on the appearance difference features (i.e., local features) of local blocks (rectangular blocks) in the seat image, in order to determine the similarity between the current state of the target seat and each category.

[0034] Step 160: Calculate the posterior probability at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time.

[0035] This application calculates the posterior probability at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time, so as to obtain the optimal posterior estimated probability at the current time.

[0036] Step 170: Determine the category of the target seat based on the posterior probability at the current moment.

[0037] The category corresponding to the maximum value of the posterior probability at the current moment is selected as the final category of the target seat, and the deployment force level of the airbag is controlled (multi-stage ignition) or completely suppressed according to the final category of the target seat.

[0038] This application provides a vehicle occupant state classification method based on multi-source information fusion. The method involves acquiring warning information from an inertial measurement unit (IMU); responding to the warning information by collecting seat images and parameters within the target vehicle; extracting global contextual features and local block appearance difference features of the target seat based on the seat images and parameters; calculating the prior probability of the target seat based on the global contextual features and seat parameters; calculating the likelihood of the target seat relative to each category based on the local block appearance difference features; calculating the posterior probability of the target seat at the current moment based on the prior probability and likelihood of the target seat at the previous moment; determining the category of the target seat based on the posterior probability at the current moment; and when the vehicle experiences warning information such as emergency braking or collision, extracting global contextual features and local block appearance difference features based on the seat images and parameters within the target vehicle, calculating the prior probability, the likelihood of the seat relative to each category, and combining the prior probability and likelihood of the current moment with the posterior probability of the previous moment to comprehensively calculate the posterior probability at the current moment, ultimately determining the category of the target seat. This method adapts to changes in the external environment, provides accurate category information for subsequent intelligent safety, and thus improves vehicle safety.

[0039] In one embodiment, step 130 can be implemented as follows: scale normalization is performed on the seat image; a filter bank is used to scan the scale-normalized seat image with a sliding window to extract global context features; based on the seat image and the reflectivity reference image corresponding to the seat parameters, the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image is calculated; based on the negative normalized correlation coefficient, the appearance difference features of the target local block are determined.

[0040] Specifically, this application scales the input image (e.g., 320×240), and then uses a dense filter bank (including horizontal, vertical, and diagonal types and multiple sizes) to scan the entire seat image in a sliding window manner to generate a high-dimensional global feature vector to represent the global contextual features of the scene. The seat image can be decomposed into the product of two components, i.e., I(x,y)=R(x,y)L(x,y), where I(x,y) is the seat image, R(x,y) is the reflectivity component, representing the inherent optical properties of the object surface (such as the texture and color of leather), which is a constant to be solved, and L(x,y) is the illumination component, representing the intensity and distribution of the external light source, which is a quantity that varies with the environment. The reflectance reference image is a reflectance component that is independent of illumination and only reflects the inherent properties of the seat surface. This application calculates the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image based on the reflectance reference image corresponding to the current seat parameters and the seat image itself. Furthermore, based on the negative normalized correlation coefficient, the appearance difference features of the target local blocks are determined. Specifically, for the current frame image I... k The reflectivity reference image corresponding to the current seat parameters is registered and aligned with the current frame image. For any candidate local block p (rectangular region) in the current frame image, it is divided into four quadrants {q1, q2, q3, q4}. Within each quadrant, I... k The negative normalized correlation coefficient with the reflectance reference image is used as a measure of appearance difference. Its value ranges from [-1, 1], with a value closer to 1 indicating a greater difference. For each candidate block p, the negative normalized correlation coefficient values ​​in its four quadrants are combined into a 4-dimensional feature descriptor vector: f(I k (p))=[γ1,γ2,γ3,γ4] T .

[0041] In one embodiment, step 130 can be implemented as follows: for each seat parameter, experimental images are collected under different lighting conditions; based on the lighting conditions and experimental images, a reflectance reference image for the corresponding seat parameter is calculated.

[0042] For each seat parameter, this application employs different lighting conditions. L k Captured seat images of the same seat {I1,I2,...,I N (Experimental images), seeking an optimal reflectance reference image R. *This method aims to best interpret all seat images. Specifically, a dataset of over 1000 images was collected, covering various lighting conditions (such as sunlight, shadows, indoor nighttime lighting, supplemental infrared illumination, etc.), with only empty seats in the scene. Optimization algorithms such as gradient descent or alternating direction multiplier method were used to alternately optimize and solve for the reflectivity image R and a series of illumination images {L1, L2, ..., L...}. N The optimization process continues until the energy function converges, ultimately outputting a unique reflectivity reference image R that is independent of illumination. * This image will serve as a constant benchmark for the online classification phase under this specific seating configuration.

[0043] In one embodiment, step 130 can be implemented by: calculating the difference between the product of all experimental images and the reflectance reference image and the corresponding illumination conditions; solving the difference minimization problem to obtain the reflectance reference image.

[0044] Specifically, this application obtains the optimal reflectivity reference image by solving the following minimization problem:

[0045] ;

[0046] in, To preserve data fidelity, the original image is reconstructed from the results of forced decomposition. As a regularization term, based on the prior knowledge of illumination smoothness, it can be set as the gradient norm of the illumination components. ), used to force the smoothness of the illumination component; λ is a regularization parameter used to balance the weights of the data fidelity term and the regularization term.

[0047] In one embodiment, step 140 can be implemented by inputting global context features and seat parameters into a context-aware prior model to obtain the prior probability of the target seat.

[0048] Specifically, this application uses a Gaussian mixture model as a context-aware prior model. The global context features and seat parameters are input into the Gaussian mixture model to obtain the probability distribution of the target seat, that is, the probability distribution of the target seat in each category in the current scene.

[0049] In one embodiment, step 150 can be implemented as follows: for each category, the likelihood of the target seat to the corresponding category is calculated based on the appearance difference features of the local blocks and their Gaussian distribution.

[0050] For each category, this application calculates the likelihood between the target seat and the corresponding category based on the appearance differences of local blocks and their Gaussian distribution, in order to determine the category of the target seat in the current scene. Specifically, this application uses the following formula to calculate the likelihood between the target seat and the corresponding category:

[0051] ;

[0052] in, Let the seat parameter be θ t Seat Image I in State t The likelihood between class c and class c, μ k c Let Σ be the mean vector. k c Θ is the covariance matrix; k c For the threshold or bias term, N(μ) k c ,Σ k c The distribution is a 4-dimensional Gaussian distribution. Image I of the seat t The likelihood of the k-th local block and the image of class c in that local block, K c Image I of the seat t The number of local blocks in the text.

[0053] In one embodiment, step 160 can be implemented by using Bayesian filtering to update the posterior probability of the target seat at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time.

[0054] This application constructs a Bayesian filtering model to integrate the prior probability and likelihood at the current time step with the posterior probability at the previous time step, in order to update the posterior probability at the current time step.

[0055] In one embodiment, step 170 can be implemented by selecting the category corresponding to the maximum value of the posterior probability at the current time as the category of the target seat.

[0056] After calculating the posterior probability at the current time, this application selects the category corresponding to the maximum value of the posterior probability at the current time as the category of the target seat to ensure the accuracy of the category recognition of the target seat.

[0057] In one embodiment, step 120 can be implemented by: acquiring an image of the target seat, its front and rear position on the guide rail, and the angle between the seat back and the vertical plane.

[0058] This application uses a position sensor, such as a Hall effect sensor or a potentiometer, installed on the seat track to detect the front and rear position of the seat on the guide rail in real time. It also uses a seat back angle sensor, such as an angle encoder, installed on the seat to detect the angle between the seat back and the vertical plane in real time.

[0059] Figure 2 This is a schematic diagram of the structure of a vehicle occupant status classification system based on multi-source information fusion provided in an exemplary embodiment of this application. Figure 2 As shown, the vehicle occupant state classification system 20 based on multi-source information fusion includes: a warning information acquisition module 21, used to acquire warning information from the inertial measurement unit; a seat data acquisition module 22, used to acquire seat images and seat parameters inside the target vehicle in response to the warning information; a seat feature extraction module 23, used to extract global context features and local block appearance difference features of the target seat based on the seat images and seat parameters; a prior probability calculation module 24, used to calculate the prior probability of the target seat based on the global context features and seat parameters; a likelihood calculation module 25, used to calculate the likelihood of the target seat with each category based on the appearance difference features of the local blocks; a posterior probability calculation module 26, used to calculate the posterior probability of the target seat at the current time based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time; and a category determination module 27, used to determine the category of the target seat based on the posterior probability at the current time.

[0060] This application provides a vehicle occupant state classification system based on multi-source information fusion. The system acquires warning information from the inertial measurement unit via a warning information acquisition module 21; a seat data acquisition module 22 responds to the warning information and acquires seat images and parameters within the target vehicle; a seat feature extraction module 23 extracts global context features and local block appearance difference features of the target seat based on the seat images and parameters; a prior probability calculation module 24 calculates the prior probability of the target seat based on the global context features and seat parameters; a likelihood calculation module 25 calculates the likelihood of the target seat with each category based on the appearance difference features of the local blocks; and a posterior probability calculation module 26 calculates the likelihood of the target seat with respect to the current occupant state based on the appearance difference features of the local blocks. The prior probability and likelihood of the current time step, along with the posterior probability of the previous time step, are used to calculate the posterior probability of the current time step. The category determination module 27 determines the category of the target seat based on the posterior probability of the current time step. When the vehicle experiences warning information such as emergency braking or collision, global context features and appearance difference features of local blocks are extracted based on the seat image and seat parameters inside the target vehicle. The prior probability, the likelihood of the seat and each category are calculated. Combining the prior probability and likelihood of the current time step with the posterior probability of the previous time step, the posterior probability of the current time step is calculated comprehensively to finally determine the category of the target seat. This adapts to changes in the external environment and provides accurate category information for subsequent intelligent safety, thereby improving vehicle safety.

[0061] In one embodiment, the seat feature extraction module 23 can be further configured to: perform scale normalization processing on the seat image; use a filter bank to scan the scale-normalized seat image with a sliding window to extract global context features; calculate the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image based on the seat image and the reflectivity reference image corresponding to the seat parameters; and determine the appearance difference features of the target local block based on the negative normalized correlation coefficient.

[0062] In one embodiment, the seat feature extraction module 23 can be further configured to: acquire experimental images under different lighting conditions for each seat parameter; and calculate the reflectance reference image of the corresponding seat parameter based on the lighting conditions and the experimental images.

[0063] In one embodiment, the seat feature extraction module 23 can be further configured to: calculate the difference between the product of all experimental images and the reflectance reference image and the corresponding illumination conditions; solve the difference minimization problem to obtain the reflectance reference image.

[0064] In one embodiment, the prior probability calculation module 24 can be further configured to input global context features and seat parameters into the context-aware prior model to obtain the prior probability of the target seat.

[0065] In one embodiment, the likelihood calculation module 25 described above can be further configured to: for each category, calculate the likelihood of the target seat with respect to the corresponding category based on the appearance difference features of the local blocks and their Gaussian distribution.

[0066] In one embodiment, the aforementioned posterior probability calculation module 26 can be further configured to: update the posterior probability of the target seat at the current time using Bayesian filtering based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time.

[0067] In one embodiment, the category determination module 27 can be further configured to select the category corresponding to the maximum value of the posterior probability at the current time as the category of the target seat.

[0068] In one embodiment, the seat data acquisition module 22 can be further configured to: acquire the seat image of the target seat, its front and rear position on the guide rail, and the angle between the seat back and the vertical plane.

[0069] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0070] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0071] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0072] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0073] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0074] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0075] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0076] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0077] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A vehicle occupant status classification method based on multi-source information fusion, characterized in that, include: Obtain early warning information from the inertial measurement unit; In response to the warning information, images and parameters of the seats inside the target vehicle are collected; Based on the seat image and the seat parameters, extract the global context features and local block appearance difference features of the target seat; Based on the global context features and the seat parameters, the prior probability of the target seat is calculated; Based on the appearance differences of the local blocks, the likelihood of the target seat with each category is calculated; Based on the prior probability and likelihood of the target seat at the current moment and the posterior probability at the previous moment, the posterior probability at the current moment is calculated. The category of the target seat is determined based on the posterior probability at the current moment; The step of extracting global context features and local block appearance difference features of the target seat based on the seat image and the seat parameters includes: The seat image is then subjected to scale normalization. A filter bank is used to scan the scale-normalized seat image with a sliding window in order to extract the global context features; Based on the seat image and the reflectance reference image corresponding to the seat parameters, calculate the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image; Based on the negative normalized correlation coefficient, the appearance difference features of the target local block are determined.

2. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, The method for determining the reflectivity reference image corresponding to the seat parameters includes: For each seat parameter, experimental images were collected under different lighting conditions; Based on the lighting conditions and the experimental images, a reflectance reference image corresponding to the seat parameters is calculated.

3. The vehicle occupant status classification method based on multi-source information fusion according to claim 2, characterized in that, The step of calculating the reflectance reference image corresponding to the seat parameters based on the illumination conditions and the experimental image includes: Calculate the difference between the product of all the experimental images and the reflectance reference image and the corresponding illumination conditions; Solve the difference minimization problem to obtain the reflectance reference image.

4. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, The calculation of the prior probability of the target seat based on the global context features and the seat parameters includes: The global context features and the seat parameters are input into the context-aware prior model to obtain the prior probability of the target seat.

5. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, The calculation of the likelihood of the target seat with each category based on the appearance difference features of the local blocks includes: For each category, the likelihood of the target seat to the corresponding category is calculated based on the appearance difference features of the local blocks and their Gaussian distribution.

6. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, The calculation of the posterior probability at the current moment based on the prior probability and likelihood of the target seat at the current moment and the posterior probability at the previous moment includes: Based on the prior probability and likelihood of the target seat at the current time and the posterior probability at the previous time, Bayesian filtering is used to update the posterior probability of the target seat at the current time.

7. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, Determining the category of the target seat based on the posterior probability at the current moment includes: The category corresponding to the maximum value of the posterior probability at the current moment is selected as the category of the target seat.

8. The vehicle occupant status classification method based on multi-source information fusion according to claim 1, characterized in that, The acquisition of seat images and seat parameters inside the target vehicle includes: The system acquires images of the target seat, its front-to-back position on the guide rail, and the angle between the seat back and the vertical plane.

9. A vehicle occupant status classification system based on multi-source information fusion, characterized in that, include: The early warning information acquisition module is used to acquire early warning information from the inertial measurement unit; The seat data acquisition module is used to respond to the warning information and acquire images and parameters of the seats inside the target vehicle; The seat feature extraction module is used to extract global context features and local block appearance difference features of the target seat based on the seat image and the seat parameters; The prior probability calculation module is used to calculate the prior probability of the target seat based on the global context features and the seat parameters; The likelihood calculation module is used to calculate the likelihood of the target seat with each category based on the appearance difference features of the local blocks; The posterior probability calculation module is used to calculate the posterior probability at the current moment based on the prior probability and likelihood of the target seat at the current moment and the posterior probability at the previous moment. The category determination module is used to determine the category of the target seat based on the posterior probability at the current time. The seat feature extraction module is further configured as follows: The seat image is then subjected to scale normalization. A filter bank is used to scan the scale-normalized seat image with a sliding window in order to extract the global context features; Based on the seat image and the reflectance reference image corresponding to the seat parameters, calculate the negative normalized correlation coefficient between the four quadrants of each target local block in the seat image; Based on the negative normalized correlation coefficient, the appearance difference features of the target local block are determined.

Images