Method and system for seat belt status detection

Abstract

The present disclosure provides a method and system for seat belt status detection. The method includes: obtaining a first image in the infrared spectrum of at least a portion of a seat belt area from an image acquisition device (200), wherein the seat belt is expected to be visible in the seat belt area when the seat belt is worn by a vehicle occupant; obtaining a second image in the visible light spectrum of at least a portion of the seat belt area used to obtain the first image from the image acquisition device; sending the first image to a first classifier (320); sending the second image to a second classifier (330); determining a first probability estimate from the first classifier; determining a second probability estimate from the second classifier; combining the first probability estimate and the second probability estimate using respective scores assigned to the first image and assigned to the second image to determine a combined probability, each score indicating the confidence of the respective image for detecting the seat belt status; comparing the combined probability with a predetermined threshold to determine the seat belt status.

Description

Technical Field

[0001] The present invention relates to the field of seat belt status detection in vehicles. Background Art

[0002] Most vehicles are equipped with a system for automatically detecting whether a vehicle occupant is wearing a seat belt.

[0003] For this purpose, it is known to use a seat belt buckle sensor mounted in the buckle receiver, which detects whether the seat belt buckle has been inserted. Another type of sensor measures how much the seat belt has been pulled out to determine whether a person is wearing the seat belt.

[0004] While such sensors are reliable and low-cost, they are limited in that they cannot identify certain misuses of seat belts. For example, the sensor cannot detect that an occupant has buckled his seat belt by inserting the seat belt buckle into the buckle receptacle of an adjacent seat, or that the seat belt is not being worn properly. For example, the seat belt may be misplaced behind the body, or under the shoulders and arms.

[0005] Automakers and suppliers are currently developing image-based systems for seat belt status detection. Such detection systems typically include an image sensor that collects images of a seat belt area where the seat belt is expected to be visible when worn by a vehicle occupant and a processor that analyzes the image to determine the seat belt status.

[0006] US2007 / 0195990A1 discloses an image-based system for detecting the status of a vehicle seat belt. A plurality of reflective indicators are distributed on the seat belt assembly, and an infrared light source illuminates the seat belt assembly. An image sensor receives reflected infrared light from the plurality of reflective indicators and captures an image. Then, an image processor analyzes the captured image, identifies the plurality of indicators, and matches the identified indicators with a predefined set of indicators to characterize the seat belt status as "seat belt worn" or "seat belt not worn".

[0007] US2010 / 0087990A1 discloses an image-based occupant classification system for determining occupant classification and seat belt status. The system has: an image sensor that captures, for example, infrared or near-infrared images; and an image processor that analyzes the images captured by the image sensor to determine the status of the seat belt by detecting whether a portion of the seat belt is disposed across the front of the occupant, or whether the tongue of the seat belt is engaged with an associated buckle.

[0008] US 2018 / 0326944A1 discloses another system for detecting and monitoring a seat belt of an occupant. The system includes a light source that emits light of a predetermined wavelength onto a seat belt assembly, the seat belt assembly including a reflective pattern that reflects the predetermined wavelength at a preferred brightness; a 3D time-of-flight camera that captures an image of the seat belt assembly; and a computer processor that calculates a depth measurement of the distance of the reflective pattern on the seat belt assembly based on the image.

[0009] CN110084184A discloses another image-based seat belt status detection system. The system includes a wide-angle camera and a processor provided with an image judgment module, and the image judgment module includes a facial feature recognition block, a shoulder feature recognition block and a seat belt feature recognition block. In operation, the wide-angle camera collects images of the seat belt status. When the brightness is insufficient, an infrared emission source is used and an infrared image is collected. The image module determines to process the collected images, extract facial features and shoulder features, calculate the distance from the right shoulder to the facial center line and the distance from the left shoulder to the facial center line, and compare the two shoulder distances to preliminarily determine whether the seat belt is worn. More precisely, when the seat belt is worn by the passenger, the shoulder distance on one side of the seat belt should be shorter than the shoulder distance on the side without the seat belt. When the seat belt is initially determined to be worn, the feature points of the seat belt are fitted to a straight line, and different features of the straight line (such as slope, length, etc.) are compared with a threshold range to determine whether the seat belt is worn correctly.

[0010] CN110458093A discloses a seat belt detection method based on a driver monitoring system. After obtaining a target DMS image, the angle value of the first part of the seat belt extending from the uppermost end of the seat belt to the driver's shoulder is determined from the target DMS image and compared with an angle threshold to determine whether the driver wears the seat belt correctly. The second part of the seat belt extending from the driver's shoulder to the lowermost end of the seat belt can also be identified and sent to a trained classifier for identification. The classification result allows determining whether the driver wears his seat belt correctly. The image can be an infrared image.

[0011] A major disadvantage of known seat belt status detection systems is that, depending on the current conditions in the cabin (eg brightness), the colour of the occupant's clothing, the colour of the seat belts etc., the detection result may be erroneous.

[0012] Therefore, there is a need to improve the reliability of image-based seat belt status detection and avoid false positive detections as much as possible. Summary of the invention

[0013] The present disclosure relates to a computer-implemented method for seat belt status detection in a vehicle, the method comprising the following steps performed by a processing device:

[0014] obtaining, from an image acquisition device, at least one first image in an infrared spectrum of at least a portion of a seat belt region in which the seat belt is expected to be visible when the seat belt is worn by an occupant of the vehicle;

[0015] obtaining from the image acquisition device at least one second image in the visible light spectrum of at least a portion of the seat belt area used to obtain the at least one first image;

[0016] sending the at least one first image to a first classifier dedicated to images in the infrared spectrum;

[0017] sending the at least one second image to a second classifier dedicated to images in the visible spectrum;

[0018] determining from the first classifier a first probability estimate that the at least one first image includes a properly worn seat belt;

[0019] determining from the second classifier a second probability estimate that the at least one second image includes a properly worn seat belt;

[0020] combining the first probability estimate and the second probability estimate using respective scores assigned to the at least one first image and to the at least one second image to determine a combined probability, the respective scores indicating a confidence level of the respective image for detecting a seat belt status;

[0021] The combined probability is compared to a predetermined threshold to determine the seat belt status.

[0022] The above method allows to dynamically decide whether the seat belt is worn correctly based on the image and according to the current situation in the vehicle. The fusion or combination of probability estimates obtained from multiple images in IR and visible light allows to dynamically decide which modality in IR or visible light is more trusted in the current situation and thus obtain a robust and reliable seat belt status detection regardless of the current environmental conditions, time of day, weather, clothing color, brightness, etc. The number of false positive detections can be significantly reduced.

[0023] The present disclosure also relates to:

[0024] A seat belt status detection system for a vehicle;

[0025] A computer program, the computer program comprising instructions, the instructions causing the above system to perform the steps of the above method;

[0026] A computer readable medium having a computer program stored thereon;

[0027] A vehicle comprises the above-mentioned seat belt status detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Other features, objects and advantages of the present disclosure will become more apparent from reading the detailed description of non-limiting embodiments made with reference to the accompanying drawings.

[0029] Figure 1 An image-based seat belt status detection system according to an embodiment is shown.

[0030] FIG. 2A to FIG. 2B Indicates that according to the implementation mode, Figure 1 Flow chart of a seat belt status detection method executed by a system.

[0031] Figure 3 An infrared image corresponding to a front view of the interior of a vehicle is shown, showing a scene including a driver and a passenger seated on two front seats of the vehicle.

[0032] Figure 4 Representation and Figure 3 A scene image of the same scene represented in but in visible light, such as an RGB or color image.

[0033] Figure 5 Another visible light RGB or color image of the interior of a vehicle is shown including three occupants (two in the front seats and one in the back seat) and a box generated from a cropped image of the visible light image defining the seat belt zone in accordance with an embodiment. DETAILED DESCRIPTION

[0034] Figure 1 The image-based seat belt status detection system 100 according to the embodiment is shown. The system 100 includes an image acquisition device 200 and a processing device 300.

[0035] The image acquisition device 200 is operable to acquire images in the infrared IR portion and / or the near infrared NIR portion of the electromagnetic spectrum, such as wavelengths between 750nm and 1000nm, and images in the visible light portion of the electromagnetic spectrum, such as wavelengths between 380nm and 750nm. In an embodiment, the image acquisition device 200 may have four spectral channels for red, green, blue and IR and / or NIR wavelengths. For example, the image acquisition device 200 is an RGB-IR camera. The device 200 may simultaneously acquire an IR image and a visible light or color image of the same scene at a given point in time. When there is one or more occupants in the vehicle, the acquired images include one or more seat belt areas, each seat belt area being an area where the seat belt is expected to be visible when the seat belt is worn by an occupant in the vehicle. For example, as Figures 3 to 5As shown, the image captured by the image capture device 200 corresponds to a front view of the interior of the vehicle including two front seats and part of the rear seat. Figure 3 and Figure 4 Images of the same scene acquired in the infrared spectrum as well as in the visible light spectrum are shown. Figure 5 An image of another scene acquired in the vehicle in the visible spectrum is shown.

[0036] In the following description, an image acquired in the infrared or near infrared spectrum will be referred to as an “IR image”, and an image acquired in the visible spectrum will be referred to as an “RGB image” or a “color image.” However, the present disclosure is not limited to IR and RGB.

[0037] The processing device 300 has the function of receiving and processing the initial image from the image acquisition device 200, and based on the received initial image, detecting whether the vehicle occupant is wearing the seat belt correctly, in other words, detecting the seat belt status.

[0038] The processing device 300 may include a processor 310, a first classifier 320, a second classifier 330, a sorting module 340, a fusion module 350, and a seat belt status detection module 360. The first classifier 320, the second classifier 330, the sorting module 340, the fusion module 350, and the seat belt status detection module 360 ​​may be implemented by software or a computer program running on the processor 310.

[0039] The first classifier 320 and the second classifier 330 may have the function of identifying a seat belt in an image to determine whether the seat belt exists or is visible in the image, and / or have the function of identifying different situations of seat belt misuse in the image to detect whether the seat belt in the image is misused or not worn correctly. The first classifier 320 and the second classifier 330 may be two separate modules, one module for IR and / or NIR channels and the other module for visible light channels, such as RGB channels. The first classifier 320 is dedicated to processing IR and / or NIR images, and the second classifier 330 is dedicated to processing RGB or color images. In operation, the first classifier 320 and the second classifier 330 receive an image as input and may determine a probability estimate that the image includes a seat belt that is worn correctly.

[0040] The first classifier 320 and the second classifier 330 can be neural networks trained to perform seat belt recognition and / or recognize different situations of seat belt misuse in images. In this case, the first classifier 320 is trained with infrared images and the second classifier 330 is trained with RGB or color images.

[0041] In another embodiment, the first classifier 320 and the second classifier 330 may implement another algorithm suitable for classification problems, such as a support vector machine (SVM), a randomized decision forest, and the like.

[0042] The ranking module 340 has the functionality to assign scores to images. The scores assigned to individual images represent the confidence of the corresponding image for detecting the seat belt status. It represents how credible or meaningful the image is for detecting the seat belt status. The main purpose of the ranking module 340 is to analyze different aspects of the image that are relevant to determining whether the content in the image is likely to be helpful in determining the seat belt status. In an embodiment, the process of assigning scores to images can be based on the analysis of information in the image. The analysis of information in the image allows information to be extracted from the image that is useful for evaluating how reliably the image detects seat belts and / or seat belt status. The ranking module 340 may include one or more functional blocks that are operable to perform different tasks of analyzing the information in the image, such as:

[0043] Block 341 for analyzing statistical properties of the image, such as brightness, contrast, etc.;

[0044] a block 342 for detecting occlusions, which detects if the visibility of the seat belt is occluded in the image (e.g., by an arm);

[0045] a block 343 for identifying a seat belt color profile, which identifies a predetermined profile of seat belt color in the image;

[0046] Block 344 for clothing detection, which detects clothing occluding the back region in the image.

[0047] The sorting module 340 may include other functional blocks for implementing other types of tasks for analyzing information in the image and / or optionally in the vehicle environment. Analysis of information in the environment allows determining current environmental conditions, such as brightness, weather conditions, time of day, etc.

[0048] The sorting module 340 may also include a scoring block 345, which receives the image analysis results from the function blocks 341 to 344 for each image to be scored, and calculates the score to be assigned to the image based on a predetermined rule. The score assigned to each image may correspond to a weighting factor, as explained later. The score of the image is sent to the fusion module 350.

[0049] The fusion module 350 has the function of combining or fusing the probability estimates of the multiple images determined by the first classifier 320 and the second classifier 330 to calculate a combined probability, which will be explained in more detail below. The combined probability can then be sent by the fusion module 350 to the seat belt status detection module 360.

[0050] The seat belt status detection module 360 ​​is responsible for comparing the combined probability with a predetermined detection threshold to detect the seat belt status, which indicates whether the vehicle occupant is wearing the seat belt correctly.

[0051] In an embodiment, the system 100 may further include an image generator 370 operable to:

[0052] Generating a plurality of first images according to the initial IR image from the image acquisition device 200, wherein the plurality of first images, such as IR or NIR images, may be referred to as sub-images, corresponding to different portions of the seat belt area;

[0053] Multiple second images corresponding to different parts of the seat belt area are generated based on the initial RGB or color image from the image acquisition device 200. The multiple second images, such as visible light images, RGB images or color images, correspond to different parts of the seat belt area.

[0054] The first image and the second image may be cropped images from the initial IR image and the initial visible light image, respectively. Figure 5 As shown, the first image and the second image are bounded by a bounding box, which is, for example, a rectangle. The initial image from the image acquisition device 200 may be a front wide-angle image of the interior of the vehicle, showing, for example, two front seats and part of the rear seat of the vehicle. Figure 5 An initial visible light image is shown, and for each of the two occupants sitting in the front seats, N sub-images or second images are cropped from the initial image, where N=4. The number of first images generated from the initial IR image and the number of second images generated from the color image can be the same. However, they can also be different. The sub-images from the initial image can be partially overlapping.

[0055] The system 100 may also include a body key point detection module 380 for detecting and locating body key points of one or more passengers of the vehicle visible in the image. The body key points may include one or more points of the left and right shoulders, hips, elbows, hands and / or faces of the passengers. The detected body key points are intended to be used by the image generator 370 to identify and locate one or more seat belt areas, where the seat belt is expected to be visible in the seat belt area when the seat belt is worn by the passengers in the vehicle. For example, the seat belt area can be defined by an area around a line extending from a given shoulder point (depending on the seat, the left shoulder or the right shoulder) to the opposite hip point of the passenger, which line can be curved.

[0056] The system 100 may also include a selection module 390 for selecting images among all first IR images and all second RGB or color images generated by the image generator 370 based on one initial IR image and one initial RGB image acquired simultaneously or approximately simultaneously, respectively. The task of the selection module 390 is to determine which images from the plurality of images of the first IR image will be sent to the first classifier 320, and which images from the plurality of images of the second RGB image will be sent to the second classifier 330. The purpose of image selection is to limit the amount of computation required for seat belt status detection, since only the selected images are sent to the first classifier 320, the second classifier 330 for classification. Image selection allows saving computational resources, since the tasks performed by the classifiers are performed only on the selected images, and not on the images that are not selected.

[0057] In one example of the detection process or algorithm, the selection module 390 may determine that all first images and all second images may be sent to the corresponding first classifier and second classifier, if the computational budget permits, in other words, if the available computational resources are sufficient. In another example of the process, the selection module 390 may ignore or not select one or more specific images (IR images and / or color images) having the lowest score provided by the ranking module 340 or having a score below a predetermined critical threshold. A low score may indicate that the corresponding image is untrustworthy for seat belt status detection, for example because the position of the image or sub-image in the initial image or the form of the image (IR or RGB) has no information value for detecting the seat belt status at the current point in time.

[0058] The image generator 370 , the body key point detection module 380 , and the selection module 390 for image selection may be implemented by software or a computer program running on the processor 310 .

[0059] exist Figure 2A and Figure 2B A computer-implemented method for seat belt status detection in a vehicle performed by the system 100 is shown in FIG. 1 and will now be described.

[0060] The method includes executing the following detection process or algorithm under the control of the processor 310.

[0061] At a given time point, in step S1 , the image acquisition device 200 acquires an initial IR image IR_IM0 and an initial RGB or color image RGB_IM0 of the same scene in the vehicle, such as a wide-angle view of two front seats and part of the rear seats.

[0062] In step S2 , two images, namely, the initial IR image IR_IM0 and the initial RGB image RGB_IM0 , are sent from the image acquisition device 200 to the processing device 300 .

[0063] In step S3 , the body key point detection module 380 processes each of the initial IR image IR_IM0 and the initial RGB image RGB_IM0 and detects body key points of one or more occupants of the vehicle visible in the initial image, such as left / right shoulders, hips, hands, etc.

[0064] In step S4, the image generator 370 generates a plurality of sub-images from each of the initial images IR_IM0 and RGB_IM0 using the detected body key points provided by the body key point detection module 380. The sub-image in the IR or NIR spectrum is referred to as a "first image", and the sub-image in the RGB or color spectrum is referred to as a "second image". More specifically, based on the body key points detected in each of the initial images IR_IM0, RGB_IM0, the image generator 370 determines a seat belt area for each occupant visible in the initial image, the seat belt area corresponding to the area in the image where the seat belt is expected to be visible when the seat belt is properly worn by the occupant in the vehicle. For example, a bend line extending from the shoulder to the opposite hip can be determined, along which the seat belt is expected to be positioned. In the case where two occupants are seated in the two front seats of the vehicle and are visible in each of the IR image IR_IM0 and the RGB image RGB_IM0, the body key point detection module 380 detects the body key points of the two front occupants, and the image generator 370 determines two corresponding seat belt areas in each of the IR image IR_IM0 and the RGB image RGB_IM0. Then, the image generator 370 generates:

[0065] A plurality of first IR images or sub-images IR_IM0 from an initial IR image IR_IM0 1i …, where i = 1, 2, …, which correspond to different parts of each seat belt area, and

[0066] Multiple second RGB images or sub-images RGB_IM from the initial RGB image RGB_IM0 2j …, where j=1, 2, …, which correspond to different parts of each seat belt area.

[0067] The sub-images may be positioned adjacent to each other along a line defining the seat belt area.

[0068] Alternatively, the image generator 370 may use the detected body key points to determine a buckle region in each initial image IR_IM0 and RGB_IM0 for each occupant in the image, which is expected to include a seat belt buckle assembled with a buckle receiver. In this case, the image generator may also generate a first IR image or sub-image corresponding to the curved region from the initial IR image IR_IM0, and generate a second RGB image or sub-image corresponding to the curved region in the initial RGB image RGB_IM0.

[0069] The first image may be cropped from the initial IR image IR_IM0, and the second image may be cropped from the initial RGB image RGB_IM0. For example, the cropped images may be bounded by a bounding box (e.g., a rectangle), such as Figure 5 This allows limiting the processing to the relevant parts of the initial image.

[0070] Generate a first IR image or sub-image IR_IM from the initial IR image and from the initial RGB image respectively 1i and a second RGB image or sub-image RGB_IM 2j Not limited to the use of body key points. In another embodiment, the position or orientation of the first IR image or sub-image and the second RGB image or sub-image is based on a fixed position in the initial image, or other parameters suggesting which position in the image is most informative for the extraction of the seat belt status.

[0071] In a variation, the image generator 370 may generate one sub-image or first image based on the initial IR image IR_IM0 and one sub-image or second image based on the initial color image RGB_IM0.

[0072] In step S5, the sorting module 340 performs image analysis on each of the images (including the first IR image and the second RGB image) generated by the image generator 370 to extract information from each of the generated images. Therefore, the sorting module 340 can analyze different aspects in each of the generated images to assign scores to the analyzed images, and the different aspects are related to determining whether the content in the analyzed image is likely to help determine the seat belt status. The sorting module 340 can perform one or more tasks of analyzing information in the image, such as:

[0073] Statistical analysis of image characteristics, such as brightness, contrast, etc.

[0074] Occlusion detection, used to detect whether a seat is not visible because it is blocked by an object;

[0075] seatbelt color profile recognition to determine if the seatbelt in the image matches a predetermined color profile of the vehicle's seatbelt;

[0076] Clothing detector, used to detect clothing (e.g., jackets) that occlude the back area in an image.

[0077] The task of analyzing the information in the image may be performed by a digital image processing unit.

[0078] Then, in step S6, the ranking module uses the information extracted through the image analysis to assign scores to the individual images generated by the image generator 370. The extracted information may include the results of the information analysis task.

[0079] In step S7, the selection module 390 selects all first IR images IR_IM generated by the image generator 370 based on the scores. 1i and the second RGB image RGB_IM 2j The image is selected from the IR images. For example, images with scores above a predetermined threshold are selected, or the top p IR images with the highest scores are selected, where p is a predetermined number. The selection may be configurable. For example, it may be adjusted according to available computing resources and / or computing budget. The fewer computing resources are available, the more important the selection is. The selected images may include one or more first IR images IR_IM 1i and one or more second RGB images RGB_IM 2j In some cases, all generated images IR_IM can be selected 1i and RGB_IM 2j It may happen that only one or more first IR images IR_IM are selected. 1i , and not the second RGB image, and vice versa. Performing image selection among all generated sub-images also allows dynamically determining which position in the initial IR and color images is more reliable in the current situation.

[0080] Then, in step S8, only the selected images are transferred to the first classifier 320 and the second classifier 330. More specifically, when the selected images include one or more first IR images IR_IM 1i and one or more second RGB images RGB_IM 2j When mixing, the selected one or more first IR images IR_IM 1i is sent to the first classifier 320, and the selected one or more second RGB images RGB_IM 2j is sent to the second classifier 330 .

[0081] In step S9, the first classifier 320 processes each received first IR image and determines a first probability estimate that the first IR image includes a properly worn seat belt.

[0082] In step S10 , the second classifier 330 processes the respective received second RGB images simultaneously and determines a second probability estimate that the second RGB image includes a properly worn seat belt.

[0083] Then, in step S11, the fusion module 350 combines the first probability estimate and the second probability estimate determined by the first classifier 320 and the second classifier 330 for the received first IR image and the second RGB image using the corresponding scores assigned to the images, each score indicating how reliable the corresponding image is for detecting the seat belt, and determines the combined probability. In an embodiment, the probability estimates of different IR or RGB images can be weighted using the corresponding scores of the images. For example, the combined probability is calculated using the following formula:

[0084]

[0085] Among them, pcombinations is the combination probability; α i is the score assigned to the image with index i; p i is the probability estimate of image index i determined by the corresponding classifier.

[0086] Alternatively or additionally, the fusion module 350 can use a predetermined rule to combine the probability estimates of the at least one first image and the at least one second image. For example, the predetermined rule can be based on experimental and / or statistical knowledge about which sub-image combinations are more trustworthy (in other words, better at detecting the seat belt status) than other sub-image combinations.

[0087] Then, in step S12, the detection module 360 ​​compares the combined probability with a predetermined detection threshold to determine the seat belt status detection result. Figure 2B If the branch in step S13 is "yes", it is detected that the seat belt is worn correctly and the seat belt state is fastened. If the combined probability is less than the threshold, see Figure 2B If the branch in is "No", it is detected in step S14 that the seat belt is not worn correctly, and the seat belt status is not fastened or misused.

[0088] The detection threshold may be configurable. For example, it may be adjusted depending on the level of reliability required for the detection. The more reliable the detection, the higher the detection threshold.

[0089] The above process including steps S1 to S12 may be repeated at successive time points, for example iteratively or cyclically.

[0090] In this case, the result or output of the combining step S11 can be stabilized through a feedback loop, where the scores of the first IR image and the second RGB image, as well as the outputs of the first classifier 320 and the second classifier 330, are provided as inputs for the next execution of the process.

[0091] The step S4 of generating sub-images based on the respective initial images in IR and visible light is optional. The step S7 of image selection is also optional.

[0092] In an embodiment, the selection of sub-images in the initial images is not based on image scores, but on the positions of the sub-images in the initial images, and the positions of the selected images change in each cycle or for each new pair of initial images. The purpose of this sub-image selection is to reduce the computational resources required for seat belt status detection.

[0093] We consider the following illustrative and non-limiting examples described below:

[0094] Step S1 is executed iteratively or cyclically such that the image acquisition device 200 acquires a sequence of pairs of initial images {IR_IM0(t0), RGB_IM0(t0)}, {IR_IM0(t1), RGB_IM0(t1)}, {IR_IM0(t2), RGB_IM0(t2)}, {IR_IM0(t3), RGB_IM0(t3)}, {IR_IM0(t4), RGB_IM0(t4)},... including IR images and color images at consecutive points in time t0, t1, t2, ….

[0095] In each cycle, after step S1, steps S2 to S6 are executed as described above. In step S4, N first images or sub-images are generated based on the initial IR image, and N second images or sub-images are generated based on the initial color image. The N first / second images correspond to N different sub-image positions pos1, …, pos N . For example, for the initial color image, as Figure 5 shown, N = 4.

[0096] After step S6, the selection step S7 is executed. In the selection step S7, M first images are selected from the N first images and M second images are selected from the N second images, where M < N. For example, M = 2. And the positions of the sub-images of the selected images change in each cycle, for example, by a shift of one position. All sub-image positions can be selected after several cycles. For example, the selection is as follows:

[0097] At t0: Sub-images are selected at positions pos1, pos2;

[0098] At t1: select a sub-image at positions pos2, pos3;

[0099] At t2: select a sub-image at positions pos3, pos4;

[0100] At t3: select a sub-image at positions pos4, pos1;

[0101] At t4: select a sub-image at positions pos1, pos2;

[0102] At t5: select a sub-image at positions pos2, pos3;

[0103] And so on.

[0104] The present disclosure also relates to:

[0105] A computer program or instruction code corresponding to a software program, which includes instructions for causing the seat belt status detection system 100 to perform the steps of the previously disclosed method;

[0106] A computer-readable medium storing the above-mentioned computer program;

[0107] A vehicle includes a seat belt status detection system 100 .

[0108] As previously described, seat belt status detection uses one or more first images in the infrared spectrum and one or more second images in the visible light spectrum, which are sent to a first classifier dedicated to images in the infrared spectrum and a second classifier dedicated to images in visible light, respectively. More generally, seat belt status detection can use one or more first images in a first spectrum and one or more second images in a second spectrum different from the first spectrum, which are sent to a first classifier dedicated to images in the first spectrum and a second classifier dedicated to images in the second spectrum, respectively. The first spectrum and the second spectrum are two different parts of the electromagnetic spectrum, preferably two separate parts of the electromagnetic spectrum.

Claims

1. A computer-implemented method for detecting the state of a seat belt in a vehicle, the method comprising the following steps performed by a processing device (300): At least one first image (IR_IM) of at least a portion of the seat belt area is obtained from the image acquisition device (200) in the infrared spectrum. 1i ), in which the seat belt is intended to be visible when the seat belt is worn by an occupant of the vehicle; Obtaining from the image acquisition device (200) a method for obtaining the at least one first image (IR_IM 1i ) of at least a portion of the seat belt area in the visible light spectrum (RGB_IM 2j ); An image analysis step (S5) is used to obtain the at least one first image (IR_IM 1i ) and the at least one second image (RGB_IM 2j ) to extract information from each of them; A score assignment step (S6) for assigning a score to each of the at least one first image and the at least one second image by calculating a score to be assigned to the image according to a predetermined rule based on the information extracted from the corresponding image, wherein each score indicates the credibility of the corresponding image for detecting the seat belt state; The at least one first image (IR_IM 1i ) sending (S8) to a first classifier (320) dedicated to images in said infrared spectrum; The at least one second image (RGB_IM 2j ) sending (S8) to a second classifier (330) dedicated to images in said visible light spectrum; Determine (S9) the at least one first image (IR_IM 1i ) includes a first probability estimate of a seat belt being correctly worn; The at least one second image (RGB_IM 2j ) includes a second probability estimate of the seat belt being correctly worn; Using the at least one first image (IR_IM 1i ) and assigning the at least one second image (RGB_IM 2j ) to combine (S11) the first probability estimate and the second probability estimate to determine a combined probability; Comparing (S12) the combined probability with a predetermined threshold to determine (S13, S14) the seat belt state.

2. The method according to claim 1, wherein: The image analysis step includes, for each image, one or more of the following image analysis tasks performed by a digital image processing unit: Analyzing the statistical characteristics of the image; Detecting occlusion to detect that the visibility of the seat belt in the image is occluded; Identifying a seat belt color profile to identify a predetermined profile of the seat belt color in the image; Detecting clothing to detect clothing that occludes the back region in the image.

3. The method according to claim 1 or 2, wherein: The step of obtaining the at least one first image comprises acquiring (S1) an initial infrared image (IR_IM0) of the interior of the vehicle by the image acquisition device (200), and generating (S4) a plurality of first images (IR_IM0) of different parts of the seat belt area from the initial infrared image (IR_IM0). 1i ), and the step of obtaining the at least one second image comprises acquiring (S1) an initial visible light image (RGB_IM0) of the interior of the vehicle by the image acquisition device (200), and generating (S4) a plurality of second images (RGB_IM0) of different parts of the seat belt area from the initial visible light image (RGB_IM0). 2j ).

4. The method according to claim 3, wherein: The step of obtaining the at least one first image further includes selecting one or more of the plurality of first images (IR_IM0); the step of obtaining the at least one second image further includes selecting one or more of the plurality of second images (RGB_IM0); and only the selected first images and the selected second images are respectively sent to the first classifier (320) and the second classifier (330).

5. The method according to claim 4, wherein: The selection is based on the scores assigned to the first image and the second image.

6. The method according to claim 3, the method further comprising: Detecting (S3) body key points visible in each of the initial infrared image (IR_IM0) and the initial visible light image (RGB_IM0) of one or more occupants of the vehicle; In each of the initial infrared image (IR_IM0) and the initial visible light image (RGB_IM0), determining (S4) the seat belt region based on the detected body key points.

7. The method according to claim 4, wherein: Cyclically repeating the steps of acquiring the initial infrared image (IR_IM0) and acquiring the initial visible light image (RGB_IM0), and in each cycle, generating N first images from the initial infrared image and N second images from the initial visible light image, and then in the selection, selecting M first images from the N first images and M second images from the N second images, where M < N, and wherein the image positions of the selected images change in each cycle.

8. The method according to claim 7, wherein: Stabilizing the result of the combined step through a feedback loop, wherein the scores of the at least one first image and the at least one second image and the outputs of the first classifier and the second classifier are provided as inputs for the next execution of the cycle.

9. The method according to claim 1, wherein: In the step of combining (S11), the at least one first image (IR_IM 1i ) and the at least one second image (RGB_IM 2j ) are weighted to calculate the combined probability.

10. The method according to claim 1, wherein: The combined step combines the probability estimates of the at least one first image and the probability estimates of the at least one second image using a predetermined rule.

11. A seat belt status detection system for a vehicle, the seat belt status detection system comprising: An image acquisition device (200) operable to acquire a first image in an infrared spectrum and a second image in a visible light spectrum; Processing device (300) operable to perform the steps of the method according to any one of claims 1 to 10. 12 . A computer program product, comprising instructions for causing the seat belt status detection system according to claim 11 to execute the steps of the method according to any one of claims 1 to 10.

13. A computer-readable medium storing the computer program product according to claim 12. 14 . A vehicle, comprising the seat belt status detection system according to claim 11 .

Images