Systems and methods for vehicle monitoring using machine vision and encoded surface patterns

Abstract

Vehicle systems for monitoring vehicle occupancy using encoded surface patterns are described herein. A system may include: at least one image sensor (e.g. camera) positioned within the vehicle configured to detect various encoded surface patterns; and at least one processor operatively coupled to the at least one image sensor that is configured to determine at least one characteristic of at least one occupant and / or the at least one vehicle component or surface based at least in part on the detected encoded surface pattern.

Description

^ ^ Attorney Docket No. 10213-286WO1SYSTEMS AND METHODS FOR VEHICLE MONITORING USING MACHINE VISION AND ENCODED SURFACE PATTERNS CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 609,559, entitled “SYSTEMS AND METHODS FOR VEHICLE MONITORING USING MACHINE VISION AND ENCODED SURFACE PATTERNS”, filed on December 13, 2023, the content of which is incorporated by reference herein in its entirety. BACKGROUND

[0002] Seat belts are standard equipment for almost every kind of vehicle in which occupants are transported in today's transportation systems. Not only are original equipment manufacturers (OEMs) required to meet strict standards for seat belt engineering and installation, but in many scenarios, vehicle occupants are required to wear seat belts as a matter of law. Even with manufacturing regulations and use laws in place, however, overall vehicle safety is entirely dependent upon vehicle occupants using seat belts properly. Visual inspection by outside authorities is not completely reliable given that a vehicle interior is only partially visible from outside of a vehicle. Individuals attempting to circumvent seat belt use laws also position seat belts inside a vehicle in a way that gives an appearance of seat belt use but allows the vehicle occupant more latitude in range of movement (i.e. fastening the seat belt behind the user's back or pulling the seat belt only partially across the user's body and manipulating the seat belt spool to maintain the seat belt in an extended position without requiring a fixed latching). BRIEF SUMMARY OF THE DISCLOSURE

[0003] State-of-the-art seatbelts are designed to meet occupant restraint, durability, environmental, and aesthetic requirements. However, trends towards the use of active and passive electromagnetic occupant monitoring in vehicle, particularly infrared cameras, provide an opportunity for augmenting seatbelts with deterministic electromagnetic tuned patterns (e.g. infrared (IR) reflecting / absorbing patterns for cameras) which are designed to improve the ability to detect, classify and track features of the seatbelt system. However, the inclusion of such patterns should not compromise the current state-of-the-art functions or properties of the belt or ^^ ^ Attorney Docket No. 10213-286WO1vehicle interior fabrics. The patterns should be easily detectable by the sensor system but not detectable by the human occupant and fulfill aesthetic requirements.

[0004] Embodiments of the present disclosure relate to creation and applications of a-priori designed repeating pseudo-random patterns responsive in a specific electromagnetic region (e.g. Visible, IR, ultraviolet (UV), etc.). Implementations may use state-of-the-art dyed threads formed into patterns by weaving machinery such that the patterns are configured to optimize their detection with imaging equipment. This disclosure incorporates using fabrics, woven materials, other textiles or even films incorporating patterned sections that are detectable upon illumination with particular kinds of electromagnetic radiation of selected frequency ranges. Accordingly, embodiments herein may be configured for use in conjunction with classification and tracking systems that utilize properties of occupant monitoring systems (e.g. camera center wavelength and pass band, resolution, longitudinal and lateral field of view, range, frame rate, etc.). In non-limiting embodiments, the a-priori pattern design may be developed to be non- discernible to human occupants and yet retain information which can be used by a remote sensing system to discern information about vehicle components. For example, seat belt geometry and other vehicle components can be monitored for position and / or motion dynamics and thus infer information about activities within the vehicle cabin and occupants within the cabin. In an analogous manner, patterns that are woven into or printed onto any vehicle interior surface within a sensing field of a camera can include patterns which can be used by machine vision algorithms to gain information about the cabin and occupant environment (e.g. location / size of occupants, objects, position and orientation of adjustable seats, trim, etc.).

[0005] The patterns may typically be designed based on the limitations of the state-of-the-art threads, looms and post-processing currently used for seatbelt weaving. Alternatively, the patterns could be achieved through thermal or electrostatic transfer printing techniques and thus adhere to the limitations of the state-of-the-art fabric surface printing techniques.

[0006] Embodiments of the present disclosure optimize woven fabric patterns for detection, classification and tracking digital encoding patterns to support occupant monitoring performance, for example, monitoring patterns in seatbelt webbing, airbag fabrics, trim piece fabrics, and other vehicle components. In some examples, such patterns are not detectable by human perception. Example encoded surface patterns may minimize and randomize effects of contrast thread in a baseline matrix thus normalizing and balancing strength and durability of the 2 ^^ ^ Attorney Docket No. 10213-286WO1final fabric. Similarly, such techniques can be applied to thermally printed, painted, or cast surfaces using state-of-the-art manufacturing processes. In some implementations, quick response (QR) codes and machine vision are used for encoding detailed information in human visually imperceptible pseudo-random patterns.

[0007] In some implementations, a system for occupancy monitoring and sensing within a vehicle is provided. The system can include: at least one vehicle component or surface (e.g. a seatbelt) including an encoded surface pattern; at least one image sensor (e.g. a camera) positioned within the vehicle configured to detect the encoded surface pattern; and at least one processor operatively coupled to the at least one image sensor that is configured to determine at least one characteristic of at least one occupant and / or the at least one vehicle component or surface based at least in part on the detected encoded surface pattern (e.g. based on a location of the encoded surface pattern in 2D or 3D space).

[0008] In some implementations, the encoded surface pattern is detected or analyzed using a two-dimensional (2D) or three-dimensional (3D) machine vision segmentation operation and / or machine learning algorithm.

[0009] In some implementations, the machine vision segmentation operation is used to determine at least one of: an estimated belted occupant height, weight, body morphology, respiration patterns, movement patterns, and occupant position (e.g. sitting in-position, leaning forward, leaning side to side, legs not on the floor, occupant lying down on seat or multiple seats, occupant slouching).

[0010] In some implementations, the at least one processor is further configured to: determine a confidence value in relation to the at least one characteristic based at least in part on the detected encoded surface pattern.

[0011] In some implementations, the at least one processor is further configured to: in response to determining that the confidence value meets or exceeds a predetermined threshold or range, transmit a control signal.

[0012] In some implementations, the control signal is configured to: generate an alert (e.g. via a display, audible) or verify an expected system occupancy / monitoring output. ^^ ^ Attorney Docket No. 10213-286WO1

[0013] In some implementations, the encoded surface pattern includes a pattern that reflects or absorbs a predetermined wavelength range at a preferred luminance (e.g. a narrow infrared band).

[0014] In some implementations, the at least one characteristic includes at least one of a location and / or size of the at least one occupant, location and / or size of a vehicle object, a seat position and / or orientation.

[0015] In some implementations, the at least one characteristic is determined based at least in part on a detected change (e.g. partial view) to the encoded surface pattern.

[0016] In some implementations, the encoded surface pattern includes at least one of a machine- readable code, a digital code (e.g. QR code, barcode), and a deterministic pseudo-random code.

[0017] In some implementations, the encoded surface pattern further includes a seatbelt weaving pattern (e.g. Twill, World Twill, 4 pane, 5 pane) or fabric.

[0018] In some implementations, the at least one vehicle component includes a seatbelt, and wherein the detected encoded surface pattern is used to determine at least one of the position of the seatbelt or incorrect fitment or placement of the seatbelt (e.g. for belt twist detection).

[0019] In some implementations, the encoded surface pattern is configured to facilitate machine vision segmentation (e.g.2D or 3D separation) of an occupant from the at least one vehicle component (e.g. seatbelt) or for detection and tracking of occupant movement relative to seatbelt or other vehicle surfaces (e.g. moving torso, arms, hands, head, legs, or the like).

[0020] In some implementations, the at least one vehicle component or surface includes a Child Restraint System (CRS), and wherein the encoded surface pattern is configured to facilitate machine vision segmentation in relation to the CRS (e.g. is the CRS in the correct position, is it forward facing, rear facing, booster) and / or a seatbelt (e.g. is the seatbelt attached to the CRS for fixation).

[0021] In some implementations, the at least one vehicle component or surface includes a seating surface, and wherein the encoded surface pattern is used to determine at least one of a position and orientation relative to other patterns on other textile surfaces and / or objects in the vehicle (e.g. headliner, floor, door surfaces, bags, luggage). ^^ ^ Attorney Docket No. 10213-286WO1

[0022] In some implementations, a system for occupancy monitoring and sensing within a vehicle is provided. The system can include: a plurality of vehicle components or surfaces, wherein each vehicle component or surface includes a corresponding encoded surface pattern; at least one image sensor (e.g. camera) positioned within the vehicle configured to detect each encoded surface pattern; and at least one processor operatively coupled to the at least one image sensor that is configured to detect and track at least one occupant of the vehicle and / or a respective status of each of the plurality of vehicle components or surfaces based at least in part on each detected encoded surface pattern.

[0023] In some implementations, each encoded surface pattern is detected or analyzed using a 2D or 3D machine vision segmentation operation or technique and / or machine learning algorithm.

[0024] In one embodiment, a computer-implemented method is provided. The computer- implemented method can comprise: detecting, by at least one processor and using an image sensor positioned within a vehicle, at least one encoded surface pattern corresponding with a vehicle component or surface; determining, by the at least one processor, at least one characteristic of at least one occupant and / or at least one vehicle component or surface based at least in part on the at least one detected encoded surface pattern; determining, by the at least one processor, a confidence value in relation to the at least one determined characteristic; and responsive to determining that the confidence value meets or exceeds a predetermined threshold or range, generate a control signal or generate an alert to verify an expected system occupancy / monitoring output.

[0025] In some implementations, the at least one encoded surface pattern is detected using a 2D or 3D machine vision segmentation operation and / or machine learning algorithm.

[0026] In some implementations, the at least one encoded surface pattern is configured to facilitate machine vision segmentation of an occupant from the at least one vehicle component or for detection and tracking of occupant movement relative to seatbelt or other vehicle surfaces. ^^ ^ Attorney Docket No. 10213-286WO1BRIEF DESCRIPTION OF THE DRAWINGS

[0027] These and other features, aspects, and advantages of the present disclosure will become apparent from the following description and the accompanying exemplary embodiments shown in the drawings, which are briefly described below.

[0028] FIG. 1 is a schematic illustration of a vehicle interior having seats installed within respective fields of view of cameras in the vehicle in accordance with embodiments set forth herein.

[0029] FIG. 2A shows a basic twill pattern in accordance with embodiments set forth herein.

[0030] FIG. 2B shows example weaving patterns in accordance with embodiments set forth herein.

[0031] FIG. 2C shows example random woven patterns in accordance with embodiments set forth herein.

[0032] FIG. 3A illustrates an example standard seatbelt webbing.

[0033] FIG. 3B is an image showing the appearance of an example seatbelt with a reflective encoded surface pattern.

[0034] FIG. 3C is another image showing a seatbelt with highly absorbing contrasting thread.

[0035] FIG. 4 is a schematic illustration showing example QR codes in accordance with embodiments of the present disclosure.

[0036] FIG. 5A, FIG.5B, and FIG. 5C illustrate example methods for generating encoded surface patterns in accordance with certain embodiments described herein.

[0037] FIG. 6 is a flowchart diagram illustrating example operations in accordance with various embodiments of the present disclosure. DETAILED DESCRIPTION

[0038] The figures illustrate the exemplary embodiments in detail. However, it should be understood that the application is not limited to the details or methodology set forth in the description or illustrated in the figures. It should also be understood that the terminology is for the purpose of description only and should not be regarded as limiting. ^^ ^ Attorney Docket No. 10213-286WO1

[0039] Sensor(s) can be used to detect positions of numerous components of a seat belt assembly and track seat belt use within a vehicle. In one embodiment, the sensor is an active optical 3D time of flight imaging system which emits a known waveform (e.g. sinusoidal, pseudo-random, and the like) of electromagnetic wavelength(s) of light which are collocated and / or synchronized with a 2D imager detector array, where the amplitude of the detected signal is proportional to the reflected light at the light wavelength(s). Such a sensor can collect both the reflected light intensity of surfaces in the field of view of the imager and the distance of the surface from the imager detector.

[0040] The light is emitted and hits the surface of all objects within a line of sight. As a function of the geometric arrangement and compositional materials of the object, a portion of the light is reflected back towards an imager detector array. Signal processing of the detected signals can be used to reconstruct 3D information (intensity image and depth image) which can be used in machine vision algorithms to detect, and / or classify, and / or track information about the objects within the scene. In one non-limiting example embodiment, the light source target wavelength may be selected as 950 nm, and the source of the selected light could be a light emitting diode (LED) array or vertical-cavity surface-emitting laser(s) (VCSEL) with dispersion / filtering optics to disperse light within a known spatial area. Without limiting this disclosure to one kind of equipment set-up, the imager array may be, for example, a silicon multi-pixel array synchronized and sensitive to the above-described 950 nm light emitted from a corresponding light source. However, the sensor and associated sources and detectors could also be based on other electromagnetic methods such as passive optical imagers (2D, using ambient lighting) radar, ultrasonic, microwave, and numerous detection technologies for use with other electromagnetic wavelengths as are necessary for the selected sensor and detector.

[0041] In example embodiments, the seat belt material and / or the mechanical mounting of the seat belts (e.g. seat belt payout aperture) and / or mechanical features on the seat belt (e.g. d-rings, retention buttons, etc.) are composed of materials and / or augmented with an appropriate encoded surface pattern such that features within the pattern have a controlled, deterministic reflectivity in the sensor wavelength region(s). For example, the seatbelt material can be coated (or sewn or woven) with an interchanging pattern of high and low reflectivity materials at the selected sensor wavelength(s). The encoded surface pattern can be selected to provide improved ability to detect and track information about a seat belt, or any other vehicle component, by either visual ^^ ^ Attorney Docket No. 10213-286WO1inspection or by image detection in an automated computer vision system. Machine vision methods can be optimized to detect, classify, and track these patterns. Pattern features may be selected for optimal contrast to detect / track extent of seat belt payout, depth of seat belt, and other comparative data sets, such as which belt position is the closest position to the camera (e.g. to identify the occupant's chest). Embodiments described herein detect, monitor, and / or track seat belt payout apertures and seat belt patterns, wherever located in an image created from a camera field of view. For example, these encoded surface patterns can be located in seats, on roofs or in vehicle side structures to detect positions of seat belts or portions thereof relative to occupant body anatomy (e.g. shoulder / head). In cases where a vehicle component, such as a belt, may be obscured by occupant appendages or objects brought into a vehicle by the occupant, (e.g. clothing, blankets, luggage, cargo, or anything that the occupant places over an expected area for a seat belt), obscured portions of vehicle components and safety equipment can be accounted for in this system. The system and methods described herein also identify reference points for vehicle components and safety equipment within a space that is significantly less likely to be obscured in a vehicle, providing known structures from which to evaluate seat belt and other vehicle component use and operation. By identifying reference structures that are visible within a vehicle, the system and methods disclosed herein take advantage of partially visible portions of vehicle components, seats, a seat belt assembly and the like along with occupant classification methods, to predict proper or improper occupant position and seat belt use. The detailed description below explains more embodiments of the methods and systems for seat belt monitoring in accordance the figures referenced therein.

[0042] In some embodiments, a system for occupancy monitoring and sensing within a vehicle includes at least one vehicle component or surface (e.g. seatbelt) comprising an encoded surface pattern, at least one image sensor (e.g. camera) positioned within the vehicle configured to detect the encoded surface pattern, and at least one processor operatively coupled to the at least one image sensor that is configured to determine at least one characteristic of at least one occupant and / or the at least one vehicle component or surface based at least in part on the detected encoded surface pattern (e.g. based on a location of the encoded surface pattern in 2D or 3D space). In some examples, the encoded surface pattern is detected using a 2D or 3D machine vision segmentation operation and / or machine learning algorithm. In some implementations, multiple patterns can be embedded simultaneously. For example, a first pattern can be reflective ^^ ^ Attorney Docket No. 10213-286WO1at 950 nm (i.e. infrared) and a second pattern can be reflective at 680 nm (i.e. red visible light). The machine vision segmentation operation or machine learning algorithm can facilitate determining one or more of an estimated belted occupant height, weight, body morphology, seatbelt fitment (e.g., tight fit, loose fit, improper position across chest), respiration patterns, movement patterns, and occupant position (e.g. sitting in-position, leaning forward, leaning side to side, legs not on the floor, occupant lying down on seat or multiple seats, occupant slouching). In some embodiments, using a two-dimensional (2D) imaging array that is configured to detect encoded surface patterns on or within vehicle components provides for faster computation steps in computerized implementations of this disclosure, as compared to the above noted three- dimensional imaging systems. Accordingly, this disclosure takes advantage of the efficiencies allowed in using QR codes and other detectable patterns with simpler computational requirements and hardware.

[0043] FIG. 1 is an overview schematic of a vehicle according to this disclosure including rows of seats 13A, 13B, 13C within the interior 10, or cabin, of a vehicle. The term “vehicle” as used herein includes all of the broadest plain meanings for the term within the context of transportation (i.e. any references to an automobile are for example purposes only and do not limit this disclosure to any one embodiment). The vehicle of FIG.1 incorporates a driver's seat 13A adjacent a steering wheel 19 and a common driver's control panel 17 (possibly including a viewing screen). The vehicle control system is not shown separately but would include processors, memory, electronic circuits, and sensors necessary to establish a safe driving environment in the vehicle interior 10. The computers 27 in the vehicle may communicate with occupant classification systems 21 used to determine the entry / exit location, anatomy, age, adult / child / infant status, and other quantitative characteristics of each occupant in the vehicle. The vehicle of FIG.1 would typically include standard OEM equipment such as seat belt assemblies shown in more detail in other figures. The vehicle of FIG.1, however, illustrates installation of image sensors, i.e. cameras 12A, 12B, 12C having respective light sources 16A, 16B, 16C and positioned in the vehicle interior 10 to establish respective fields of view of occupants, seats (13A, 13B, 13C), seat belt assemblies (18A 18B, 18C), and other structures in the vehicle. In this non-limiting example, the cameras / image sensors 12A, 12B, 12C have been installed on the ceiling 15 of the vehicle and atop the driver's control panel 17. The vehicle includes the associated circuitry to connect the cameras 12A, 12B, 12C, light sources 16A, 16B, ^^ ^ Attorney Docket No. 10213-286WO116C, and associated arrays / sensors (hereinafter “image sensors” to a vehicle control system operating via a computer bank 11.

[0044] FIG. 2A, FIG.2B, and FIG. 2C depict example encoded surface patterns in accordance with embodiments described herein.

[0045] FIG. 2A shows a basic twill pattern.

[0046] FIG. 2B shows example weaving patterns. In particular, FIG.2B shows the following weaves: a plain weave (1), twill 2 / 1 (2), basket 2 / 1 (3), twill 3 / 1 (4), twill 2 / 2 (5), warp satin (4 healds) (6), basket 2 / 27, warp satin (6 healds) (8), twill 5 / 11 / 11 / 11 / 1 (9), leno 3 / 3 (10), twill 7 / 11 / 11 / 1 (11), twill 4 / 21 / 11 / 11 / 1 (12), weave based on plain weave (13), diamond specular broken twill 1 / 13 / 2 (14), diamond negative broken twill 1 / 12 / 2 (15).

[0047] FIG. 2C shows random woven patterns. In various implementations, encoded surface patterns, for example, a repeating “pseudo-random” pattern can be applied where the pattern has further defined rules which are based on the development of surface features which are optimally visible to a remote sensor. An encoded surface pattern can be a code or string indicating a component and corresponding attributes (e.g. location). An example encoded surface pattern can embed the following information: “[I am the] front seat passenger side seatbelt for VIN: XXXXXXXXXXXXX”. In certain implementations, an encoded surface pattern can be used for authentication of surface materials and may indicate that a surface material is an original OEM material or an aftermarket material, for example. The selected patterns are designed based on the remote sensor specifications (e.g. camera location, bore site angle, field of view, resolution, and / or the like) and the position and geometric degrees of freedom of the vehicle component (e.g. seatbelt or other vehicle interior fabric). Desired pseudo-random patterns can be achieved by: 1) using a baseline state-of-the-art thread (e.g. for infrared (IR) camera in 940 nanometers (nm), highly reflecting, but in a visible light wavelength yielding a black appearance (i.e. threading that meets all regulatory requirements), and 2) using a highly contrasting thread woven in proximity to the surface of the baseline thread in a deterministic pseudo-random pattern, as discussed in more detail below.

[0048] FIG. 3A illustrates an example standard seatbelt webbing (baseline) showing the appearance of a seatbelt 302 based on a standard visible light camera. FIG.3B is an image 300 showing the appearance of the seatbelt 303 for a 940 nm center wavelength infrared camera with ^^ ^ Attorney Docket No. 10213-286WO1active 940 nm IR illumination. The appearance of red pixels in FIG.3B shows areas of reflective intensity which saturate the imager, which means the reflected light intensity is higher than the processing range of the analog to digital converter. Any detected light intensity that is greater than the designed maximum will saturate the imager and thus be assigned the maximum digital value supported by the sensor system (for example, if the supported digital range is 8 bit (min = 0, max= 256), the saturation level will be 256. In contrast, the non-saturated segments of the seatbelt appear white. FIG. 3C is another image 301 showing a seatbelt 305 with highly absorbing contrasting thread.

[0049] Established state-of-the-art machine vision pattern matching techniques are described in “Application of artificial neural network in determining the fabric weave pattern” by Das et al. Zastita Materijala 63 (3) 291 - 299 (2022) (https: / / doi.org / 10.5937 / zasmat2203291D), the content of which is incorporated by reference herein in its entirety. Such techniques are used in industries such as quality control manufacturing to verify the accuracy and precision of manufactured textile patterns to target patterns (e.g. through machine learning such as neural networks) and associated template pattern development a-priori, training of detection algorithms and associated performance validation. In a similar manner, these techniques can be adapted and customized for the remote sensor occupant monitoring system (e.g. infrared camera) used to survey a vehicle cabin interior occupants, and seating, and occupant and restraint geometries can be simulated a-prior based on intended vehicle degrees of freedom for certain components. The methods, apparatuses, and systems of this disclosure, therefore, select pseudo-random fabric patterns (e.g. seatbelt compositions) which maximize the probability of pattern detection, classification and tracking of this component while minimizing the contrast thread required and also meeting aesthetic requirements. This disclosure also includes using valuable information that may be gleaned from encoded surface patterns that are only partially visible due to obstructions or positioning (e.g., a twisted seat belt might leave only a portion of a QR code visible for decoding). As discussed further below, computerized implementations of this disclosure may utilize artificial intelligence to not only decode patterns and portions of patterns, but also to ensure that the detected information is accurately assessed for further action, such as by the use of confidence levels for all calculations. Accordingly, any calculations that do not meet threshold levels of confidence for determining vehicle component positions and safety measurements can by properly addressed by overall vehicle control systems. For example, upon ^^ ^ Attorney Docket No. 10213-286WO1detecting a lack of confidence in certain measurements, the control systems may default to fail safe instructions.

[0050] Artificial Intelligence and Machine Learning

[0051] The term “artificial intelligence” is defined herein to include any technique that enables one or more computing devices or computing systems (i.e. a machine) to mimic human intelligence. Artificial intelligence (AI) includes, but is not limited to, knowledge bases, machine learning, representation learning, and deep learning. The term “machine learning” is defined herein to be a subset of AI that enables a machine to acquire knowledge by extracting patterns from raw data. Machine learning techniques include, but are not limited to, logistic regression, support vector machines (SVMs), decision trees, Naïve Bayes classifiers, and artificial neural networks. The term “representation learning” is defined herein to be a subset of machine learning that enables a machine to automatically discover representations needed for feature detection, prediction, or classification from raw data. Representation learning techniques include, but are not limited to, autoencoders. The term “deep learning” is defined herein to be a subset of machine learning that enables a machine to automatically discover representations needed for feature detection, prediction, classification, etc. using layers of processing. Deep learning techniques include, but are not limited to, artificial neural network or multilayer perceptron (MLP).

[0052] Machine learning models include supervised, semi-supervised, and unsupervised learning models. In a supervised learning model, the model learns a function that maps an input (also known as feature or features) to an output (also known as target or targets) during training with a labeled data set (or dataset). In an unsupervised learning model, the model learns patterns (e.g. structure, distribution, etc.) within an unlabeled data set. In a semi-supervised model, the model learns a function that maps an input (also known as feature or features) to an output (also known as target or target) during training with both labeled and unlabeled data.

[0053] Artificial Neural Networks: An artificial neural network (ANN) is a computing system including a plurality of interconnected neurons (e.g. also referred to as “nodes”). This disclosure contemplates that the nodes can be implemented using a computing device (e.g. a processing unit and memory as described herein). The nodes can be arranged in a plurality of layers such as input layer, output layer, and optionally one or more hidden layers. An ANN having hidden ^^ ^ Attorney Docket No. 10213-286WO1layers can be referred to as deep neural network or multilayer perceptron (MLP). Each node is connected to one or more other nodes in the ANN. For example, each layer is made of a plurality of nodes, where each node is connected to all nodes in the previous layer. The nodes in a given layer are not interconnected with one another, i.e. the nodes in a given layer function independently of one another. As used herein, nodes in the input layer receive data from outside of the ANN, nodes in the hidden layer(s) modify the data between the input and output layers, and nodes in the output layer provide the results. Each node is configured to receive an input, implement an activation function (e.g. binary step, linear, sigmoid, tanH, or rectified linear unit (ReLU) function), and provide an output in accordance with the activation function. Additionally, each node is associated with a respective weight. ANNs are trained with a dataset to maximize or minimize an objective function. In some implementations, the objective function is a cost function, which is a measure of the ANN’s performance (e.g. error such as L1 or L2 loss) during training, and the training algorithm tunes the node weights and / or bias to minimize the cost function. This disclosure contemplates that any algorithm that finds the maximum or minimum of the objective function can be used for training the ANN. Training algorithms for ANNs include, but are not limited to, backpropagation. It should be understood that an artificial neural network is provided only as an example machine learning model. This disclosure contemplates that the machine learning model can be any supervised learning model, semi- supervised learning model, or unsupervised learning model. Optionally, the machine learning model is a deep learning model. Machine learning models are known in the art and are therefore not described in further detail herein.

[0054] A convolutional neural network (CNN) is a type of deep neural network that has been applied, for example, to image analysis applications. Unlike a traditional neural networks, each layer in a CNN has a plurality of nodes arranged in three dimensions (width, height, depth). CNNs can include different types of layers, e.g. convolutional, pooling, and fully-connected (also referred to herein as “dense”) layers. A convolutional layer includes a set of filters and performs the bulk of the computations. A pooling layer is optionally inserted between convolutional layers to reduce the computational power and / or control overfitting (e.g. by down sampling). A fully-connected layer includes neurons, where each neuron is connected to all of the neurons in the previous layer. The layers are stacked similar to traditional neural networks. ^^ ^ Attorney Docket No. 10213-286WO1

[0055] Logistic Regression: A logistic regression (LR) classifier is a supervised classification model that uses the logistic function to predict the probability of a target, which can be used for classification. LR classifiers are trained with a data set (also referred to herein as a “dataset”) to maximize or minimize an objective function, for example a measure of the LR classifier’s performance (e.g. error such as L1 or L2 loss), during training. This disclosure contemplates that any algorithm that finds the maximum or minimum of the objective function can be used. LR classifiers are known in the art and are therefore not described in further detail herein.

[0056] Naïve Bayes: A Naïve Bayes’ (NB) classifier is a supervised classification model that is based on Bayes’ Theorem, which assumes independence among features (i.e. presence of one feature in a class is unrelated to presence of any other features). NB classifiers are trained with a data set by computing the conditional probability distribution of each feature given label and applying Bayes’ Theorem to compute conditional probability distribution of a label given an observation. NB classifiers are known in the art and are therefore not described in further detail herein.

[0057] KNN: A k-NN classifier is a supervised classification model that classifies new data points based on similarity measures (e.g. distance functions). k-NN classifier is a non-parametric algorithm, i.e. it does not make strong assumptions about the function mapping input to output and therefore has flexibility to find a function that best fits the data. k-NN classifiers are trained with a data set (also referred to herein as a “dataset”) by learning associations between all samples and classification labels in the training dataset. k-NN classifiers are known in the art and are therefore not described in further detail herein.

[0058] Ensemble: A majority voting ensemble is a meta-classifier that combines a plurality of machine learning classifiers for classification via majority voting. In other words, the majority voting ensemble’s final prediction (e.g. class label) is the one predicted most frequently by the member classification models. Majority voting ensembles are known in the art and are therefore not described in further detail herein.

[0059] FIG. 4 is a schematic illustration showing example QR codes in accordance with embodiments of the present disclosure. Using QR codes as a reference, digital information can be included along the longitudinal and lateral dimensions of a structure, for example, through the ^^ ^ Attorney Docket No. 10213-286WO1entirety of a seatbelt. These patterns can be developed to enhance algorithms for measuring belt payout, stature, position, movement, belt twist, and other relevant belt and occupant information.

[0060] FIG. 5A, FIG.5B, and FIG. 5C illustrate example methods for generating encoded surface patterns in accordance with certain embodiments described herein.

[0061] In the encoded surface pattern shown in FIG. 5A, the left most panel 501A and right most panel 505A will appear bright in IR, and modulated middle columns 503A (e.g. as shown, six modulated middle columns) can be used to encode specific text that can be used to determine seat belt payout and / or size estimation.

[0062] In the encoded surface pattern shown in FIG. 5B, the rightmost panel 505B has an offset stripe of black down the middle. Modulated middle columns 503B can be used to encode specific text that can be used to determine seatbelt payout and / or size estimation, best twist detection, etc. based on enhanced background rejection (e.g. based on a b / d / b pattern on one side of the seatbelt surface that should always present in the same way).

[0063] As illustrated in FIG.5C, an example encoded surface pattern comprises an array of blocks. As shown, a first block 501C comprises a code, string, and / or text (American Standard Code for Information Interchange (ASCII) code). A separator 503C, for example, ASCII separator is used for demarcation of the code, string, and / or text. The example encoded surface pattern can directly encode data for the purposes of determining seatbelt payout, for example. In some examples, the array can be matched to a stored pattern as it is rotated, twisted, or the like.

[0064] Referring now to FIG. 6, a flowchart diagram illustrating example operations 600 in accordance with various embodiments of the present disclosure is provided.

[0065] In some examples, the method 600 may be performed by a processing circuitry (for example, but not limited to, an application-specific integrated circuit (ASIC), a central processing unit (CPU)). In some examples, the processing circuitry may be electrically coupled to and / or in electronic communication with other circuitries of a vehicle system and / or occupancy monitoring system, such as, a memory (such as, for example, random access memory (RAM) for storing computer program instructions), and / or a display circuitry (for rendering readings on a display).

[0066] In some examples, one or more of the procedures described in FIG. 6 may be embodied by computer program instructions, which may be stored by a memory (such as a non-transitory ^^ ^ Attorney Docket No. 10213-286WO1memory) of a system employing an embodiment of the present disclosure and executed by a processing circuitry (such as a processor) of the system. These computer program instructions may direct the system to function in a particular manner, such that the instructions stored in the memory circuitry produce an article of manufacture, the execution of which implements the function specified in the flow diagram step / operation(s). Further, the system may comprise one or more other circuitries. Various circuitries of the system may be electronically coupled between and / or among each other to transmit and / or receive energy, data and / or information.

[0067] In some examples, embodiments may take the form of a computer program product on a non-transitory computer-readable storage medium storing computer-readable program instructions (e.g. computer software). Any suitable computer-readable storage medium may be utilized, including non-transitory hard disks, CD-ROMs, flash memory, optical storage devices, or magnetic storage devices.

[0068] The example method 600 begins at step / operation 602. At step / operation 602, a processing circuitry (e.g. at least one processor operatively coupled to at least one image sensor within a vehicle) detects an encoded surface pattern. For example, by detecting a pattern that reflects or absorbs a predetermined wavelength range at a preferred luminance (e.g. narrow infrared band). As discussed herein, the encoded surface pattern can be disposed on a surface of a vehicle component or vehicle surface (e.g. seatbelt, vehicle seat, or the like). In the example of a seatbelt, the encoded surface pattern can comprise a seatbelt weaving pattern (e.g. Twill, World Twill, 4 pane, 5 pane) or fabric. The encoded surface pattern can comprise at least one of a machine-readable code, a digital code (e.g. QR code, barcode), and a deterministic pseudo- random code.

[0069] Subsequent to step / operation 602, the example method 600 proceeds to step / operation 604. At step / operation 604, the processing circuitry determines at least one characteristic of one or more occupants or the vehicle component or vehicle surface based at least in part on the detected encoded surface pattern. For example, using a 2D or 3D machine vision segmentation operation / technique and / or machine learning algorithm. An example vehicle system can be configured to detect and / or track at least one occupant of the vehicle and / or a respective status of each of a plurality of vehicle components or surfaces (e.g. seatbelt, vehicle seat) based at least in part on a corresponding encoded surface pattern. ^^ ^ Attorney Docket No. 10213-286WO1

[0070] In some embodiments, the at least one vehicle component or surface comprises a seating surface, and the encoded surface pattern is used to determine at least one of a position and orientation relative to other patterns on other textile surfaces and / or objects in the vehicle (e.g. headliner, floor, door surfaces, bags, luggage). In some embodiments, the encoded surface pattern is configured to facilitate machine vision segmentation (e.g. 2D or 3D separation) of an occupant from the at least one vehicle component (e.g. seatbelt) or for detection and tracking of occupant movement relative to seatbelt or other vehicle surfaces (e.g. moving torso, arms, hands, head, legs, or the like). In some embodiments, the at least one vehicle component comprises a seatbelt, and wherein the detected encoded surface pattern is used to determine at least one of the position of the seatbelt or incorrect fitment or placement of the seatbelt (e.g. for belt twist detection). In some implementations, the placement of an encoded surface pattern (e.g. a QR code) may be used to identify with certainty a current position of a vehicle structure that is enclosed within another vehicle component. For example, and without limitation, a QR code may be placed on or proximately to a vehicle seat or other vehicle component in order to identify in real-time where items such as air bags, hardware, or safety electronics are positioned inside the vehicle seat or other vehicle component. In some implementations, the at least one vehicle component or surface comprises a Child Restraint System (CRS), and wherein the encoded surface pattern is configured to facilitate machine vision segmentation in relation to the CRS (e.g. is the CRS in the correct position, is it forward facing, rear facing, booster) and / or a seatbelt (e.g. is the seatbelt attached to the CRS for fixation). In some examples, the at least one characteristic is determined based at least in part on a detected change (e.g. partial view) to the encoded surface pattern. An encoded surface pattern on a seat may appear differently based on how the seat is stowed (e.g. fully folded or positioned at various degrees of freedom). Thus, the appearance of the encoded surface pattern may be indicative of a particular seat position. In some implementations, the at least one characteristic is related to the manufacturer of the vehicle component, for example a designation that the vehicle component is an OEM component or an aftermarket or counterfeit component.

[0071] Subsequent to step / operation 604, the example method 600 proceeds to step / operation 606. At step / operation 606, the processing circuitry determines a confidence value in relation to the at least one characteristic, i.e. based at least in part on the detected encoded surface pattern. The confidence value may be a measure of whether or not a determined characteristic (e.g. ^^ ^ Attorney Docket No. 10213-286WO1location and / or size of the at least one occupant, location and / or size of a vehicle object, a seat position and / or orientation) is true (e.g. confirmed, validated). The confidence value can be a predetermined or user-configurable threshold. In some implementations, an encoded surface pattern can comprise multiple patterns that are embedded simultaneously (e.g. a first pattern reflective to infrared light and a second pattern reflective to visible light). In such examples, multiple cameras (e.g. an infrared camera and a visible light camera) can be used for simultaneous detection and classification confidence. In such examples, using multiple cameras to generate separate confidence values and correlating the results to make a final determination can increase system confidence and accuracy.

[0072] Subsequent to step / operation 606, the example method 600 proceeds to step / operation 608. At step / operation 608, the processing circuitry determines whether the confidence value meets or exceeds a predetermined threshold or range. By way of example, if the confidence threshold is 70%, then a confidence value of 70% or above indicates that a determined characteristic (e.g. seat position, occupant location, seatbelt status) is true or can be validated. In this example, a confidence value below 70% indicates that the determined characteristic is not true or cannot be validated.

[0073] Subsequent to step / operation 608, if the processing circuitry determines that the confidence value meets or exceeds the confidence threshold or range, the method 600 proceeds to step / operation 610 and the processing circuitry generates a control signal, such as, but not limited to, an alert (e.g. via a display, audible) or verifies an expected system occupancy / monitoring output. If the processing circuitry determines that the confidence value does not meet or exceed the confidence threshold or range, the method 600 returns to step / operation 602 and the processing circuitry continues to monitor and / or track occupant(s) and / or vehicle components or surfaces. In this manner, the system can enhance the accuracy of a machine vision system that has to make dynamic decisions in real-time, for example, to ensure vehicle safety. In some implementations, the vehicle system can trigger / implement a safety protocol in response to detecting and verifying an unsafe condition (e.g. improper CRS installation with an infant in the vehicle) based at least in part on detected encoded surface pattern(s).

[0074] For purposes of this disclosure, the term “coupled” means the joining of two components (electrical, mechanical, or magnetic) directly or indirectly to one another. Such joining may be 18 ^^ ^ Attorney Docket No. 10213-286WO1stationary in nature or movable in nature. Such joining may be achieved with the two components (electrical or mechanical) and any additional intermediate members being integrally defined as a single unitary body with one another or with the two components or the two components and any additional member being attached to one another. Such joining may be permanent in nature or alternatively may be removable or releasable in nature.

[0075] The present disclosure has been described with reference to example embodiments, however persons skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the claimed subject matter. For example, although different example embodiments may have been described as including one or more features providing one or more benefits, it is contemplated that the described features may be interchanged with one another or alternatively be combined with one another in the described example embodiments or in other alternative embodiments. Because the technology of the present disclosure is relatively complex, not all changes in the technology are foreseeable. The present disclosure described with reference to the example embodiments and set forth in the following claims is manifestly intended to be as broad as possible. For example, unless specifically otherwise noted, the claims reciting a single particular element also encompass a plurality of such particular elements.

[0076] It is also important to note that the construction and arrangement of the elements of the system as shown in the preferred and other exemplary embodiments is illustrative only. Although only a certain number of embodiments have been described in detail in this disclosure, those skilled in the art who review this disclosure will readily appreciate that many modifications are possible (e.g. variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.) without materially departing from the novel teachings and advantages of the subject matter recited. For example, elements shown as integrally formed may be constructed of multiple parts or elements shown as multiple parts may be integrally formed, the operation of the assemblies may be reversed or otherwise varied, the length or width of the structures and / or members or connectors or other elements of the system may be varied, the nature or number of adjustment or attachment positions provided between the elements may be varied. It should be noted that the elements and / or assemblies of the system may be constructed from any of a wide variety of materials that provide sufficient strength or durability. ^^ ^ Attorney Docket No. 10213-286WO1

[0077] Accordingly, all such modifications are intended to be included within the scope of the present disclosure. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the preferred and other exemplary embodiments without departing from the spirit of the present subject matter.

[0078] In example implementations, at least some portions of the activities may be implemented in software provisioned on a networking device. In some embodiments, one or more of these features may be implemented in computer hardware, provided external to these elements, or consolidated in any appropriate manner to achieve the intended functionality. The various network elements may include software (or reciprocating software) that can coordinate image development across domains such as time, amplitude, depths, and various classification measures that detect movement across frames of image data and further detect particular objects in the field of view in order to achieve the operations as outlined herein. In still other embodiments, these elements may include any suitable algorithms, hardware, software, components, modules, interfaces, or objects that facilitate the operations thereof.

[0079] Furthermore, computer systems described and shown herein (and / or their associated structures) may also include suitable interfaces for receiving, transmitting, and / or otherwise communicating data or information in a network environment. Additionally, some of the processors and memory elements associated with the various nodes may be removed, or otherwise consolidated such that single processor and a single memory element are responsible for certain activities. In a general sense, the arrangements depicted in the Figures may be more logical in their representations, whereas a physical architecture may include various permutations, combinations, and / or hybrids of these elements. It is imperative to note that countless possible design configurations can be used to achieve the operational objectives outlined here. Accordingly, the associated infrastructure has a myriad of substitute arrangements, design choices, device possibilities, hardware configurations, software implementations, equipment options, etc.

[0080] In some example embodiments, one or more memory elements (e.g. memory can store data used for the operations described herein. This includes the memory being able to store instructions (e.g. software, logic, code, etc.) in non-transitory media, such that the instructions are executed to carry out the activities described in this Specification. A processor can execute any type of computer readable instructions associated with the data to achieve the operations detailed ^^ ^ Attorney Docket No. 10213-286WO1herein in this Specification. In one example, processors (e.g. processor) could transform an element or an article (e.g. data) from one state or thing to another state or thing. In another example, the activities outlined herein may be implemented with fixed logic or programmable logic (e.g. software / computer instructions executed by a processor) and the elements identified herein could be some type of a programmable processor, programmable digital logic (e.g. a field programmable gate array (FPGA), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM)), an ASIC that includes digital logic, software, code, electronic instructions, flash memory, optical disks, CD-ROMs, DVD ROMs, magnetic or optical cards, other types of machine-readable mediums suitable for storing electronic instructions, or any suitable combination thereof.

[0081] These devices may further keep information in any suitable type of non-transitory storage medium (e.g. random-access memory (RAM), read only memory (ROM), field programmable gate array (FPGA), erasable programmable read only memory (EPROM), electrically erasable programmable ROM (EEPROM), etc.), software, hardware, or in any other suitable component, device, element, or object where appropriate and based on particular needs. Any of the memory items discussed herein should be construed as being encompassed within the broad term 'memory element.' Similarly, any of the potential processing elements, modules, and machines described in this Specification should be construed as being encompassed within the broad term 'processor.'

[0082] The following patents, applications, and publications, as listed below and throughout this document, are hereby incorporated by reference in their entirety herein. [1] Edita Mal^iauskien^ et al. "Influence of Weave into Slippage of Yarns in Woven Fabric", March 2011Materials Science 17(1), DOI:10.5755 / j01.ms.17.1.248 [2] https: / / evasweaving.wordpress.com / tag / random-weaves / . Accessed December 8, 2023 ^

Claims

^ ^ Attorney Docket No. 10213-286WO1CLAIMS What is claimed is:

1. A system for occupancy monitoring and sensing within a vehicle comprising: at least one vehicle component or surface comprising an encoded surface pattern; at least one image sensor positioned within the vehicle configured to detect the encoded surface pattern; and at least one processor operatively coupled to the at least one image sensor that is configured to determine at least one characteristic of at least one occupant and / or the at least one vehicle component or surface based at least in part on the detected encoded surface pattern.

2. The system of claim 1, wherein the encoded surface pattern is detected or analyzed using a two-dimensional (2D) or three-dimensional (3D) machine vision segmentation operation and / or machine learning algorithm.

3. The system of claim 2, wherein the 2D or 3D machine vision segmentation operation is used to determine at least one of: an estimated belted occupant height, weight, body morphology, respiration patterns, movement patterns, and occupant position.

4. The system of any one of claims 1-3, wherein the at least one processor is further configured to: determine a confidence value in relation to the at least one characteristic based at least in part on the detected encoded surface pattern.

5. The system of claim 4, wherein the at least one processor is further configured to: in response to determining that the confidence value meets or exceeds a predetermined threshold or range, transmit a control signal.

6. The system of claim 5, wherein the control signal is configured to: ^^ ^ Attorney Docket No. 10213-286WO1generate an alert or verify an expected system occupancy / monitoring output.

7. The system of any one of claims 1-6, wherein the encoded surface pattern comprises a pattern that reflects or absorbs a predetermined wavelength range at a preferred luminance.

8. The system of any one of claims 1-7, wherein the at least one characteristic comprises at least one of a location and / or size of the at least one occupant, location and / or size of a vehicle object, a seat position and / or orientation.

9. The system of any one of claims 1-8, wherein the at least one characteristic is determined based at least in part on a detected change to the encoded surface pattern.

10. The system of any one of claims 1-9, wherein the encoded surface pattern comprises at least one of a machine-readable code, a digital code, and a deterministic pseudo-random code.

11. The system of any one of claims 1-10, wherein the encoded surface pattern further comprises a seatbelt weaving pattern or fabric.

12. The system of any one of claims 1-11, wherein the at least one vehicle component comprises a seatbelt, and wherein the detected encoded surface pattern is used to determine at least one of the position of the seatbelt or incorrect fitment or placement of the seatbelt.

13. The system of any one of claims 1-12, wherein the encoded surface pattern is configured to facilitate machine vision segmentation of an occupant from the at least one vehicle component or for detection and tracking of occupant movement relative to seatbelt or other vehicle surfaces.

14. The system of any one of claims 1-13, wherein the at least one vehicle component or surface comprises a Child Restraint System (CRS), and wherein the encoded surface pattern is configured to facilitate machine vision segmentation in relation to the CRS and / or a seatbelt.

15. The system of any one of claims 1-14, wherein the at least one vehicle component or surface comprises a seating surface, and wherein the encoded surface pattern is used to ^^ ^ Attorney Docket No. 10213-286WO1determine at least one of a position and orientation relative to other patterns on other textile surfaces and / or objects in the vehicle.

16. A system for occupancy monitoring and sensing within a vehicle comprising: a plurality of vehicle components or surfaces, wherein each vehicle component or surface comprises a corresponding encoded surface pattern; at least one image sensor positioned within the vehicle configured to detect each encoded surface pattern; and at least one processor operatively coupled to the at least one image sensor that is configured to detect and track at least one occupant of the vehicle and / or a respective status of each of the plurality of vehicle components or surfaces based at least in part on each detected encoded surface pattern.

17. The system of claim 16, wherein each encoded surface pattern is detected or analyzed using a 2D or 3D machine vision segmentation operation or technique and / or machine learning algorithm.

18. A computer-implemented method comprising: detecting, by at least one processor and using an image sensor positioned within a vehicle, at least one encoded surface pattern corresponding with a vehicle component or surface; determining, by the at least one processor, at least one characteristic of at least one occupant and / or at least one vehicle component or surface based at least in part on the at least one detected encoded surface pattern; determining, by the at least one processor, a confidence value in relation to the at least one determined characteristic; and responsive to determining that the confidence value meets or exceeds a predetermined threshold or range, generate a control signal or generate an alert to verify an expected system occupancy / monitoring output.

19. The computer-implemented method of claim 18, wherein the at least one encoded surface pattern is detected using a 2D or 3D machine vision segmentation operation and / or machine learning algorithm. ^^ ^ Attorney Docket No. 10213-286WO120. The computer-implemented method of claim 18 or 19, wherein the at least one encoded surface pattern is configured to facilitate machine vision segmentation of an occupant from the at least one vehicle component or for detection and tracking of occupant movement relative to seatbelt or other vehicle surfaces. ^^^ ^^

Images