Predictive maintenance of automotive transmissions

Through artificial neural networks, the sensor data and operating parameters of the automotive transmission are monitored, abnormal status is identified, and maintenance warnings are generated, which solves the problems of high maintenance costs and safety risks of automotive transmissions in the prior art, and realizes early fault detection and predictive maintenance.

CN118894053APending Publication Date: 2024-11-05MICRON TECHNOLOGY INC
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411012538.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-12-18
Filing Date
2020-12-14
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

The prior art is difficult to effectively monitor and predict abnormal states of automotive transmissions, resulting in high maintenance costs and potential safety risks.

Method used

Artificial neural network (ANN) is used to combine sensor data and operating parameters to monitor the status of the car transmission in real time through data storage devices and inference engines, identify abnormal patterns and generate maintenance warnings.

Benefits of technology

It realizes early fault detection and predictive maintenance of automotive transmissions, reduces repair costs, and improves vehicle safety and reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118894053A_ABST
    Figure CN118894053A_ABST
Patent Text Reader

Abstract

The invention relates to a system, method and apparatus for predictive maintenance of an automotive transmission of a vehicle. For example, a transmission has at least one sensor for measuring a temperature of transmission oil, a torque applied to a shaft of the transmission, a vibration sensor, and / or a microphone. During a period in which the vehicle is assumed to be in normal operation, sensor data generated by a transmission sensor is used to train an artificial neural network to identify a normal pattern of the sensor data. The trained artificial neural network is then used to determine if the current sensor data from the transmission sensor is abnormal. Maintenance alerts for the vehicle may be generated in response to determining an operational anomaly of the transmission from the artificial neural network and the current sensor data.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Information about divisional applications

[0002] This application is a divisional application of the invention patent application with application date of December 14, 2020, application number 202011474360.7, and invention name “Predictive Maintenance of Automobile Transmission”. Technical Field

[0003] At least some embodiments disclosed herein relate generally to vehicles, and more particularly, but not limited to, to monitoring predictive maintenance of automotive transmission systems. Background Art

[0004] An automotive transmission is a gearbox that uses gears and gear trains to convert speed and torque between a drive shaft and a driven shaft. Typically, an automotive transmission can last more than 150,000 miles or about 7 years under normal conditions.

[0005] However, different driving and / or usage styles can have different effects on the life of a vehicle's transmission. For example, overheating or lack of maintenance of the transmission fluid can shorten the life of the transmission. For example, speeding or sudden stops can shorten the life of the transmission. For example, using the transmission to hold the vehicle in a parked state without using the parking brake can shorten the life of the transmission. Driving with a bad transmission can be dangerous.

[0006] Recent developments in the field of autonomous driving technology allow computing systems to operate control elements of a motor vehicle, at least under some conditions, without assistance from a human operator of the vehicle.

[0007] For example, sensors such as cameras and radars may be mounted on a motor vehicle to detect conditions of the vehicle's surroundings as it travels on a road. A computing system mounted on the vehicle analyzes the sensor inputs to identify the conditions and generates control signals or commands for autonomous adjustments to the vehicle's direction and / or speed, with or without any input from a human operator of the vehicle.

[0008] In some arrangements, when the computing system identifies a situation in which the computing system cannot continue to operate the vehicle in a safe manner, the computing system alerts the human operator of the vehicle and requests the human operator to take over control of the vehicle and drive manually instead of allowing the computing system to autonomously drive the vehicle.

[0009] Autonomous driving and / or advanced driver assistance systems (ADAS) may use artificial neural networks (ANNs) to recognize events and / or objects captured in sensor inputs. Examples of sensor inputs include images from digital cameras, lidar, radar, ultrasonic sonar, etc.

[0010] Generally speaking, an artificial neural network (ANN) uses a network of neurons to process inputs to the network and generate outputs from the network.

[0011] For example, each neuron in the network receives a set of inputs. Some of the inputs to a neuron may be the outputs of a particular neuron in the network; and some of the inputs to a neuron may be inputs provided to the neural network. The input / output relationship between neurons in the network represents the neuron connectivity in the network.

[0012] For example, each neuron may have a bias, an activation function, and a set of synaptic weights for its inputs, respectively. The activation function may be in the form of a step function, a linear function, a log-sigmoid function, etc. Different neurons in the network may have different activation functions.

[0013] For example, each neuron may produce a weighted sum of its inputs and its bias and then produce an output that is a function of the weighted sum, calculated using the neuron's activation function.

[0014] The relationship between the input and output of an ANN is generally defined by an ANN model, which contains data representing the connectivity of neurons in the network and the bias, activation function, and synaptic weights of each neuron. Using a given ANN model, a computing device calculates the output of the network from a given set of inputs to the network.

[0015] For example, input to the ANN network may be generated based on camera input; and output from the ANN network may be identification of an item such as an event or object.

[0016] Spiking neural networks (SNNs) are a type of ANN that highly mimics natural neural networks. SNN neurons generate pulses as output when the activation level of the neuron is high enough. The activation level of SNN neurons mimics the membrane potential of natural neurons. The output / pulse of SNN neurons can change the activation level of other neurons that receive the output. The time-varying current activation level of SNN neurons is usually modeled using differential equations and is regarded as the state of the SNN neuron. Incoming pulses from other neurons can push the activation level of the neuron up to reach the threshold of the pulse. Once a neuron sends a pulse, its activation level is reset. Before sending a pulse, the activation level of the SNN neuron can decay over time, such as controlled by a differential equation. The temporal element of the behavior of SNN neurons makes SNNs suitable for processing spatiotemporal data. The connectivity of SNNs is usually sparse, which helps reduce the computational workload.

[0017] In general, ANNs can be trained using supervised methods, where parameters in the ANN are adjusted to minimize or reduce the error between the known outputs resulting from the corresponding inputs and the calculated outputs resulting from applying the inputs to the ANN. Examples of supervised learning / training methods include reinforcement learning and learning with error correction.

[0018] Alternatively or in combination, the ANN may be trained using unsupervised methods, where the exact output resulting from a given set of inputs is unknown until training is complete. The ANN may be trained to classify items into multiple categories or to classify data points into clusters.

[0019] A variety of training algorithms are available for complex machine learning / training paradigms. Summary of the invention

[0020] According to one aspect of the present application, a vehicle is provided. The vehicle comprises: an automobile transmission; at least one sensor configured on the automobile transmission to generate sensor data; a computer system configured to determine operating parameters of the vehicle, the computer system having: one or more memory components configured to store the sensor data; and a reasoning engine configured to determine whether the operation of the automobile transmission is abnormal using an artificial neural network, the sensor data, and the operating parameters; wherein the computer system is configured to generate a maintenance warning in response to determining the abnormal operation of the automobile transmission according to the artificial neural network.

[0021] According to another aspect of the present application, a method is provided. The method includes: receiving sensor data from at least one sensor configured on an automobile transmission of the vehicle in a data storage device configured in the vehicle; determining an operating parameter of the vehicle associated with the sensor data from the at least one sensor configured on the automobile transmission; determining by the data storage device whether the operation of the automobile transmission is abnormal based on the sensor data from the at least one sensor configured on the automobile transmission and the operating parameter using an artificial neural network; and generating a maintenance warning for the vehicle in response to determining that the operation of the automobile transmission is abnormal according to the artificial neural network.

[0022] According to another aspect of the present application, a data storage device configured in a vehicle is provided. The data storage device includes: a host interface configured to receive sensor data from at least one sensor configured on an automobile transmission of the vehicle; one or more memory components configured to cyclically store the sensor data in an input area; a controller configured to determine operating parameters of the vehicle associated with the sensor data from the at least one sensor configured on the automobile transmission; and an inference engine configured to use an artificial neural network and determine whether the operation of the automobile transmission is abnormal based on the sensor data from the at least one sensor and the operating parameters, wherein a maintenance warning of the vehicle is generated in response to determining the abnormal operation of the automobile transmission according to the artificial neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Embodiments are illustrated by way of example and not limitation in the accompanying drawings in which like reference numerals indicate similar elements.

[0024] Figure 1 A system for monitoring predictive maintenance of an automotive transmission of a vehicle is presented according to one embodiment.

[0025] Figure 2 A system having a vehicle configured to collect and process sensor data is shown according to some embodiments.

[0026] Figure 3 An autonomous vehicle is shown according to one embodiment.

[0027] Figures 4 to 6 The training of an artificial neural network for prediction according to some embodiments is described.

[0028] Figure 7 A data storage device having a neural network accelerator according to one embodiment is shown.

[0029] Figure 8 A memory component for accelerating neural network computations is presented according to one embodiment.

[0030] Fig. 9 Memory capacity configured to support neural network computations is shown according to one embodiment.

[0031] Fig.10 The configuration of a memory area of ​​an artificial neural network (ANN) model according to one embodiment is described.

[0032] Fig.11 The configuration of a memory area for input of an artificial neuron according to one embodiment is described.

[0033] Fig.12 The configuration of a memory area for outputs from an artificial neuron according to one embodiment is described.

[0034] Fig.13 Communication between an autonomous vehicle and a data storage device is shown according to one embodiment.

[0035] Fig.14 Communications within a data storage device according to one embodiment are shown.

[0036] Fig.15 A data storage device configured to monitor an automotive transmission of a vehicle is shown according to one embodiment.

[0037] Fig.16 A method for generating a predictive maintenance warning for an automotive transmission of a vehicle is presented according to one embodiment. DETAILED DESCRIPTION

[0038] At least some embodiments disclosed herein provide systems, methods, and apparatus for predictive maintenance of an automotive transmission of a motor vehicle or another vehicle, with or without an advanced driver assistance system (ADAS).

[0039] In at least some embodiments discussed herein, an artificial neural network is configured to monitor and analyze sensor data generated by sensors configured on an automotive transmission to predict whether maintenance or repair is needed.

[0040] Sensors may be configured on an automotive transmission to monitor its operating conditions. For example, a temperature sensor may be configured in the transmission to measure the operating temperature of the transmission oil. For example, a torque sensor may be attached to the input shaft to measure the torque applied to the input shaft; alternatively, in combination, a torque sensor may be applied to the output shaft to measure the torque transmitted by the transmission to the output shaft. For example, a vibration sensor may be configured in the transmission oil to monitor the movement of the transmission oil. For example, a microphone may be attached to the transmission to monitor the noise generated by the automotive transmission.

[0041] The artificial neural network can be trained to recognize common transmission problems from sensor data and / or predict the need for transmission maintenance. The artificial neural network can be trained to determine whether the sensor data has a normal pattern or an abnormal pattern that is informed that no immediate intervention is needed or an abnormal pattern that is informed that immediate intervention is required.

[0042] Based on the classification of the sensor data determined by the artificial neural network, the vehicle may be configured to alert the driver / passenger that an abnormality has occurred and / or cause the autonomous driving system or advanced driver assistance system to safely slow down and / or stop the vehicle.

[0043] When the artificial neural network classifies a set of sensor data of the transmission as abnormal, the sensor data with the abnormal pattern can be automatically stored in the data storage device for subsequent diagnosis in the maintenance facility. Alternatively or in combination, the abnormal sensor data can be transmitted from the vehicle via a wireless connection to a server for diagnosis. Sensor data with recognized normal patterns can be discarded to reduce the need for large storage capacity.

[0044] When the diagnostic results are available in a maintenance facility, the artificial neural network (or another artificial neural network) may be further trained to predict the diagnostic results from sensor data having abnormal patterns that may be stored in and / or downloaded from the data storage device.

[0045] Early detection of transmission damage or failure can result in less expensive repairs and potentially avoid catastrophic accidents.

[0046] For example, sensor data from a transmission and / or other sensor data and / or operating parameters of a vehicle may be provided as an input data stream to a data storage device. The data storage device cyclically records the input data stream in an input area to maintain the most recent segment of the input data stream. During a period in which the transmission is expected to operate normally, the sensor data identifies a normal pattern. An artificial neural network (ANN) in the data storage device, such as a spiking neural network (SNN), may be trained to recognize normal patterns in the sensor data stored in the input area. After training, the data storage device may be switched to a prediction mode in which the trained ANN is used to determine whether the current sensor data in the input area has a normal pattern identified from the training cycle.

[0047] When suspected erroneous behavior is detected, the controller of the data storage device can prevent the recorded sensor data from being erased or discarded (e.g., by saving the data to a separate output area, partition, and / or cloud). In addition, the vehicle's computer system can generate a warning about the suspected erroneous behavior to the driver, owner, and / or maintenance facility.

[0048] After investigation or diagnosis in a maintenance facility, the data may be labeled as normal or erroneous and thus used to further train an artificial neural network (eg, SNN) for improved future detections.

[0049] When diagnosing a problem in a car transmission after a maintenance visit, the diagnostic and recorded data can be used to train an artificial neural network to recognize defects from similar data. This artificial neural network can be configured on a server to remotely diagnose problems with the car transmission when the vehicle uploads its abnormal sensor data to the server. Alternatively, this artificial neural network can be downloaded to the vehicle on-site via an over-the-air update to allow the vehicle to generate diagnostic results from the abnormal sensor data.

[0050] Optionally, the sensor data provided to the data storage device for training of the ANN and / or prediction of the need for maintenance service may further include data representing operating conditions of the vehicle, such as the vehicle's ambient temperature, the vehicle's speed, the vehicle's location, the road on which the vehicle is traveling, inputs from other sensors of the vehicle, inference results generated based on inputs from some of the sensors, and the status of components of the vehicle (such as the vehicle's throttle, brakes, steering mechanism, infotainment system and / or communication device).

[0051] Figure 1 A system for monitoring predictive maintenance of an automotive transmission of a vehicle is presented according to one embodiment.

[0052] exist Figure 1 In the invention, a vehicle transmission (101) is configured with one or more sensors (103) for determining the operating conditions of the vehicle transmission (101).

[0053] For example, the sensors (103) of the transmission may include one or more temperature sensors disposed in the transmission oil. Temperature measurements from the sensors (103) identify the operating temperature of the gears and gear trains in the transmission.

[0054] For example, the sensors (103) of the transmission may include one or more strain or stress sensors that can be attached to one or more shafts of the transmission to measure the torque of the shafts. The measured torque is indicative of the operating condition of the transmission.

[0055] For example, a vibration sensor or microphone may be disposed in the transmission oil to measure vibrations of the transmission oil.

[0056] For example, a vibration sensor or microphone may be disposed on the transmission housing to monitor the noise generated by the automotive transmission.

[0057] exist Figure 1 In the embodiment of the present invention, inputs sensed by the transmission sensor (103) and / or other input data (eg, from a vehicle's computer system) are transmitted to a data storage device (112).

[0058] The data storage device (112) has a medium (108) configured to store sensor data (121) from a transmission sensor (103). A controller (151) of the data storage device (112) can store the sensor data (121) in an input area, partition or namespace in a round-robin manner so that the most recent set of sensor data (121) is available in the data storage device (112).

[0059] The data storage device (112) includes an inference engine (109) that can analyze sensor data (121) using an ANN (125) configured in the data storage device (112). The ANN (125) is configured to classify the sensor data (121) as normal or abnormal. The abnormal sensor data (121) can be further classified into unknown abnormal sensor data, abnormal sensor data that is informed to require immediate intervention, and abnormal sensor data that is informed not to require immediate intervention.

[0060] For example, during a period in which the vehicle's automotive transmission is assumed to have normal operation, the sensor data (121) collected in the data storage device (112) may be used to train the ANN (125) to recognize normal patterns in the sensor data (121).

[0061] For example, a vehicle may be assumed to have normal operation within a predetermined period of time from when the vehicle left the manufacturer. For example, a vehicle's automotive transmission may be assumed to have normal operation when operated within a predetermined initial number of miles of the vehicle.

[0062] Subsequently, as the likelihood of the vehicle having abnormal operation increases, the inference engine (109) is configured to use the ANN (125) to determine whether the current sensor data (121) deviates from the identified normal pattern. When a new or abnormal pattern is detected, the data storage device (112) can generate an alert for maintenance service. After maintenance service, the new or abnormal pattern can be labeled as normal, abnormal sensor data that is notified to require immediate intervention, or abnormal sensor data that is notified to not require immediate intervention; and the ANN (125) can be further trained to predict the label from the sensor data (121) having the new pattern.

[0063] For example, the data storage device (112) may be configured to Figure 2 In a vehicle described in . The reasoning engine (109) can train the ANN (125) to recognize normal patterns of sensor data (121) collected during a period in which a predetermined vehicle transmission has a probability of normal operation above a threshold. Subsequently, the reasoning engine (109) can use the ANN (125) to determine whether current sensor data (121) from the sensor and control unit has a normal pattern; and when the current sensor data (121) does not have a normal pattern, the vehicle can present a warning for a maintenance visit.

[0064] Figure 2 A system having a vehicle (111) configured to collect and process sensor data is shown according to some embodiments.

[0065] Figure 2A vehicle (111) has a data storage device (112), a sensor (122), an ANN (125), and an ADAS (128) configured to process sensor data including input from the sensor (122) to generate a control signal for the vehicle (111).

[0066] For example, the sensor (122) may include Figure 1 A sensor (103) for a vehicle transmission (101) as described herein.

[0067] In general, one or more sensors (e.g., 122) may be configured on the vehicle (111) to generate sensor data input to the ADAS (128) and / or the data storage device (112). The data storage device (112) and / or the ADAS (128) may be configured to generate inference results using the ANN (125). The inference results may include control signals for operating or driving the vehicle (111), recommendations for maintenance services for the vehicle (111), etc.

[0068] In some embodiments, at least a portion of the data generated by the sensors (e.g., 122) is used in both the ADAS (128) for driving assistance and the ANN (125) for maintenance prediction. Optionally, the output of the ANN (125) can be used in both the data storage device (112) and the ADAS (128). The ANN (125) can be part of the ADAS (128).

[0069] Some sensors (122) may be configured in digital cameras, lidars, radars, ultrasonic sonars, etc. Other types of sensors may also be used, such as brake sensors, speed sensors, acceleration sensors, airbag sensors, GPS (Global Positioning System) receivers, audio sensors / microphones, vibration sensors, force / stress sensors, deformation sensors, motion sensors, temperature sensors, etc. Some sensors (122) may be configured to primarily monitor the environment of the vehicle (111); and other sensors (122) may be configured to primarily monitor the operating conditions of one or more components of the vehicle (111), such as the internal combustion engine, exhaust system, electric motor, brakes, tires, battery, etc.

[0070] The ANN (125) of the vehicle (111) is configured to process sensor input data from the sensor (122) to control the vehicle (111) and / or the data storage device (112) and / or generate an alert for maintenance service.

[0071] Generally, the output of the time-varying sensor (122) is provided as a sensor data stream to an ADAS (128) and / or an ANN (125) to provide driving assistance (eg, autonomous driving) and maintenance prediction.

[0072] At least a portion of the sensor data stream may be provided to a data storage device (112) for storage and / or for processing. For example, a portion of the ANN (125) may be implemented in the data storage device (112). An inference engine (109) of the data storage device (112) may process the sensor data stream to generate inference results for further processing by the ADAS (128). Thus, the input data stream of the data storage device (112) may include at least a portion of the sensor data stream from the sensor (122); and the output data stream may include inference results generated for the ADAS (128) of the vehicle (111) using the ANN (125) in the data storage device (112). The operating conditions of the vehicle (111) and therefore the workload of the data storage device (112) may be determined from patterns in the input / output data streams of the data storage device (112).

[0073] The ANN (125) in the vehicle (111) and / or the data storage device (112) may include an SNN configured to classify time-based changes in sensor data and / or detect deviations from a known pattern of sensor data of the vehicle (111). When the ANN (125) detects a deviation from a known pattern, the sensor data corresponding to the deviation may be stored in the data storage device (112) for further analysis and / or for further training of the ANN (125).

[0074] The data storage device (112) of the vehicle (111) may be configured to record sensor data over a period of time. The recorded sensor data may be used in the ANN (125) for predictive maintenance and / or for further training of the ANN (125). A maintenance service facility (e.g., 127) may download the sensor data (121) from the data storage device (112) and provide the sensor data (121) and corresponding inference result data (123) to the server (119) to facilitate training of the ANN (125).

[0075] Optionally or in combination, the data storage device (112) is configured with a machine learning module to customize and / or train the ANN (125) installed in the vehicle (111) and / or in the data storage device (112).

[0076] The vehicle (111) may have a wireless communication device that communicates with a remote server (119) via a wireless signal (113) and a communication network (117). The remote server (119) is typically configured at a location away from a road (114) on which the vehicle (111) is traveling. For example, the vehicle (111) may provide some sensor data (121) to the server (119) and receive updates for the ANN (125) from the server (119).

[0077] The communication network (117) may be a mobile phone network having one or more base stations (e.g., 115) for receiving wireless signals (e.g., 113). Alternatively or in combination, the communication network (117) may include the Internet, wherein wireless local area network signals (e.g., 113) transmitted by the vehicle (111) are received in access points (e.g., 115) for further transmission to a server (119). In some embodiments, the vehicle (111) communicates with the server (119) using a communication link (116) of a satellite (118) and a communication balloon.

[0078] The server (119) may also communicate with one or more maintenance service facilities (eg, 127) to receive sensor data (121) and / or desired inference result data (123) of a vehicle (eg, 111).

[0079] For example, the desired inference result data (123) may be generated by a human operator inspecting sensor data (121) (e.g., images from a sensor (122)) and / or related conditions of a vehicle (111). For example, the desired inference result data (123) may include inspection records and / or service records of components of a vehicle (e.g., 111). For example, the inspection records and / or service records may indicate the degree of wear, identification of failed or malfunctioning components, etc., inspected during service at a maintenance service facility (e.g., 127). The sensor data (121) of a vehicle (e.g., 111) obtained during a period associated with the desired inference result data (123) may be used to train an ANN (125) at a server (119) to improve the reasoning capabilities of the ANN (125).

[0080] The updated ANN (125) may be installed in the vehicle (111) at the maintenance service facility (127). Alternatively, the updated ANN (125) may be transmitted to the vehicle (111) to update the vehicle (111) over the air.

[0081] A portion of the ANN (125) responsible for processing input from the sensor (122) may be configured in the data storage device (112). The inference engine (109) of the data storage device (112) processes the input from the sensor (122) to generate an inference result that is transmitted from the data storage device (112) to the ADAS (128). Based on the input from the sensor (122) and / or the inference result to the ADAS (128), the inference engine (109) of the data storage device (112) may generate an inference result to optimize the performance of the data storage device (112) in processing input data streams and output data streams by adjusting the operation of buffering / caching, collection of obsolete items, wear leveling, etc.

[0082] Figure 3An autonomous vehicle (111) is shown according to one embodiment. For example, Figure 2 The vehicle (111) in the system can be used Figure 3 An autonomous vehicle (111) is implemented.

[0083] In general, the vehicle (111) may include an infotainment system (149), a communication device (139), one or more sensors (122), and a computer system (131) connected to some controls of the vehicle (111), such as a steering control (141) for guiding the vehicle (111), a brake control (143) for stopping the vehicle (111), an acceleration control (145) for the speed of the vehicle (111), etc. In some embodiments, Figure 2 The vehicle (111) in the system has a similar configuration and / or similar components.

[0084] Figure 3 The vehicle (111) is configured with an advanced driver assistance system (ADAS) (128). The ADAS (128) of the vehicle (111) may have an artificial neural network (ANN) (125) for object detection, recognition, identification and / or classification based on images generated by the sensor (122). A portion of the ANN (125) may be implemented in Figure 1 in a data storage device (112).

[0085] The computer system (131) of the vehicle (111) may include one or more processors (133), a data storage device (112), and a memory (135) storing firmware (or software) (147) (including computer instructions and data models of the ADAS (128)).

[0086] The sensors (122) of the vehicle (111) may include visible light cameras, infrared cameras, lidar, radar or sonar systems, peripheral sensors, global positioning system (GPS) receivers, satellite positioning system receivers, brake sensors and / or airbag sensors. In addition, the sensors (122) of the vehicle (111) may include audio sensors (such as microphones) configured to monitor noise from various components and locations in the vehicle (111), vibration sensors, pressure sensors, force sensors, stress sensors and / or deformation sensors configured to measure loads on components of the vehicle (111), accelerometers and / or gyroscope sensors that measure the movement of some components of the vehicle (111), etc. Such sensors can be used to monitor the operating status and / or health of components for predictive maintenance.

[0087] The sensor (122) may provide a real-time sensor data stream to the computer system (131). The sensor data generated by the sensor (122) of the vehicle (111) may include images of objects captured using a camera that uses imaging light visible to the human eye, or a camera that uses imaging infrared light, or a sonar, radar, or LIDAR system. Preferably, the images are processed by the inference engine (109) of the data storage device (112) to generate inference results as an output data stream of the data storage device (112) and thereby reduce the computational workload of the host computer system (131).

[0088] For example, the camera can be used to obtain road information on which the vehicle (111) is traveling, which can be processed by the ANN (125) to generate control signals for the vehicle (111). For example, the camera can be used to monitor the operating status / health of components of the vehicle (111), which can be processed by the ANN (125) to predict or schedule maintenance services.

[0089] The infotainment system (149) of the vehicle (111) can be used to present data and / or inference results from the sensor (122). For example, a compressed image with reduced resolution and refresh rate can be generated in the sensor (122) and transmitted to the infotainment system (149) for presentation to an occupant of the vehicle (111). Optionally, the communication device (139) can establish a connection to a mobile device of an occupant of the vehicle (111) for presentation.

[0090] When the vehicle (111) is configured with an ADAS (128), the output of the ADAS (128) may be used to control the acceleration of the vehicle (111), the speed of the vehicle (111), and / or the direction of the vehicle (111) during autonomous driving.

[0091] Figures 4 to 6 The training of an artificial neural network (125) for prediction according to some embodiments is described.

[0092] exist Figure 4 In the invention, a supervised machine learning module (171) is used to train an artificial neural network (125) to minimize the difference between the predictions (129) generated from the sensor data (121) and the desired inference result data (123).

[0093] For example, the sensor data (121) may include an image showing an object; and the desired / expected inference result data (123) may identify the image area occupied by the object, features of the object, a classification of the object, the identity of the object, etc.

[0094] For example, the sensor data (121) may include images of the surroundings of the vehicle (111); and the desired / expected inference result data (123) may include preferred control inputs for steering control, braking control, and acceleration control.

[0095] The desired / expected reasoning result data (123) may be generated by a human operator. For example, the sensor data (121) may be used to construct a virtual reality representation of a situation encountered by the vehicle (111), which includes images from the sensor (122) showing the environment of the vehicle (111); and the desired / expected reasoning result data (123) may include a response generated by the human operator in response to the virtual reality representation of the situation.

[0096] The supervised machine learning module (171) can adjust the artificial neural network (125) to reduce / minimize the difference between the predictions (129) generated based on the sensor data (121) and the desired / expected inference result data (123) generated by the human operator.

[0097] Figure 4 The supervised learning (171) can be applied in the server (119) based on the sensor data of a group of vehicles and the corresponding desired / expected inference result data (123) to generate a general ANN for the group of vehicles.

[0098] Figure 4 The supervised learning (171) can be applied in the vehicle (111) based on the sensor data and the inference result data (123) of the vehicle to generate a customized / personalized ANN (125). For example, the general ANN (125) can be first used in the vehicle (111); and the sensor data of the vehicle (111) and the desired / expected inference result data (123) specific to the vehicle (111) can be used to further train the ANN (125) of the vehicle for customization / personalization of the ANN (125) in the vehicle (111).

[0099] exist Figure 5 In the invention, an unsupervised machine learning module (175) is used to train or improve an artificial neural network (125) to facilitate anomaly detection (173). The unsupervised machine learning module (175) is configured to adjust the ANN (e.g., SNN) to produce normal classifications, clusters, or recognized patterns in the sensor data (121), so that the degree of deviation from the normal classifications, clusters, or recognized patterns in the sensor data (121) can be used to signal the anomaly detection (173).

[0100] For example, anomaly detection (173) may be used to save sensor data (121) associated with the anomaly for further analysis. In response to anomaly detection (173) in the vehicle (111), the computer system (131) may issue a read command to the sensor (122) to retrieve image data associated with the anomaly from the sensor (122) and store the retrieved image data in the data storage device (112). The image data associated with the anomaly may be temporarily saved in a memory device of the sensor (122) and loaded to the data storage device (112) within a period of time using available communication bandwidth between the sensor (122) and the data storage device (112) without affecting normal operation of the ADAS (128).

[0101] While the vehicle (111) is in the maintenance service facility (127), image data (and other sensor data) associated with the anomaly may be retrieved from the data storage device (112) to generate desired / expected inference result data (123) for use Figure 4 The ANN (125) is further trained by supervised learning (171).

[0102] Optionally, supervised machine learning (171) may be used to train the ANN (125), such as Figure 6 Supervised learning (171) can be used to minimize the classification difference between the prediction (179) made using the ANN (125) based on the sensor data (121) and the expected classification (177).

[0103] For example, in the absence of accidents, near-miss events, or user inputs indicating abnormal conditions, a "normal" classification may be assumed. Accidents, near-miss events, or user inputs may be used to identify expected "abnormal" classifications of sensor data that led to the accidents, events, or user inputs. Supervised machine learning (171) may be used to train an artificial neural network (125) to make classifications (179) that have reduced / minimized variance from expected classifications (177).

[0104] Optionally, the inference engine (109) of the data storage device (112) may be configured to accelerate computation of a portion of an artificial neural network (ANN) (125) implemented in the data storage device (112).

[0105] For example, the inference engine (109) may include a neural network accelerator (159) that is specialized for performing at least a portion of the computations involved in the artificial neural network (ANN) (125), such as dot products, multiplications, and accumulation operations on vectors and tensors.

[0106] Figure 7 A data storage device (112) having a neural network accelerator (159) is shown according to one embodiment. For example, Figure 7 The data storage device (112) can be used to Figure 2 Or in the vehicle (111) described in 3.

[0107] exist Figure 7 In the embodiment, the data storage device (112) has a memory configured to communicate with a host processor (e.g. Figure 3 For example, the communication between the host processor (e.g., 133) and the host interface (157) may be based at least in part on the communication protocol of a peripheral component interconnect express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a universal serial bus (USB) bus, and / or a storage area network (SAN).

[0108] For example, the host computer system (131) may communicate with the host interface (157) to retrieve inference results generated by the data storage device (112) from an input data stream containing sensor data generated by sensors (122) of the vehicle (111).

[0109] For example, the host interface (157) may be used to receive sensor data (121) of the vehicle (111) from the sensor (122); and the sensor data (121) may optionally be stored in the data storage device (112) for analysis of subsequent accident or near-accident events.

[0110] exist Figure 7 , each of the memory components ( 161 - 163 ) may be a memory integrated circuit configured to store data.

[0111] The neural network accelerator (159) and the controller (151) may be implemented via logic circuits formed on one or more integrated circuit dies stacked on the integrated circuit dies of the memory components (161 to 163). Through-silicon vias between the integrated circuit dies of the neural network accelerator (159) and the controller (151) and the integrated circuit dies of the memory components (161 to 163) may be used to provide high communication bandwidth for processing data stored in the memory components (161 to 163) to generate inference results. The inference results may be stored in the local memory (153) of the controller (151) and / or some of the memory components (161 to 163) for retrieval by a host system such as a computer system (131) of the vehicle (111). For example, different memory components (161 to 163) or different sections of a memory component (e.g., 161 or 163) may use through-silicon vias to facilitate parallel access to different portions of the neural network accelerator (159) and the controller (151).

[0112] In general, some memory integrated circuits are volatile and require power to maintain stored data; and some memory integrated circuits are non-volatile and retain stored data even when not powered. The memory components (161-163) may include volatile memory and / or non-volatile memory. The memory components (161-163) may implement different types of memory or the same type of memory.

[0113] Examples of non-volatile memory include flash memory, memory cells formed based on NAND logic gates, NOR logic gates, phase change memory (PCM), magnetic memory (MRAM), resistance random access memory, cross-point storage, and memory devices. Cross-point memory devices may use transistor-free memory elements, each of which has memory cells and selectors stacked together in a row. The memory element columns are connected via two wire layers extending in a vertical direction, wherein the wires of one layer extending in one direction in the layer are positioned above the memory element column, and the wires of the other layer are in the other direction and in the layer positioned below the memory element column. Each memory element can be individually selected at the intersection of a wire on each of the two layers. Cross-point memory devices are fast and non-volatile, and can be used as a unified memory pool for processing and storage. Further examples of non-volatile memory include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM) memory, etc. Examples of volatile memory include dynamic random access memory (DRAM) and static random access memory (SRAM).

[0114] The data storage device (112) may have a controller (151) including a volatile local memory (153) and at least one processing device (155).

[0115] The local memory of the controller (151) may be an embedded memory configured to store instructions for executing various processes, operations, logic flows, and routines for controlling the operation of the processing device (155), including handling communications between the data storage device (112) of the vehicle (111) and the processor (e.g., 133) and other functions described herein. Optionally, the local memory (153) of the controller (151) may include a read-only memory (ROM) for storing microcode and / or memory registers and / or volatile memory such as dynamic random access memory (DRAM) and static random access memory (SRAM) for storing (e.g.,) memory pointers, fetch data, etc.

[0116] exist Figure 7In the invention, the data storage device (112) includes a neural network accelerator (159) coupled to a controller (151) and / or memory components (161-163).

[0117] For example, the neural network accelerator (159) may be configured to perform matrix arithmetic calculations more efficiently than the processing device (155) of the controller (151). Calculations involving the ANN (125) have matrix multiplication and accumulation operations, which can be computationally intensive for general-purpose processors (e.g., 133, 155). Using the neural network accelerator (159) to perform matrix arithmetic calculations can reduce the data transmitted to the processor (133) of the vehicle (111) and reduce the computational workload of the processor (133, 155).

[0118] When the ANN (125) comprises a spiking neural network (SNN), simulation of the differential equations used to control the activation levels of the SNN neurons can be computationally intensive for a general purpose processor (e.g., 133, 155). Optionally, a neural network accelerator (159) can use dedicated hardware to simulate the differential equations and thereby improve the computational efficiency of implementing the SNN.

[0119] In some implementations, the neural network accelerator (159) is an integrated circuit device separate from the controller (151) and / or the memory components (161-163). Alternatively or in combination, the neural network accelerator (159) is integrated with the controller (151) in an integrated circuit die. Alternatively or in combination, a portion of the neural network accelerator (159) may be integrated on an integrated circuit die of at least one of the memory components (161-163), such as Figure 8 In the description.

[0120] Figure 8 A memory component (160) for accelerating neural network computations according to one embodiment is shown. For example, Figure 7 Each or several of the memory components (161 to 163) in Figure 8 The memory component (160) is implemented.

[0121] exist Figure 8 In the embodiment of the present invention, the memory component (160) may be formed on an integrated circuit die. An input / output (I / O) interface (169) of the memory component (160) is configured to process input / output signals of the memory component (160). For example, the input / output signals may include an address signal for specifying a location in the media unit (165) and a data signal representing data written to or retrieved from a location in the media unit (165) specified by the address signal.

[0122] exist Figure 8In the embodiment, the neural network accelerator (159) is coupled to the control logic (167) and / or the media unit (165) to perform calculations for evaluating the output of a portion of the ANN (125) and / or training the ANN (125).

[0123] For example, the input / output interface (169) may receive an address identifying a matrix stored in a media unit and to be operated upon when passing through the neural network accelerator (159). The memory component (160) may provide the computational results of the neural network accelerator (159) as output data in response to the address, store the output data in a buffer for further operation, and store the output data in a location in the media unit (165) specified via the address signal. Thus, the computation performed by the neural network accelerator (159) may be within the memory component (160), which is proximate to the media unit (165) in which the matrix data is stored.

[0124] For example, state data of SNN neurons may be stored in the media unit (165) according to a predetermined pattern. The neural network accelerator (159) may automatically update the state of the SNN neurons according to a differential equation for controlling the activation level of the SNN neurons over time. Optionally, the neural network accelerator (159) is configured to process neuron spikes in the neural network. Alternatively, the data storage device (112) and / or the neural network accelerator (159) of the processor (133) may be configured to process neuron spikes and / or input accumulation of the SNN.

[0125] For example, the sensor (122) generates sensor data (e.g., images) at a predetermined frequency. Each image is stored in the memory components (161-163) in a cyclic manner, where the newest image overwrites the oldest image. The memory components (161-163) further store a portion of the ANN (125) of the vehicle (111) responsible for processing the images from the sensor (122). The controller (151) processes the images in the memory components (161-163) according to the portion of the ANN (125) to generate an inference result. The inference result is stored in the memory components (161-163) and / or the local memory (153) of the controller (151) for reading by a host system, such as the computer system (131) of the vehicle (111).

[0126] For example, the data storage device (112) receives a sensor data stream from at least one sensor (122) configured on the vehicle (111) and generates an inference result based on the sensor data stream according to a portion of the ANN (125) stored in the memory components (161-163). A neural network accelerator (159) configured within the data storage device (112) performs at least a portion of the calculation based on the artificial neural network (125) and the sensor data stream.

[0127] Optionally, the neural network accelerator (159) may be configured on an integrated circuit die separate from the controller (151) and / or separate from the memory components (161-163).

[0128] Optionally, the neural network accelerator (159) may be configured on an integrated circuit die that includes a controller (151) of the data storage device (112) or a memory component (160, 161, or 163) of the data storage device (112).

[0129] The neural network accelerator (159) may be configured to perform computations using data stored in the data storage device (112), such as matrix arithmetic computations for an ANN and / or differential equation simulations for an SNN.

[0130] Examples of matrix arithmetic calculations include matrix multiplication and accumulation operations. After a calculation that produces a result of a matrix arithmetic calculation using data stored in the data storage device (112), the neural network accelerator (159) can provide the result as an output of the data storage device (112) in a data retrieval operation (e.g., in response to a read command). Alternatively or in combination, the result of the matrix arithmetic calculation can be buffered in the data storage device (112) as an operand for the next matrix arithmetic performed in conjunction with a data matrix retrieved from the non-volatile memory via a read command received in the host interface (157).

[0131] When the artificial neural network (ANN) (125) includes a spiking neural network (SNN), the neural network accelerator (159) can be configured to simulate differential equations that control activation levels of neurons in the spiking neural network (SNN). Optionally, the memory component (160) is configured to store states of neurons in the spiking neural network according to a predetermined pattern; and the neural network accelerator is configured to automatically update the states of the neurons over time according to the differential equation. For example, the neural network accelerator (159) can be configured to train the spiking neural network (SNN) to detect anomalies via unsupervised machine learning.

[0132] Computations performed by a neural network accelerator (159) based on an artificial neural network (ANN) (125) involve different types of data with different usage patterns of a data storage device (112).

[0133] For example, making predictions using an artificial neural network (ANN) (125) includes using data specifying a model of the artificial neural network (ANN) (125), input data provided to the artificial neuron, and output data generated by the artificial neuron.

[0134] The memory capacity of the data storage device (112) can be partitioned into different portions for different types of ANN-related data. The different portions can be individually configured to optimize access and storage of the corresponding data according to the mode in which it is used by the neural network accelerator (159) and / or the processor (133) of the computer system (131) in which the data storage device (112) is configured.

[0135] A model of an artificial neural network (ANN) (125) may include parameters that specify static properties of individual artificial neurons in the ANN (125) and connectivity of neurons in the ANN (125). The model data of the ANN (125) is static and does not change during predictive calculations performed using the ANN (125). Therefore, the usage pattern of the model data is mostly read. However, the model data of the ANN (125) may change when an updated ANN (125) is installed. For example, the vehicle (111) may download the updated ANN (125) from the server (119) to the data storage device (112) of the vehicle (111) to update its predictive capabilities. The model data of the ANN (125) may also change during or after training the ANN (125) using machine learning techniques (e.g., 171 or 175). It is preferred to configure separate partitions or regions of the memory of the data storage device (112) to store the model data, wherein the partitions or regions operate according to configuration parameters of the memory cells that optimize a particular usage pattern of the model data (e.g., mostly read, infrequently updated). For example, when the memory cells are implemented using flash memory based on NAND logic gates, the memory cells in the ANN model partition / region can be configured to operate in a multi-level cell (MLC) mode, a triple-level cell (TLC) mode, or a quad-level cell (QLC) mode, where each memory cell stores two, three, or four bits to increase storage capacity.

[0136] Input data provided to the artificial neurons in the ANN (125) may include external inputs and internal inputs. The external inputs are typically generated by sensors (e.g., 122) of the vehicle (111) rather than the artificial neurons in the ANN (125). The external inputs may be stored in a cyclic manner so that input data for the most recent period of a predetermined driving length may be found in the data storage device (112). Therefore, it is preferred to configure a separate memory partition or region of the data storage device (112) to store the external input data, wherein the partition or region operates according to configuration parameters of the memory cells that optimize the storage mode of the external input data (e.g., enhanced endurance, cyclic rewrite). For example, when the memory cells are implemented using flash memory based on NAND logic gates, the memory cells in the ANN input partition / region may be configured to operate in a single-level cell (SLC) mode, wherein each memory cell stores one data bit to improve endurance of cyclic rewrite operations.

[0137] In some embodiments, an artificial neuron may have state variables that change over time in response to input during a prediction calculation. For example, the activation level of a spiking neuron may change over time and be considered a dynamic state variable of the spiking neuron. In some embodiments, this state variable data of the artificial neuron has a storage usage pattern similar to external input data; therefore, the state variable data may be stored in a partition or region configured for external input data. In other embodiments, the state variable data of the artificial neuron is maintained in a buffer and is not stored as frequently as the external input; therefore, another partition / region may be configured for storing the dynamic state variable data of the artificial neuron.

[0138] Output data generated by artificial neurons in the ANN (125) may be buffered for further access by a neural network accelerator (159) and / or a processor (133) of a computer system (131). The output data may include external outputs and internal outputs. External inputs are generated by artificial neurons as outputs from the ANN (125), such as the result of classification or prediction performed by the ANN (125). The outputs of the ANN (125) are typically further processed by a processor (133) of the computer system (131). External inputs may be periodically saved (e.g., in a manner similar to storing state variable data). Internal outputs and / or some external outputs may be internal inputs to artificial neurons in the ANN (125). Generally, it is not necessary to store internal outputs from a buffer of a data storage device to a memory component. In some embodiments, when the buffer capacity of the data storage device (112) is insufficient to hold all state variable data and / or internal outputs, the data storage device (112) may use a swap partition / region to expand the capacity of the buffer. The swap partition / region may be configured to optimize random access and to improve durability.

[0139] The external output and / or dynamic state of the neuron may be stored in a separate output partition or region in a cyclic manner, so that the external output data and / or dynamic state of the neuron may be stored periodically and the most recent sets of external output and / or dynamic state may be found in the data storage device (112). The external output and / or dynamic state of the neuron may be selectively stored because some of such data may be regenerated by the ANN from external inputs stored in an input partition or region. Preferably, the output partition or region is configured to store one or more sets of external output and / or dynamic state that cannot be created by external inputs stored in an input partition or region. When storing data in an input / output partition or region in a cyclic manner, the oldest stored data set is erased to make room for the most recent data set. The ANN input / output partition / region may be configured to optimize a sequential write stream to copy data from a buffer of the data storage device to a memory cell in a memory component of the data storage device.

[0140] Fig. 9 Memory capacity (181) configured to support neural network computations is shown according to one embodiment. For example, Figure 7 The memory capacity (181) of the memory components (161 to 163) of the data storage device (112) may be based on Fig. 9 to support neural network computations.

[0141] Fig. 9 The memory capacity (181) may be implemented using a set of memory components (eg, 161-163) of the data storage device (112).

[0142] A set of regions (183, 185, 187, ...) may be created on a memory capacity (181) of a data storage device (112). Each of the regions (e.g., 183, 185, or 187) corresponds to a named portion of the memory capacity (181). Logical addresses are defined within each region. An address mapping table (191) is configured to map logical addresses defined in the regions (183, 185, 187, ...) to memory components (e.g., Figure 7 Mapping is performed between the physical addresses of the memory cells in 161 to 163) described in the specification.

[0143] The address mapping table (191) may include region optimization settings (192) for regions (183, 185, and 187).

[0144] For example, the ANN model region (183) may be a memory / storage partition configured for model data of an artificial neural network (ANN) (125). The region optimization setting (192) optimizes memory operations in the ANN model region (183) according to the data usage pattern of the ANN model (e.g., centered around mostly reads, infrequent updates).

[0145] For example, the neuron input region (185) may be a memory / storage partition configured for external input data of an artificial neural network (ANN) (125). The region optimization setting (192) optimizes memory operations in the neuron input region (185) according to data usage patterns of the external input data (e.g., enhanced endurance for cyclic rewriting of continuous input data streams supporting sequential writes).

[0146] For example, the neuron output region (187) may be a memory / storage partition configured for external output data provided from an artificial neural network (ANN) (125). The region optimization setting (192) optimizes memory operations in the neuron output region (187) according to data usage patterns of the external output data (e.g., improving durability of periodically rewriting data with random read / write accesses).

[0147] The data storage device (112) includes a buffer configured to store temporary / intermediate data of an artificial neural network (ANN) (125), such as internal inputs / outputs of artificial neurons in the ANN (125).

[0148] Optionally, a swap area may be configured in the memory capacity (181) to expand the capacity of the buffer (152).

[0149] Optionally, the address mapping table (191) includes a mapping between logical memory addresses received in the host interface (157) to access data of the artificial neurons and the identities of the artificial neurons. Thus, a read or write command for accessing one type of data for an artificial neuron in one region may cause the controller 151 to access another type of data for an artificial neuron in another region.

[0150] For example, in response to a request to write external input data for a neuron into the memory capacity (181) of the data storage device (112), the address mapping table (191) may be used to calculate the address of the model parameters of the neuron in the ANN model area (183) and read the model parameters into the buffer (152) to allow the neural network accelerator (159) to perform calculations of the output of the neuron. The output of the neuron may be saved in the buffer (152) as an internal input to other neurons (e.g., to reduce write amplification). In addition, the identities of other neurons connected to the neuron may also be retrieved from the ANN model area (183) into the buffer (152), which allows the neural network accelerator (159) and / or processor to further process the propagation of the output in the ANN (125). Retrieving the model data from the ANN model area (183) may be performed in parallel with storing the external input data in the neuron input area (185). Thus, the processor (133) of the computer system (131) of the vehicle (111) does not have to explicitly send in a read command to retrieve model data from the ANN model area (183).

[0151] Similarly, in response to reading the output data of the neuron, the address mapping table (191) can be used to calculate the address of the model parameters of the neuron stored in the ANN model area (183) and read the model parameters into the buffer (152) to allow the neural network accelerator (159) to apply the internal input in the buffer (152) to perform the calculation of the output of the neuron. The calculated output can be provided as a response to reading the output data of the neuron, and the data storage device (112) does not have to store the output data in the memory components (e.g., 161 to 163). Therefore, the processor (133) and / or the neural network accelerator (159) can control the calculation of the neuron by writing input to the neuron and / or reading output from the neuron.

[0152] In general, the incoming external input data for the ANN (125) may be raw sensor data (121) that is directly generated by a sensor (e.g., 122) and has not been processed by the processor (133) and / or the neural network accelerator (159). Alternatively, indirect sensor data (121) of a signal from a sensor (122) that has been processed by the processor (133) of the ANN (125) may be provided as the external input data. The incoming external input data may be received in the host interface (157) and written to the neuron input area (185) in a round-robin manner and automatically buffered in the buffer (152) of the neural network accelerator (159) to generate neuron outputs using the model stored in the ANN model area (183). The output generated by the neural network accelerator (159) may be further buffered as internal input to further apply the model in the ANN model area (183). When the external output becomes available, the data storage device (112) may report that the write request has been completed with an indication of the availability of the external output. Optionally, the controller 151 and / or the neural network accelerator (159) may generate an internal read command that propagates a signal in the ANN (125) when an external output is generated. Alternatively, the host processor (133) may control the propagation of the signal in the ANN (125) by selectively reading the output of the neuron; and the data storage device (112) may actively buffer the data required to accelerate the ANN calculation in the buffer (152).

[0153] Fig.10 The configuration of the memory area (183) of an artificial neural network (ANN) model according to one embodiment is described. For example, Fig.10 The configuration can be Fig. 9 The logical memory capacity (181) is implemented in Figure 7 In the data storage device (112). For example, Fig.10 The settings (193) can be Fig. 9 Part of the regional optimization settings (192).

[0154] Fig.10The configuration maps the ANN model region (183) to at least one memory component A (161). Preferably, the at least one memory component A (161) can be used in parallel by the controller (151) with the memory components (e.g., 163) of other regions (e.g., 185 and 187) hosting ANN data. For example, the memory component A (161) can be in an integrated circuit package separate from the integrated circuit packages of other regions (e.g., 185 and 187). Alternatively, the memory components (161-163) are formed on separate integrated circuit dies embedded in the same integrated circuit package. Alternatively, the memory components (161-163) can be formed on separate regions of the integrated circuit die, where the separate regions can be operated substantially in parallel (e.g., for reading, for erasing, and / or for writing).

[0155] exist Fig.10 In , setting (193) is optimized for a usage pattern of mostly reads and infrequent updates.

[0156] Fig.11 The configuration of the region (185) for the input of an artificial neuron according to one embodiment is described. For example, Fig.11 The configuration can be implemented in Figure 7 and / or the data storage device (112) described in 9. For example, Fig.11 The settings (195) can be Fig. 9 Part of the regional optimization settings (192).

[0157] Fig.11 The configuration maps the neuron input region (185) to at least one memory component B (163). Preferably, the at least one memory component B (163) can be used in parallel by the controller (151) with the memory components (e.g., 161) of other regions (e.g., 183 and 187) hosting ANN data. For example, the memory component B (163) can be in an integrated circuit package separate from the integrated circuit packages of the other regions (e.g., 183 and 187). Alternatively, the memory components (161-163) are formed on separate integrated circuit dies embedded in the same integrated circuit package. Alternatively, the memory components (161-163) can be formed on separate regions of the integrated circuit die, where the separate regions can be operated substantially in parallel (e.g., for reading, for erasing, and / or for writing).

[0158] exist Fig.11 In the embodiment, setting (195) is optimized for a usage mode that enhances the durability of cyclic sequential overwrite when recording a continuous input data stream sampled at fixed time intervals.

[0159] Fig.12The configuration of the region (187) for output from an artificial neuron according to one embodiment is illustrated. For example, Fig.12 The configuration can be implemented in Figure 7 and / or the data storage device (112) described in 9. For example, Fig.12 The settings (197) can be Fig. 9 Part of the regional optimization settings (192).

[0160] Fig.12 The configuration maps the neuron output region (187) to at least one memory component C (162). Preferably, the at least one memory component C (162) can be used in parallel by the controller (151) with the memory components (e.g., 161 and 163) of other regions (e.g., 183 and 185) hosting ANN data. For example, the memory component C (162) can be in an integrated circuit package separate from the integrated circuit packages of the other regions (e.g., 183 and 185). Alternatively, the memory components (161-163) are formed on separate integrated circuit dies embedded in the same integrated circuit package. Alternatively, the memory components (161-163) can be formed on separate regions of the integrated circuit die, where the separate regions can be operated substantially in parallel (e.g., for reading, for erasing, and / or for writing).

[0161] exist Fig.12 In the example, the settings (197) are optimized for a usage pattern of buffered data that is periodically overwritten with random access. For example, the memory cells are configured via the optimized settings (193-197) to be updated / overwritten at a higher frequency in the neuron output region (187) than in the ANN model region (183) but less frequently than in the neuron input region (185).

[0162] The communication protocol / interface may be configured to allow the data storage device to perform neural network acceleration on the fly with reduced data traffic to the host system.

[0163] For example, a host processor (eg, 133) of the vehicle (111) may provide a write command to the data storage device (112) to store a model of the artificial neural network in a model partition (eg, 183).

[0164] To use the ANN model in classification and / or prediction, the host processor (e.g., 133) of the vehicle (111) may optionally stream input data of the ANN (125) into the neuron input partition (e.g., 185). The neural network accelerator (159) of the storage device (112) may automatically apply the image from the sensor (122) and (if present) the input data from the host processor (133) to the model stored in the ANN model partition (e.g., 183) according to the address mapping table (191). The data storage device (112) makes the calculated output available for propagation in the ANN (125). Preferably, the calculated output can be made available to the neural network accelerator (159) through the buffer (152) without storing the intermediate output in the memory components (e.g., 161 to 163). Therefore, data communication between the host processor (e.g., 133) and the data storage device (112) for transmitting the output of the neuron can be reduced. When the output has been propagated to the output neuron in the ANN (125), the data storage device (112) may provide a response to the request from the host processor (e.g., 133). The response indicates that the external output from the neuron in the ANN (125) is available. In response, the host processor (e.g., 133) of the vehicle (111) may optionally issue a read command to retrieve the external output for further processing.

[0165] Fig.13 Communication between an autonomous vehicle (111) and a data storage device (112) is shown according to one embodiment. For example, Fig.13 The communications described in Figure 1 The data storage device (112) described in or 7 is implemented in Figure 2 or 3 in the vehicle (111).

[0166] exist Fig.13 In the embodiment of the present invention, the processor (133) of the host system can be configured with a simplified set of instructions (201) for performing neural network calculations because some of the calculations involving the ANN (125) are performed by the neural network accelerator (159) within the data storage device (112). There is no need to transfer model data back to the processor (133) during the use of the ANN (125) for prediction and / or classification.

[0167] The sensor (122) may generate a continuous input stream (205) as part of the sensor data (121) of the vehicle (111). The sensor data (121) in the input stream (205) may be generated at fixed predetermined time intervals (e.g., during operation of the vehicle (111)).

[0168] The input stream (205) is applied to an input neuron in the ANN (125). The input neuron in the ANN (125) is configured to accept external input to the ANN (125); and the output neuron is configured to provide external output from the ANN (125).

[0169] The processor (133) can execute instructions (201) to process output data (207) from the data storage device (112) and some sensor data (121).

[0170] For example, the processor (133) may write sensor data (121) as part of an input stream (205) into a neuron input region (185) and read output data (207) generated by a neural network accelerator (159) using ANN data in a model region (183) from an output region (187).

[0171] The data storage device (112) stores the input stream (205) in the neuron input area (185) in a cyclic manner, wherein the oldest input group corresponding to the oldest time of data sampling of the data group currently stored in the neuron input area (185) is erased to store the latest input group.

[0172] For each input data set, the neural network accelerator (159) applies the model of the ANN (125) stored in the ANN model area (183). The neural network accelerator (159) (or the processor (133)) can control the propagation of signals within the neural network. When the output neuron of the ANN (125) generates its output in response to the input data set, the data storage device (112) can provide an indication to the processor (133) that the neuron output is ready for retrieval. The indication can be configured in response to a request by the processor (133) to write the input data set to the neuron input area (185). The processor (133) can optionally retrieve the output data (207) (e.g., based on conditions and / or criteria programmed in the instructions).

[0173] In some embodiments, the trigger parameters are configured in the data storage device (112). When the output parameters in the external output (217) meet the requirements specified by the trigger parameters, the data storage device provides a response to the processor (133) request to write the input data set to the neuron input area (185).

[0174] Fig.14 Communication within a data storage device (112) according to one embodiment is shown. For example, Fig.14 The communication can be implemented in Figure 1 Or the data storage device (112) described in 7.

[0175] exist Fig.14In the embodiment of the present invention, the model area (183) stores a model (213) of the ANN (125). In response to receiving a set of external inputs (215) from the input stream (205) in the buffer (152) at a certain time, the data storage device (112) can write the external inputs (215) to the input area (185) and simultaneously retrieve a neuron model (212) that contains a portion of the ANN model (213) corresponding to the parameters of the input neurons and / or the identities of neurons connected to the input neurons. The buffer (152) allows the neural network accelerator (159) to combine the neuron model (212) and the external inputs (225) to produce an output (227) of the input neurons.

[0176] In general, the neuron output (227) may include a portion of the internal output (216) for further propagation within the ANN (125) and / or a portion of the external output (217) of the processor (133).

[0177] The internal output (216) is stored in the buffer (152) as the internal input (216) for further propagation in the ANN (125) in a manner similar to the generation of the neuron output (227) from the external input (215). For example, a portion of the internal input (216) may cause the controller (151) and / or the neural network accelerator (159) to retrieve the corresponding neuron model (212) associated with the internal input, so that the internal input is applied to the corresponding neuron model (212) in the neural network accelerator (159) to generate its neuron output (227).

[0178] When a full set of external outputs (217) is available in the buffer (152), the external outputs (217) may be stored in the output area (187).

[0179] Optionally, the storage device (112) does not store every set of external outputs (217) corresponding to a set of stored external inputs (215) sampled at a certain moment. For example, the storage device (112) may be configured to store a set of external outputs (217) each time a predetermined number of sets of external inputs (e.g., 215) have been counted. Alternatively or in combination, the processor (133) may determine whether to store the external outputs (217). For example, the storage device (112) may be configured to store the external outputs (217) in response to the processor (133) retrieving the external outputs (217) for further processing. For example, the storage device (112) may be configured to store the external outputs (217) in response to a write command from the processor (133) after the external outputs (217) are processed in the processor (133).

[0180] Fig.15A data storage device (112) is shown configured to monitor an automotive transmission (101) of a vehicle (111) according to one embodiment. For example, Fig.15 The data storage device can be used Figure 1 or 7 data storage device (112) technology to implement Figure 2 Or in the vehicle (111) described in 3.

[0181] exist Fig.15 In the invention, the data storage device (112) includes a neural network accelerator (159) and an ANN model (213) stored in memory components (161 to 163).

[0182] The controller (151) is configured to use a neural network accelerator (159) to perform maintenance prediction by using an ANN model (213) to classify sensor data (121) in conjunction with operating parameters (233).

[0183] The sensor data (121) includes inputs generated by a sensor (103) disposed on the vehicle transmission (101). Optionally, the sensor data (121) may include inputs from other sensors disposed on the vehicle (111).

[0184] Optionally, the sensor data (121) for a period of time is stored in the memory components (161-163). The neural network accelerator (159) is configured to retrieve the sensor data (121) from the memory components (161-163) to perform calculations to identify the classification of the sensor data (121) as normal or abnormal when the data storage device is idle (e.g., when the vehicle is in a parked state).

[0185] The data storage device (112) may be set to a training mode during a period of time when the automotive transmission of a predetermined vehicle has a probability of normal operation above a threshold (e.g., within a predetermined time or number of miles from the vehicle's initial factory delivery). Subsequently, the data storage device (112) may be set to a prediction mode.

[0186] The sensor data (121) received in the host interface (157) along with the operating parameters (233) may initially be stored in the input area (185).

[0187] When the data storage device (112) is in training mode, the pattern of sensor data (121) and operating parameters (233) may be assumed to be normal. The controller (151) uses the neural network accelerator (159) to train the ANN (125) to classify the sensor data (121) and associated operating parameters (233) into clusters that are considered normal. The controller (151) may gradually train the ANN to recognize normal clusters over the duration of the training cycle. The oldest training data that has been used to train the ANN (125) may be erased from the input area (185) to store new training data.

[0188] When the data storage device (112) is in the prediction mode, the controller (151) uses the neural network accelerator (159) to determine whether the ANN (125) classifies the sensor data (121) and the associated operating parameters (233) into any cluster called normal. When the ANN (125) cannot classify the sensor data (121) and the associated operating parameters (233) into any cluster called normal, the sensor data (121) and the associated operating parameters (233) can be identified as abnormal; and the controller (151) can save the abnormal sensor data (121) and the associated operating parameters (233) to the output area (187).

[0189] The computer system (131) of the vehicle (111) may examine the output area (187) to identify abnormal sensor data (121) and associated operating parameters (233). When a set of abnormal sensor data (121) and associated operating parameters (233) are found in the output area (187), the computer system (131) of the vehicle (111) may issue a maintenance warning to a user of the vehicle (111). For example, the maintenance warning may be presented in the infotainment system (149) of the vehicle (111) and / or in the instrument panel of the vehicle (111).

[0190] While the vehicle (111) is in the maintenance service facility (127), a connection to the computer system (131) can be used to download abnormal sensor data (121) and associated operating parameters (233) from the data storage device (112). The downloaded data can be uploaded to the server (119) for analysis. In addition, the vehicle (111) can be inspected for defects in the automotive transmission. Optionally, the abnormal sensor data (121) and associated operating parameters (233) can be wirelessly (113) transmitted to the server (119) via the communication network (117) for remote diagnosis and / or analysis. Based on the inspection and analysis, the sensor data (121) and associated operating parameters (233) downloaded from the data storage device (112) can be confirmed as abnormal, or reclassified as normal. In addition, the condition of the automotive transmission can be classified as abnormal and notified that immediate intervention is required, or classified as abnormal and notified that no immediate intervention is required. The confirmation or reclassification can cause the data storage device (112) to further train the ANN (125) to improve its prediction accuracy.

[0191] Additionally, when a problem is discovered from a maintenance visit, the ANN (125) may be further trained to predict the problem based on the sensor data (121) and associated operating parameters (233) in the output area (187).

[0192] In general, operating conditions of the vehicle (111) may be identified via operating parameters (233). The operating parameters (233) of the vehicle (111) may include the ambient temperature of the vehicle (111), the speed of the vehicle (111), the location of the vehicle (111), the road on which the vehicle (111) is traveling, inputs from some sensors (122) of the vehicle (111), inference results generated based on inputs from some sensors (122), the state of vehicle controls, and the state of components of the vehicle (111), such as the infotainment system (149) and / or the communication device (139) of the vehicle (111). The operating parameters (233) of the vehicle (111) may include the state and operation of an advanced driver assistance system (ADAS) (128) and / or other applications / components running in the vehicle (111).

[0193] Examples of operational parameters (233) may include inference results generated from an input data stream of the data storage device (112) and / or provided in an output data stream of the data storage device (112).

[0194] Fig.16 A method for generating a predictive maintenance warning for an automotive transmission (101) of a vehicle (111) is presented according to one embodiment. For example, Fig.16 The method can be combined with Figure 2 or 3 vehicles (111) Figure 3The computer system (131) or ADAS (128) is implemented in Figure 1 , 7 , 13 or 15 in the data storage device (112).

[0195] At block 251, a data storage device (112) disposed in a vehicle (111) receives sensor data (121) from at least one sensor (103) disposed on a vehicle transmission (101) of the vehicle (111).

[0196] For example, at least one sensor (103) configured on the automobile transmission may include a temperature sensor configured in the transmission oil of the automobile transmission (101), a sensor configured on the input shaft of the automobile transmission (101) to measure the strain, stress or torque applied to the input shaft, a sensor configured on the output shaft of the automobile transmission (101) to measure the strain, stress or torque applied to the output shaft, a sensor configured in the transmission oil of the automobile transmission (101) to measure vibration, or a microphone configured on the chassis of the automobile transmission (101) to measure noise, or any combination thereof.

[0197] For example, a host interface (157) or another communication interface of the data storage device (112) may be configured to receive sensor data (121) from at least one sensor (103) or a processor of a computer system (131) of the vehicle (111).

[0198] At block 253, the data storage device (112) determines an operating parameter (233) of the vehicle (111) associated with sensor data (121) from at least one sensor (103) disposed on the vehicle transmission (101).

[0199] For example, operating parameters (233) may include the ambient temperature of autonomous vehicle (111), the speed of autonomous vehicle (111), the position of autonomous vehicle (111), input from sensors (122) configured on autonomous vehicle (111), the status of vehicle controls (e.g., 141, 143, 145), the status of infotainment system (149) of autonomous vehicle (111), or the status of advanced driver assistance system (128) of autonomous vehicle (111), or any combination thereof.

[0200] The operating parameters (233) may be determined by the computer system (131) of the vehicle (111) and provided to the data storage device (112) in an input data stream. The vehicle (111) may provide input from other sensors (122) to the data storage device (112); and some operating parameters (233) may be determined by the data storage device (112) from input data from sensors (122) disposed on the vehicle (111).

[0201] At block 255, the data storage device (112) determines whether operation of the vehicle transmission (101) is abnormal based on sensor data (121) from at least one sensor (103) disposed on the vehicle transmission (101) and operating parameters using the artificial neural network (125).

[0202] For example, the artificial neural network (125) may include a spiking neural network.

[0203] For example, during an initial period of use of the vehicle (111) (e.g., within a predetermined time or number of miles from the vehicle's factory delivery), the vehicle (111) may be assumed to be operating normally; and the sensor data (121) may be considered to exhibit a normal pattern. The data storage device (112) may be configured in a first mode, wherein the neural network accelerator (159) is configured to train an artificial neural network (125) to classify the sensor data stored in the data storage device (112) as normal. Subsequently, the data storage device (112) is configured in a second mode, wherein the neural network accelerator (159) is configured to determine whether the sensor data stored in the data storage device (112) is normal or abnormal using the artificial neural network (125) trained in the first mode.

[0204] For example, the neural network accelerator (159) is configured to perform matrix arithmetic calculations more efficiently than the controller (151).

[0205] For example, the storage capacity (181) of the data storage device (112) may include one or more memory components (161, ..., 163). The storage capacity (181) may be configured to have an input area (185) and an output area (187).

[0206] Sensor data received from at least one sensor (103) disposed on a vehicle transmission (101) may be cyclically stored in an input area (185) of a data storage device (112), wherein the oldest portion of the sensor data stored in the input area (185) is erased to store an incoming portion of the sensor data (121) received from the at least one sensor (103) disposed on the vehicle transmission (101).

[0207] For example, in response to the artificial neural network (125) classifying sensor data (121) of at least one sensor (103) configured on the automobile transmission (101) as abnormal based on operating parameters (233) of the vehicle (111), the controller (151) of the data storage device (112) stores the sensor data (121) of the vehicle (111) and the associated operating parameters (233) in an output area (187) of the data storage device (112).

[0208] At block 257, a maintenance warning for the vehicle (111) is generated in response to determining, based on the artificial neural network (125), that an operational anomaly of the automotive transmission (101) is present.

[0209] For example, the computer system (131) of the vehicle (111) may check the output area (187) of the data storage device (112) for the presence of data classified as abnormal by the artificial neural network (125). The computer system (131) of the vehicle (111) may present a warning for a maintenance visit in response to detecting the presence of data classified as abnormal by the artificial neural network (125).

[0210] Optionally, the computer system (131) of the vehicle (111) transmits sensor data (121) in the output area (187) classified as abnormal by the artificial neural network (125) from the vehicle (111) to the server (119) via the wireless connection (113). The transmitted data can be analyzed in the server (119) to generate a diagnosis result. Alternatively, when the vehicle (111) visits the maintenance service facility (127), the abnormal sensor data (121) can be downloaded from the output area (187) at the maintenance service facility (127) and uploaded to the server (119) for analysis. When the diagnosis result is available, the artificial neural network (125) in the data storage device (112) and / or the server (119) can be further trained to predict the diagnosis result based on the abnormal sensor data (121).

[0211] The server ( 119 ), computer system ( 131 ), and / or data storage device ( 112 ) may each be implemented as one or more data processing systems.

[0212] The present invention includes methods and apparatus for performing the above methods, including a data processing system for performing these methods and a computer-readable medium containing instructions that, when executed on the data processing system, cause the system to perform these methods.

[0213] A typical data processing system may include an interconnect (such as a bus and system core logic) that interconnects a microprocessor and memory. A microprocessor is typically coupled to a cache memory.

[0214] The interconnect interconnects the microprocessor and memory together and also interconnects the microprocessor and memory to input / output (I / O) devices via an I / O controller. The I / O devices may include a display device and / or peripheral devices such as a mouse, keyboard, modem, network interface, printer, scanner, camera, and other devices known in the art. In one embodiment, when the data processing system is a server system, some I / O devices such as a printer, scanner, mouse, and / or keyboard are optional.

[0215] The interconnect may include one or more buses connected to each other through various bridges, controllers and / or adapters. In one embodiment, the I / O controller includes a USB (Universal Serial Bus) adapter for controlling USB peripherals and / or an IEEE-1394 bus adapter for controlling IEEE-1394 peripherals.

[0216] The memory may include one or more of the following: ROM (Read Only Memory), volatile RAM (Random Access Memory), and non-volatile memory such as hard disk, flash memory, etc.

[0217] Volatile RAM is typically implemented as dynamic RAM (DRAM) that requires continuous power to refresh or maintain the data in the memory. Nonvolatile memory is typically a magnetic hard drive, a magnetic optical drive, an optical drive (such as DVD RAM), or other types of memory systems that maintain data even after power is removed from the system. Nonvolatile memory can also be random access memory.

[0218] The non-volatile memory may be a local device directly coupled to the rest of the components in the data processing system. Non-volatile memory that is remote from the system may also be used, such as a network storage device coupled to the data processing system through a network interface (such as a modem or Ethernet interface).

[0219] In the present invention, some functions and operations are described as being executed by software codes or caused by software codes to simplify the description. However, such expressions are also used to specify that the functions are caused by a processor such as a microprocessor executing code / instructions.

[0220] Alternatively or in combination, the functions and operations described herein may be implemented using dedicated circuitry, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), with or without software instructions. Embodiments may be implemented using hardwired circuitry without or in combination with software instructions. Thus, the techniques are neither limited to any specific combination of hardware circuitry and software nor to any particular source for the instructions executed by the data processing system.

[0221] While one embodiment may be implemented on fully functional computers and computer systems, various embodiments are capable of being distributed as computing products in various forms and can be used regardless of the specific type of machine or computer readable media used to actually cause the distribution.

[0222] At least some aspects disclosed may be embodied at least in part in software. That is, the techniques may be implemented in a computer system or other data processing system in response to its processor (e.g., a microprocessor) executing sequences of instructions contained in a memory (e.g., ROM, volatile RAM, non-volatile memory, cache, or remote storage device).

[0223] The routines executed to implement the embodiments may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions (referred to as a "computer program"). A computer program generally comprises one or more sets of instructions stored at various times in various memories and storage devices in a computer, and when read and executed by one or more processors in a computer, the set of instructions causes the computer to perform the operations required to execute the elements involved in various aspects.

[0224] Machine-readable media can be used to store software and data that, when executed by a data processing system, causes the system to perform various methods. Executable software and data can be stored in various locations including, for example, ROM, volatile RAM, non-volatile memory, and / or cache. Portions of this software and / or data can be stored in any of these storage devices. In addition, data and instructions can be obtained from a centralized server or a peer-to-peer network. Different portions of data and instructions can be obtained from different centralized servers and / or peer-to-peer networks at different times and in different communication sessions or the same communication session. Data and instructions can be obtained completely before executing the application. Alternatively, portions of data and instructions can be obtained dynamically only when needed for execution. Therefore, there is no need for data and instructions to be completely on machine-readable media at a specific moment.

[0225] Examples of computer-readable media include, but are not limited to, non-transitory, recordable and non-recordable media such as volatile and non-volatile memory devices, read-only memory (ROM), random access memory (RAM), flash memory devices, floppy disks and other removable disks, magnetic disk storage media, optical storage media such as compact disk read-only memory (CD ROM), digital versatile disks (DVD), etc., etc. A computer-readable medium may store instructions.

[0226] The instructions may also be embodied in digital and analog communication links using electrical, optical, acoustic or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). However, propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) are not tangible machine-readable media and cannot be configured to store instructions.

[0227] Generally speaking, a machine-readable medium includes any mechanism that provides (i.e., stores and / or transmits) information in a form accessible by a machine (e.g., a computer, a network device, a personal digital assistant, a manufacturing tool, any device with a set of one or more processors, etc.).

[0228] In various embodiments, hard-wired circuitry may be used in conjunction with software instructions to implement the technology. Thus, the technology is neither limited to any specific combination of hardware circuitry and software nor to any specific source for the instructions executed by the data processing system.

[0229] The above description and drawings are for illustration only and should not be construed as limiting. Numerous specific details are described to provide a thorough understanding. However, in certain examples, well-known or conventional details are not described to avoid obscuring the description. References to one or an embodiment in the present invention are not necessarily references to the same embodiment; and such references mean at least one.

[0230] In the above description, the invention has been described with reference to specific exemplary embodiments of the invention. It should be understood that various modifications may be made to the invention without departing from the broader spirit and scope set forth in the appended claims. The specification and drawings should accordingly be regarded as intended to be illustrative rather than restrictive.

Claims

1. An automobile vehicle, comprising: axis; an assembly configured to drive the shaft; at least one sensor configured to monitor the component during operation of the component to generate sensor data; a processor configured to determine an output of an artificial neural network in response to the sensor data as input and to generate a maintenance warning in response to the output indicating abnormal operation of the component; and A device is configured to present the maintenance warning.

2. The automotive vehicle of claim 1, wherein the shaft is a first shaft; and the automotive vehicle further comprises: A second shaft, wherein the assembly is disposed between the first shaft and the second shaft to transfer torque from the second shaft to the first shaft.

3. The automotive vehicle of claim 1, wherein the device comprises an infotainment system.

4. The automotive vehicle of claim 1, wherein the device comprises a dashboard.

5. The automotive vehicle according to claim 1, further comprising: A memory component is configured to store the sensor data.

6. The automotive vehicle of claim 5, wherein the at least one memory component is further configured to store a model of the artificial neural network; and the output is generated according to the model.

7. The automotive vehicle of claim 6, wherein the at least one memory component is further configured to have an input area to cyclically store the sensor data to provide input to the artificial neural network to periodically generate output.

8. The automotive vehicle of claim 7, wherein the at least one memory component is further configured to have an output area; and the processor is further configured to copy a portion of the sensor data indicating that the operation is abnormal from the input area to the output area.

9. The automotive vehicle of claim 8, wherein the artificial neural network is trained using sensor data from the at least one sensor collected during a predetermined period associated with normal operation of the component to establish a pattern of the normal operation for detecting the abnormal operation outside of the pattern.

10. The automotive vehicle of claim 9, wherein the at least one sensor comprises a temperature sensor, a torque sensor, a vibration sensor, or a microphone, or any combination thereof.

Citation Information

Cited By

  • System and method for assessing vehicle damage

    US20240161066A1