Method for controlling automatic acceleration and braking of a vehicle and corresponding vehicle control system

Abstract

A method for controlling automatic acceleration and braking of a vehicle (10), the method comprising: Obtaining an image using at least one vehicle camera (26) of a host vehicle (10); Extracting machine learning model feature inputs based on the obtained image; Detecting one or more objects (112, 114) in the obtained image, wherein the one or more objects (112, 114) comprise at least one pedestrian (112); Assigning attention weights to regions of the obtained image according to positions of the one or more objects (112, 114) in the obtained image; Combining the attention weights with corresponding machine learning model feature inputs according to the regions of the obtained image; and executing a machine learning model to generate an output of a prediction of a crossing intention associated with the at least one pedestrian; characterized by Controlling automatic braking of the host vehicle (10) according to a position of the at least one pedestrian (112) in response to outputting a prediction of crossing intent that exceeds a crossing intent threshold; where: (i) combining the attention weights and the machine learning model feature inputs comprises generating a weighted sum of the machine learning model feature inputs according to the attention weights; and / or (ii) running the machine learning model includes: Obtaining multiple key values ​​according to the machine learning model feature inputs, executing a classification query according to the machine learning model feature inputs, and correlating a classification query output with the multiple key values.

Description

Summary

[0001] The present invention relates generally to a method for controlling automatic acceleration and braking of a vehicle and to a corresponding vehicle control system.

[0002] A generic method is essentially described in GB 2 573 738 A.

[0003] Further prior art can be found in the documents US 2007 / 0 206 849 A1 and CN 1 14 494 942 A.

[0004] Some vehicles have in-vehicle cameras configured to capture images of an area in front of the vehicle along a path of travel. Automated driving systems can be configured to control vehicle acceleration, braking, etc., based on objects in front of the vehicle. Pedestrians are often standing and waiting to cross a road on which the vehicle is traveling and can pose a collision risk if a pedestrian crosses the road while the vehicle is moving. Summary

[0005] According to the invention, a method for controlling automatic acceleration and braking of a vehicle is presented, which is characterized by the features of claim 1.

[0006] In other features, assigning the attention weights comprises assigning a first intensity value to a first region of the obtained image corresponding to the at least one pedestrian and assigning a second intensity value to a second region of the obtained image not corresponding to the at least one pedestrian, wherein the first intensity value is greater than the second intensity value.

[0007] In other features, the machine learning model includes a multi-layer perceptron, and executing the machine learning model includes generating the output of a prediction of a crossing intention according to an output of the multi-layer perceptron.

[0008] In other features, extracting machine learning model feature inputs based on the obtained image includes feeding the obtained image to multiple visual transformer layers to generate the machine learning model feature inputs.

[0009] In other features, executing the machine learning model includes combining the attention weights with a correlation of the classification query output and the plurality of key values, and executing a normalized exponential function on a combination of the attention weights and the correlation of the classification query output and the plurality of key values ​​to produce the output of a prediction of a crossing intention.

[0010] In other features, executing the machine learning model includes combining an output of the normalized exponential function with the plurality of key values ​​to generate an embedding vector, and feeding the embedding vector to a multi-layer perceptron to generate the output of a prediction of crossing intention.

[0011] In other features, feeding training data and test data to the machine learning model, comparing multiple crossing intent prediction outputs of the machine learning model based on the training data with labeled crossing intent outputs of the test data, determining whether an accuracy of a comparison is greater than or equal to a specified accuracy threshold, adjusting parameters of the machine learning model and retraining the machine learning model in response to determining that the accuracy of the comparison is less than the specified accuracy threshold, and storing the machine learning model for use in generating a crossing intent prediction output in response to determining that the accuracy of the comparison is greater than or equal to the specified accuracy threshold.

[0012] In other features, the image includes at least a 45-degree field of view from the at least one vehicle camera. In other features, the one or more objects include at least one pedestrian crossing, a traffic light, or another vehicle.

[0013] Furthermore, a vehicle control system for controlling vehicle braking based on image processing of a vehicle camera is presented, which is characterized by the features of claim 8.

[0014] In other features, the vehicle control module is configured to assign the attention weights by assigning a first intensity value to a first region of the obtained image corresponding to the at least one pedestrian and assigning a second intensity value to a second region of the obtained image not corresponding to the at least one pedestrian, wherein the first intensity value is greater than the second intensity value.

[0015] For other features, the vehicle control module is configured to assign the attention weights and the machine learning model feature inputs by generating a weighted sum of the machine learning model feature inputs according to the attention weights.

[0016] In other features, the vehicle control module is configured to extract the machine learning model feature inputs based on the obtained image by feeding the obtained image to multiple visual transformer layers to generate the machine learning model feature inputs.

[0017] In other features, the vehicle control module is configured to execute the machine learning model by combining the attention weights with a correlation of the classification query output and the plurality of key values, and execute a normalized exponential function on a combination of the attention weights and the correlation of the classification query output and the plurality of key values ​​to generate the output of a prediction of a crossing intention.

[0018] In other features, the vehicle control module is configured to execute the machine learning model by combining an output of the normalized exponential function with the plurality of key values ​​to generate an embedding vector and feeding the embedding vector to a multi-layer perceptron to generate the intersection prediction output.

[0019] In other features, the image includes at least a forty-five degree field of view from the at least one vehicle camera, and the one or more objects include at least one crosswalk, a traffic light, or another vehicle.

[0020] Further areas of applicability of the present invention will become apparent from the detailed description, claims, and drawings. The detailed description and specific examples are provided for illustrative purposes only. Short description of the drawings

[0021] The present invention will be more fully understood from the detailed description and the accompanying drawings, in which: Fig. 1 is an illustration of an exemplary vehicle including a front vehicle camera configured to capture images of pedestrians and traffic objects in front of the vehicle; Fig. 2 is a block diagram of an exemplary system for predicting a pedestrian's crossing intent based on image processing of a vehicle camera image; Fig. 3 is a block diagram illustrating an exemplary architecture of a machine learning model of the system of Fig. 2 illustrates; Fig. 4 is a flowchart illustrating an exemplary process for predicting a pedestrian's crossing intention based on image processing of a vehicle camera image: Fig. 5A and Fig. 5B are graphical representations of exemplary recurrent neural networks for predicting a pedestrian's crossing intention based on image processing of a vehicle camera image; Fig. 6 is a graphical representation of layers of an exemplary long short-term memory (LSTM) machine learning model; and Fig. 7 is a flowchart illustrating an example process for training a machine learning model.

[0022] Reference numerals may be reused in the drawings to identify similar and / or identical elements. Detailed description

[0023] Pedestrians often stand and wait to cross a road on which the vehicle is currently traveling, which can pose a collision risk if a pedestrian crosses the road while the vehicle is moving. Some example embodiments herein include systems and methods for estimating pedestrian crossing intentions using a vehicle camera image with a wide field of view (FOV). The wide FOV image effectively captures the interactions between the pedestrian and various traffic objects, such as the road, a crosswalk, traffic lights, other vehicles, bus stops, etc.

[0024] However, the wide FOV image also contains additional pedestrians and unrelated information. The example embodiments described herein may focus a machine learning model (e.g., a neural network) on the pedestrian and the traffic objects of interest to estimate the pedestrian's crossing intent.

[0025] Referring now to Fig. 1, a vehicle 10 comprises front wheels 12 and rear wheels 13. In Fig. 1, a drive unit 14 selectively delivers torque to the front wheels 12 and / or the rear wheels 13 via drive systems 16 and 18, respectively. The vehicle 10 may include various types of drive units. For example, the vehicle may be an electric vehicle such as a battery electric vehicle (BEV), a hybrid vehicle or a fuel cell vehicle, an internal combustion engine (ICE), or another type of vehicle.

[0026] Some examples of the drive unit 14 may include any suitable electric motor, a power inverter, and a motor controller configured to control power switches within the power inverter to adjust the motor speed and torque during propulsion and / or regeneration. A battery system provides or receives power from the electric motor of the drive unit 14 via the power inverter during propulsion or regeneration.

[0027] While the vehicle 10 in Fig. 1 has a drive unit 14, the vehicle 10 may have other configurations. For example, two separate drive units may drive the front wheels 12 and the rear wheels 13, one or more separate drive units may drive individual wheels, etc. As will be appreciated, other vehicle configurations and / or drive units may also be used.

[0028] The vehicle control module 20 may be configured to control the operation of one or more vehicle components, such as the drive unit 14 (e.g., by commanding torque settings of an electric motor of the drive unit 14). The vehicle control module 20 may receive inputs to control components of the vehicle, such as signals received from a steering wheel, an accelerator paddle, a vehicle camera, etc. The vehicle control module 20 may monitor vehicle telematics data, such as vehicle speed, vehicle location, vehicle braking and acceleration, etc., for safety purposes.

[0029] The vehicle control module 20 may receive signals from any suitable components for monitoring one or more aspects of the vehicle, including one or more vehicle sensors (such as cameras, microphones, pressure sensors, wheel position sensors, location sensors such as GPS antennas, etc.). Some sensors may be configured to monitor the vehicle's current motion, vehicle acceleration, steering torque, etc.

[0030] As in Fig. 1, the vehicle 10 includes a front vehicle camera 26 configured to capture images of a field of view in front of the vehicle 10. The field of view may be a wide field of view (such as at least a thirty-degree field of view, a forty-five-degree field of view, a sixty-degree field of view, a ninety-degree field of view, etc.) to capture objects on the sides of a road on which the vehicle 10 is traveling.

[0031] The vehicle 10 may include an optional rear vehicle camera 24 and an optional side vehicle camera 28. In various implementations, the vehicle 10 may include more or fewer of these optional vehicle cameras. The vehicle 10 may include any suitable laser, lidar sensor, etc., used to detect objects in the environment of the vehicle 10.

[0032] In some example embodiments, a vehicle object detector may be configured to detect a closest in-path vehicle (CIPV) (e.g., another vehicle ahead of the current path of vehicle 10), a vulnerable road user (VRU) (e.g., a pedestrian or cyclist), etc. The vehicle control module 20 may be configured to control the movement of vehicle 10 based on a detected CIP target vehicle, such as by increasing or decreasing the automatic acceleration of vehicle 10, automatically applying the brakes of vehicle 10 (such as in response to an imminent braking event in a collision), etc.

[0033] The vehicle control module 20 may communicate with another device or apparatus via a wireless communication interface, which may include one or more wireless antennas for transmitting and / or receiving wireless communication signals. For example, the wireless communication interface may communicate via any suitable wireless communication protocol, including, but not limited to, vehicle-to-everything (V2X) communication, Wi-Fi communication, wireless area network (WAN) communication, cellular communication, personal area network (PAN) communication, short-range wireless communication (e.g., Bluetooth), etc.The wireless communication interface may communicate with a remote computing device over one or more wireless and / or wired networks. With regard to vehicle-to-vehicle (V2X) communication, the vehicle 10 may include one or more V2X transceivers (e.g., V2X signal transmitting and / or receiving antennas).

[0034] The vehicle 10 also includes a user interface. The user interface may include any suitable display (such as on a dashboard, console, or elsewhere), a touchscreen or other input device, speakers for generating audio signals, etc.

[0035] Fig. Figure 2 is a block diagram of an exemplary system 100 for predicting a pedestrian's crossing intention based on image processing of a vehicle camera image. The components of system 100 may be implemented, for example, by the vehicle control module 20 of Fig. 1 can be executed.

[0036] The vehicle control module 20 is configured to receive a vehicle camera image 102 from a vehicle camera, such as the front vehicle camera 26. At 104, the vehicle control module 20 is configured to extract features for image processing using a machine learning model. As explained further below, the vehicle camera image 102 may, for example, be fed to multiple visual transformer layers to generate a plurality of output features 106. In some examples, the plurality of output features 106 may be arranged in a three-dimensional array with M by N rows and columns and C channels.

[0037] In a separate process from feature extraction, the vehicle camera image 102 is fed to a traffic object detection model 108 to detect traffic and pedestrian objects in the vehicle camera image 102. For example, one or more object detection algorithms, machine learning models, etc., may process the vehicle camera image 102 to identify pedestrians and traffic objects in the image. Example traffic objects may include, but are not limited to, traffic lights, crosswalks, sidewalks, vehicles, traffic signs, etc.

[0038] Fig. Figure 2 illustrates a multi-region object detection output 110 with identified objects in hatched boxes. For example, a detected pedestrian 112 may be identified in a vertical bounding box corresponding to three regions of the object detection output 110, and a detected pedestrian crossing object 114 may be identified by a horizontal bounding box corresponding to six other regions of the object detection output 110.

[0039] The vehicle control module 20 is configured to assign attention weights to regions of the vehicle camera image 102 containing detected objects. For example, the vehicle camera image 102 may be divided into a grid of regions (e.g., squares, patches, etc.) in the object detection output 110, and regions in the grid corresponding to detected objects (such as the crosswalk object 114) or pedestrians (such as the pedestrian 112) may be assigned higher attention weights than regions that do not have corresponding detected objects.

[0040] The vehicle control module 20 is configured to combine the attention weights with the extracted features (e.g., the multiple output features 106) for each image region at 116. For example, a weighted sum may be generated by applying the assigned attention weight for each region to the extracted output features of the image region. In this example, a higher intensity due to the higher attention weights may bias or influence the machine learning model to focus on areas of the image where detected pedestrians or traffic objects are located.

[0041] The vehicle control module 20 is configured to execute a machine learning model to generate an output predicting a crossing intent. For example, the multiple output features 106, as modified by the attention weights from the object detection output 110, may be fed to a multi-layer perceptron (MLP) 118 or other suitable machine learning model to generate an output 120 predicting a pedestrian's crossing intent.

[0042] The pedestrian crossing intent prediction output 120 may indicate a probability that an identified pedestrian (such as pedestrian 112) intends to cross a roadway on which the host vehicle is currently traveling. For example, the vehicle control module 20 may determine whether the pedestrian crossing intent prediction output 120 is greater than a specified threshold (such as a 30% probability of crossing intent, a 50% probability of crossing intent, an 80% probability of crossing intent, etc.).

[0043] The crossing intent may be used by various systems in the vehicle 10, such as displaying the identified pedestrian on a display of the vehicle along with an indicator of the predicted crossing intent (so that the driver recognizes a pedestrian who may be planning to cross the road in front of the vehicle), generating an audible or tactile warning to the driver, controlling automatic braking to apply the vehicle brakes to decelerate the vehicle as the vehicle approaches the identified pedestrian, etc.

[0044] For example, if automatic braking is enabled in the vehicle, the vehicle control module 20 may be configured to apply the brakes of the vehicle 10 according to the position of the detected pedestrian (e.g., based on the time to collision with the detected pedestrian 112, etc.).

[0045] Fig. 3 is a block diagram illustrating an exemplary architecture 200 of a machine learning model of the system of Fig. 2. A vehicle camera image 202 is fed to multiple visual transformer layers 204 to produce extracted output features 206 (which may be arranged in an array of M by N rows and columns, each element having C channels).

[0046] The vehicle camera image 202 is also fed to a traffic object detection model 108 to generate an object detection output 210. The object detection output 210 includes a detected pedestrian 212 and a detected crosswalk object 214, but may also include other types of detected objects.

[0047] At 222, the vehicle control module 20 is configured to assign attention weights for crossing intent to various regions of the vehicle camera image 202. For example, regions of the vehicle camera image corresponding to detected traffic objects, such as the pedestrian crossing object 214, may have a higher probability of being relevant for accurately predicting whether the pedestrian 212 intends to cross the road.

[0048] As in Fig. 3, a plurality of keys 224 may be obtained from the extracted output features 206, and a classification query (CLS) 226 may be run on the extracted output features 206. The plurality of keys 224 (which may be represented by (NM)xC parameters) may be correlated with the CLS query 226 (which may be represented by a channel Cx1) to generate a correlated output (NM).

[0049] The correlated output can be combined with the attention weights and fed to a normalized exponential function 230 (e.g., a normalized softmax function) to produce an output of dimension (NM).

[0050] This output can be combined with values ​​228 of the extracted output features 206 to generate an embedding vector of dimension (C). The embedding vector is fed to a classification MLP head 218 to generate the crossing intention prediction output 220.

[0051] Fig. 4 is a flowchart illustrating an exemplary method for predicting a pedestrian's crossing intention based on image processing of a vehicle camera image. Fig. 4 can be executed by, for example, the vehicle control module 20 of Fig. 1. At 404, the process begins by receiving an image from a vehicle camera, such as the front vehicle camera 26, from Fig. 1. is received.

[0052] The vehicle control module is configured to extract features for image processing by a machine learning model at 408. For example, the image obtained from the vehicle camera may be fed to multiple visual transformer layers to generate multiple output features. In some examples, the multiple output features may be arranged in a three-dimensional array with M by N rows and columns and C channels.

[0053] The vehicle control module is configured to detect traffic and pedestrian objects in the received image at 412. For example, one or more object detection algorithms, machine learning models, etc., can process the received image to identify pedestrians and traffic objects in the image. Example traffic objects may include, but are not limited to, traffic lights, crosswalks, sidewalks, vehicles, traffic signs, etc.

[0054] The vehicle control module is configured to assign attention weights to 416 regions of the image containing detected objects. For example, the image can be divided into a grid of regions (e.g., squares, patches, etc.), and regions in the grid corresponding to detected objects or pedestrians can be assigned higher attention weights than regions that do not contain corresponding detected objects.

[0055] The vehicle control module is configured to combine the attention weights with the extracted features for each image region at 420. For example, a weighted sum may be generated by applying the assigned attention weight for each region to the extracted output features of the image region. In this example, higher intensity due to the higher attention weights may bias or influence the machine learning model to focus on areas of the image where detected pedestrians or traffic objects are located.

[0056] The vehicle control module is configured to execute a machine learning model at 424 to generate an output predicting a crossing intent. For example, the output features, as modified by the attention weights, may be fed to a multi-layer perceptron (MLP) or other suitable machine learning model to generate an output predicting a pedestrian's crossing intent.

[0057] The output of a pedestrian crossing intent prediction may indicate the probability that an identified pedestrian intends to cross a road on which the host vehicle is currently traveling. At 428, the controller determines whether the output of a pedestrian crossing intent prediction is greater than a specified threshold (e.g., a 30% probability of crossing intent, a 50% probability of crossing intent, an 80% probability of crossing intent, etc.).

[0058] If the output of a pedestrian crossing intent prediction is not greater than the specified crossing intent threshold, control returns to 404 to obtain another image from the vehicle camera. If, at 428, the output of a pedestrian crossing intent prediction is greater than the specified crossing intent threshold, control proceeds to 432 to assign a crossing intent to the identified pedestrian.

[0059] Crossing intent can be used by various systems in the vehicle, such as displaying the identified pedestrian on a vehicle display along with a predicted crossing intent indicator (so that the driver detects a pedestrian who may be planning to cross the road in front of the vehicle), generating an audible or tactile warning to the driver, controlling automatic braking to apply vehicle brakes to slow the vehicle as the vehicle approaches the identified pedestrian, etc.

[0060] For example, the vehicle control module is configured to determine at 436 whether automatic braking is enabled in the vehicle. If so, control proceeds to 440 to apply the vehicle's brakes according to the position of the detected pedestrian (e.g., based on the time to collision with the identified pedestrian, etc.). Control then returns to 404 to obtain another image from the vehicle camera.

[0061] In various implementations, machine learning models can be used to generate outputs predicting a pedestrian's crossing intention. Examples of different types of machine learning models that can be used for automated processing of vehicle camera images are described below in Fig. 5-7; however, the exemplary embodiments are not limited thereto (and may include transformer layers, multilayer perceptrons, etc.).

[0062] Fig. 5A and Fig. 5B shows an example of a recurrent neural network used to generate models such as those described above using machine learning techniques. Machine learning is a method used to design complex models and algorithms suitable for prediction (e.g., patient-provider matching predictions). Models generated using machine learning, such as those described above, can produce reliable, repeatable decisions and outcomes and uncover hidden insights by learning from historical relationships and trends in the data.

[0063] The purpose of using a recurrent neural network-based model and training it using machine learning, as described above, can be to directly predict dependent variables without mathematically representing relationships between the variables. A neural network model comprises a large number of virtual neurons operating in parallel and arranged in layers. The first layer is the input layer and receives raw, or unprocessed, input data. Each subsequent layer modifies the output of a previous layer and sends it to the next layer. The final layer is the output layer and generates the system's output.

[0064] Fig. Figure 5A shows a fully connected neural network, where each neuron in a particular layer is connected to every neuron in the next layer. In the input layer, each input node is associated with a numerical value, which can be any real number. In each layer, each connection emanating from an input node has an associated weight, which can also be any real number (see Fig. 5B). In the input layer, the number of neurons is equal to the number of features (columns) in a dataset. The output layer can have multiple continuous outputs.

[0065] The layers between the input and output layers are hidden layers. The number of hidden layers can be one or more (one hidden layer is likely sufficient for most applications). A neural network without hidden layers can represent linearly separable functions or decisions. A neural network with one hidden layer can perform a continuous mapping from one finite space to another. A neural network with two hidden layers can approximate any smooth mapping with arbitrary accuracy.

[0066] The number of neurons can be optimized. At the beginning of training, a network configuration is more likely to have too many nodes. Some nodes that would not noticeably affect the network's performance can be removed from the network during training. For example, nodes with weights approaching zero after training can be removed (this process is called pruning). The number of neurons can lead to underfitting (inability to adequately capture signals in the dataset) or overfitting (insufficient information to train all neurons; the network operates on a training dataset but not a test dataset).

[0067] Various methods and criteria can be used to measure the performance of a neural network model. For example, the root mean squared error (RMSE) measures the average distance between the observed values ​​and the model predictions. The coefficient of determination (R2) measures the correlation (not the accuracy) between observed and predicted outcomes. This method may not be reliable if the data has a large variance. Other performance measures include irreducible noise, model bias, and model variance. High model bias for a model indicates that the model is unable to capture the true relationship between the predictors and the outcome. Model variance can indicate whether a model is stable (a small perturbation in the data significantly changes the model fit).The neural network can receive inputs, such as vectors, that can be used to generate models that can be used with vendor matching, risk model processing, or both, as described here.

[0068] Fig. 6 illustrates an example long short-term memory (LSTM) neural network 602 used to generate models such as those described above using machine learning techniques, although other example embodiments may include other types of machine learning models, including transformer layers, multi-layer perceptrons, etc. The generic example LSTM neural network 602 may be used to implement a machine learning model, and various implementations may utilize other types of machine learning networks (such as transformer layers, multi-layer perceptrons, etc.). The LSTM neural network 602 includes an input layer 604, a hidden layer 608, and an output layer 612. The input layer 604 includes inputs 604a, 604b, ... 604n. The hidden layer 608 contains the neurons 608a, 608b... 608n.The output layer 612 contains the outputs 612a, 612b... 612n.

[0069] Each neuron of hidden layer 608 receives an input from input layer 604 and outputs a value to the corresponding output in output layer 612. For example, neuron 608a receives an input from input 604a and outputs a value to output 612a. Each neuron besides neuron 608a also receives an output from a previous neuron as input. For example, neuron 608b receives inputs from input 604b and output 612a. In this way, the output of each neuron is passed to the next neuron in hidden layer 608. The last output 612n in output layer 612 outputs a probability associated with inputs 604a-604n. Although input layer 604, hidden layer 608, and output layer 612 are each depicted as containing three elements, each layer may contain any number of elements.

[0070] In various embodiments, each layer of the LSTM neural network 602 must contain the same number of elements as each of the other layers of the LSTM neural network 602. In some example embodiments, a convolutional neural network may be implemented. Similar to LSTM neural networks, convolutional neural networks include an input layer, a hidden layer, and an output layer. However, in a convolutional neural network, the output layer contains one fewer output than the number of neurons in the hidden layer, and each neuron is connected to each output. Furthermore, each input in the input layer is connected to every neuron in the hidden layer. In other words, the input 604a is connected to each of the neurons 608a, 608b, ... 608n.

[0071] In various implementations, each input node in the input layer can be assigned a numerical value, which can be any real number. In each layer, each connection emanating from an input node has an associated weight, which can also be any real number. In the input layer, the number of neurons is equal to the number of features (columns) in a dataset. The output layer can have multiple continuous outputs.

[0072] As mentioned above, the layers between the input and output layers are hidden layers. The number of hidden layers can be one or more (one hidden layer may be sufficient for many applications). A neural network without hidden layers can represent linearly separable functions or decisions. A neural network with one hidden layer can perform a continuous mapping from one finite space to another. A neural network with two hidden layers can approximate any smooth mapping with arbitrary accuracy. The neural network of Fig. 6 may receive inputs, e.g., vectors, that may be used to generate models that may be used with vehicle camera images to generate outputs predicting a pedestrian's crossing intent, as described herein.

[0073] Fig. Figure 7 illustrates an example process for generating a machine learning model. At 707, the controller receives data from a database 702 (e.g., a data warehouse). The data may include any suitable data for developing machine learning models.

[0074] At 711, the controller separates the data obtained from the database 702 into training data 715 and test data 719. The training data 715 is used to train the model at 723, and the test data 719 is used to test the model at 727. Typically, depending on the desired model development parameters, the set of training data 715 is selected to be larger than the set of test data 719. For example, the training data 715 may comprise about seventy percent of the data obtained from the database 702, about eighty percent of the data, about ninety percent, etc. The remaining thirty percent, twenty percent, or ten percent is then used as test data 719.

[0075] Separating a portion of the acquired data as test data 719 enables testing of the trained model against the actual output data to enable more accurate training and development of the model at 723 and 727. The model may be trained at 723 using any suitable machine learning techniques, including those described herein, such as random forest, generalized linear models, a decision tree, and neural networks.

[0076] At 731, the controller evaluates the model test results. For example, at 727, the trained model may be tested using the test data 719, and the results of the output data from the tested model may be compared to the actual outputs of the test data 719 to determine a level of accuracy. The model results may be evaluated using any suitable analysis for machine learning models, such as the example techniques described below.

[0077] After evaluating the model test results at 731, the model can be deployed at 735 if the model test results are satisfactory. Deploying the model may involve using the model to generate predictions for a large input data set with unknown outputs. If the evaluation of the model test results at 731 is not satisfactory, the model can be further developed using different parameters, using different modeling techniques, using different model types, etc. The procedure for machine learning models of Fig. 7 may receive inputs, such as vectors, that may be used to generate models that may be used with vendor matching, risk model processing, or both, as described herein.

[0078] Throughout this application, including the definitions below, the term "module" or "controller" may be replaced by the term "circuit." The term "module" may refer to, be part of, or include an application-specific integrated circuit (ASIC); a digital, analog, or mixed analog / digital discrete circuit; a digital, analog, or mixed analog / digital integrated circuit; a combinational logic circuit; a field-programmable gate array (FPGA); processor circuitry (shared, dedicated, or group) executing code; memory circuitry (shared, dedicated, or group) storing code executed by the processor circuitry; other suitable hardware components that provide the described functionality; or a combination of some or all of the above components, such as in a system-on-chip.

[0079] The module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any given module of the present invention may be distributed among multiple modules connected via interface circuits. For example, multiple modules may enable load balancing. In another example, a server module (also known as a remote or cloud module) may perform some functions for a client module.

[0080] The term code, as used above, may include software, firmware, and / or microcode, and may refer to programs, routines, functions, classes, data structures, and / or objects. The term shared processor circuitry includes a single processor circuit that executes some or all of the code from multiple modules. The term group processor circuitry includes a processor circuit that, in combination with additional processor circuitry, executes some or all of the code from one or more modules. References to multiple processor circuits include multiple processor circuits on individual chips, multiple processor circuits on a single chip, multiple cores of a single processor circuit, multiple threads of a single processor circuit, or any combination of the above.The term shared memory circuit refers to a single memory circuit that stores some or all of the code from multiple modules. The term group memory circuit refers to a memory circuit that, in combination with additional memories, stores some or all of the code from one or more modules.

[0081] The term "memory circuit" is a subset of the term "computer-readable medium." As used herein, the term "computer-readable medium" does not include transitory electrical or electromagnetic signals propagating through a medium (such as a carrier wave); therefore, the term "computer-readable medium" can be considered tangible and non-transitory.Non-limiting examples of a non-transitory, tangible computer-readable medium include non-volatile memory circuits (such as a flash memory circuit, an erasable programmable read-only memory circuit, or a mask read-only memory circuit), volatile memory circuits (such as a static random access memory circuit or a dynamic random access memory circuit), magnetic storage media (such as an analog or digital magnetic tape or a hard disk drive), and optical storage media (such as a CD, a DVD, or a Blu-ray Disc).

[0082] The devices and methods described in this application may be implemented, in part or in whole, by a special-purpose computer created by configuring a general-purpose computer to perform one or more specific functions embodied in computer programs. The functional blocks, flowchart components, and other elements described above serve as software specifications that can be translated into computer programs through the routine work of a person skilled in the art or programmer.

[0083] The computer programs contain processor-executable instructions stored on at least one non-transitory, tangible, computer-readable medium. The computer programs may also contain or rely on stored data. The computer programs may include a basic input / output system (BIOS) that interacts with the computer's special-purpose hardware, device drivers that interact with specific special-purpose devices of the computer, one or more operating systems, user applications, background services, background applications, etc.

[0084] The computer programs may contain: (i) a description text to be analyzed, such as HTML (Hypertext Markup Language), XML (Extensible Markup Language) or JSON (JavaScript Object Notation), (ii) assembly code, (iii) object code generated from the source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. By way of example only, source code may be written using syntax from languages ​​including C, C++, C#, ObjectiveC, Swift, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5 (Hypertext Markup Language 5th Revision), Ada, ASP (Active Server Pages), PHP (PHP: Hypertext Preprocessor), Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, MATLAB, SIMULINK and Python®. legend

[0085] In the drawings, N stands for No and Y stands for Yes.

Claims

[1] A method for controlling automatic acceleration and braking of a vehicle (10), the method comprising: Obtaining an image using at least one vehicle camera (26) of a host vehicle (10); Extracting machine learning model feature inputs based on the obtained image; Detecting one or more objects (112, 114) in the obtained image, wherein the one or more objects (112, 114) comprise at least one pedestrian (112); Assigning attention weights to regions of the obtained image according to positions of the one or more objects (112, 114) in the obtained image; Combining the attention weights with corresponding machine learning model feature inputs according to the regions of the obtained image; and executing a machine learning model to generate an output of a prediction of a crossing intention associated with the at least one pedestrian; characterized by Controlling automatic braking of the host vehicle (10) according to a position of the at least one pedestrian (112) in response to outputting a prediction of crossing intent that exceeds a crossing intent threshold; where: (i) combining the attention weights and the machine learning model feature inputs comprises generating a weighted sum of the machine learning model feature inputs according to the attention weights; and / or (ii) running the machine learning model includes: Obtaining multiple key values ​​according to the machine learning model feature inputs, executing a classification query according to the machine learning model feature inputs, and correlating a classification query output with the multiple key values. [2] The method of claim 1, wherein assigning the attention weights comprises: Assigning a first intensity value to a first region of the obtained image corresponding to the at least one pedestrian; and Assigning a second intensity value to a second region of the obtained image that does not correspond to the at least one pedestrian, wherein the first intensity value is greater than the second intensity value. [3] The method of claim 1, wherein: the machine learning model contains a multi-layer perceptron; and executing the machine learning model comprises generating the output of a prediction of a crossing intention according to an output of the multi-layer perceptron. [4] The method of claim 1, wherein extracting machine learning model feature inputs based on the obtained image comprises feeding the obtained image to a plurality of visual transformer layers to generate the machine learning model feature inputs. [5] The method of claim 1, wherein executing the machine learning model comprises: Combining the attention weights with a correlation of the classification query output and the multiple key values; and Executing a normalized exponential function on a combination of the attention weights and the correlation of the classification query output and the plurality of key values ​​to produce the output of a prediction of crossing intention. [6] The method of claim 5, wherein executing the machine learning model comprises: Combining an output of the normalized exponential function with the plurality of key values ​​to generate an embedding vector; and Feeding the embedding vector to a multi-layer perceptron to produce the output of a prediction of crossing intention. [7] The method of claim 1, further comprising: Feeding training data and test data to the machine learning model; Comparing multiple crossing intention prediction outputs of the machine learning model based on the training data with labeled crossing intention outputs of the test data; Determining whether an accuracy of a comparison is greater than or equal to a specified accuracy threshold; Adjusting parameters of the machine learning model and retraining the machine learning model in response to determining that the accuracy of the comparison is less than the specified accuracy threshold; and Storing the machine learning model for use in generating an output of a prediction of crossing intent in response to determining that the accuracy of the comparison is greater than or equal to the specified accuracy threshold. [8] Vehicle control system for controlling vehicle braking based on image processing of a vehicle camera (26), the vehicle control system comprising: at least one vehicle camera (26) configured to obtain an image of an area in front of a host vehicle (10); and a vehicle control module (20) of the host vehicle (10), wherein the vehicle control module (20) is configured to: Machine learning model feature inputs extracted based on the obtained image; detecting one or more objects (112, 114) in the obtained image, wherein the one or more objects (112, 114) comprise at least one pedestrian (112); assigning attention weights to regions of the obtained image according to positions of the one or more objects (112, 114) in the obtained image; combines the attention weights with corresponding machine learning model feature inputs according to the regions of the obtained image; and executes a machine learning model to generate an output of a prediction of a crossing intention associated with the at least one pedestrian; characterized by , that the vehicle control module (20) is further configured to: in response to outputting a prediction of a crossing intention that exceeds a crossing intention threshold, controlling automatic braking of the host vehicle (10) according to a position of the at least one pedestrian (112); and (i) combines the attention weights and the machine learning model feature inputs by generating a weighted sum of the machine learning model feature inputs according to the attention weights; and / or (ii) it executes the machine learning model by obtaining a plurality of key values ​​according to the machine learning model feature inputs, executing a classification query according to the machine learning model feature inputs, and correlating a classification query output with the plurality of key values.

Images