Motor intention determination method, motor intention determination device, computer storage medium, computer device, and computer program

Abstract

The embodiments of the present disclosure provide a motion intention determination method, an apparatus, a device, and a storage medium. The motion intention determination method includes: acquiring a traffic image; determining vehicle light information and orientation information of a vehicle in the traffic image based on the traffic image; and determining a motion intention of the vehicle based on the vehicle light information and the orientation information. In this way, the motion intention of the vehicle can be predicted more accurately.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This disclosure is based on and claims priority from a Chinese patent application bearing application number 202210122826.X, filing date February 9, 2022, and title "Method, device, equipment, and storage medium for determining motor intentions," the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD The embodiments of the present disclosure relate to the field of intelligent driving technology, including, but not limited to, a method, an apparatus, a device, and a storage medium for determining driving intentions. [Background technology]

[0003] In recent years, with the introduction of running lights and sidelights, vehicle lighting has become more complex, and it is no longer possible to accurately determine whether a vehicle is braking or turning based solely on the overall brightness of the two left and right lights. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present disclosure provide a technical solution for determining movement intention.

[0005] The technical solutions of the embodiments of the present disclosure are realized as follows:

[0006] An embodiment of the present disclosure provides a method for determining a motor intention, the method including: acquiring a traffic image; determining, based on the traffic image, vehicle light information and orientation information of a vehicle in the traffic image; and determining, based on the vehicle light information and the orientation information, a motor intention of the vehicle.

[0007] In some embodiments, determining vehicle light information and vehicle orientation information of a vehicle in the traffic image based on the traffic image includes determining position information of target vehicle lights that are turned on in the traffic image based on the traffic image, determining appearance information of a vehicle in the traffic image based on the traffic image, and determining orientation information of the head of the vehicle based on the appearance information of the vehicle, and determining a motion intention of the vehicle based on the vehicle light information and the orientation information includes determining a motion intention of the vehicle based on position information of the target vehicle lights of the vehicle and orientation information of the head of the vehicle. In this way, by combining the position information of the target vehicle lights and the vehicle orientation information, a vehicle turn can be predicted more accurately.

[0008] In some embodiments, the target vehicle light is a single turn signal light, and determining the vehicle's motion intention based on the position information of the target vehicle light of the vehicle and the headway information of the vehicle includes: determining turn information indicated by the turn signal light based on the position information of the single turn signal light and the headway information of the vehicle; and determining the vehicle's turn intention based on the turn information. In this way, by analyzing the position of the single turn signal light and the headway information of the vehicle, the turn information indicated by the turn signal light can be accurately obtained, and the vehicle's turn intention can be accurately predicted.

[0009] In some embodiments, determining the vehicle's motion intention based on the position information of the target vehicle light of the vehicle and the head direction information of the vehicle includes determining that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle light being multiple turn signal lights. In this way, by identifying whether multiple turn signal lights are simultaneously turned on, it is possible to accurately predict whether the vehicle is in a braking state.

[0010] In some embodiments, the method for determining a motor intention further includes determining, based on the traffic image, vehicle type information of a vehicle in the traffic image, and determining the motor intention of the vehicle based on the vehicle light information and the orientation information includes determining the motor intention of the vehicle based on the vehicle light information, orientation information, and vehicle type information. In this way, by combining the vehicle light information, orientation information, and vehicle type information, it is possible to accurately obtain turning information indicated by the turn signal lights of a vehicle, i.e., to predict the motor intention of the vehicle.

[0011] In some embodiments, when determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes reducing the reliability of the motion intention in response to the orientation information indicating that the vehicle is facing sideways. In this way, by reducing the reliability of the motion intention when the orientation information indicates that the vehicle is facing sideways, it is possible to improve prediction accuracy of the vehicle's motion intention.

[0012] In some embodiments, when determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes obtaining an application demand for predicting the vehicle's motion intention and determining a reliability threshold matching the application demand. After determining the vehicle's motion intention, the motion intention determination method further includes determining a motion intention whose reliability is greater than the reliability threshold as the determined motion intention of the vehicle. In this way, the reliability threshold can be set according to the application demand, thereby allowing the predicted motion intention to better meet the user's demand.

[0013] In some embodiments, the determination of the vehicle light information, the orientation information, and the vehicle's motion intention is performed by a neural network, and a first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and orientation information, and a second classifier in the neural network is obtained by training using sample images labeled with the vehicle's motion intention. In this way, by identifying the vehicle's motion intention using a classification network including multiple classifiers, the accuracy of motion intention prediction can be improved.

[0014] In some embodiments, the second classifier includes at least one of a base classifier for classifying the basic vehicle movement intention and an extended classifier for classifying the extended vehicle movement intention, where the base classifier is obtained by training based on sample images labeled with the overall vehicle light status, and the extended classifier is obtained by training based on sample images labeled with the turn signal light status of the vehicle. In this way, the base classifier and the extended classifier assist each other in the training process, and in the training process, the network first considers the overall vehicle light status and then further considers the turn signal light changes in the overall vehicle light status, thereby more accurately predicting the vehicle movement intention.

[0015] In some embodiments, determining vehicle light information and vehicle orientation information of vehicles in the traffic image based on the traffic image using the neural network includes: determining an attention mask of the traffic image using a convolutional layer of the neural network; determining spatial features of the traffic image based on the attention mask; merging the spatial features and temporal features of the traffic image to obtain image features of the traffic image; and determining vehicle light information and vehicle orientation information of the vehicles based on the image features using the first classifier. In this way, multi-task learning can be used to assist a classifier for vehicle light state classification using a classifier for vehicle orientation, vehicle type, etc., thereby further improving the accuracy of predicting the display status of vehicle lights.

[0016] In some embodiments, determining the vehicle's intention based on the vehicle light information and the orientation information includes inputting the vehicle light information and the orientation information to the second classifier and outputting the vehicle's predicted intention via the second classifier, respectively determining a first confidence level for the predicted intention and a second confidence level for the classification result in response to a mismatch between the predicted intention and the classification result output by the first classifier, and determining the vehicle's intention based on the prediction result corresponding to a relatively higher confidence level out of the first confidence level for the predicted intention and the second confidence level for the classification result. In this way, if the prediction results of multiple classifiers collide, the one with a relatively higher confidence level can be selected as the final prediction result, thereby enabling a more accurate prediction of the vehicle's intention.

[0017] An embodiment of the present disclosure provides a movement intention determination device, the device including: an image acquisition unit configured to acquire a traffic image; an information determination unit configured to determine, based on the traffic image, vehicle light information and orientation information of a vehicle in the traffic image; and an intention determination unit configured to determine, based on the vehicle light information and the orientation information, a movement intention of the vehicle.

[0018] Correspondingly, an embodiment of the present disclosure provides a computer storage medium, which stores computer-executable instructions, and which, after the computer-executable instructions are executed, can realize the steps of the method described above.

[0019] An embodiment of the present disclosure further provides a computer program product, the computer program product including a computer program or instructions that, when executed on an electronic device, causes the electronic device to perform the steps in any possible embodiment of the first aspect above.

[0020] An embodiment of the present disclosure provides a computing device, the computing device comprising a memory and a processor, wherein computer-executable instructions are stored in the memory, and the computing device is capable of implementing the steps of the method described above when the processor executes the computer-executable instructions in the memory.

[0021] The embodiments of the present disclosure provide a method, device, apparatus, and storage medium for determining a vehicle's motor intention, which determine vehicle light information and direction information from a captured traffic image, and combine the vehicle light information and direction information to determine whether the vehicle will perform a motor intention such as braking or turning, thereby enabling a more accurate prediction of the vehicle's motor intention.

[0022] In order to make the above objects, features and advantages of the embodiments of the present disclosure more comprehensible, preferred embodiments will be specifically described below with reference to the accompanying drawings. [Brief explanation of the drawings]

[0023] [Figure 1A] 1 is a schematic diagram of a system architecture to which a movement intention determination method according to an embodiment of the present disclosure can be applied. [Figure 1B] 1 is a schematic flowchart of an implementation of a movement intention determination method according to an embodiment of the present disclosure. [Figure 2]1 is a schematic flowchart of another implementation of a movement intention determination method according to an embodiment of the present disclosure. [Figure 3] 1 is a schematic diagram of an application scenario of a motor intention determination method according to an embodiment of the present disclosure; [Figure 4] 1 is a schematic diagram of an implementation framework of a motor intention determination method according to an embodiment of the present disclosure. [Figure 5] 1 is a structural schematic diagram of a movement intention determination device according to an embodiment of the present disclosure; [Figure 6] FIG. 1 is a structural schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0024] In order to more clearly explain the technical solutions of the embodiments of the present disclosure, the drawings required for the embodiments are briefly introduced above, and the drawings herein are incorporated into the specification and constitute a part of this specification, these drawings show embodiments consistent with the present disclosure, and are used to explain the technical solutions of the present disclosure together with the specification. The following drawings only show some embodiments of the present disclosure, so they should not be considered as limiting the scope, and it should be understood that those skilled in the art can also obtain other related drawings based on these drawings without requiring creative efforts.

[0025] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the specific technical solutions of the invention will be described in more detail below with reference to the drawings in the embodiments of the present disclosure. The following examples are for illustrating the present disclosure, but are not intended to limit the scope of the present disclosure.

[0026] It will be understood that while "some embodiments" in the following description describe a subset of all possible embodiments, "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other if they do not conflict.

[0027] The terms "first, second, third" in the following description do not represent a particular order of objects, but merely distinguish between similar objects. It should be understood that the terms "first, second, third" may be interchanged, where permitted, to allow the implementation of the present disclosure described in some embodiments to be performed in an order other than that illustrated or described in some embodiments.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The terms used herein are merely for the purpose of describing examples of the present disclosure and are not intended to be limiting of the present disclosure.

[0029] Before describing the embodiments of the present disclosure in more detail, the nouns and terms referred to in the embodiments of the present disclosure will be explained, and the nouns and terms referred to in the embodiments of the present disclosure will be interpreted as follows.

[0030] 1) Convolutional Neural Networks (CNN): A type of feedforward neural network with a deep structure that includes convolutional calculations. It has the ability to learn features and can perform translation-invariant classification of input information according to its hierarchical structure.

[0031] 2) Ego vehicle: A vehicle that includes sensors that sense the surrounding environment. The vehicle coordinate system is fixedly connected to the ego vehicle, where the x-axis is the forward direction of the vehicle, the y-axis points to the left of the forward direction of the vehicle, and the z-axis is perpendicular to the ground and points upward, conforming to a right-handed coordinate system. The origin of the coordinate system is located on the ground below the midpoint of the rear axle.

[0032] The following describes exemplary applications of the motor intention determination device provided by the embodiments of the present disclosure, and the device provided by the embodiments of the present disclosure may be implemented as a laptop computer, tablet computer, or other in-vehicle device with image collection capabilities, or as a server. The following describes exemplary applications when the device is implemented as a terminal or a server.

[0033] 1A is a schematic diagram of a system architecture of a motion intention determination method according to an embodiment of the present disclosure. As shown in FIG. 1A, the system architecture includes an image capture device 11, a network 12, and an in-vehicle control terminal 13. To support one exemplary application, the image capture device 11 and the in-vehicle control terminal 13 establish a communication connection via the network 12. First, the image capture device 11 reports traffic images captured via the network 12 to the in-vehicle control terminal 13. The in-vehicle control terminal 13 identifies the traffic images to determine vehicle light information and orientation information of vehicles in the traffic images, and combines the vehicle light information and orientation information to determine the vehicle's motion intention.

[0034] For example, the image capture device 11 may include a visual processing device having visual information processing capabilities. The network 12 may employ a wired or wireless connection method. If the image capture device 11 is a visual processing device, the vehicle control terminal 13 may be connected to the visual processing device via a wired connection method, and data communication may be performed via a bus, for example.

[0035] Alternatively, in some scenarios, the image capture device 11 may be a visual processing device equipped with a video collection module or a host equipped with a camera, in which case the augmented reality data display method of the embodiment of the present disclosure may be performed by the image capture device 11, and the above system architecture may not include the network 12 and the in-vehicle control terminal 13.

[0036] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the specific technical solutions of the invention will be described in more detail below with reference to the drawings in the embodiments of the present disclosure. The following embodiments are intended to illustrate the present disclosure, but are not intended to limit the scope of the present disclosure.

[0037] It will be understood that while "some embodiments" in the following description describe a subset of all possible embodiments, "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other if they do not conflict.

[0038] The method can be applied to a computer device, and the functions realized by the method can be realized by a processor in the computer device calling a program code; of course, the program code may be stored in a computer storage medium, and it can be understood that the computer device includes at least a processor and a storage medium.

[0039] 1B is a schematic flowchart of the implementation of the exercise intention determination method according to an embodiment of the present disclosure. As shown in FIG. 1B, the description will be made with reference to the steps shown in FIG.

[0040] In S101, a traffic image is acquired.

[0041] In some embodiments, the traffic images may be images collected on any road, images with complex screen content, or images with simple screen content. For example, images of street scenarios collected late at night or images of street scenarios collected during the day. The traffic images may include vehicles, where vehicles include vehicles with various functions (e.g., trucks, cars, motorcycles, etc.) and vehicles with various numbers of wheels (e.g., four-wheeled vehicles, two-wheeled vehicles, etc.). Passenger cars may be used as an example in the following description. For example, the traffic images are images collected of passenger cars on a road.

[0042] In step S102, based on the traffic image, the vehicle light information and the vehicle orientation information in the traffic image are determined.

[0043] In some embodiments, a traffic image is input to a trained neural network, and a fully convolutional network in the network is used to perform feature extraction on the traffic image to obtain image features. The image features are input to multiple classifiers in the neural network to identify vehicle light information for each vehicle light and vehicle orientation information. The vehicle light information includes the display status and position of the vehicle lights. For example, assuming that the vehicle is a standard passenger car, the vehicle lights of the standard passenger car include headlights, fog lights, backup lights, license plate lights, position lights, running lights, turn signals, dome lights, high-level brake lights, high beams / low beams, warning lights, and trunk lights. The vehicle light information of the standard passenger car is the display status and position of each vehicle light included in the vehicle body. The orientation information is the orientation of the head of the vehicle and is configured to represent the vehicle orientation of the vehicle, including whether the head is facing forward or backward with respect to the host vehicle whose image was collected, i.e., the head is facing forward means that the tail of the vehicle faces the host vehicle, and the tail of the vehicle is presented in the traffic image collected in this manner, and the head is facing backward means that the head of the vehicle faces the host vehicle, and the head of the vehicle is presented in the traffic image collected in this manner. The orientation information further includes the lateral direction of the vehicle, for example, whether the head of the vehicle faces the left side of the road or the right side of the road, i.e., the vehicle is lying on the road.

[0044] In step S103, the vehicle's intention to move is determined based on the vehicle light information and the direction information.

[0045] In some embodiments, the vehicle's vehicle light display status and vehicle head direction information are combined to determine the vehicle's maneuvering behavior indicated by the vehicle's lights. That is, the maneuvering behavior includes the vehicle's left turn, right turn, forward movement, backward movement, or braking. Traffic images are input to a neural network classifier for classifying vehicle light information and a classifier for predicting vehicle head direction, resulting in multiple classification results output by the vehicle light classifier and the vehicle direction classifier. The outputs of these multiple classifiers are compared to determine whether or not there is a risk of collision, and finally, vehicle light information and direction information with relatively high reliability are output. In this way, comprehensive consideration of vehicle light information and direction information allows for more accurate prediction of the vehicle's maneuvering behavior.

[0046] In the embodiment of the present disclosure, the vehicle's light information and direction information are determined in the traffic image, and the light information and direction information are combined to determine whether the vehicle will perform a motion intention such as braking or turning, thereby enabling a more accurate analysis of the vehicle's motion intention.

[0047] In some embodiments, the vehicle's two-class orientation is analyzed to determine whether the turn signal light is a left turn signal light or a right turn signal light, and then the vehicle's motion intention is determined, that is, the above S102 can be realized by the following steps S121 to S123 (not shown).

[0048] In S121, position information of the target vehicle lights that are turned on in the traffic image is determined based on the traffic image.

[0049] Here, a neural network is used to detect the lit vehicle lights of vehicles in a traffic image, and if the vehicle lights are turn signal lights, the coordinates of the detection frame of the turn signal lights are determined, thereby obtaining the position information of the target turn signal light.

[0050] In some possible implementations, the target vehicle light to be turned on is first determined in the vehicle light information. Here, a target vehicle light whose display state is turned on is selected in the vehicle light information. The target vehicle light can be any vehicle light on the vehicle. If the target vehicle light is a turn signal light, position information of the turn signal light is determined based on the orientation information. Here, the orientation information can include the vehicle head orientation, i.e., whether the vehicle head is facing forward or backward. For example, for the same vehicle, if the vehicle head is facing forward, the position of the left turn signal light in the image is on the left side, and if the vehicle head is facing backward, the position of the left turn signal light in the image is on the right side. In this way, the orientation of the vehicle head is combined to determine whether the target vehicle light in the turn signal light information is a left turn signal light or a right turn signal light. If the target vehicle light is not a turn signal light, the target vehicle light can be represented in the traffic image in the form of a detection frame, and the coordinates of the detection frame are the position information of the target vehicle light. In this way, by taking into account the vehicle head direction information, it is possible to accurately predict whether the target turn signal light is a left turn signal light or a right turn signal light.

[0051] In step S122, based on the traffic image, appearance information of the vehicles in the traffic image is determined.

[0052] In some embodiments, a single-frame traffic image is input to a neural network to extract appearance information of the vehicle, where the appearance information includes the appearance that the vehicle presents in the traffic image, for example, if the traffic image is collected behind a vehicle, the appearance information is the appearance corresponding to the tail of the vehicle (including the turn signal lights and high-level brake lights behind the vehicle, etc.).

[0053] In step S123, information on the direction of the head of the vehicle is determined based on the vehicle appearance information.

[0054] In some embodiments, the orientation of the vehicle head can be determined by analyzing the image content of the vehicle shown in the traffic image, i.e., whether the vehicle is moving forward, backward, or sideways.

[0055] The above steps S121 to S123 provide a method for determining the vehicle head direction information and the position information of the target vehicle light. By using a single frame image in this way to predict the vehicle direction information, the network model can be simplified and processing delay can be reduced.

[0056] After determining the position information of the target vehicle light and the direction information of the vehicle head, the vehicle's motion intention is determined in the next step S124.

[0057] In step S124, the vehicle's intention to move is determined based on the position information of the target vehicle lights of the vehicle and the direction information of the head of the vehicle.

[0058] Here, the turning of the vehicle, i.e., the target turning, is predicted based on the vehicle direction information and the target turn signal light that is on. For example, if the target turn signal light is a left turn signal light and the vehicle is facing forward, the target turning is a left turn ahead. In this way, by combining the position information of the target vehicle light and the vehicle direction information, the turning of the vehicle can be predicted more accurately.

[0059] In some embodiments, if the target vehicle light is a single turn signal light, the vehicle's motion intention can be predicted by analyzing the position information of the single turn signal light and the vehicle head direction information. That is, the above step S124 can be realized by the following steps:

[0060] In a first step, turning information indicated by a turn signal light is determined based on the position information of the single turn signal light and the head direction information of the vehicle.

[0061] In some embodiments, the position information of a single turn signal light is the position of the single turn signal light on a vehicle shown in a traffic image, and the turn information indicated by the turn signal light may be a left turn, a right turn, a double flash, etc. In one specific example, when the vehicle head is facing forward in a traffic image, if the position information of the single turn signal light is to the left of the position in the image, it means that the turn signal light is a left turn signal, and it is further determined that the turn information indicated by the turn signal light is a left turn. Similarly, when the vehicle head is facing backward, if the position information of the single turn signal light is to the left of the position in the image, it means that the turn signal light is a right turn signal, and it is further determined that the turn information indicated by the turn signal light is a right turn.

[0062] In a second step, a turning intention of the vehicle is determined based on the turning information.

[0063] In some embodiments, the turn indicated by the turn signal light can be obtained according to the turn information indicated by the turn signal light, and the next turn of the vehicle can be predicted, i.e., the turning intention of the vehicle can be determined.

[0064] In the embodiments of the present disclosure, by analyzing the position of a single turn signal light and the orientation of the vehicle head, the turning information indicated by the turn signal light can be accurately obtained, and the turning intention of the vehicle can be accurately predicted.

[0065] In some embodiments, when the target vehicle lights are multiple turn signal lights, it is possible to analyze whether the vehicle is in a braking state by analyzing whether the vehicle light information includes brake light information, that is, step S124 above can be realized by a process that determines that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being multiple turn signal lights.

[0066] In some embodiments, if the vehicle light information does not include brake light information, it means that brake light information is not collected in the vehicle light information.

[0067] In some possible implementations, first, it is determined whether the vehicle light information includes brake light information. If the brake light information is not included, the display status of the left and right turn signal lights of the vehicle is determined in the vehicle light information. For example, since a truck or bus does not have a dome light, its braking status can be determined by the left and right turn signal lights. In response to both the left and right turn signal lights being lit, it is determined that the vehicle is in a braking state. That is, if the target vehicle lights that are lit are multiple turn signal lights, that is, if multiple turn signal lights are lit simultaneously, it can be predicted that the vehicle is in a braking state. In this way, even if the vehicle light information does not include brake light information, it can be accurately predicted that the vehicle is in a braking state if it is identified that multiple turn signal lights are lit simultaneously.

[0068] In another embodiment, if the vehicle light information includes brake light information, it is possible to predict whether the vehicle is in a braking state by analyzing the blinking state of the brake light information.

[0069] In some embodiments, determining a vehicle's turn by identifying the vehicle's type and combining the type with target vehicle light position information and vehicle orientation information can be achieved by the following steps:

[0070] In a first step, based on the traffic image, vehicle type information of the vehicles in the traffic image is determined.

[0071] In some embodiments, a neural network is used to identify the vehicle type. Image features of the traffic image are input to a vehicle type classifier to identify the vehicle type, for example, whether the vehicle is a car, truck, or bus.

[0072] In a second step, the vehicle's motion intention is determined based on the vehicle light information, direction information, and vehicle type information.

[0073] In some embodiments, the vehicle type information, vehicle light information, and orientation information are combined to determine whether the target vehicle light is a single turn signal light, and if so, to further determine whether it is a left turn signal light or a right turn signal light. For example, the vehicle type information can first be used to determine the appearance and position of the turn signal light of the vehicle light, so that it can be determined whether the position information of the target vehicle light in the vehicle light information is the position information of the turn signal light. Next, once it is determined that the target vehicle light is a single turn signal light, the vehicle orientation information can be combined to accurately predict whether the vehicle's turn signal light is a left turn signal light or a right turn signal light, and the turning information indicated by the vehicle's turn signal light can be accurately obtained, i.e., the vehicle's movement intention can be predicted.

[0074] In some embodiments, to improve accuracy of the predicted vehicle movement intention, when determining the vehicle movement intention, a confidence level of the vehicle movement intention is determined, and in response to the orientation information indicating that the vehicle is facing sideways, the confidence level of the movement intention is reduced.

[0075] In some possible implementations, when the orientation classifier identifies that the vehicle orientation is sideways, the reliability of the vehicle intention is reduced when the orientation information indicates that the vehicle is sideways, in order to improve the prediction accuracy of the vehicle's motion intention. Here, when the vehicle leans, it is difficult to distinguish between a left turn and a right turn, so the reliability of the predicted motion intention is reduced. When the vehicle is sideways, only the vehicle light status on one side of the vehicle is visible, and multiple situations may occur, so the reliability of the prediction is reduced in such situations.

[0076] In some embodiments, a neural network is used to identify vehicle light information, vehicle headway information, and vehicle movement intentions in traffic images. A trained neural network is obtained, and traffic images are input to multiple classifiers in the network to predict the display status of each vehicle light and the direction of the vehicle headway, and then predict the vehicle movement intentions.

[0077] In some embodiments, a first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and orientation information, and a second classifier in the neural network is obtained by training using sample images labeled with vehicle movement intentions.

[0078] In some possible implementations, the first classifier includes at least one classifier used to classify the vehicle lamp information, vehicle type, and orientation information of each vehicle lamp of the sample vehicle. For example, the vehicle lamp information, vehicle type, and orientation information of each vehicle lamp are obtained by classification based on three different classifiers, respectively, or the vehicle lamp information, vehicle type, and orientation information of each vehicle lamp are obtained by classification based on the same classifier. The second classifier is used to classify the motion intention of the sample vehicle, inputting collected traffic images into a neural network and identifying the display state of each vehicle lamp of the vehicle using multiple classifiers in the neural network, thereby obtaining the vehicle lamp information of the vehicle. The orientation classifier in the network identifies the orientation of the vehicle's head, thereby obtaining the vehicle's head orientation information.

[0079] In the neural network training process, the second classifier is trained using mini-batches, i.e., two portions of data with different labeling types are used to train the second classifier. The second classifier includes at least one of a base classifier for classifying the vehicle's basic motion intention and an extended classifier for classifying the vehicle's extended motion intention, where the base classifier is trained based on sample images labeled with the vehicle's overall lighting status and the extended classifier is trained based on sample images labeled with the vehicle's turn signal lighting status, both the base motion intention and the extended motion intention represent the vehicle's motion intention, and the reliability of the base motion intention is lower than the reliability of the extended motion intention. Alternatively, the base motion intention is a vehicle's motion intention roughly predicted based on the vehicle's overall lighting status, and the extended motion intention is a vehicle's motion intention accurately predicted based on the display status of the vehicle's turn signal lighting. That is, the training sample data for the base classifier is data labeled with the display status of all vehicle lights on the vehicle, and the training sample data for the extended classifier is data labeled with the brightness and absence of left / right turn signals.

[0080] If the second classifier includes a base classifier, it roughly predicts the basic driving intention of the vehicle based on the overall display state of the vehicle's lights.

[0081] If the second classifier includes an extended classifier, it accurately predicts the extended motoring intention of the vehicle based on the display state of the vehicle's turn signal lights.

[0082] When the second classifier includes a base classifier and an extended classifier, the base classifier is first used to roughly predict the basic motoring intention of the vehicle based on the overall vehicle light display state. Based on the basic motoring intention, the vehicle's turn signal light display state is combined with the extended classifier to more accurately predict the vehicle's extended motoring intention. In this way, the basic motoring intention based on the overall vehicle light display state and the extended motoring intention based on the left / right turn signal light display state are trained, and the base classifier and the extended classifier assist each other in the training process. In the training process, the network first considers the overall vehicle light state and then further considers the turn signal light direction change in the overall vehicle light, thereby more accurately predicting the vehicle's motoring intention.

[0083] In some possible implementations, the traffic image is input to a first classifier of a neural network to identify at least dome light information and turn signal light information of the vehicle to obtain vehicle lamp information.

[0084] Here, after feature extraction is performed on the traffic image, it is input to a first classifier, and the left turn signal light classifier, right turn signal light classifier, dome light classifier, etc. in the first classifier classify the vehicle light display states based on the extracted image features, thereby obtaining the left turn signal light display state, right turn signal light display state, dome light display state, etc. The vehicle light display states and vehicle light positions identified by each vehicle light classifier in the first classifier are defined as the vehicle light information. In this way, by identifying at least the vehicle's turn signal light information and dome light information, not only can the amount of data to be identified be reduced, but also a rich basis can be provided for predicting the vehicle's movement intention.

[0085] In some embodiments, the neural network is used to perform feature extraction on traffic images, and multiple classifiers are used to predict vehicle light information and orientation information based on the extracted image features. That is, step 102 above can be realized by the steps shown in Figure 2. Figure 2 is a schematic flowchart of another implementation of a movement intention determination method according to an embodiment of the present disclosure. The following description will be made with reference to the steps shown in Figure 2.

[0086] In step S201, the convolutional layers of the neural network are used to determine an attention mask of the traffic image.

[0087] In some embodiments, a traffic image is input to a fully convolutional neural network to predict an attention mask for the image.

[0088] In step S202, spatial features of the traffic image are determined based on the attention mask.

[0089] In some embodiments, the traffic image and the attention mask are multiplied element-wise, and the product is output to a CNN for spatial feature extraction to obtain spatial features.

[0090] In step S203, the spatial features and the temporal features of the traffic image are merged to obtain image features of the traffic image.

[0091] In some embodiments, the extracted spatial features are input into a special recurrent neural network (RNN), a long short-term memory (LSTM), and merged with the temporal features, and the merged features are used as image features, which can be used to subsequently identify the vehicle headlight status and vehicle head orientation based on the features.

[0092] In step S204, the first classifier is used to determine the vehicle's lamp information and the vehicle's orientation information based on the image features.

[0093] In some embodiments, the image features are input to the first classifiers to obtain at least predicted vehicle light information for each vehicle light of the vehicle and predicted vehicle orientation information. The image features are input to the first classifiers to predict vehicle light status and vehicle head orientation. For example, the first classifiers include a classifier for classifying the display status of a dome light, a classifier for classifying the display status of a left turn signal light, a classifier for classifying the display status of a right turn signal light, a classifier for classifying the vehicle type, and a classifier for classifying the vehicle head orientation. The classification results of the first classifiers include the display status of each vehicle light of the vehicle, the vehicle type, and vehicle head orientation information. Among the classification results for the vehicle light information of the same vehicle light, predicted vehicle light information with a reliability equal to or greater than a reliability threshold is selected as the vehicle light information. Similarly, among the classification results, predicted orientation information with a reliability equal to or greater than a reliability threshold is selected as the vehicle orientation information. For example, the classification result for the left turn signal light includes bright, dark, and none, and if the reliability of the brightness is greater than the reliability threshold, the state in which the left turn signal light is bright is taken as the vehicle light information of the left turn signal light of the vehicle.Similarly, the information about the direction includes forward, backward, and sideways, and if the reliability of the direction is greater than the reliability threshold, the direction information of the vehicle is that the vehicle head is facing forward.

[0094] In an embodiment of the present disclosure, by using multi-task learning, a classifier for vehicle light state classification can be aided using a vehicle orientation classifier, a vehicle type classifier, etc., to further improve the accuracy of predicting the display state of the vehicle lights. For example, an orientation classifier can be used to help the vehicle light model determine left and right, and a vehicle type classifier can be used to help the vehicle light information classifier determine the vehicle light position and light shape.

[0095] In some embodiments, the following two methods can be used to select a superior exercise intention from multiple predicted exercise intentions: That is, the above step S103 can be realized by the following steps.

[0096] In Method 1, by analyzing the user's demand, a classifier with a relatively high reliability can output the exercise intention, that is, a more accurate exercise intention can be predicted by the following steps.

[0097] In the first step, the application demand for predicting the vehicle's motion intention is obtained.

[0098] In some embodiments, the application requirement may be set independently by a user, such as a set maximum number of false brake detections or a preset turn supervision for a vehicle. In one specific example, the application requirement may be that the number of false detections for a right-turning vehicle is less than 5.

[0099] In a second step, a reliability threshold that matches the application demand is determined.

[0100] In some embodiments, the confidence threshold is set based on the application needs. For example, if the application needs to have fewer than 5 false positives for right-turning vehicles, the confidence threshold can be set to a larger value (e.g., the confidence threshold is set to 0.9). If the application needs to have fewer than 20 false positives for right-turning vehicles, the confidence threshold can be set to 0.8, etc.

[0101] In a third step, a movement intention whose reliability is greater than the reliability threshold is determined as the determined movement intention of the vehicle.

[0102] In some embodiments, after determining the vehicle's motion intention, a motion intention whose reliability is greater than the reliability threshold is determined as the determined motion intention of the vehicle. The predicted motion intention may be a left turn, a right turn, forward, backward, or brake. Among these predicted motion intentions, a predicted motion intention whose reliability is greater than the reliability threshold that matches the application demand is determined as the vehicle's motion intention. In this way, the reliability threshold can be set according to the application demand, thereby allowing the predicted motion intention to better meet the user's demand.

[0103] In Method 2, we analyze whether there are conflicts between the multiple classification results obtained and output the output of a classifier with a relatively high degree of reliability. That is, we can predict motor intentions more accurately by the following steps.

[0104] In a first step, the vehicle light information and the direction information are input to the second classifier, and the second classifier outputs a predicted movement intention of the vehicle.

[0105] In some embodiments, when the second classifier is a base classifier, the base classifier predicts the vehicle's motion intention based on all vehicle light information as a whole, combining the vehicle's orientation to obtain an overall predicted motion intention of the vehicle. When the second classifier is an extended classifier, the extended classifier predicts the vehicle's motion intention based on the state of the turn signal lights in the vehicle light information, combining the orientation information to obtain an extended motion intention of the vehicle.

[0106] In a second step, in response to a mismatch between the predicted motor intention and the classification result output by the first classifier, a first reliability of the predicted motor intention and a second reliability of the classification result are determined, respectively.

[0107] In some embodiments, a mismatch between the predicted intention and the classification result output by the first classifier refers to a conflict between the predicted intention and the classification result output by the first classifier. In a specific example, the second classifier is a base classifier, the predicted intention is a left turn, and the first classification classifier outputs the left turn signal classifier as "off," the right turn signal classifier as "on," and the dome light classifier as "off." Thus, based on the flashing status of each vehicle light output by the first classifier, it is concluded that the vehicle's intention is a right turn. In this case, the output results of the multiple classifiers collide. In this case, a reliability of the predicted intention to collide and a second reliability of the classification result are obtained. The second reliability of the classification result can be understood as the reliability of determining the vehicle's intention based on the classification result, or as the reliability of the overall classification result.

[0108] In a third step, the vehicle's intention to move is determined based on the prediction result corresponding to a relatively higher reliability out of the first reliability of the predicted intention to move and the second reliability of the classification result.

[0109] In some embodiments, the intention with the highest confidence level is selected as the intention for the vehicle from among the predicted intention output by the second classifier and the intention determined by the classification results. In this way, if the prediction results of multiple classifiers conflict, the intention with the highest confidence level is selected as the final prediction result, thereby making it possible to more accurately predict the intention for the vehicle.

[0110] In the following, an exemplary application of the embodiments of the present disclosure in a practical application scenario will be described, taking determining vehicle light status based on multi-task learning and multi-stage learning as an example.

[0111] In advanced driving assistance systems (ADAS) and autonomous driving tasks, it is necessary to detect the lighting status of other vehicles to determine their intentions and future driving trajectories, and to help the vehicle perform tasks such as collision warning and strategy planning. In terms of related technologies, ADAS products are basically lacking in the area of ​​dynamic prediction, and even in autonomous driving systems, there are almost no dynamic prediction models for lighting for multiple driving scenarios.

[0112] In the related art, it is not accurate enough to determine which turn signal indicates a left or right turn based on the lighting position of the identified light. As shown in Figure 3, the right turn signal of the vehicle, the left front turn signal in image 31, and the right rear turn signal in image 32 are all on.

[0113] With the advent of running lights and sidelights, vehicle lights have become more complex, making it impossible to determine whether a vehicle is braking or turning based solely on the overall brightness of the two left and right lights. Based on this, an embodiment of the present disclosure provides a vehicle light status prediction method that uses deep learning to perform a vehicle light intention determination task, cognitively decomposing vehicle intentions represented by the brightness or darkness of a single light and those represented by the overall flashing state of the vehicle lights. The two levels are simultaneously supported by vehicle direction and vehicle type classifiers. This multi-task and multi-level support process is beneficial to the learning of the vehicle light network and can significantly improve the inference accuracy of the final model.

[0114] In some embodiments, the present disclosure provides a method for predicting whether a vehicle will turn left or right based on the vehicle's turn signal lights by acquiring information about the vehicle's flashing position and the vehicle's forward direction, as shown in an image. In some embodiments, in response to the characteristics of the vehicle's turn signal lights and brake lights in different positions and colors in some regions, the present disclosure provides a method for predicting the vehicle light status, which may be implemented by the following process:

[0115] First, multi-task learning is used to input a vehicle image into a single-frame model for end-to-end training, and simultaneously output the heading / tail dome light / left light / right light / vehicle type / left and right turn signals to determine the instantaneous state of a single vehicle light.

[0116] In some possible implementations, because the position and shape of vehicle lights are complex and variable, and there are many types of light and dark combinations, a classifier is set up independently for each light, i.e., the left / right vehicle lights, left / right turn signals, and tail dome lights are classified as light and dark or off. Based on this, additional information can help determine the position and shape type of the vehicle lights, for example, a vehicle direction classifier can help determine left / right vehicle lights, and a vehicle type classifier can help determine the type of vehicle lights.

[0117] Second, multi-level learning is used to further determine braking status and turning intent based on the model's learning of a single light state.

[0118] Here, the braking state and turning intent judgment have multiple levels. Based on the state inference of multiple single vehicle lights, the turning intent and braking intent represented by the entire vehicle lights are obtained. Furthermore, the state of the left vehicle light, the state of the right vehicle light, and the state of the dome light are combined to judge the intention of the entire vehicle. In addition, the occlusion and movement of the vehicle also bring great uncertainty to the judgment of a single frame.

[0119] In some embodiments, the exercise intention determination method may be implemented by the following steps.

[0120] In the first step, multi-task training is performed using a single frame input of a vehicle to obtain multiple classifiers including orientation, vehicle type, dome light status, left light status, and right light status.

[0121] In some possible implementations, motor intention can be determined by the method shown in Fig. 4. As shown in Fig. 4, Fig. 4 is a schematic diagram of an implementation framework of a motor intention determination method according to an embodiment of the present disclosure. A single-frame image 400 of a vehicle is input to a vehicle detector 401 to identify the vehicle in the image, and the detection frame of the identified vehicle is input to a CNN 402 to extract features and obtain a feature map 403. The dimension of the feature map 403 is 7x7x2048. The feature map 403 is processed to obtain a 2048-dimensional feature vector 404.

[0122] In the second step, the network is trained using a mini-batch method based on the classifier from the first step.

[0123] Here, in one batch, half of the data uses data labeled as bright, dark, or off for the entire left / right vehicle lights (here, "overall" refers to the entire light, and any sublights (which remain on even when brake lights / fog lights / turn signals are on)), and the other half uses data labeled as bright, dark, or off for the left / right turn signals. We train a basic vehicle intent based on the entire state of the left and right vehicle lights, and an extended vehicle intent based on the left / right turn signals.

[0124] The two sets of classifiers mutually assist each other during training. Mutual assistance refers to the mutual promotion of different correlated tasks in multi-task learning. This process is completed automatically during model training. In some possible implementations, the network model can first input the overall lamp status, then aggregate the turn signal statuses of the entire lamp group, and finally output the vehicle intent of the lamps. This not only improves the final result, but also solves the problem of inability to train due to repeated labeling (e.g., relabeling data where the status of each lamp is unlabeled). In this way, multi-level learning is used to scale the difficulty of tasks from light to difficult, matching natural cognitive levels from judging the status of a single lamp to judging the overall braking status and turning intent, which is beneficial for model training. Furthermore, training the network using a mini-batch method largely solves the training difficulties caused by different labeling of data, allowing the same model to capture multiple features with different labeling information, thereby significantly reducing labeling costs.

[0125] As shown in Fig. 4, the extracted feature vector 404 is input to multiple fully connected layers (fc) for classification, where fully connected layer 451 is used to classify whether the vehicle's dome light is on or off, fully connected layer 452 is used to classify whether the vehicle's left turn signal light is on or off, fully connected layer 453 is used to classify whether the vehicle's right turn signal light is on or off, fully connected layer 454 is used to classify the vehicle's direction (e.g., forward or backward), fully connected layer 455 is used to classify the vehicle type (e.g., car, truck, bus, taxi, ambulance, or other vehicle), fully connected layer 456 is used to classify the basic vehicle intention based on the overall state of the vehicle's left and right lights for vehicles facing forward, and fully connected layer 457 is used to classify the basic vehicle intention based on the overall state of the vehicle's left and right lights for vehicles facing backward. That is, the fully connected layer 456 and the fully connected layer 457 train a basic vehicle intention based on the overall state of the left and right lights, which is suitable for simple scenarios. The fully connected layer 458 is used to classify an extended vehicle intention based on the left and right turn signal lights of the vehicle for vehicles facing forward, and the fully connected layer 459 is used to classify an extended vehicle intention based on the left and right turn signal lights of the vehicle for vehicles facing backward. That is, the fully connected layer 458 and the fully connected layer 459 train an extended vehicle intention based on the left and right turn signal lights of the vehicle, which is suitable for complex scenarios. The outputs of the classifiers formed by the fully connected layers 451 to 459 are merged to generate a probability distribution of vehicles in multiple categories, for example, the probability of vehicle intention ( JPEG0007805470000001.jpg513), the probability that the vehicle will turn left ( JPEG0007805470000002.jpg69), the probability that the vehicle will turn right ( JPEG0007805470000003.jpg611), and the probability of vehicle orientation ( JPEG0007805470000004.jpg510).

[0126] In some embodiments, the above-described manner of determining motor intention using the method shown in FIG. 4 is merely a possible embodiment, and the manner of determining motor intention in the embodiments of the present disclosure is not limited thereto. For example, motor intention can also be determined using a residual network or a deep neural network, which will not be described in detail here.

[0127] In the third step, post-processing and logical addition can be conveniently performed based on the first and second steps. For example, the following multiple post-processing and logical addition can be performed:

[0128] a. Determine whether it is a left or right light based on the vehicle's two-category orientation (front / back).

[0129] b. Trucks / buses do not have dome lights or running lights, and decide whether to brake or double flash based on the brightness or absence of the left and right lights.

[0130] c. There is usually an advertising or taxi identification light on the dome light of the taxi, and braking is judged based on the timing status of the left and right lights.

[0131] d. When the predictions of multiple classifiers conflict, select the prediction with the highest confidence, or e. In order to minimize false positives of turning and braking in the application layer prediction, the final prediction result is output with a confidence level higher than a certain threshold. In this way, flexible post-processing makes the training method more adaptable, and setting a confidence threshold can reduce false positives.

[0132] In the fourth step, the training process is supplemented.

[0133] In some possible implementations, the first and second steps may be merged into one step and directly trained using the mini-batch method. If the labels of the datasets match, mini-batch may not be used, and the dataset may be used to perform one-step training on the network to obtain a trained network model.

[0134] In an embodiment of the present disclosure, first, an image captured by a camera attached to the vehicle is acquired, then, based on the image, it is determined whether the lit turn signal lights of the other vehicle indicate a left turn or a right turn, and finally, based on the image, it is further determined whether the lit turn signal lights of the other vehicle indicate a left turn or a right turn using the vehicle light information and forward direction (forward / backward) of the other vehicle displayed in the image. In this way, when determining the intention to turn left or right, by combining the information on the lit position of the vehicle lights and the forward direction (front / back) information of the other vehicle displayed in the image, it is possible to enhance the robustness of the determination of whether the vehicle is turning left or right.

[0135] Those skilled in the art can understand that in the above method of the specific embodiment, the order in which each step is written does not mean a strict execution order and does not constitute any restriction on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0136] Based on the same inventive concept, the embodiments of the present disclosure further provide a motor intention determination device corresponding to the motor intention determination method. The principle of solving the problem of the device in the embodiments of the present disclosure is similar to the above-mentioned motor intention determination method in the embodiments of the present disclosure, so the implementation of the device can refer to the implementation of the method.

[0137] An embodiment of the present disclosure provides a movement intention determining device. Figure 5 is a structural schematic diagram of a movement intention determining device according to an embodiment of the present disclosure. As shown in Figure 5, the movement intention determining device 600 includes: an image capture unit 601 configured to capture traffic images; an information determining unit 602 configured to determine, based on the traffic image, vehicle light information and orientation information of the vehicles in the traffic image; and an intention determining unit 603 configured to determine a motion intention of the vehicle based on the vehicle light information and the direction information.

[0138] In some embodiments, the information determiner 602: a position information determining sub-unit configured to determine position information of illuminated target lights of the vehicle in the traffic image based on the traffic image; an appearance information determination subunit configured to determine, based on said traffic images, appearance information of vehicles in said traffic images; a direction information determination subunit configured to determine direction information of the head of the vehicle based on the appearance information of the vehicle; The intention determination unit 603 further The vehicle's motion intention is determined based on position information of the target vehicle light of the vehicle and information on the direction of the head of the vehicle.

[0139] In some embodiments, the target vehicle light is a single turn signal light, and the intention determining unit 603 is a turn information determination subunit configured to determine turn information indicated by a turn signal light based on position information of the single turn signal light and head direction information of the vehicle; and an intention determination subunit configured to determine a turning intention of the vehicle based on the turning information.

[0140] In some embodiments, the intention determiner 603 is and a braking state determination subunit configured to determine that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being a plurality of turn signal lights.

[0141] In some embodiments, the device further comprises: a vehicle type information determination unit configured to determine vehicle type information of vehicles in the traffic image based on the traffic image; The intention determination unit 603 further The vehicle is configured to determine a motion intention of the vehicle based on the vehicle light information, orientation information, and vehicle type information.

[0142] In some embodiments, when determining the vehicle's movement intention, the device further comprises: a confidence determination unit configured to determine a confidence of the vehicle's movement intention; and a reliability adjustment unit configured to reduce a reliability of the movement intention in response to the orientation information indicating that the vehicle is facing sideways.

[0143] In some embodiments, the device further comprises: a demand acquisition unit configured to acquire an application demand for predicting the vehicle's motion intention; a reliability threshold matching unit configured to determine a reliability threshold matching the application demand; After determining the vehicle's motion intention, the intention determination unit 603 further: A movement intention having a reliability greater than the reliability threshold is determined as a determined movement intention of the vehicle.

[0144] In some embodiments, the determination of the vehicle light information, the orientation information, and the vehicle's movement intention is performed by a neural network, a first classifier in the neural network being obtained by training using sample images labeled with the vehicle light information and the orientation information, and a second classifier in the neural network being obtained by training using sample images labeled with the vehicle's movement intention.

[0145] In some embodiments, the second classifier comprises at least one of a base classifier for classifying a base movement intention of the vehicle and an extended classifier for classifying an extended movement intention of the vehicle, wherein the base classifier is obtained by training based on sample images labeled with the overall vehicle light status, and the extended classifier is obtained by training based on sample images labeled with the light status of the vehicle's turn signal lights.

[0146] In some embodiments, the information determiner 602 is further configured to determine, based on the traffic image, using the neural network, vehicle light information of vehicles in the traffic image and orientation information of the vehicles, wherein the information determiner 602 is configured to: a mask determination subunit configured to determine an attention mask of the traffic image using a convolutional layer of the neural network; a spatial feature determination subunit configured to determine spatial features of the traffic image based on the attention mask; a feature merging subunit configured to merge the spatial features and the temporal features of the traffic image to obtain image features of the traffic image; and an information determining subunit configured to determine, based on the image features, vehicle light information and vehicle orientation information using the first classifier.

[0147] In some embodiments, the intention determiner 603 is an information input sub-unit configured to input the vehicle light information and the orientation information to the second classifier and output a predicted movement intention of the vehicle via the second classifier; a confidence determination subunit configured to determine a first confidence level of the predicted motor intention and a second confidence level of the classification result in response to a mismatch between the predicted motor intention and the classification result output by the first classifier; and a reliability comparison subunit configured to determine the vehicle's motion intention based on the prediction result corresponding to a relatively larger reliability out of the first reliability of the predicted motion intention and the second reliability of the classification result.

[0148] It should be noted that the above description of the apparatus embodiment is similar to the above description of the method embodiment, and has the same beneficial effects as the method embodiment. For technical details not disclosed in the apparatus embodiment of the present disclosure, please refer to the description of the method embodiment of the present disclosure.

[0149] In this and other embodiments of the present disclosure, a "module" may be a circuit, a processor, a program, software, or the like, and may of course be a unit or may be non-modular.

[0150] It should be noted that in the embodiments of the present disclosure, the above-described motor intention determination method may be implemented in the form of a software functional module and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solutions of the embodiments of the present disclosure, either essentially or in part contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of the present disclosure are not limited to any specific combination of hardware and software.

[0151] Correspondingly, embodiments of the present disclosure further provide a computer program product including computer-executable instructions, which, after execution, can realize steps of the motor intention determination method provided by the embodiments of the present disclosure. Correspondingly, embodiments of the present disclosure further provide a computer storage medium, which stores computer-executable instructions and, when executed by a processor, realizes steps of the motor intention determination method provided by the above-mentioned embodiments. Correspondingly, embodiments of the present disclosure provide a computer device, and FIG. 6 is a structural schematic diagram of a computer device according to an embodiment of the present disclosure. As shown in FIG. 6, the computer device 700 includes one processor 701, at least one communication bus, a communication interface 702, at least one external communication interface, and a memory 703. Here, the communication interface 702 is configured to realize connection communication between these components. The communication interface 702 can include a display, and the external communication interface can include a standard wired interface and a wireless interface. The processor 701 is configured to execute an image processing program in the memory to realize steps of the motor intention determination method provided by the above-mentioned embodiments.

[0152] The above descriptions of the embodiments of the motor intention determination device, computer device, and storage medium are similar to the descriptions of the above method embodiments, and have the same technical details and beneficial effects as the corresponding method embodiments. Due to space limitations, the descriptions of the above method embodiments may be referenced, and therefore will not be described in detail here. For technical details not disclosed in the embodiments of the motor intention determination device, computer device, and storage medium of the present disclosure, please refer to the descriptions of the method embodiments of the present disclosure. It should be understood that the term "one embodiment" or "an embodiment" used throughout the specification means that specific features, structures, or characteristics associated with an embodiment are included in at least one embodiment of the present disclosure. Therefore, the appearance of "in one embodiment" or "in one embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In various embodiments of the present disclosure, the magnitude of the numbers of the above processes does not imply an order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitations on the implementation process of the embodiments of the present disclosure. The numbers of the embodiments of the present disclosure above are for illustrative purposes only and do not represent the superiority or inferiority of the embodiments.

[0153] It should be understood that, as used herein, the terms "comprises," "having," or any variation thereof, are intended to cover a non-exclusive inclusion, whereby a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also other elements not expressly listed or inherent in such process, method, article, or apparatus. Absent further limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

[0154] In some embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods may be realized in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical and functional division. In actual implementation, other division methods are possible. For example, multiple units or components may be combined or integrated into other systems, or some features may be omitted or not implemented. Furthermore, the mutual coupling, direct coupling, or communication connection between each component shown or discussed may be an indirect coupling or communication connection via some interfaces, devices, or units, and may be in an electrical, mechanical, or other form.

[0155] The units described as separate components may or may not be physically separate, and components represented as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the objectives of the solutions of the present embodiments. Furthermore, the functional units in each embodiment of the present disclosure may all be integrated into a single processing unit, each individual unit may be a single unit, or two or more units may be integrated into a single unit. The integrated units may be implemented in the form of hardware or a functional unit that combines hardware and software. Those skilled in the art will understand that all or some of the steps for implementing the above method embodiments may be achieved by instructing relevant hardware through a program. The program may be stored in a computer-readable storage medium, and when the program is executed, the steps of the above method embodiments are performed. The storage medium may include various media capable of storing program code, such as a mobile storage device, a read-only memory (ROM), a magnetic disk, or an optical disk.

[0156] Alternatively, in the present disclosure, the above-mentioned integrated units may be realized in the form of software functional modules and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solutions of the embodiments of the present disclosure, either essentially or in part contributing to the prior art, may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute all or part of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks. The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any modifications or substitutions that anyone skilled in the art can easily conceive within the technical scope disclosed in the present disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be governed by the scope of protection of the claims. [Industrial Applicability]

[0157] An embodiment of the present disclosure provides a method, an apparatus, a device, and a storage medium for determining a motor intention, the method including: acquiring a traffic image; determining, based on the traffic image, vehicle light information and orientation information of a vehicle in the traffic image; and determining, based on the vehicle light information and orientation information, a motor intention of the vehicle.

Claims

1. A motor intention determination method executed by an electronic device, comprising: acquiring traffic images; determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, Determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image, determining position information of the illuminated target lights of the vehicle in the traffic image based on the traffic image; determining appearance information of vehicles in the traffic image based on the traffic image; determining head direction information of the vehicle based on the appearance information of the vehicle; determining a motion intention of the vehicle based on the vehicle light information and the direction information, determining a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and direction information of the head of the vehicle; Determining the vehicle's intention to move based on position information of a target vehicle light of the vehicle and direction information of the head of the vehicle, determining that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being a plurality of turn signal lights. Exercise intention determination method.

2. The target vehicle light is a single turn signal light, and determining the vehicle's motion intention based on position information of the target vehicle light and information on the direction of the head of the vehicle, determining turn information indicated by a turn signal light based on position information of the single turn signal light and head orientation information of the vehicle; and determining a turning intention of the vehicle based on the turning information. The exercise intention determination method according to claim 1 .

3. A motor intention determination method executed by an electronic device, comprising: acquiring traffic images; determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, The exercise intention determination method further includes: determining vehicle type information of vehicles in the traffic image based on the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, and determining the vehicle's intention to move based on the vehicle light information, direction information, and vehicle type information. Exercise intention determination method.

4. A motor intention determination method executed by an electronic device, comprising: acquiring traffic images; determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, When determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes: and reducing a reliability of the movement intention in response to the orientation information indicating that the vehicle is facing sideways. Exercise intention determination method.

5. A motor intention determination method executed by an electronic device, comprising: acquiring traffic images; determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, When determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes: Obtaining an application demand for predicting the vehicle's motion intention; determining a reliability threshold that matches the application demand; After determining the vehicle's motion intention, the motion intention determination method further includes: and determining a motion intention having a reliability greater than the reliability threshold as the determined motion intention of the vehicle. Exercise intention determination method.

6. A motor intention determination method executed by an electronic device, comprising: acquiring traffic images; determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information, the determination of the vehicle light information, the orientation information and the vehicle's movement intention is performed by a neural network; The first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and direction information, and the second classifier in the neural network is obtained by training using sample images labeled with vehicle movement intentions. Exercise intention determination method.

7. The second classifier includes at least one of a base classifier for classifying a basic vehicle movement intention and an extended classifier for classifying an extended vehicle movement intention, wherein the base classifier is obtained by training based on sample images labeled with the overall vehicle light state, and the extended classifier is obtained by training based on sample images labeled with the light state of a turn signal light of the vehicle. The exercise intention determination method according to claim 6.

8. determining, based on the traffic image, vehicle light information and vehicle orientation information of the vehicles in the traffic image using the neural network, determining an attention mask of the traffic image using a convolutional layer of the neural network; determining spatial features of the traffic image based on the attention mask; merging the spatial features with the temporal features of the traffic image to obtain image features of the traffic image; and determining vehicle light information and vehicle orientation information based on the image features using the first classifier. The exercise intention determination method according to claim 6.

9. determining a motion intention of the vehicle based on the vehicle light information and the direction information, inputting the vehicle light information and the direction information into the second classifier, and outputting a predicted movement intention of the vehicle by the second classifier; determining a first confidence level of the predicted motor intention and a second confidence level of the classification result in response to a mismatch between the predicted motor intention and the classification result output by the first classifier; and determining the vehicle's intention based on the prediction result corresponding to a relatively higher reliability out of the first reliability of the predicted intention and the second reliability of the classification result. The exercise intention determination method according to claim 8.

10. A motor intention determination device, comprising: an image capture unit configured to capture traffic images; an information determining unit configured to determine, based on the traffic image, vehicle light information and orientation information of the vehicles in the traffic image; an intention determination unit configured to determine a motion intention of the vehicle based on the vehicle light information and the direction information, The information determination unit determining position information of the target lights of the vehicle in the traffic image based on the traffic image; determining appearance information of vehicles in the traffic images based on the traffic images; determining head direction information of the vehicle based on the appearance information of the vehicle; The intention determination unit determining a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and information on a direction of the head of the vehicle; The intention determination unit determining that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being a plurality of turn signal lights; Motor intention determination device.

11. A computer storage medium having stored thereon computer-executable instructions for causing a computer to execute the motor intention determination method according to any one of claims 1 to 9.

12. A computing device comprising: a memory for storing computer-executable instructions; and a processor for executing the computer-executable instructions in the memory to perform the movement intention determination method according to any one of claims 1 to 9.

13. A computer program that causes a computer to execute the exercise intention determination method according to any one of claims 1 to 9.

Images