Exercise intention determination method, device, equipment, and storage medium

Abstract

The embodiments of the present disclosure provide a motion intention determination method, an apparatus, a device, and a storage medium. The motion intention determination method includes: acquiring a traffic image; determining vehicle light information and orientation information of a vehicle in the traffic image based on the traffic image; and determining a motion intention of the vehicle based on the vehicle light information and the orientation information. In this way, the motion intention of the vehicle can be predicted more accurately.

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This disclosure is filed based on a Chinese patent application having application number 202210122826.X, filing date February 9, 2022, and title "Method, device, apparatus and storage medium for determining motor intention," and claims priority to the Chinese patent application, the entire contents of which are incorporated by reference into this disclosure.

[0002] The embodiments of the present disclosure relate to the field of intelligent driving technology, including, but not limited to, a method, an apparatus, a device, and a storage medium for determining driving intentions. [Background technology]

[0003] In recent years, with the introduction of running lights and sidelights, vehicle lighting has become more complex, and it is no longer possible to accurately determine whether a vehicle is braking or turning based on the overall brightness of the two left and right lights alone. Summary of the Invention [Means for solving the problem]

[0004] The embodiments of the present disclosure provide a technical solution for determining movement intention.

[0005] The technical solutions of the embodiments of the present disclosure are realized as follows.

[0006] An embodiment of the present disclosure provides a method for determining a movement intention, the method including: acquiring a traffic image; determining, based on the traffic image, vehicle light information of a vehicle in the traffic image and orientation information of the vehicle; and determining, based on the vehicle light information and the orientation information, a movement intention of the vehicle.

[0007] In some embodiments, determining vehicle light information of a vehicle in the traffic image and orientation information of the vehicle based on the traffic image includes determining position information of a target vehicle light turned on in the traffic image based on the traffic image, determining appearance information of a vehicle in the traffic image based on the traffic image, and determining orientation information of the head of the vehicle based on the appearance information of the vehicle, and determining a motion intention of the vehicle based on the vehicle light information and the orientation information includes determining a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and orientation information of the head of the vehicle. In this way, by combining the position information of the target vehicle light and the orientation information of the vehicle, a more accurate prediction of a vehicle turn can be made.

[0008] In some embodiments, the target vehicle light is a single turn signal light, and determining the motion intention of the vehicle based on the position information of the target vehicle light of the vehicle and the head direction information of the vehicle includes: determining turning information indicated by the turn signal light based on the position information of the single turn signal light and the head direction information of the vehicle, and determining the turning intention of the vehicle based on the turning information. In this way, by analyzing the position of the single turn signal light and the head direction, the turning information indicated by the turn signal light can be accurately obtained, and the turning intention of the vehicle can be accurately predicted.

[0009] In some embodiments, determining the motion intention of the vehicle based on the position information of the target vehicle lights of the vehicle and the head orientation information of the vehicle includes determining that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being multiple turn signal lights. In this way, by identifying whether multiple turn signal lights are simultaneously turned on, it is possible to accurately predict whether the vehicle is in a braking state.

[0010] In some embodiments, the motion intention determining method further includes: determining, based on the traffic image, vehicle type information of a vehicle in the traffic image; and determining the motion intention of the vehicle based on the vehicle light information and the orientation information includes determining the motion intention of the vehicle based on the vehicle light information, orientation information and vehicle type information. In this way, by combining the vehicle light information, orientation information and vehicle type information, turning information indicated by the turn signal lights of a vehicle can be accurately obtained, i.e., the motion intention of the vehicle can be predicted.

[0011] In some embodiments, when determining the motion intention of the vehicle, a reliability of the motion intention of the vehicle is determined, and the motion intention determination method further includes reducing a reliability of the motion intention in response to the orientation information indicating that the vehicle is facing sideways. In this manner, by reducing a reliability of the motion intention when the orientation information indicates that the vehicle is facing sideways, it is possible to improve prediction accuracy of the vehicle motion intention.

[0012] In some embodiments, when determining the motion intention of the vehicle, a reliability of the motion intention of the vehicle is determined, and the motion intention determination method further includes obtaining an application demand for predicting the motion intention of the vehicle and determining a reliability threshold matching the application demand, and after determining the motion intention of the vehicle, the motion intention determination method further includes determining a motion intention whose reliability is greater than the reliability threshold as the determined motion intention of the vehicle. In this way, the reliability threshold can be set according to the application demand, thereby making the predicted motion intention better meet the demand of the user.

[0013] In some embodiments, the determination of the vehicle light information, the orientation information, and the vehicle's motion intention is performed by a neural network, and a first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and orientation information, and a second classifier in the neural network is obtained by training using sample images labeled with the vehicle's motion intention. In this way, the accuracy of motion intention prediction can be improved by identifying the vehicle's motion intention using a classification network including multiple classifiers.

[0014] In some embodiments, the second classifier includes at least one of a base classifier for classifying a basic motion intention of the vehicle and an extended classifier for classifying an extended motion intention of the vehicle, where the base classifier is obtained by training based on sample images labeled with the overall vehicle light state of the vehicle, and the extended classifier is obtained by training based on sample images labeled with the vehicle light state of the turn signal lights. In this way, the base classifier and the extended classifier assist each other in the training process, and in the training process, the network first considers the overall vehicle light state, and further considers the turn signal light turning in the overall vehicle light, so as to more accurately predict the vehicle motion intention.

[0015] In some embodiments, determining vehicle light information and vehicle orientation information of vehicles in the traffic image based on the traffic image using the neural network includes: determining an attention mask of the traffic image using a convolutional layer of the neural network, determining spatial features of the traffic image based on the attention mask, merging the spatial features and temporal features of the traffic image to obtain image features of the traffic image, and determining vehicle light information and vehicle orientation information of the vehicles based on the image features using the first classifier. In this way, multi-task learning can be used to assist a classifier for vehicle light state classification using a classifier for vehicle orientation, vehicle type, etc., and further improve the accuracy of predicting the display state of vehicle lights.

[0016] In some embodiments, determining the motion intention of the vehicle based on the vehicle light information and the orientation information includes: inputting the vehicle light information and the orientation information to the second classifier and outputting the predicted motion intention of the vehicle by the second classifier; determining a first reliability of the predicted motion intention and a second reliability of the classification result in response to a mismatch between the predicted motion intention and the classification result output by the first classifier; and determining the motion intention of the vehicle based on a prediction result corresponding to a relatively higher reliability out of the first reliability of the predicted motion intention and the second reliability of the classification result. In this way, when prediction results of multiple classifiers collide, the one with a relatively higher reliability can be selected as a final prediction result, thereby making it possible to more accurately predict the motion intention of the vehicle.

[0017] An embodiment of the present disclosure provides a movement intention determination device, the device comprising: an image acquisition unit configured to acquire a traffic image; an information determination unit configured to determine, based on the traffic image, vehicle light information of a vehicle in the traffic image and orientation information of the vehicle; and an intention determination unit configured to determine a movement intention of the vehicle based on the vehicle light information and the orientation information.

[0018] Correspondingly, an embodiment of the present disclosure provides a computer storage medium, which stores computer-executable instructions, and which, after execution of the computer-executable instructions, can realize the steps of the method described above.

[0019] An embodiment of the present disclosure further provides a computer program product, the computer program product including a computer program or instructions, which when executed on an electronic device, causes the electronic device to perform steps in any possible embodiment of the first aspect above.

[0020] An embodiment of the present disclosure provides a computing device, the computing device comprising a memory and a processor, the memory storing computer-executable instructions, and capable of implementing steps of the method described above when the processor executes the computer-executable instructions in the memory.

[0021] The embodiments of the present disclosure provide a method, device, apparatus and storage medium for determining a motion intention, which determines vehicle light information and orientation information of a vehicle in an acquired traffic image, and determines whether the vehicle will perform a motion intention such as braking or turning by combining the vehicle light information and the orientation information, thereby making it possible to more accurately predict the vehicle's motion intention.

[0022] In order to make the above objects, features and advantages of the embodiments of the present disclosure more comprehensible, preferred embodiments will be specifically described below with reference to the accompanying drawings. [Brief description of the drawings]

[0023] [Figure 1A] FIG. 1 is a schematic diagram of a system architecture to which a movement intention determination method according to an embodiment of the present disclosure can be applied. [Figure 1B] 1 is a schematic flowchart of an implementation of a movement intention determination method according to an embodiment of the present disclosure. [Diagram 2]1 is a schematic flowchart of another implementation of a movement intention determination method according to an embodiment of the present disclosure. [Diagram 3] FIG. 2 is a schematic diagram of an application scenario of the exercise intention determination method according to an embodiment of the present disclosure. [Figure 4] FIG. 2 is a schematic diagram of an implementation framework of a movement intention determination method according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a structural schematic diagram of a motion intention determining device according to an embodiment of the present disclosure. [Figure 6] FIG. 2 is a structural schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] In order to more clearly explain the technical solutions of the embodiments of the present disclosure, the above briefly introduces drawings required for the embodiments, which are incorporated in the specification and form a part of this specification, and these drawings show embodiments consistent with the present disclosure and are used to explain the technical solutions of the present disclosure together with the specification. The following drawings only show some embodiments of the present disclosure, so they should not be considered as limiting the scope, and it should be understood that those skilled in the art can also obtain other related drawings based on these drawings without creative efforts.

[0025] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the specific technical solutions of the invention are described in more detail below with reference to the drawings in the embodiments of the present disclosure. The following embodiments are for illustrating the present disclosure, and are not intended to limit the scope of the present disclosure.

[0026] It will be understood that while "some embodiments" in the following description describe a subset of all possible embodiments, "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other if there is no conflict.

[0027] The terms "first, second, third" in the following description do not represent a particular order of objects, but merely distinguish between similar objects. It is to be understood that the terms "first, second, third" may be interchanged with a particular order or order of precedence, where permitted, such that the embodiments of the present disclosure described in some embodiments may be implemented in an order other than that illustrated or described in some embodiments.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure pertains. The terms used herein are merely for the purpose of describing the embodiments of the present disclosure and are not intended to be limiting of the present disclosure.

[0029] Before describing the embodiments of the present disclosure in more detail, the nouns and terms referred to in the embodiments of the present disclosure will be explained, and the nouns and terms referred to in the embodiments of the present disclosure will be applied to the following interpretation.

[0030] 1) Convolutional Neural Networks (CNN): A type of feedforward neural network with a deep structure that includes convolutional calculations. It has feature learning capabilities and can perform translation-invariant classification of input information according to its hierarchical structure.

[0031] 2) Ego vehicle: A vehicle that includes sensors to sense the surrounding environment. The vehicle coordinate system is fixedly connected to the ego vehicle, where the x-axis is the forward direction of the car, the y-axis points to the left of the forward direction of the vehicle, and the z-axis is perpendicular to the ground and points upward, conforming to a right-handed coordinate system. The origin of the coordinate system is located on the ground below the midpoint of the rear axle.

[0032] In the following, exemplary applications of the exercise intention determination device provided by the embodiment of the present disclosure will be described, and the device provided by the embodiment of the present disclosure may be implemented as a notebook computer, tablet computer, or other in-vehicle device with image collection function, or may be implemented as a server. In the following, exemplary applications when the device is implemented as a terminal or a server will be described.

[0033] 1A is a schematic diagram of a system architecture of a motion intention determination method according to an embodiment of the present disclosure. As shown in FIG. 1A, the system architecture includes an image capture device 11, a network 12, and an in-vehicle control terminal 13. To realize the support of one exemplary application, the image capture device 11 and the in-vehicle control terminal 13 establish a communication connection through the network 12. First, the image capture device 11 reports a traffic image captured through the network 12 to the in-vehicle control terminal 13, and the in-vehicle control terminal 13 identifies the traffic image to determine vehicle light information and orientation information of the vehicle in the traffic image, and combines the vehicle light information and the orientation information to determine the motion intention of the vehicle.

[0034] For example, the image capture device 11 may include a visual processing device having visual information processing capabilities. The network 12 may adopt a wired or wireless connection method. Here, when the image capture device 11 is a visual processing device, the vehicle control terminal 13 can be connected to the visual processing device via a wired connection method, and can perform data communication, for example, via a bus.

[0035] Or, in some scenarios, the image capture device 11 may be a visual processing device with a video collection module, or a host with a camera. In this case, the augmented reality data display method of the embodiment of the present disclosure may be performed by the image capture device 11, and the above system architecture may not include the network 12 and the in-vehicle control terminal 13.

[0036] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the specific technical solutions of the invention are described in more detail below with reference to the drawings in the embodiments of the present disclosure. The following embodiments are constructed to illustrate the present disclosure, but are not intended to limit the scope of the present disclosure.

[0037] It will be understood that while "some embodiments" in the following description describe a subset of all possible embodiments, "some embodiments" may be the same or different subsets of all possible embodiments, and may be combined with each other if there is no conflict.

[0038] The method can be applied to a computer device, and the functions realized by the method can be realized by a processor in the computer device calling a program code, and of course, the program code can be stored in a computer storage medium, and it can be understood that the computer device includes at least a processor and a storage medium.

[0039] 1B is a schematic flow chart of the implementation of the exercise intention determination method according to an embodiment of the present disclosure. As shown in FIG. 1B, the description will be given with reference to the steps shown in FIG.

[0040] In S101, a traffic image is acquired.

[0041] In some embodiments, the traffic image may be an image collected on any road, may be an image with complex screen content, or may be an image with simple screen content. For example, an image of a street scenario collected at midnight, or an image of a street scenario collected during the day. The traffic image may include vehicles, where the vehicles include vehicles with various functions (e.g., trucks, cars, motorcycles, etc.) and vehicles with various numbers of wheels (e.g., four-wheeled vehicles, two-wheeled vehicles, etc.). A passenger car may be described as an example below. For example, the traffic image is an image collected of a passenger car on a road.

[0042] In step S102, based on the traffic image, vehicle light information of vehicles in the traffic image and orientation information of the vehicles are determined.

[0043] In some embodiments, a traffic image is input to a trained neural network, and a fully convolutional network in the network is used to perform feature extraction on the traffic image to obtain image features. The image features are input to multiple classifiers in the neural network to identify vehicle light information of each vehicle light of the vehicle and orientation information of the vehicle. The vehicle light information includes the display state of the vehicle light and the position of the vehicle light on the vehicle. For example, the vehicle is a normal passenger car, and the vehicle lights of the normal passenger car include headlights, fog lights, backup lights, license plate lights, position lights, running lights, turn signals, dome lights, high-level brake lights, high beams / low beams, warning lights, and trunk lights. The vehicle light information of the normal passenger car is the display state and position of each vehicle light included in the vehicle body. The orientation information is the orientation of the head of the vehicle and is configured to represent the vehicle orientation of the vehicle, including head-forward or head-backward with respect to the own vehicle whose image was collected, i.e. head-forward means that the tail of the vehicle faces the own vehicle and it is the tail of the vehicle that is presented in the traffic image collected in this way, and head-backward means that the head of the vehicle faces the own vehicle and it is the head of the vehicle that is presented in the traffic image collected in this way. The orientation information further includes the lateral direction of the vehicle, for example, the head of the vehicle faces the left side of the road or the right side of the road, i.e. the vehicle is lying on the road.

[0044] In step S103, the vehicle's motion intention is determined based on the vehicle light information and the direction information.

[0045] In some embodiments, the vehicle light display state of the vehicle and the vehicle head direction information are combined to determine the motion manner indicated by the vehicle lights turned on. That is, the motion intention includes the vehicle turning left, turning right, moving forward, moving backward, or braking. A traffic image is input to a classifier for classifying vehicle light information in a neural network and a classifier for predicting the vehicle head direction, to obtain classification results output by a plurality of vehicle light classifiers and a classification result output by a vehicle direction classifier. It is determined whether the output results of these plurality of classifiers collide with each other, and finally, vehicle light information and direction information with relatively high reliability are output. In this way, the vehicle light information and direction information can be comprehensively considered to predict the vehicle motion intention more accurately.

[0046] In the embodiment of the present disclosure, the vehicle light information and the direction information of the vehicle are determined in the traffic image, and the vehicle light information and the direction information are combined to determine whether the vehicle has a motion intention such as braking or turning, etc. In this way, the motion intention of the vehicle can be analyzed more accurately.

[0047] In some embodiments, by analyzing the two types of vehicle orientations, it is determined whether the turn signal light is a left turn signal light or a right turn signal light, and then the vehicle's motion intention is determined, that is, the above S102 can be realized by the following steps S121 to S123 (not shown).

[0048] In S121, position information of the illuminated target vehicle lights of the vehicle in the traffic image is determined based on the traffic image.

[0049] Here, a neural network is used to detect the light of a vehicle in a traffic image, and if the light is a turn signal light, the coordinates of the detection frame of the turn signal light are determined, thereby obtaining the position information of the target turn signal light.

[0050] In some possible implementations, first, determine the target vehicle light to be turned on in the vehicle light information. Here, select a target vehicle light whose display state is turned on in the vehicle light information. The target vehicle light may be any vehicle light on the vehicle. Then, if the target vehicle light is a turn signal light, determine the position information of the turn signal light based on the orientation information. Here, the orientation information may include the orientation of the head of the vehicle, i.e., whether the head of the vehicle faces forward or the head of the vehicle faces backward. For example, for the same vehicle, when the head of the vehicle faces forward, the position of the left turn signal light in the image is the left side, and when the head of the vehicle faces backward, the position of the left turn signal light in the image is the right side. In this way, combine the orientation of the head of the vehicle to determine whether the target turn signal light in the turn signal light information is a left turn signal light or a right turn signal light. If the target vehicle light is not a turn signal light, the target vehicle light can be represented in the form of a detection frame in the traffic image, and the coordinates of the detection frame are the position information of the target vehicle light. In this way, by taking into account the headway information, it is possible to accurately predict whether the target turn signal light is a left turn signal light or a right turn signal light.

[0051] In step S122, based on the traffic image, appearance information of vehicles in the traffic image is determined.

[0052] In some embodiments, a single-frame traffic image is input to a neural network to extract appearance information of the vehicle, where the appearance information includes the appearance that the vehicle presents in the traffic image, for example, if the traffic image is collected behind a vehicle, the appearance information is the appearance corresponding to the tail of the vehicle (including the turn signal lights and high brake lights at the rear of the vehicle, etc.).

[0053] In step S123, information on the direction of the head of the vehicle is determined based on the external appearance information of the vehicle.

[0054] In some embodiments, by analyzing the image content of a vehicle depicted in a traffic image, the orientation of the vehicle head can be determined, i.e., whether the vehicle's heading is forward, reverse or sideways.

[0055] The above steps S121 to S123 provide a method for determining the vehicle head direction information and the position information of the target vehicle light. By predicting the vehicle direction information using a single frame image in this manner, the network model can be simplified and the processing delay can be reduced.

[0056] After the position information of the target vehicle lights and the direction information of the vehicle head are determined, the vehicle motion intention is determined in the next step S124.

[0057] In step S124, the motion intention of the vehicle is determined based on the position information of the target vehicle lights of the vehicle and the direction information of the head of the vehicle.

[0058] Here, the turning of the vehicle, i.e., the target turning, is predicted based on the vehicle's direction information and the target indicator light that is turned on. For example, if the target indicator light is a left indicator light and the vehicle is facing forward, the target turning is a left turn ahead. In this way, the position information of the target vehicle light and the direction information of the vehicle can be combined to more accurately predict the turning of the vehicle.

[0059] In some embodiments, when the target vehicle light is a single turn signal light, the vehicle's motion intention can be predicted by analyzing the position information of the single turn signal light and the head direction information of the vehicle. That is, the above step S124 can be realized by the following steps:

[0060] In a first step, turning information indicated by a turn signal light is determined based on position information of the single turn signal light and head direction information of the vehicle.

[0061] In some embodiments, the position information of a single turn signal light is the position of the single turn signal light on the vehicle shown in the traffic image, and the turning information indicated by the turn signal light may be a left turn, a right turn, or a double flash, etc. In one specific example, in a traffic image, when the vehicle head faces forward, if the position information of the single turn signal light is on the left side of the position in the image, it means that the turn signal light is a left turn signal, and it is further determined that the turning information indicated by the turn signal light is a left turn signal. Similarly, when the vehicle head faces backward, if the position information of the single turn signal light is on the left side of the position in the image, it means that the turn signal light is a right turn signal, and it is further determined that the turning information indicated by the turn signal light is a right turn signal.

[0062] In a second step, a turning intention of the vehicle is determined based on the turning information.

[0063] In some embodiments, the turn indicated by the turn indicator can be obtained according to the turn information indicated by the turn indicator, and the next turn of the vehicle can be predicted, i.e., the turning intention of the vehicle can be determined.

[0064] In the embodiments of the present disclosure, by analyzing the position of a single turn signal light and the vehicle head orientation, the turning information indicated by the turn signal light can be accurately obtained, and the turning intention of the vehicle can be accurately predicted.

[0065] In some embodiments, when the target vehicle lights are multiple turn signal lights, it is possible to analyze whether the vehicle is in a braking state by analyzing whether the vehicle light information includes brake light information, that is, the above step S124 may be realized by a process that determines that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights are multiple turn signal lights.

[0066] In some embodiments, if the vehicle light information does not include brake light information, this means that brake light information is not collected in the vehicle light information.

[0067] In some possible implementations, first determine whether there is brake lamp information in the vehicle light information, and then determine the display state of the left and right turn signal lights of the vehicle in the vehicle light information if there is no brake lamp information. For example, since a truck or bus does not have a dome light, its braking state can be determined by the left and right turn signal lights, and in response to both the left and right turn signal lights being turned on, it is determined that the vehicle is in a braking state. That is, if the target vehicle lights that are turned on are multiple turn signal lights, that is, if multiple turn signal lights are in a turned on state at the same time, it can be predicted that the vehicle is in a braking state. In this way, if the vehicle light information does not include brake lamp information, it can be accurately predicted that the vehicle is in a braking state if it is identified that multiple turn signal lights are in a turned on state at the same time.

[0068] In another embodiment, if the vehicle light information includes brake light information, it is possible to predict whether the vehicle is in a braking state by analyzing the blinking state of the brake light information.

[0069] In some embodiments, determining a vehicle turn by identifying the vehicle type and combining the vehicle type with the target vehicle light position information and vehicle orientation information can be achieved by the following steps.

[0070] In a first step, based on the traffic image, vehicle type information of the vehicles in the traffic image is determined.

[0071] In some embodiments, a neural network is used to identify the vehicle type. Image features of a traffic image are input to a vehicle type classifier to identify the vehicle type, for example, the vehicle is a car, truck, or bus.

[0072] In a second step, a motion intention of the vehicle is determined based on the vehicle light information, the direction information and the vehicle type information.

[0073] In some embodiments, the vehicle type information, the vehicle light information, and the orientation information are combined to determine whether the target vehicle light is a single turn signal light, and if so, to further determine whether it is a left turn signal light or a right turn signal light. For example, the vehicle type information can be used to first determine the appearance and position of the turn signal light of the vehicle light, so that it can be determined whether the position information of the target vehicle light in the vehicle light information is the position information of the turn signal light. Then, when it is determined that the target vehicle light is a single turn signal light, the vehicle orientation information can be combined to accurately predict whether the turn signal light of the vehicle is a left turn signal light or a right turn signal light, and the turning information indicated by the turn signal light of the vehicle can be accurately obtained, that is, the motion intention of the vehicle can be predicted.

[0074] In some embodiments, to improve accuracy of the predicted vehicle movement intention, a confidence level of the vehicle movement intention is determined when determining the vehicle movement intention, and a confidence level of the movement intention is reduced in response to the orientation information indicating that the vehicle is facing sideways.

[0075] In some possible implementations, when the orientation classifier identifies that the vehicle orientation is sideways, in order to improve the prediction accuracy of the vehicle's motion intention, the reliability of the motion intention is reduced when the orientation information indicates that the vehicle is sideways. Here, when the vehicle tilts, it is difficult to distinguish between the vehicle turning left and right, so the reliability of the predicted motion intention is reduced. When the vehicle is sideways, only the vehicle light status on one side of the vehicle is visible, and multiple situations may occur, so the predicted reliability is reduced in such situations.

[0076] In some embodiments, a neural network is used to identify vehicle light information, vehicle head direction information, and vehicle movement intention in a traffic image. A trained neural network is obtained, and traffic images are input to multiple classifiers of the network to predict the display state of each vehicle light and the direction of the vehicle head, and further to predict the vehicle movement intention.

[0077] In some embodiments, a first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and orientation information, and a second classifier in the neural network is obtained by training using sample images labeled with vehicle motion intentions.

[0078] In some possible implementations, the first classifier includes at least one classifier that is used to classify the vehicle light information, vehicle type, and orientation information of each vehicle light of the sample vehicle, respectively. For example, the vehicle light information, vehicle type, and orientation information of each vehicle light are obtained by classification based on three different classifiers, respectively, or the vehicle light information, vehicle type, and orientation information of each vehicle light are obtained by classification based on the same classifier. The second classifier is used to classify the motion intention of the sample vehicle, inputting the collected traffic images into a neural network, respectively identifying the display state of each vehicle light of the vehicle by a plurality of classifiers in the neural network, thereby obtaining the vehicle light information of the vehicle. The orientation classifier in the network identifies the orientation of the head of the vehicle, thereby obtaining the orientation information of the head of the vehicle.

[0079] In the process of training the neural network, a mini-batch is used to train the second classifier, i.e., two parts of data with different labeling types are used to train the second classifier. The second classifier includes at least one of a base classifier for classifying a basic motion intention of the vehicle and an extended classifier for classifying an extended motion intention of the vehicle, where the base classifier is obtained by training based on sample images labeled with an overall vehicle light state of the vehicle, and the extended classifier is obtained by training based on sample images labeled with a vehicle light state of a turn signal light of the vehicle, both the basic motion intention and the extended motion intention represent the motion intention of the vehicle, and the reliability of the basic motion intention is lower than the reliability of the extended motion intention. Or, the basic motion intention is a roughly predicted vehicle motion intention based on an overall vehicle light state, and the extended motion intention is a precisely predicted vehicle motion intention based on a display state of a turn signal light of the vehicle. That is, the training sample data for the base classifier is data labeled with the display states of all vehicle lights on the vehicle, and the training sample data for the extended classifier is data labeled with the brightness and absence of left / right turn signals.

[0080] If the second classifier includes a base classifier, it roughly predicts the base driving intention of the vehicle based on the overall display state of the vehicle's lights.

[0081] When the second classifier includes an extended classifier, it accurately predicts the extended motion intention of the vehicle based on the indication state of the vehicle's turn signal lights.

[0082] When the second classifier includes a base classifier and an extended classifier, the base classifier is first used to roughly predict the basic motion intention of the vehicle based on the overall vehicle light display state of the vehicle. Based on the basic motion intention, the display state of the turn signal lights of the vehicle is combined, and the extended classifier is further used to more accurately predict the extended motion intention of the vehicle. In this way, the basic motion intention based on the overall vehicle light display state and the extended motion intention based on the display state of the left / right turn signal lights are trained, so that the base classifier and the extended classifier mutually assist each other in the training process, and in the training process, the network first considers the overall vehicle light state, and further considers the turn signal light turning in the overall vehicle light, so as to more accurately predict the motion intention of the vehicle.

[0083] In some possible implementations, the traffic image is inputted into a first classifier of a neural network to identify at least dome light information and turn signal light information of the vehicle, and obtain vehicle lamp information.

[0084] Here, after feature extraction is performed on the traffic image, it is input to a first classifier, and the left turn signal light classifier, right turn signal light classifier, dome light classifier, etc. in the first classifier classify the display state of the vehicle lights based on the extracted image features, thereby obtaining the display state of the left turn signal light, the display state of the right turn signal light, the display state of the dome light, etc. The vehicle light display state and the vehicle light position identified by each vehicle light classifier in the first classifier are the vehicle light information. In this way, by identifying at least the turn signal light information and the dome light information of the vehicle, not only can the amount of data to be identified be reduced, but also a rich basis can be provided for predicting the vehicle's motion intention.

[0085] In some embodiments, the neural network is used to perform feature extraction on the traffic image, and multiple classifiers are used to predict vehicle light information and orientation information based on the extracted image features. That is, the above step 102 can be realized by the steps shown in Figure 2. Figure 2 is a schematic flow chart of another implementation of the movement intention determination method according to the embodiment of the present disclosure. The following description will be made with reference to the steps shown in Figure 2.

[0086] In step S201, a convolutional layer of the neural network is used to determine an attention mask of the traffic image.

[0087] In some embodiments, a traffic image is input to a fully convolutional neural network to predict an attention mask for the image.

[0088] In step S202, spatial features of the traffic image are determined based on the attention mask.

[0089] In some embodiments, element-wise multiplication is performed on the traffic image and the attention mask, and the product is output to a CNN for spatial feature extraction to obtain spatial features.

[0090] In step S203, the spatial features and the temporal features of the traffic image are merged to obtain image features of the traffic image.

[0091] In some embodiments, the extracted spatial features are input into a special recurrent neural network (RNN), long short-term memory (LSTM), and merged with the temporal features, and the merged features are taken as image features, which are useful for subsequently identifying the vehicle headlight status and vehicle head orientation based on the features.

[0092] In step S204, the first classifier is used to determine vehicle light information and vehicle orientation information based on the image features.

[0093] In some embodiments, the image features are input to the first classifiers to obtain at least predicted vehicle light information for each vehicle light of the vehicle and predicted orientation information for the vehicle. The image features are input to the first classifiers to predict vehicle light states and vehicle head orientation. For example, the first classifiers include a classifier for classifying the display state of a dome light, a classifier for classifying the display state of a left turn signal light, a classifier for classifying the display state of a right turn signal light, a classifier for classifying the type of vehicle, and a classifier for classifying the vehicle head orientation. The classification results of the first classifiers include display states of each vehicle light of the vehicle, the vehicle type, and vehicle head orientation information. Among the classification results of the vehicle light information of the same vehicle light, predicted vehicle light information with a reliability equal to or greater than a reliability threshold is selected as the vehicle light information of the vehicle, and similarly, among the classification results, predicted orientation information with a reliability equal to or greater than a reliability threshold is selected as the vehicle orientation information. For example, the classification result for the left turn signal light includes bright, dark, and none, where if the confidence of the brightness is greater than the confidence threshold, the state in which the left turn signal light is bright is taken as the vehicle light information of the left turn signal light of the vehicle.Similarly, the information on the orientation includes forward, backward, and sideways, where if the confidence of the orientation is greater than the confidence threshold, the orientation information of the vehicle is that the head of the vehicle is facing forward.

[0094] In the embodiment of the present disclosure, by using multi-task learning, a vehicle orientation classifier, a vehicle type classifier, etc. can be used to assist the classifier for vehicle light state classification, and further improve the accuracy of predicting the display state of the vehicle light. For example, an orientation classifier is used to help the vehicle light model determine left and right, and a vehicle type classifier is used to help the vehicle light information classifier determine the vehicle light position and the vehicle light shape.

[0095] In some embodiments, a superior exercise intention can be selected from a plurality of predicted exercise intentions by the following two methods. That is, the above step S103 can be realized by the following steps.

[0096] In method 1, by analyzing the user's demand, a classifier with a relatively high reliability can output the exercise intention, that is, a more accurate exercise intention can be predicted by the following steps.

[0097] In a first step, an application demand for predicting the vehicle's motion intention is obtained.

[0098] In some embodiments, the application requirements may be set autonomously by a user, such as a set maximum number of false brake detections or a preset turn supervision for a vehicle, etc. In one specific example, the application requirements may be that the number of false detections for a right-turning vehicle is less than 5.

[0099] In a second step, a reliability threshold that matches the application demand is determined.

[0100] In some embodiments, the confidence threshold is set based on the application demand. For example, if the application demand is that the number of false positives for right-turning vehicles is less than 5, the confidence threshold can be set to a larger value (e.g., the confidence threshold is set to 0.9). If the application demand is that the number of false positives for right-turning vehicles is less than 20, the confidence threshold can be set to 0.8, etc.

[0101] In a third step, a motion intention whose reliability is greater than the reliability threshold is determined as a determined motion intention of the vehicle.

[0102] In some embodiments, after determining the motion intention of the vehicle, a motion intention whose reliability is greater than the reliability threshold is determined as the determined motion intention of the vehicle. The predicted motion intention may be a left turn, a right turn, forward, backward, or brake, etc. Among these predicted motion intentions, a predicted motion intention whose reliability is greater than a reliability threshold matching an application demand is determined as the motion intention of the vehicle. In this way, the reliability threshold can be set according to the application demand, thereby making the predicted motion intention better meet the user demand.

[0103] In method 2, the output of a classifier with a relatively high reliability is output by analyzing whether there is a conflict between the multiple classification results obtained. That is, a more accurate motor intention can be predicted by the following steps.

[0104] In a first step, the vehicle light information and the orientation information are input to the second classifier, and a predicted motion intention of the vehicle is output by the second classifier.

[0105] In some embodiments, when the second classifier is a base classifier, the base classifier predicts the motion intention of the vehicle by combining the orientation of the vehicle based on all vehicle light information as a whole to obtain an overall predicted motion intention of the vehicle. When the second classifier is an extended classifier, the extended classifier predicts the motion intention of the vehicle by combining the orientation information based on the state of the turn signal lights in the vehicle light information to obtain an extended motion intention of the vehicle.

[0106] In a second step, in response to a mismatch between the predicted movement intention and the classification result output by the first classifier, a first confidence level of the predicted movement intention and a second confidence level of the classification result are respectively determined.

[0107] In some embodiments, the mismatch between the predicted motion intention and the classification result output by the first classifier means that the predicted motion intention and the classification result output by the first classifier collide. In a specific example, the second classifier is a base classifier, the predicted motion intention is to turn left, and the output of the left turn signal classifier in the first classification is off, the output of the right turn signal classifier is on, and the output of the dome light classifier is off, and thus, based on the blinking state of the vehicle lights of each part output by the first classifier, it is concluded that the motion intention of the vehicle is to turn right. At this time, the output results of the multiple classifiers collide. In this case, the reliability of the colliding predicted motion intention and the second reliability of the classification result are respectively obtained. Here, the second reliability of the classification result can be understood as the reliability of determining the vehicle motion intention based on the classification result, or can be understood as the reliability of the entire classification result in the classification result.

[0108] In a third step, the vehicle's motion intention is determined based on the prediction result corresponding to a relatively larger reliability out of the first reliability of the predicted motion intention and the second reliability of the classification result.

[0109] In some embodiments, the one with the highest reliability is selected as the vehicle's motion intention from among the predicted motion intention output by the second classifier and the motion intention determined by the classification result. In this way, when the prediction results of multiple classifiers collide, the one with a relatively high reliability is selected as the final prediction result, making it possible to more accurately predict the vehicle's motion intention.

[0110] In the following, an exemplary application of the embodiments of the present disclosure in a practical application scenario is described, and determining vehicle light status based on multi-task learning and multi-stage learning is taken as an example.

[0111] In advanced driving assistance systems (ADAS) and autonomous driving tasks, it is necessary to detect the light status of other vehicles to determine the intentions and future driving trajectories of other vehicles, and help the vehicle perform tasks such as collision warning and strategy planning. In terms of related technologies, ADAS products are basically blank in the aspect of dynamic prediction, and even in autonomous driving systems, there are almost no dynamic prediction models of light for multiple driving scenarios.

[0112] In the related art, it is not accurate enough to determine which turn signal is turning left or right based on the light position of the identified light. As shown in Figure 3, the right turn signal of the vehicle, the left front turn signal in image 31, and the right rear turn signal in image 32 are all turned on.

[0113] With the emergence of running lights and sidelights, vehicle lights have become complicated, and the overall brightness and darkness of the two left and right lights alone cannot determine whether the vehicle will brake or turn. Based on this, the embodiment of the present disclosure provides a method for predicting vehicle light status, and uses deep learning to perform vehicle light intention judgment tasks, which can cognitively decompose the vehicle intention represented by the brightness and darkness and absence of a single light and the vehicle intention represented by the overall blinking state of the vehicle lights, and the two levels are simultaneously assisted by the vehicle orientation and vehicle type classifiers. In this way, the multi-task and multi-level assisted processing is beneficial to the learning of the vehicle light network, and the inference accuracy of the final model can also be greatly improved.

[0114] In the embodiments of the present disclosure, the left and right turns indicated by the turn signal lights of the vehicle are predicted by obtaining the blinking position information of the vehicle lights shown in the image and the forward direction of the vehicle. In some embodiments, for the characteristics that the positions and colors of the turn signal lights and brake lights of the vehicles in some regions are different, the embodiments of the present disclosure propose a method for predicting the vehicle light state, which can be realized by the following process.

[0115] First, multi-task learning is used to input a vehicle image into a single-frame model for end-to-end training, and simultaneously output the heading / tail dome light / left light / right light / vehicle type / left and right turn signals to make a judgment on the instantaneous state of a single vehicle light.

[0116] In some possible implementations, because the position and shape of vehicle lights are complex and changeable, and there are many types of light and dark combinations, a classifier is set up separately for each light, i.e., to determine the bright and dark and off classifications of left / right vehicle lights, left / right turn signals, and tail dome lights. Based on this, additional information can help determine the position and shape type of vehicle lights, for example, a vehicle direction classifier can help determine left and right vehicle lights, and a vehicle type classifier can help determine the type of vehicle lights.

[0117] Second, multi-level learning is used to further determine braking status and turning intent based on the model's learning of a single light's condition.

[0118] Here, the judgment of braking state and turning intent has multiple levels, and the turning intent and braking intent represented by the whole vehicle lights are obtained based on the state inference of multiple single vehicle lights. Furthermore, the left vehicle light state / right vehicle light state / dome light state are combined to judge the whole vehicle intent. In addition, the occlusion and movement of the vehicle also bring great uncertainty to the judgment of a single frame.

[0119] In some embodiments, the exercise intention determination method may be implemented by the following steps.

[0120] In the first step, we perform multi-task training using a single frame input of a vehicle to obtain multiple classifiers including orientation / vehicle type / dome light status / left light status / right light status.

[0121] In some possible implementations, the motion intention can be determined by the method shown in Fig. 4. As shown in Fig. 4, Fig. 4 is a schematic diagram of an implementation framework of the motion intention determination method according to an embodiment of the present disclosure. A single frame image 400 of a vehicle is input to a vehicle detector 401 to identify the vehicle in the image, and the detection frame of the identified vehicle is input to a CNN 402 to extract features and obtain a feature map 403. The dimension of the feature map 403 is 7 x 7 x 2048. The feature map 403 is processed to obtain a 2048-dimensional feature vector 404.

[0122] In the second step, the network is trained using the mini-batch method based on the classifier from the first step.

[0123] Here, in one batch, half of the data uses data labeled as bright, dark, or absent for the entire left / right vehicle lights (here, entire refers to the entire light, and any sub-lights (which are on even if brake lights / fog lights / turn signals are on) are on), and the other half uses data labeled as bright, dark, or absent for the left / right turn signals. We train a basic vehicle intent based on the entire left / right vehicle light state, and an extended vehicle intent based on the left / right turn signals.

[0124] The two sets of classifiers mutually assist each other during training, where mutual assistance refers to the mutual promotion of different tasks that are correlated in multi-task learning, and the process of mutual promotion is automatically completed during the training of the model. In some possible implementations, the network model can first input the state of the entire lamp, then collect the state of the turn signal lights in the entire lamp, and finally output the vehicle intention of the car light. In this way, not only can the final result be improved, but also the problem of being unable to train due to repeated labeling (e.g., re-labeling data where the state of each light is not labeled) can be solved. In this way, the difficulty of the task is set from shallow to difficult using multi-level learning, which matches the natural cognitive level from judging the state of a single car light to judging the overall braking state and turning intention, and is favorable for model learning. And by training the network in the mini-batch manner, the training difficulty caused by different labeling of data can be largely solved, allowing the same model to obtain multiple features in different labeling information, thereby greatly reducing the labeling cost.

[0125] As shown in FIG. 4, the extracted feature vector 404 is input to multiple fully connected layers (fc) for classification, where the fully connected layer 451 is used to classify the on and off state of the vehicle's dome light, the fully connected layer 452 is used to classify the on and off state of the vehicle's left turn signal light, the fully connected layer 453 is used to classify the on and off state of the vehicle's right turn signal light, the fully connected layer 454 is used to classify the vehicle's orientation (e.g., forward and backward), the fully connected layer 455 is used to classify the vehicle type (e.g., passenger car, truck, bus, taxi, ambulance or other vehicle), the fully connected layer 456 is used to classify the basic vehicle intention based on the overall state of the left and right lights of the vehicle for vehicles with a forward vehicle orientation, and the fully connected layer 457 is used to classify the basic vehicle intention based on the overall state of the left and right lights of the vehicle for vehicles with a backward vehicle orientation. That is, the fully connected layer 456 and the fully connected layer 457 train a basic vehicle intention based on the overall state of the left and right lights, which is suitable for simple scenarios. The fully connected layer 458 is used to classify an extended vehicle intention based on the left and right turn signals of the vehicle for a vehicle whose orientation is forward, and the fully connected layer 459 is used to classify an extended vehicle intention based on the left and right turn signals of the vehicle for a vehicle whose orientation is backward. That is, the fully connected layer 458 and the fully connected layer 459 train an extended vehicle intention based on the left and right turn signals of the vehicle, which is suitable for complex scenarios. The outputs of the classifiers formed by the fully connected layers 451 to 459 are merged to obtain a probability distribution of vehicles in multiple categories, for example, the probability of vehicle intention ( JPEG2025505573000002.jpg513), the probability that the vehicle will turn left ( JPEG2025505573000003.jpg69), the probability that the vehicle will turn right ( JPEG2025505573000004.jpg611), and the probability of vehicle orientation ( The image obtained is JPEG2025505573000005.jpg510).

[0126] In some embodiments, the above-described manner of determining a motor intention by the method shown in FIG. 4 is merely a possible embodiment, and the manner of determining a motor intention in the embodiments of the present disclosure is not limited thereto. For example, a residual network or a deep neural network may be used to determine a motor intention, which will not be described in detail here.

[0127] In the third step, post-processing and logical addition can be conveniently performed based on the first and second steps. For example, the following multiple post-processing and logical addition can be performed:

[0128] a. Determine whether it is a left or right light based on the vehicle's two-category orientation (front / back).

[0129] b. Trucks / buses do not have dome lights and running lights, and decide whether to brake or double flash based on the brightness or absence of left and right lights.

[0130] c. There is usually an advertising or taxi identification light in the dome light of the taxi, and braking is judged based on the timing state of the left and right lights.

[0131] d. When the predictions of multiple classifiers conflict, select the prediction with the highest confidence; or e. In response to the demand for fewer false positives of turns and brakes in application layer prediction, the final prediction result is output with a confidence level higher than a certain threshold. In this way, flexible post-processing makes the applicability of the training method stronger, and setting a confidence threshold can reduce false positive samples.

[0132] In the fourth step, the training process is replenished.

[0133] In some possible implementations, the first and second steps may be merged into one step and trained directly using the mini-batch method. If the labels of the dataset match, mini-batch may not be used, and the dataset may be used to perform one-step training on the network to obtain a trained network model.

[0134] In the embodiment of the present disclosure, first, an image captured by a camera attached to the vehicle is obtained, then, based on the image, it is determined whether the directional indicator light of the other vehicle is turning left or right, and finally, based on the image, it is further determined whether the directional indicator light of the other vehicle is turning left or right using the vehicle light information and forward direction (forward / backward) of the other vehicle displayed in the image. In this way, when determining the intention of turning left or right, the robustness of the determination of the left or right turn of the vehicle can be strengthened by combining the vehicle light lighting position information and forward direction (front / back) information of the other vehicle displayed in the image.

[0135] Those skilled in the art can understand that in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any restriction on the implementation process, and the specific execution order of each step should be determined by its function and possible internal logic.

[0136] Based on the same inventive concept, the embodiment of the present disclosure further provides a motor intention determination device corresponding to the motor intention determination method. The principle of solving the problem of the device in the embodiment of the present disclosure is similar to the above-mentioned motor intention determination method in the embodiment of the present disclosure, so the implementation of the device can refer to the implementation of the method.

[0137] An embodiment of the present disclosure provides a motion intention determining device. FIG. 5 is a structural schematic diagram of a motion intention determining device according to an embodiment of the present disclosure. As shown in FIG. 5, the motion intention determining device 600 includes: an image capture unit 601 configured to capture traffic images; an information determining unit 602 configured to determine, based on said traffic image, vehicle light information of vehicles in said traffic image and orientation information of said vehicles; and an intention determining unit 603 configured to determine a motion intention of the vehicle based on the vehicle light information and the orientation information.

[0138] In some embodiments, the information determiner 602 is: a position information determination subunit configured to determine position information of illuminated target lights of the vehicle in the traffic image based on the traffic image; an appearance information determination subunit configured to determine appearance information of vehicles in said traffic images based on said traffic images; and a direction information determination subunit configured to determine direction information of the head of the vehicle based on the appearance information of the vehicle, The intention determination unit 603 further The vehicle is configured to determine a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and orientation information of the head of the vehicle.

[0139] In some embodiments, the target vehicle light is a single turn signal light, and the intention determiner 603 comprises: a turn information determination subunit configured to determine turn information indicated by a turn signal light based on position information of the single turn signal light and head direction information of the vehicle; and an intention determination subunit configured to determine a turning intention of the vehicle based on the turning information.

[0140] In some embodiments, the intent determiner 603 is: and a braking state determination subunit configured to determine that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being a plurality of turn signal lights.

[0141] In some embodiments, the apparatus further comprises: a vehicle type information determination unit configured to determine vehicle type information of vehicles in the traffic image based on the traffic image, The intention determination unit 603 further The vehicle is configured to determine a motion intention of the vehicle based on the vehicle light information, the orientation information and the vehicle type information.

[0142] In some embodiments, when determining the vehicle's movement intention, the device further comprises: a confidence determination unit configured to determine a confidence of the vehicle's movement intention; and a reliability adjustment unit configured to reduce a reliability of the movement intention in response to the orientation information indicating that the vehicle is facing sideways.

[0143] In some embodiments, the apparatus further comprises: A demand acquisition unit configured to acquire an application demand for predicting the motion intention of the vehicle; a reliability threshold matching unit configured to determine a reliability threshold matching the application demand; After determining the vehicle motion intention, the intention determination unit 603 further: A motion intention having a reliability greater than the reliability threshold is determined as a determined motion intention for the vehicle.

[0144] In some embodiments, the determination of the vehicle light information, the orientation information and the vehicle's motion intention is performed by a neural network, a first classifier in the neural network being obtained by training with sample images labeled with the vehicle light information and the orientation information, and a second classifier in the neural network being obtained by training with sample images labeled with the vehicle's motion intention.

[0145] In some embodiments, the second classifier comprises at least one of a base classifier for classifying a base movement intention of the vehicle and an augmented classifier for classifying an augmented movement intention of the vehicle, where the base classifier is trained based on sample images labeled with overall vehicle light states and the augmented classifier is trained based on sample images labeled with turn signal light states of the vehicle.

[0146] In some embodiments, the information determiner 602 is further configured to determine, based on the traffic image, vehicle light information of vehicles in the traffic image and orientation information of the vehicles using the neural network, wherein the information determiner 602 is further configured to: a mask determination subunit configured to determine an attention mask of the traffic image using a convolutional layer of the neural network; a spatial features determination subunit configured to determine spatial features of the traffic image based on the attention mask; a feature merging subunit configured to merge the spatial features and the temporal features of the traffic image to obtain image features of the traffic image; and an information determining subunit configured to determine vehicle light information and vehicle orientation information of the vehicle based on the image features using the first classifier.

[0147] In some embodiments, the intent determiner 603 is: an information input sub-unit configured to input the vehicle light information and the orientation information to the second classifier and output a predicted motion intention of the vehicle by the second classifier; a confidence level determination subunit configured to determine a first confidence level of the predicted movement intention and a second confidence level of the classification result in response to a mismatch between the predicted movement intention and the classification result output by the first classifier, respectively; and a reliability comparison subunit configured to determine a motion intention of the vehicle based on a prediction result corresponding to a relatively larger reliability out of a first reliability of the predicted motion intention and a second reliability of the classification result.

[0148] It should be noted that the above description of the apparatus embodiment is similar to the above description of the method embodiment, and has the same beneficial effects as the method embodiment. For technical details not disclosed in the apparatus embodiment of the present disclosure, please refer to the description of the method embodiment of the present disclosure to understand.

[0149] In this and other embodiments of the present disclosure, a "module" may be a circuit, a processor, a program, or software, and of course, may be a unit or may be non-modular.

[0150] It should be explained that in the embodiments of the present disclosure, the above-mentioned exercise intention determination method may be realized in the form of a software functional module and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solutions of the embodiments of the present disclosure may be essentially or in part embodied in the form of a software product, which is stored in a storage medium and includes some instructions for causing a computer device (which may be a terminal, a server, etc.) to execute all or part of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash disk, a mobile hard disk, a read only memory (ROM), a magnetic disk, or an optical disk. In this way, the embodiments of the present disclosure are not limited to any specific combination of hardware and software.

[0151] Correspondingly, an embodiment of the present disclosure further provides a computer program product including computer executable instructions, which can realize the steps of the motor intention determination method provided by the embodiment of the present disclosure after the computer executable instructions are executed. Correspondingly, an embodiment of the present disclosure further provides a computer storage medium, which stores computer executable instructions, and when the computer executable instructions are executed by a processor, realizes the steps of the motor intention determination method provided by the above embodiment. Correspondingly, an embodiment of the present disclosure provides a computer device, and FIG. 6 is a structural schematic diagram of a computer device according to an embodiment of the present disclosure. As shown in FIG. 6, the computer device 700 includes one processor 701, at least one communication bus, a communication interface 702, at least one external communication interface, and a memory 703. Here, the communication interface 702 is configured to realize the connection communication between these components. The communication interface 702 can include a display, and the external communication interface can include a standard wired interface and a wireless interface. The processor 701 is configured to execute an image processing program in the memory to realize the steps of the motor intention determination method provided in the above embodiment.

[0152] The above description of the embodiments of the motion intention determination device, computer device, and storage medium is similar to the description of the above method embodiments, and has the same technical description and beneficial effects as the corresponding method embodiments. Due to space limitations, the description of the above method embodiments can be referred to, and will not be described in detail here. Technical details not disclosed in the embodiments of the motion intention determination device, computer device, and storage medium of the present disclosure can be understood by referring to the description of the method embodiments of the present disclosure. It should be understood that "one embodiment" or "one embodiment" referred to throughout the specification means that a specific feature, structure, or characteristic associated with an embodiment is included in at least one embodiment of the present disclosure. Therefore, "in one embodiment" or "in one embodiment" appearing in various places throughout the specification does not necessarily mean the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that in various embodiments of the present disclosure, the magnitude of the numbers of each of the above processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure. The numbers of the embodiments of the present disclosure above are for illustrative purposes only and do not represent superiority or inferiority of the embodiments.

[0153] It should be understood that, as used herein, the terms "comprises" and "has" or any variation thereof are intended to cover a non-exclusive inclusion, whereby a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also other elements not expressly listed or inherent to such process, method, article, or apparatus. In the absence of further limitations, an element qualified by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

[0154] In some embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods may be realized in other ways. The above-described device embodiments are merely illustrative, for example, the division of the units is only a logical functional division, and other division methods may be used in actual implementation, for example, multiple units or components may be combined or integrated into other systems, or some features may be omitted or not implemented. In addition, the mutual coupling, or direct coupling, or communication connection between each component shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical, or other forms.

[0155] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed in multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of the embodiment. In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, each unit may be a single unit, or two or more units may be integrated into one unit, and the integrated units may be realized in the form of hardware or in the form of a functional unit that combines hardware and software. Those skilled in the art can understand that all or some of the steps of implementing the above method embodiments may be completed by instructing related hardware by a program, and the program may be stored in a computer-readable storage medium, and when the program is executed, the steps of the above method embodiments are executed. The storage medium includes various media that can store program code, such as a mobile storage device, a read only memory (ROM), a magnetic disk, or an optical disk.

[0156] Alternatively, in the present disclosure, the above integrated units may be realized in the form of a software functional module and stored in one computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solutions of the embodiments of the present disclosure may be essentially or in part embodied in the form of a software product, which is stored in a storage medium and includes some instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks. The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto, and any changes or replacements that any person skilled in the art can easily conceive within the technical scope disclosed in the present disclosure should all be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be subject to the scope of protection of the claims. [Industrial Applicability]

[0157] An embodiment of the present disclosure provides a motion intention determination method, an apparatus, a device, and a storage medium, which includes: acquiring a traffic image; determining vehicle light information and orientation information of a vehicle in the traffic image based on the traffic image; and determining a motion intention of the vehicle based on the vehicle light information and the orientation information.

Claims

1. A method for determining an exercise intention performed by an electronic device, comprising: Obtaining traffic images; determining vehicle light information and vehicle orientation information of vehicles in the traffic image based on the traffic image; determining a motion intention of the vehicle based on the vehicle light information and the direction information.

2. Determining vehicle light information and vehicle orientation information of a vehicle in the traffic image based on the traffic image includes: determining position information of illuminated target lights of the vehicle in the traffic image based on the traffic image; determining appearance information of vehicles in said traffic images based on said traffic images; determining head direction information of the vehicle based on the appearance information of the vehicle; Determining a motion intention of the vehicle based on the vehicle light information and the direction information determining a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and information on a direction of a head of the vehicle, The exercise intention determination method according to claim 1 .

3. The target vehicle light is a single turn signal light, and determining the motion intention of the vehicle based on position information of the target vehicle light of the vehicle and direction information of the head of the vehicle, determining turning information indicated by a turn signal light based on position information of the single turn signal light and head orientation information of the vehicle; and determining a turning intent of the vehicle based on the turning information. The exercise intention determination method according to claim 2 .

4. Determining a motion intention of the vehicle based on position information of a target vehicle light of the vehicle and direction information of a head of the vehicle, determining that the vehicle is in a braking state in response to the vehicle light information not including brake light information and the target vehicle lights being a plurality of turn signal lights. The exercise intention determination method according to claim 2 .

5. The exercise intention determination method further includes: determining vehicle type information of vehicles in the traffic image based on the traffic image; Determining a motion intention of the vehicle based on the vehicle light information and the direction information determining a motion intention of the vehicle based on the vehicle light information, the direction information, and the vehicle type information, The exercise intention determination method according to any one of claims 1 to 4.

6. When determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes: and reducing a reliability of the movement intention in response to the orientation information indicating that the vehicle is facing sideways. The exercise intention determination method according to any one of claims 1 to 4.

7. When determining the vehicle's motion intention, a reliability of the vehicle's motion intention is determined, and the motion intention determination method further includes: Obtaining an application demand for predicting the vehicle's motion intention; determining a reliability threshold matching the application demand; After determining the vehicle motion intention, the motion intention determination method further comprises: and determining a motion intention having a reliability greater than the reliability threshold as a determined motion intention of the vehicle. The exercise intention determination method according to any one of claims 1 to 4.

8. the determination of the vehicle light information, the orientation information and the vehicle movement intention is performed by a neural network; The first classifier in the neural network is obtained by training using sample images labeled with vehicle light information and direction information, and the second classifier in the neural network is obtained by training using sample images labeled with vehicle motion intentions. The exercise intention determination method according to claim 1 .

9. The second classifier includes at least one of a base classifier for classifying a basic motion intention of the vehicle and an extended classifier for classifying an extended motion intention of the vehicle, the base classifier being obtained by training based on sample images labeled with an overall vehicle light state of the vehicle, and the extended classifier being obtained by training based on sample images labeled with a vehicle light state of a turn signal light of the vehicle. The exercise intention determination method according to claim 8.

10. Determining vehicle light information and vehicle orientation information of a vehicle in the traffic image based on the traffic image using the neural network includes: determining an attention mask of the traffic image using a convolutional layer of the neural network; determining spatial features of the traffic image based on the attention mask; merging the spatial features and the temporal features of the traffic image to obtain image features of the traffic image; and determining vehicle light information and vehicle orientation information based on the image features using the first classifier. The exercise intention determination method according to claim 8.

11. Determining a motion intention of the vehicle based on the vehicle light information and the direction information inputting the vehicle light information and the orientation information into the second classifier, and outputting a predicted motion intention of the vehicle by the second classifier; determining a first confidence level of the predicted movement intention and a second confidence level of the classification result in response to a mismatch between the predicted movement intention and the classification result output by the first classifier, respectively; determining the motion intention of the vehicle based on a prediction result corresponding to a relatively larger reliability out of a first reliability of the predicted motion intention and a second reliability of the classification result. The exercise intention determination method according to claim 10.

12. A movement intention determination device, comprising: an image capture unit configured to capture traffic images; an information determining unit configured to determine, based on the traffic image, vehicle light information of vehicles in the traffic image and orientation information of the vehicles; an intention determination unit configured to determine a motion intention of the vehicle based on the vehicle light information and the orientation information.

13. A computer storage medium having stored thereon computer-executable instructions which, when executed by a computer, can implement the steps of the movement intention determination method according to any one of claims 1 to 11.

14. A computing device comprising: a memory for storing computer-executable instructions; and a processor for executing the computer-executable instructions in the memory to perform steps of the movement intention determination method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program or instructions which, when executed on an electronic device, causes said electronic device to carry out the method for determining movement intention according to any one of claims 1 to 11.

Images