Vehicle taillight state determination method, device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a vehicle taillight state determination method and device, equipment, a medium and a program product. Comprising the following steps: acquiring a current to-be-identified video frame of a target vehicle, inputting the current to-be-identified video frame into a feature extraction sub-model of a pre-trained taillight state determination model, and determining a video frame feature corresponding to the current to-be-identified video frame; updating a feature recognition sequence corresponding to the previous video frame according to the video frame features, and determining a current feature recognition sequence corresponding to the current to-be-recognized video frame; when the number of features in the current feature recognition sequence is equal to a preset recognition number, inputting the current feature recognition sequence into a state recognition sub-model of a pre-trained taillight state determination model, and determining a current candidate taillight state corresponding to the current feature recognition sequence; and according to the current candidate tail lamp state, the historical candidate tail lamp state and a preset tail lamp state selection rule, the target vehicle tail lamp state is determined. And the accuracy of determining the target vehicle taillight state is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, equipment, medium and program product for determining the state of a vehicle taillight. Background Art

[0002] The taillight status is the main basis for drivers to judge the driving intention of the vehicle in front. It is roughly divided into left turn, right turn and double flash, which has a certain reminder effect on the driver. However, during driving, due to lack of concentration, or affected by factors such as sight distance and light intensity, the driver may not be able to effectively receive the taillight warning information of the vehicle in front, resulting in untimely prediction of the vehicle's driving status, thus causing traffic safety problems.

[0003] At present, conventional methods for determining the status of vehicle taillights can be roughly divided into two categories, based on static images and based on continuous videos. Among them, the method for determining the status of vehicle taillights based on static images often requires obtaining vehicle body images at regular intervals to identify the on-state of the taillights, and then comprehensively identifying the status of the taillights based on multiple images. Its recognition logic is determined manually and has certain limitations, and the recognition effect will be affected by lighting and the shape of the vehicle taillights. The method for determining the status of vehicle taillights based on continuous videos requires saving a certain number of video frames first, and using a neural network to perform a fusion recognition of each feature in the video frame to obtain the corresponding vehicle taillight status.

[0004] Since the status of the vehicle's taillights is represented by the regular lighting and extinguishing of the corresponding lights, a certain length of video clip is required for recognition, and one video clip can only output one recognition result. In order to ensure the accuracy of the vehicle's taillight status judgment, it is often necessary to make a judgment based on multiple output results. At this time, the amount of data that needs to be input into the neural network model is huge, and the real-time performance is poor, which makes it difficult to meet the recognition needs of the vehicle's taillight status during driving. Summary of the invention

[0005] The present invention provides a method, device, equipment, medium and program product for determining the state of a vehicle taillight, which improves the recognition continuity of the state of a vehicle taillight based on a video, and improves the speed and accuracy of determining the state of the vehicle taillight.

[0006] In a first aspect, an embodiment of the present invention provides a method for determining a vehicle taillight state, comprising:

[0007] Obtain a current video frame to be identified of the target vehicle, and input the current video frame to be identified into a feature extraction submodel of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified;

[0008] The feature recognition sequence corresponding to the previous video frame is updated according to the video frame feature, and the updated feature recognition sequence is used as the current feature recognition sequence corresponding to the current video frame to be recognized;

[0009] When the number of features in the current feature recognition sequence is equal to the preset recognition number, the current feature recognition sequence is input into the state recognition sub-model of the pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence;

[0010] The taillight state of the target vehicle is determined according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

[0011] In a second aspect, an embodiment of the present invention further provides a vehicle taillight state determination device, comprising:

[0012] A feature extraction module is used to obtain a current video frame to be identified of the target vehicle, and input the current video frame to be identified into a feature extraction sub-model of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified;

[0013] A feature sequence updating module is used to update the feature recognition sequence corresponding to the previous video frame according to the video frame features, and use the updated feature recognition sequence as the current feature recognition sequence corresponding to the current video frame to be recognized;

[0014] A candidate state determination module, for inputting the current feature recognition sequence into a state recognition sub-model of a pre-trained taillight state determination model to determine a current candidate taillight state corresponding to the current feature recognition sequence when the number of features in the current feature recognition sequence is equal to a preset recognition number;

[0015] The target state determination module is used to determine the target vehicle taillight state based on the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

[0016] In a third aspect, an embodiment of the present invention further provides a vehicle taillight state determination device, comprising:

[0017] at least one processor; and a memory communicatively coupled to the at least one processor;

[0018] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can implement the vehicle taillight state determination method of any embodiment of the present invention.

[0019] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the vehicle taillight state determination method of any embodiment of the present invention.

[0020] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, is used to execute the vehicle taillight state determination method of any embodiment of the present invention.

[0021] The embodiments of the present invention provide a method, device, equipment, medium and program product for determining the state of vehicle taillights. The method obtains a current video frame to be identified of a target vehicle, and inputs the current video frame to be identified into a feature extraction submodel of a pre-trained taillight state determination model to determine the video frame features corresponding to the current video frame to be identified; updates a feature recognition sequence corresponding to a previous video frame according to the video frame features, and uses the updated feature recognition sequence as the current feature recognition sequence corresponding to the current video frame to be identified; when the number of features in the current feature recognition sequence is equal to a preset recognition number, inputs the current feature recognition sequence into a state recognition submodel of a pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence; and determines the state of the taillights of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule. By adopting the above technical solution, the taillight state determination model is split into a feature extraction sub-model and a state recognition sub-model. According to the video frame features output by the current video frame to be identified each time, the feature recognition sequence corresponding to the previous video frame can be updated to obtain multiple feature recognition sequences. Since the difference between each feature recognition sequence is only the difference in the features added to the feature recognition sequence at the end, most of the features can be reused, so all video frames except the first video frame obtained for the target vehicle can be fully utilized. In the case of limited video frames obtained, the state recognition sub-model is used to perform more taillight state recognition, and then the final target vehicle taillight state is determined based on the extracted multiple candidate taillight states. The multi-feature recognition sequence determination in the taillight state determination model is realized, and the features in the multi-feature recognition sequence are reused, so that the recognition speed is greatly improved. And because the feature recognition sequence is continuously generated with the input of the video frame to be identified, the state recognition sub-model continuously outputs the candidate taillight state, which further improves the accuracy of identifying the target vehicle taillight state based on the candidate taillight state.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flow chart of a method for determining a vehicle taillight state provided in Embodiment 1 of the present invention;

[0025] Figure 2 A flowchart of a method for determining a vehicle taillight state provided in Embodiment 2 of the present invention;

[0026] Figure 3 A schematic diagram of the structure of a vehicle taillight status determination device provided in Embodiment 3 of the present invention;

[0027] Figure 4 A schematic diagram of the structure of a vehicle taillight status determination device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first" and "second" and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Embodiment 1

[0031] Figure 1The flowchart of a vehicle taillight state determination method provided in the first embodiment of the present invention is applicable to the situation of accurately identifying the taillight state of the preceding vehicle during driving. The method can be performed by a vehicle taillight state determination device, which can be implemented by software and / or hardware, and the vehicle taillight state determination device can be configured in a vehicle taillight state determination device. Optionally, the vehicle taillight state determination device can be an electronic device, which can be a notebook, a desktop computer, a smart tablet, a vehicle-mounted computer, etc., and the embodiment of the present invention does not limit this.

[0032] like Figure 1 As shown, a method for determining the state of a vehicle taillight provided by an embodiment of the present invention specifically includes the following steps:

[0033] S101, obtaining a current video frame to be identified of a target vehicle, and inputting the current video frame to be identified into a feature extraction sub-model of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified.

[0034] In this embodiment, the target vehicle can be specifically understood as a vehicle for which the vehicle taillight status needs to be determined. The current video frame to be identified can be specifically understood as a currently acquired video frame containing a target vehicle for which the vehicle taillight status needs to be determined. The taillight status determination model can be specifically understood as a neural network model for extracting and fusing features of multiple video frames input therein to comprehensively determine the vehicle taillight status in each video frame. The feature extraction sub-model can be specifically understood as a combination of neural network layers in the taillight status determination model for extracting features from video frames input therein.

[0035] Specifically, when it is necessary to identify the vehicle taillight status, the vehicle can be determined as the target vehicle, and a video containing the target vehicle is collected, and the video frame collected at the current moment is determined as the current video frame to be identified of the target vehicle. Then, the current video frame to be identified is input into the feature extraction sub-model of the pre-trained taillight status determination model, and the feature extraction sub-model extracts features from the input video frame to be identified and outputs the video frame features corresponding to the current video frame to be identified.

[0036] S102: updating a feature recognition sequence corresponding to a previous video frame according to the video frame feature, and using the updated feature recognition sequence as a current feature recognition sequence corresponding to a current video frame to be recognized.

[0037] In this embodiment, the previous video frame can be specifically understood as a video frame captured before the current video frame to be identified, containing the target vehicle and captured most recently with the current video frame to be identified. The feature recognition sequence can be specifically understood as a feature sequence composed of multiple features extracted by the feature extraction sub-model and used for fusion to determine the vehicle taillight state.

[0038] Specifically, in order to ensure the continuity of vehicle taillight status recognition for the target vehicle, after determining the video frame features corresponding to the current video frame to be recognized, the feature recognition sequence corresponding to the previous video frame before the current video frame to be recognized can be obtained, and the video frame features corresponding to the current video frame to be recognized can be updated to the feature recognition sequence corresponding to the previous video frame, so that the current feature recognition sequence corresponding to the current video frame to be recognized obtained after the update includes the features corresponding to the current video frame to be recognized.

[0039] It can be understood that when the current video frame to be identified is the first frame, it can be considered that there is no previous video frame before the current video frame to be identified. At this time, in order to ensure the normal update of the feature recognition sequence, the feature recognition sequence corresponding to the previous video frame can be determined as an empty sequence, that is, a sequence that does not contain video frame features.

[0040] S103. When the number of features in the current feature recognition sequence is equal to the preset recognition number, the current feature recognition sequence is input into the state recognition sub-model of the pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence.

[0041] In this embodiment, the preset recognition number can be specifically understood as the number of features that can be used to identify the vehicle taillight state once, which is pre-set according to actual conditions. It is understandable that the preset recognition number should be consistent when the taillight state determination model is trained and put into use. The state recognition sub-model can be specifically understood as a combination of neural network layers in the taillight state determination model for multi-feature fusion, recognition and classification of input features. The current candidate taillight state can be specifically understood as the state that the vehicle taillights may be in in the current feature recognition sequence corresponding to the current video frame to be identified.

[0042] Specifically, when the number of features in the current feature recognition sequence is equal to the preset recognition number, it can be considered that the current feature recognition sequence meets the input requirements of the state recognition sub-model of the pre-trained taillight state determination model, and the vehicle taillight state recognition can be performed once based on the current feature recognition sequence. At this time, the current feature recognition sequence is input into the state recognition sub-model of the pre-trained taillight state determination model, and each feature in the feature recognition sequence is fused and recognized by the state recognition sub-model, and the result output by the state recognition sub-model is determined as the current candidate taillight state corresponding to the current feature recognition sequence.

[0043] S104: Determine the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

[0044] In this embodiment, the historical candidate taillight state can be specifically understood as the vehicle taillight state of the target vehicle determined from the video frame containing the target vehicle before the current video frame to be identified is acquired. The target vehicle taillight state can be specifically understood as the vehicle taillight state with the highest actual probability in the set of video frames to be identified that is desired to be determined. The preset taillight state selection rule can be specifically understood as a rule pre-set according to actual conditions to select the actual most likely state of the vehicle taillight from multiple candidate vehicle taillight states.

[0045] Specifically, while clarifying the current candidate taillight state, the candidate vehicle taillight state corresponding to the target vehicle and determined by the video frame collected before the current video frame to be identified can also be obtained. At this time, the vehicle taillight state of the target vehicle determined by the video frame containing the target vehicle before the current video frame to be identified is determined as the historical candidate taillight state. Since the current candidate taillight state and each historical candidate taillight state may be the vehicle taillight state actually corresponding to the target vehicle, and the types corresponding to the current candidate taillight state and each historical candidate taillight state are not the same, the current candidate taillight state and each historical candidate taillight state can be screened and processed according to the preset taillight state selection rules, and the candidate vehicle taillight state with the highest actual possibility is determined as the target vehicle taillight state.

[0046] The technical solution of this embodiment is to obtain the current video frame to be identified of the target vehicle, and input the current video frame to be identified into the feature extraction sub-model of the pre-trained taillight state determination model to determine the video frame features corresponding to the current video frame to be identified; update the feature recognition sequence corresponding to the previous video frame according to the video frame features, and use the updated feature recognition sequence as the current feature recognition sequence corresponding to the current video frame to be identified; when the number of features in the current feature recognition sequence is equal to the preset recognition number, input the current feature recognition sequence into the state recognition sub-model of the pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence; determine the target vehicle taillight state according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule. By adopting the above technical solution, the taillight state determination model is split into a feature extraction sub-model and a state recognition sub-model. According to the video frame features output by the current video frame to be identified each time, the feature recognition sequence corresponding to the previous video frame can be updated to obtain multiple feature recognition sequences. Since the difference between each feature recognition sequence is only the difference in the features added to the feature recognition sequence at the end, most of the features can be reused, so all video frames except the first video frame obtained for the target vehicle can be fully utilized. In the case of limited video frames obtained, the state recognition sub-model is used to perform more taillight state recognition, and then the final target vehicle taillight state is determined based on the extracted multiple candidate taillight states. The multi-feature recognition sequence determination in the taillight state determination model is realized, and the features in the multi-feature recognition sequence are reused, so that the recognition speed is greatly improved. And because the feature recognition sequence is continuously generated with the input of the video frame to be identified, the state recognition sub-model continuously outputs the candidate taillight state, which further improves the accuracy of identifying the target vehicle taillight state based on the candidate taillight state.

[0047] Embodiment 2

[0048] Figure 2The flowchart of a vehicle taillight state determination method provided in the second embodiment of the present invention is further optimized on the basis of the above-mentioned optional technical solutions. After the video frame features of the current video frame to be identified are extracted, the video frame features of the current video frame to be identified are determined according to the number of features in the feature recognition sequence corresponding to the previous video frame of the current video frame to be identified. That is, in the case where the feature recognition sequence corresponding to the previous video frame has not been identified for the taillight state, the video frame features corresponding to the current video frame to be identified are added to the end of the feature recognition sequence corresponding to the previous video frame by using the feature addition method to form the current feature recognition sequence corresponding to the current video frame to be identified. In the case where the feature recognition sequence corresponding to the previous video frame has been identified for the taillight state, the feature recognition sequence corresponding to the previous video frame is updated in a "last in, first out" manner, that is, the video frame features at the first position of the feature recognition sequence corresponding to the previous video frame are deleted, and the video frame features corresponding to the current video frame to be identified are added to the end of the feature recognition sequence corresponding to the previous video frame to form the current feature recognition sequence corresponding to the current video frame to be identified. That is, there is only one video frame feature difference between the newly generated current feature recognition sequence and the feature recognition sequence corresponding to the previous video frame, which realizes the full reuse of video frame features between different feature recognition sequences, greatly improving the recognition speed. At the same time, a variety of methods are given to determine the target vehicle taillight state based on the current candidate taillight state, historical candidate taillight state and preset taillight state selection rules. The target vehicle taillight state is determined through the evaluation criteria such as the proportion of the number of candidate taillight states of the same type and the number of consecutive appearances, which improves the recognition accuracy and speed of the vehicle taillight state.

[0049] like Figure 2 As shown, a method for determining the state of a vehicle taillight provided by an embodiment of the present invention specifically includes the following steps:

[0050] S201. Obtain a current video frame to be identified of a target vehicle, and input the current video frame to be identified into a feature extraction sub-model of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified.

[0051] Optionally, before obtaining the current video frame to be identified of the target vehicle, it is necessary to first complete the training of a taillight state determination model for determining the vehicle taillight state. The training of the vehicle taillight state determination model can be achieved through the following steps:

[0052] 1) Obtain a set of historical video frame sequences and the historical vehicle taillight states corresponding to each historical video frame sequence.

[0053] The historical video frame sequence is a sequence extracted from a historical video and composed of a preset number of continuous historical video frames.

[0054] In this embodiment, the historical video can be specifically understood as a video containing the state of the vehicle taillights of a vehicle within a period of historical time. For example, a historical video can be a video consisting of hundreds of video frames collected from a vehicle within one minute. The historical video frame sequence can be specifically understood as a sequence consisting of multiple historical video frames obtained by continuously extracting historical video frames in a historical video, with a total number of frames being a preset recognition number. The historical vehicle taillight state can be specifically understood as the actual state of the vehicle taillights in the historical video corresponding to the historical video frame sequence.

[0055] It is understandable that the starting position of a historical video frame sequence when it is extracted in its corresponding historical video may be different. For example, for a historical video with a length of 100 frames, assuming the preset recognition number is 50, frames 0-49 of it can be extracted as a historical video frame sequence, frames 1-50 of it can be extracted as a historical video frame sequence, or frames 20-69 of it can be extracted as a historical video frame sequence. The embodiment of the present invention is not limited to this.

[0056] In an embodiment of the present invention, since historical video frame sequences with different starting positions can be extracted from the same historical video to construct training samples, historical video frame sequences with the same or different starting positions can also be extracted from different historical videos to construct training samples. For example, for a historical video in which the taillights of a vehicle are in a double flashing state, a historical video frame sequence with the first frame being off can be extracted, and a historical video frame sequence with the first frame being on can also be extracted. Since historical video frame sequences with different starting positions can be extracted to construct training samples, the taillight states of various types of vehicles can be included in the training samples, thereby improving the accuracy of the taillight state determination model obtained through training.

[0057] 2) Taking each historical vehicle taillight state as a label of the corresponding historical video frame sequence, determine a training sample set corresponding to the historical video frame sequence set.

[0058] Specifically, the historical vehicle taillight states and the historical video frame sequences are grouped according to their corresponding relationships. For each group of historical vehicle taillight states and historical video frame sequences, the historical vehicle taillight state is used as the label of the historical video frame sequence and determined as a training sample. Then, all historical video frame sequences in the historical video frame sequence set are labeled to obtain the corresponding training sample set.

[0059] 3) The initial taillight state determination model is trained using the training sample set until a preset convergence condition is met to obtain the taillight state determination model.

[0060] In this embodiment, the initial taillight state determination model is a taillight state determination model in which the weights of each internal neural network layer are not adjusted when the model is not trained, and its structure is completely consistent with the taillight state determination model. The preset convergence condition can be specifically understood as a condition predetermined according to actual conditions to determine whether the neural network model meets the standard for being put into use. Optionally, the preset convergence condition can be that all training samples have completed training or the loss error is less than a preset error threshold, etc., and the embodiment of the present invention does not limit this.

[0061] Specifically, each training sample in the training sample set is sequentially input into the initial taillight state determination model, and the initial taillight state determination model is trained as a whole until a preset convergence condition is met, thereby obtaining a trained taillight state determination model.

[0062] 4) The taillight state determination model is divided into a feature extraction sub-model and a state recognition sub-model in which the number of single input features is a preset recognition number.

[0063] Specifically, in order to ensure that the number of features used in feature fusion recognition is consistent with the number of features that can be extracted from the training samples input during the training process, the taillight state determination model can be divided into a feature extraction sub-model for video frame feature extraction and a state recognition sub-model in which the number of single input features is a preset recognition number according to the function of each neural network layer.

[0064] Exemplarily, assuming that the feature length of the feature extraction sub-model for a single image, that is, a single video frame, is N, when the number of input features of the state recognition sub-model is clearly the preset recognition number M, the input feature length of the state recognition sub-model should be M*N.

[0065] S202, determining whether the number of features in the feature recognition sequence corresponding to the previous video frame of the current video frame to be recognized is less than the preset recognition number, if so, executing S203; if not, executing S204.

[0066] Specifically, the feature recognition sequence corresponding to the previous video frame of the current video frame to be recognized is obtained, and the size relationship between the number of features contained in the feature recognition sequence and the preset recognition number is judged. When the number of features is less than the preset recognition number, it can be considered that the feature recognition sequence corresponding to the previous video frame does not meet the requirements of the input state recognition sub-model to determine the vehicle taillight state. At this time, the feature recognition sequence needs to be supplemented, and S203 is executed at this time; when the number of features is equal to the preset recognition number, it can be considered that the feature recognition sequence corresponding to the previous video frame has recognized the vehicle taillight state. At this time, the number of features in the feature recognition sequence does not need to be increased, and only a certain video frame feature in the feature recognition sequence needs to be replaced and updated by the video frame feature corresponding to the current video frame to be recognized, and S204 is executed at this time.

[0067] S203, adding the video frame feature to the end of the feature recognition sequence corresponding to the previous video frame, determining the current feature recognition sequence, and executing S205.

[0068] Specifically, adding the video frame features to the end of the feature recognition sequence corresponding to the previous video frame is equivalent to adding the video frame features to the position corresponding to the latest acquisition moment in the feature recognition sequence corresponding to the previous video frame according to the acquisition order. The video frame features in the resulting current feature recognition sequence are still sorted in order of acquisition time, which meets the data processing requirements of the state recognition sub-model, and then S205 can be executed.

[0069] S204, adding the video frame feature to the end of the feature recognition sequence corresponding to the previous video frame, and deleting the video frame feature at the first position of the feature recognition sequence corresponding to the previous video frame, determining the current feature recognition sequence, and executing S205.

[0070] Specifically, adding the video frame feature to the end of the feature recognition sequence corresponding to the previous video frame is equivalent to adding the video frame feature to the position corresponding to the latest acquisition time in the feature recognition sequence corresponding to the previous video frame according to the acquisition order. At the same time, since the number of features in the feature recognition sequence corresponding to the previous video frame in this state has reached the preset recognition number, in order to ensure that the feature recognition sequence after adding the video frame feature can still be input into the state recognition sub-model for processing, the video frame feature at the first position in the feature recognition sequence corresponding to the previous video frame and farthest from the current moment can be deleted to obtain the current feature recognition sequence that meets the data processing requirements of the state recognition sub-model, and then S205 can be executed.

[0071] It can be understood that if the number of features in the current feature recognition sequence obtained after the video frame features corresponding to the current video frame to be recognized are added to the feature recognition sequence corresponding to the previous video frame reaches the preset recognition number for the first time, then the current feature recognition sequence can be considered to be the initial feature recognition sequence in the process of determining the vehicle taillight status, and subsequent update methods all adopt the update method in S204.

[0072] For example, assuming that only 60 video frames to be identified can be obtained for the target vehicle, and the preset identification number is 50, then after the 50th video frame to be identified is input into the feature extraction sub-model as the current video frame to be identified, the first feature recognition sequence that can be input into the state recognition sub-model will be generated, and after the 51st video frame to be identified is input into the feature extraction sub-model, the features corresponding to the output 51st video frame to be identified will be added to the end of the previous feature recognition sequence, and the features at the initial position of the previous feature recognition sequence will be removed to obtain a new feature recognition sequence containing the features of the 51st video frame to be identified. That is, when each video frame to be identified in the set of video frames to be identified is input into the taillight state determination model, 11 feature recognition sequences will be generated.

[0073] It can be understood that in the embodiment of the present invention, the newly generated feature recognition sequence differs from the previously generated feature recognition sequence by only one video frame feature, that is, the video frame feature with the earliest acquisition time in the previous feature recognition sequence is deleted, and the newly generated feature recognition sequence is obtained by adding the latest output video frame feature, thereby achieving full reuse of video frame features between different feature recognition sequences, thereby greatly improving the recognition speed.

[0074] S205. When the number of features in the current feature recognition sequence is equal to the preset recognition number, the current feature recognition sequence is input into the state recognition sub-model of the pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence.

[0075] S206: Determine the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

[0076] Optionally, determining the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule can be implemented in different ways. In the embodiments of the present invention, only the following implementation ways are taken as examples:

[0077] 1) The candidate taillight state with the highest proportion among the current candidate taillight state and the historical candidate taillight state is determined as the target vehicle taillight state.

[0078] Specifically, after all the video frames to be identified corresponding to the target vehicle are processed, or after multiple video frames to be identified corresponding to the target vehicle are processed, multiple candidate taillight states including the current candidate taillight state and multiple historical post-selected taillight states can be obtained. In order to improve the accuracy of determining the vehicle taillight state, the majority priority principle can be adopted to determine the candidate taillight state with the highest number among the candidate taillight states as the target vehicle taillight state.

[0079] 2) When the number of candidate vehicle taillight states of the same type in the current candidate taillight state and the historical candidate taillight states reaches a preset number, stop acquiring the current video frame to be identified of the target vehicle and determine the candidate vehicle taillight state of the same type as the target vehicle taillight state.

[0080] Specifically, since the current candidate taillight state and the historical candidate taillight state are continuously output by the state recognition sub-model, and the current candidate taillight state and the historical candidate taillight state are of different types, namely, left turn, right turn or double flash type, the candidate vehicle taillight states belonging to the left turn, right turn and double flash type can be accumulated during the output process of the state recognition sub-model. If the accumulated number of candidate vehicle taillight states of a certain type reaches the preset number of states, it can be considered that the candidate vehicle taillight state of this type is dominant among the multiple types of vehicle taillight states corresponding to the target vehicle. At this time, the candidate vehicle taillight state of this type is determined as the target vehicle taillight state. At the same time, when the target vehicle taillight state is determined, it is possible that multiple video frames to be identified of the target vehicle have not been completely acquired and input into the taillight state determination model. At this time, the acquisition of the current video frame to be identified of the target vehicle can be stopped, so as to reduce the required number of identifications, improve the identification speed, and reduce the amount of data processing while ensuring the accuracy of the vehicle taillight state determination.

[0081] 3) When the candidate vehicle taillight states of the same type in the current candidate taillight state and the historical candidate taillight states appear consecutively for a preset number of times, stop acquiring the current video frame to be identified of the target vehicle and determine the candidate vehicle taillight states of the same type as the target vehicle taillight states.

[0082] Specifically, since the current candidate taillight state and the historical candidate taillight state are continuously output by the state recognition sub-model, and the current candidate taillight state and the historical candidate taillight state are of different types, namely, left turn, right turn or double flash type, in the output process of the state recognition sub-model, if the candidate vehicle taillight states output within a continuous period of time are all of the same type, it can be considered that the type is more likely to actually belong to the vehicle taillight state corresponding to the target vehicle, and at this time, the candidate vehicle taillight state of this type is determined as the target vehicle taillight state. At the same time, it is possible that multiple video frames to be identified of the target vehicle have not been completely acquired and input into the taillight state determination model. At this time, the acquisition of the current video frame to be identified of the target vehicle can be stopped, so as to reduce the required number of identifications, improve the identification speed, and reduce the amount of data processing while ensuring the accuracy of the vehicle taillight state determination.

[0083] The technical solution of this embodiment is to determine the updating method of the video frame features of the current video frame to be identified for the feature recognition sequence corresponding to the previous video frame according to the number of features in the feature recognition sequence corresponding to the previous video frame of the current video frame to be identified after the video frame features of the current video frame to be identified are extracted. That is, when the feature recognition sequence corresponding to the previous video frame has not been identified for the taillight state, the video frame features corresponding to the current video frame to be identified are added to the end of the feature recognition sequence corresponding to the previous video frame by means of feature addition, so as to form the current feature recognition sequence corresponding to the current video frame to be identified. When the feature recognition sequence corresponding to the previous video frame has been identified for the taillight state, the feature recognition sequence corresponding to the previous video frame is updated by means of "last in first out", that is, the video frame features at the first position of the feature recognition sequence corresponding to the previous video frame are deleted, and the video frame features corresponding to the current video frame to be identified are added to the end of the feature recognition sequence corresponding to the previous video frame, so as to form the current feature recognition sequence corresponding to the current video frame to be identified. That is, there is only one video frame feature difference between the newly generated current feature recognition sequence and the feature recognition sequence corresponding to the previous video frame, which realizes the full reuse of video frame features between different feature recognition sequences, greatly improving the recognition speed. At the same time, a variety of methods are given to determine the target vehicle taillight state based on the current candidate taillight state, historical candidate taillight state and preset taillight state selection rules. The target vehicle taillight state is determined through the evaluation criteria such as the proportion of the number of candidate taillight states of the same type and the number of consecutive appearances, which improves the recognition accuracy and speed of the vehicle taillight state.

[0084] Embodiment 3

[0085] Figure 3 A schematic diagram of the structure of a vehicle taillight status determination device provided in Embodiment 3 of the present invention is shown in FIG. Figure 3As shown, the vehicle taillight state determination device includes a feature extraction module 31, a feature sequence update module 32, a candidate state determination module 33 and a target state determination module 34.

[0086] Among them, the feature extraction module 31 is used to obtain the current video frame to be identified of the target vehicle, and input the current video frame to be identified into the feature extraction sub-model of the pre-trained taillight state determination model to determine the video frame features corresponding to the current video frame to be identified; the feature sequence updating module 32 is used to update the feature recognition sequence corresponding to the previous video frame according to the video frame features, and use the updated feature recognition sequence as the current feature recognition sequence corresponding to the current video frame to be identified; the candidate state determination module 33 is used to input the current feature recognition sequence into the state recognition sub-model of the pre-trained taillight state determination model when the number of features in the current feature recognition sequence is equal to the preset recognition number, and determine the current candidate taillight state corresponding to the current feature recognition sequence; the target state determination module 34 is used to determine the target vehicle taillight state according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

[0087] The technical solution of the embodiment of the present invention is to split the taillight state determination model into a feature extraction sub-model and a state recognition sub-model. According to the video frame features output by the current video frame to be recognized each time, the feature recognition sequence corresponding to the previous video frame can be updated to obtain multiple feature recognition sequences. Since the difference between the feature recognition sequences is only the difference in the features added to the feature recognition sequence at the end, most of the features can be reused, so all video frames except the first video frame acquired for the target vehicle can be fully utilized. In the case of limited acquired video frames, the state recognition sub-model is used to perform more taillight state recognition, and then the final target vehicle taillight state is determined based on the extracted multiple candidate taillight states. The multi-feature recognition sequence determination in the taillight state determination model is realized, and the features in the multi-feature recognition sequence are reused, so that the recognition speed is greatly improved. And because the feature recognition sequence is continuously generated with the input of the video frame to be recognized, the state recognition sub-model continuously outputs the candidate taillight state, and further improves the accuracy of recognizing the target vehicle taillight state based on the candidate taillight state.

[0088] Optionally, the feature sequence updating module 32 is specifically used for:

[0089] When the number of features in the feature recognition sequence corresponding to the previous video frame is less than the preset recognition number, the video frame feature is added to the end of the feature recognition sequence corresponding to the previous video frame to determine the current feature recognition sequence.

[0090] When the number of features in the feature recognition sequence corresponding to the previous video frame is equal to the preset recognition number, the video frame feature is added to the end of the feature recognition sequence corresponding to the previous video frame, and the video frame feature at the first position of the feature recognition sequence corresponding to the previous video frame is deleted to determine the current feature recognition sequence.

[0091] Optionally, the target state determination module 34 is specifically configured to:

[0092] The candidate taillight state with the highest proportion in the current candidate taillight state and the historical candidate taillight states is determined as the target vehicle taillight state.

[0093] Optionally, the target state determination module 34 is specifically configured to:

[0094] When the number of candidate vehicle taillight states of the same type in the current candidate taillight state and the historical candidate taillight states reaches a preset number, stop acquiring the current video frame to be identified of the target vehicle and determine the candidate vehicle taillight state of the same type as the target vehicle taillight state.

[0095] Optionally, the target state determination module 34 is specifically configured to:

[0096] When the candidate vehicle taillight state of the same type in the current candidate taillight state and the historical candidate taillight state appears continuously for a preset number of times, stop acquiring the current video frame to be identified of the target vehicle and determine the candidate vehicle taillight state of the same type as the target vehicle taillight state.

[0097] Optionally, the vehicle taillight state determination device further includes:

[0098] The model building module is specifically used to obtain a set of historical video frame sequences and historical vehicle taillight states corresponding to each historical video frame sequence before obtaining the current video frame to be identified of the target vehicle; wherein the historical video frame sequence is a sequence consisting of a preset number of continuous historical video frames extracted from a historical video. The taillight state of each historical vehicle is used as a label of the corresponding historical video frame sequence, and a training sample set corresponding to the set of historical video frame sequences is determined. The initial taillight state determination model is trained with the training sample set until the preset convergence condition is met to obtain the taillight state determination model. The taillight state determination model is divided into a feature extraction sub-model and a state recognition sub-model in which the number of single input features is the preset recognition number.

[0099] The vehicle taillight state determination device provided in the embodiment of the present invention can execute the vehicle taillight state determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0100] Embodiment 4

[0101] Figure 4 A schematic diagram of the structure of a vehicle taillight state determination device provided for Embodiment 4 of the present invention. The vehicle taillight state determination device 40 may be intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The vehicle taillight state determination device 40 may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0102] like Figure 4 As shown, the vehicle taillight state determination device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 to the random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the vehicle taillight state determination device 40 can also be stored. The processor 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0103] Multiple components in the vehicle taillight state determination device 40 are connected to the I / O interface 45, including: an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the vehicle taillight state determination device 40 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0104] The processor 41 may be a variety of general and / or dedicated processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 executes the various methods and processes described above, such as a vehicle taillight state determination method.

[0105] In some embodiments, the vehicle taillight state determination method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on the vehicle taillight state determination device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the power distribution scheme determination method described above may be performed. Alternatively, in other embodiments, the processor 41 may be configured to execute the vehicle taillight state determination method in any other appropriate manner (e.g., by means of firmware).

[0106] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the vehicle taillight state determination method provided in any embodiment of the present invention.

[0107] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on a power distribution scheme determining device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which a user can provide input to the power distribution scheme determining device. Other types of devices may also be used to provide interaction with a user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for determining the state of a vehicle taillight, characterized in that: include: Obtaining a current video frame to be identified of the target vehicle, and inputting the current video frame to be identified into a feature extraction submodel of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified; updating a feature recognition sequence corresponding to a previous video frame according to the video frame feature, and using the updated feature recognition sequence as a current feature recognition sequence corresponding to the current video frame to be recognized; When the number of features in the current feature recognition sequence is equal to the preset recognition number, inputting the current feature recognition sequence into the state recognition sub-model of the pre-trained taillight state determination model to determine the current candidate taillight state corresponding to the current feature recognition sequence; The taillight state of the target vehicle is determined according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

2. The method for determining the vehicle taillight status according to claim 1, characterized in that: The updating of the feature recognition sequence corresponding to the previous video frame according to the video frame feature, and using the updated feature recognition sequence as the current feature recognition sequence corresponding to the current video frame to be recognized, comprises: When the number of features in the feature recognition sequence corresponding to the previous video frame is less than the preset recognition number, adding the video frame feature to the end of the feature recognition sequence corresponding to the previous video frame to determine the current feature recognition sequence; When the number of features in the feature recognition sequence corresponding to the previous video frame is equal to the preset recognition number, the video frame feature is added to the end of the feature recognition sequence corresponding to the previous video frame, and the video frame feature at the first position of the feature recognition sequence corresponding to the previous video frame is deleted to determine the current feature recognition sequence.

3. The method for determining the vehicle taillight status according to claim 1 or 2, characterized in that: Before obtaining the current video frame to be identified of the target vehicle, the method further includes: Acquire a set of historical video frame sequences and historical vehicle taillight states corresponding to each historical video frame sequence; wherein the historical video frame sequence is a sequence consisting of a preset number of continuous historical video frames extracted from a historical video; Using each of the historical vehicle taillight states as a label of a corresponding historical video frame sequence, and determining a training sample set corresponding to the historical video frame sequence set; The initial taillight state determination model is trained by using the training sample set until a preset convergence condition is met to obtain the taillight state determination model; The taillight state determination model is divided into a feature extraction sub-model and a state recognition sub-model in which the number of single input features is the preset recognition number.

4. The method for determining the vehicle taillight status according to claim 3, characterized in that: The step of determining the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule includes: The candidate taillight state with the highest proportion in number among the current candidate taillight state and the historical candidate taillight states is determined as the target vehicle taillight state.

5. The method for determining the vehicle taillight status according to claim 3, characterized in that: The step of determining the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule includes: When the number of candidate vehicle taillight states of the same type in the current candidate taillight state and the historical candidate taillight states reaches a preset number, the acquisition of the current video frame to be identified of the target vehicle is stopped and the candidate vehicle taillight state of the same type is determined as the target vehicle taillight state.

6. The method for determining the vehicle taillight status according to claim 3, characterized in that: The step of determining the taillight state of the target vehicle according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule includes: When the candidate vehicle taillight state belonging to the same type in the current candidate taillight state and the historical candidate taillight state appears continuously for a preset number of times, the acquisition of the current video frame to be identified of the target vehicle is stopped and the candidate vehicle taillight state belonging to the same type is determined as the target vehicle taillight state.

7. A vehicle taillight status determination device, characterized in that: include: A feature extraction module is used to obtain a current video frame to be identified of the target vehicle, and input the current video frame to be identified into a feature extraction sub-model of a pre-trained taillight state determination model to determine a video frame feature corresponding to the current video frame to be identified; A feature sequence updating module, used to update a feature recognition sequence corresponding to a previous video frame according to the video frame feature, and use the updated feature recognition sequence as a current feature recognition sequence corresponding to the current video frame to be recognized; a candidate state determination module, configured to input the current feature recognition sequence into the state recognition sub-model of the pre-trained taillight state determination model to determine a current candidate taillight state corresponding to the current feature recognition sequence when the number of features in the current feature recognition sequence is equal to a preset recognition number; The target state determination module is used to determine the target vehicle taillight state according to the current candidate taillight state, the historical candidate taillight state and the preset taillight state selection rule.

8. A vehicle taillight status determination device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the vehicle taillight state determination method according to any one of claims 1 to 6.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the vehicle taillight state determination method according to any one of claims 1 to 6 when executed by a computer processor.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for determining the vehicle taillight state according to any one of claims 1 to 6 is implemented.

Images