A vehicle pose estimation method and system under insufficient illumination conditions

Abstract

The application discloses a vehicle posture estimation method and system under insufficient illumination conditions, and relates to the field of intelligent parking management.The method comprises the following steps: according to traffic scene video image data, performing data labeling on image data with normal illumination; performing model training according to the data-enhanced normal illumination image data, and constructing a teacher network model; performing data enhancement on the labeled normal illumination image data, and training the teacher network model according to the enhanced normal illumination image data, to generate a trained teacher network model; according to the trained teacher network model and unlabeled dark-light scene data, constructing a student network and generating pseudo labels corresponding to the dark-light scene data; and estimating the vehicle posture under insufficient illumination conditions according to the student network and the pseudo labels corresponding to the dark-light scene data, so that the difficulty of realizing the vehicle posture estimation under insufficient illumination conditions is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent parking management, and in particular to a method and system for estimating vehicle attitude under insufficient lighting conditions. Background Technology

[0002] In recent years, high-position video technology has developed rapidly. By installing high-position video cameras on the roadside, vehicle parking can be judged and managed. By detecting vehicles and their posture, and then analyzing the data with known parking space locations, the status of vehicles can be determined, such as whether the vehicle is in a parking space, whether the vehicle is parked on the line, or whether the vehicle is parked in a no-parking zone. In addition, the posture of the vehicle can be used to determine whether the vehicle is illegally parked. Therefore, it has a positive role in promoting various aspects of urban traffic management and driving safety.

[0003] For determining vehicle attitude, one approach is to use devices such as LiDAR, binocular or multi-view cameras to acquire the vehicle's 3D information, thereby estimating the vehicle's attitude and determining whether it is illegally parked. However, installing LiDAR and similar equipment is costly for roadside parking scenarios. Another approach is to directly estimate vehicle attitude using 2D images. This method requires no additional equipment and only uses video image data captured by cameras, achieving vehicle attitude estimation at a lower cost. However, in low-light conditions, the image quality is low, the visibility of the vehicle is low, and there is excessive noise in the image, resulting in lower accuracy in vehicle attitude estimation. Nevertheless, in nighttime scenarios, determining the parking status of a vehicle based on its attitude is crucial for traffic management.

[0004] Currently, there are two main methods for vehicle pose estimation in low-light scenarios. One method combines RGB images captured by a visible light camera with infrared images captured by an infrared camera. This leverages the advantage of infrared images, which are unaffected by low-light conditions, to improve the accuracy of vehicle pose estimation in low-light environments. However, the installation of thermal infrared cameras limits their application in real-world scenarios. Another method involves using paired normal and low-light image data for model learning to enhance the low-light image. However, obtaining paired normal and low-light data is very difficult, and in practical applications, it has been found that the enhanced low-light data often exhibits unrealistic image artifacts. Therefore, vehicle pose estimation based on enhanced low-light images also presents certain challenges. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a vehicle attitude estimation method and system under insufficient lighting conditions, which can overcome the limitations and implementation difficulties of existing vehicle attitude estimation methods under insufficient lighting conditions.

[0006] To achieve the above objectives, the present invention provides a vehicle attitude estimation method under low-light conditions, the method comprising:

[0007] Based on traffic scene video image data, data annotation is performed on image data with normal lighting to obtain the two-dimensional detection box and three-dimensional bounding box annotation information of vehicles in the image;

[0008] A teacher network model is constructed by training the model based on the data-enhanced normally illuminated image data.

[0009] The labeled normal lighting image data is augmented, and the teacher network model is trained based on the augmented normal lighting image data to generate the trained teacher network model.

[0010] Based on the trained teacher network model and the unlabeled dark light scene data, a student network is constructed and pseudo-labels corresponding to the dark light scene data are generated;

[0011] The vehicle attitude under insufficient lighting conditions is estimated based on the pseudo-labels corresponding to the student network and the low-light scene data.

[0012] Furthermore, the step of training the model based on the data-enhanced normally illuminated image data to construct the teacher network model includes:

[0013] Based on the data-enhanced normally lit image data, the backbone network model in the teacher network model is subjected to a superposition operation of convolutional layers, normalization layers, activation function layers, and convolutional combination layers;

[0014] In the feature aggregation network model of the teacher network model, data aggregation is performed on features located between different layers of the network.

[0015] Based on the two-dimensional detection boxes and three-dimensional bounding boxes of the vehicles in the image and the key point prediction network model in the teacher network model, the center point of eight vehicle key points, the offset of each vehicle key point relative to the center point, and the heat map are learned.

[0016] Furthermore, the steps of performing data augmentation on the labeled normal lighting image data and training the teacher network model based on the augmented normal lighting image data to generate the trained teacher network model include:

[0017] Based on the loss function L main =L C +α main L O and L com =L HThe teacher network model is trained to generate a trained teacher network model, where L C The training loss function for the center points of vehicle key points is represented by the MSE loss function; α main L is the coefficient parameter. O The loss function for the offset is the L1 loss function, L H The loss function for the key point heatmap is represented by the MSE loss function.

[0018] Furthermore, the teacher network model includes a primary teacher network model and an auxiliary teacher network model. The step of constructing a student network and generating pseudo-labels corresponding to the dark light scene data based on the trained teacher network model and unlabeled dark light scene data includes:

[0019] According to formula P all =NMS(Concat(P main [C main >s main ],P com [C com >s com Construct a student network and generate pseudo-labels corresponding to the low-light scene data, where P main P com S represents the prediction results of the primary teacher model and the auxiliary teacher model, respectively; main s com Let C represent the score thresholds of the two teacher models, respectively. main C com This represents the prediction scores of two teacher models. Concat() means concatenating and combining the predictions from the two models that are greater than a threshold, and then using NMS() to filter and remove duplicate predictions.

[0020] Further, before the step of estimating the vehicle pose under insufficient lighting conditions based on the pseudo-labels corresponding to the student network and the low-light scene data, the method includes:

[0021] According to formula L S =α sup L sup +α unsup L unsup Construct a loss function for the student network and train and update the student network, where α sup α unsup L represents the weight coefficients for supervised and unsupervised learning in the student network, respectively. sup L unsup These represent supervised and unsupervised loss functions in the student network, respectively, which are the same as the loss function used for training the main teacher model.

[0022] Furthermore, the present invention provides a vehicle attitude estimation system under low-light conditions, the system comprising:

[0023] The annotation module is used to annotate normally lit image data based on traffic scene video image data, and obtain the two-dimensional detection box and three-dimensional bounding box annotation information of vehicles in the image;

[0024] The module is used to train the model based on the data-enhanced normally illuminated image data and build a teacher network model.

[0025] The generation module is used to perform data augmentation on the labeled normal lighting image data, and to train the teacher network model based on the augmented normal lighting image data to generate the trained teacher network model.

[0026] The construction module is also used to construct a student network and generate pseudo-labels corresponding to the dark light scene data based on the trained teacher network model and the unlabeled dark light scene data.

[0027] An estimation module is used to estimate the vehicle attitude under insufficient lighting conditions based on the pseudo-labels corresponding to the student network and the low-light scene data.

[0028] Furthermore, the construction module is specifically used to perform superposition operations of convolutional layers, normalization layers, and activation function layers on the backbone network model of the teacher network model based on the data-enhanced normally lit image data; to perform data aggregation on features located between different layers in the feature aggregation network model of the teacher network model; and to learn the center points of eight vehicle key points, the offset of each vehicle key point relative to the center point, and the heatmap based on the two-dimensional detection boxes and three-dimensional bounding boxes of the vehicles in the image and the key point prediction network model of the teacher network model.

[0029] Furthermore, the generation module is specifically used to generate based on the loss function L. main =L C +α main L O and L com =L H The teacher network model is trained to generate a trained teacher network model, where L C The training loss function for the center points of vehicle key points is represented by the MSE loss function; α main L is the coefficient parameter. O The loss function for the offset is the L1 loss function, L H The loss function for the key point heatmap is represented by the MSE loss function.

[0030] Furthermore, the building module is also used to calculate according to formula P. all =NMS(Concat(P main [C main >s main ],P com [C com >s com Construct a student network and generate pseudo-labels corresponding to the low-light scene data, where P main P com s represents the prediction results of the primary teacher model and the auxiliary teacher model, respectively; main s com Let C represent the score thresholds of the two teacher models, respectively. main C com This represents the prediction scores of two teacher models. Concat() means concatenating and combining the predictions from the two models that are greater than a threshold, and then using NMS() to filter and remove duplicate predictions.

[0031] Furthermore, the building module is also used to calculate according to formula L. S =α sup L sup +α unsup L unsup Construct a loss function for the student network and train and update the student network, where α sup α unsup L represents the weight coefficients for supervised and unsupervised learning in the student network, respectively. sup L unsup These represent supervised and unsupervised loss functions in the student network, respectively, which are the same as the loss function used for training the main teacher model.

[0032] This invention provides a vehicle pose estimation method and system under low-light conditions. By constructing a set of complementary teacher network models to generate more reliable pseudo-labels, the student model can achieve good vehicle pose estimation even on images with extremely low light. It enables learning of vehicle pose estimation in low-light scenes without needing to obtain paired data of normal lighting and low-light scenes, only requiring images with normal lighting, which greatly reduces the difficulty of implementing vehicle pose estimation under low-light conditions. Attached Figure Description

[0033] Figure 1 This is a flowchart of a vehicle attitude estimation method under insufficient lighting conditions provided by the present invention;

[0034] Figure 2 This is a schematic diagram of a vehicle attitude estimation system under insufficient lighting conditions provided by the present invention. Detailed Implementation

[0035] The device structure and implementation of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0036] This invention provides a method for vehicle attitude estimation under low-light conditions, such as... Figure 1 As shown, the specific steps include the following:

[0037] 101. Based on traffic scene video image data, perform data annotation on image data with normal lighting to obtain the two-dimensional detection box and three-dimensional bounding box annotation information of vehicles in the image.

[0038] Specifically, the traffic scenario includes vehicle parking behaviors such as the process of a vehicle entering a parking space, the process of a vehicle in a parking space, and the process of a vehicle leaving a parking space, and includes targets such as parking space lines and roadside green belts; it includes vehicle data from different camera angles, different camera installation heights, different seasons, different lighting conditions, and different time periods; the two-dimensional detection box of the vehicle refers to the x and y coordinates of the upper left and lower right corners of the vehicle's rectangular frame, represented by (x_min, y_min, x_max, y_max); the three-dimensional bounding box information of the vehicle refers to the type of vehicle key points, the coordinates of the key points, and the visibility attributes of the key points, usually using the vehicle's three-dimensional bounding box to describe the vehicle's posture; the eight key points are divided according to the following rules: four key points where the vehicle is in contact with the ground and four key points where the vehicle roof is in the air.

[0039] Based on the vehicle's orientation, the key point on the left side of the vehicle's front that is in contact with the ground is designated as point 1 (bottom-front-1). Then, rotating clockwise, the other three ground-contact points are designated as bottom-front-2, bottom-back-3, and bottom-back-4. Next, the key point on the left side of the vehicle's roof in the front direction is designated as point 5 (top-front-5). Rotating clockwise, the other three roof-mounted key points are designated as top-front-6, top-back-7, and top-back-8. This results in eight types of key points describing the vehicle's posture. The coordinates of each key point are its x and y coordinates in the image coordinate system. The visibility attribute includes both visible and invisible attributes; visible points are designated as 1, and invisible points as 0. The category, coordinate position, and visibility attribute of these key points are used as label data.

[0040] 102. Train the model based on the data-enhanced normally lit image data to construct a teacher network model.

[0041] In this embodiment of the invention, step 102 may specifically include: performing a superposition operation of convolutional layers, normalization layers, and activation function layers on the backbone network model of the teacher network model based on the data-enhanced normally lit image data; performing data aggregation on the features in the feature aggregation network model of the teacher network model that are located between different layers of the network; and learning the center points of eight vehicle key points, the offset of each vehicle key point from the center point, and a heatmap based on the two-dimensional detection box and three-dimensional bounding box annotation information of the vehicles in the image and the key point prediction network model of the teacher network model.

[0042] The teacher network is divided into two complementary networks: a primary teacher network that learns the model of the vehicle center point and key point offsets, and an auxiliary teacher network that learns the model of the key point heatmap. Both the primary and auxiliary teacher networks consist of three parts: a backbone network, a feature aggregation network, and a key point prediction network. The backbone network and the feature aggregation network have the same network structure, and the only difference is in the key point prediction network, which learns the key points in a different way.

[0043] Specifically, the backbone network is used for image feature extraction. The input image size is H*W*C, where H and W represent the height and width of the image, respectively, and C represents the number of channels. In this case, C is 3, representing an RGB three-channel image. The backbone network mainly uses a convolutional layer-normalization layer-activation function layer convolutional combination method, performing multiple convolutional combination layer stacking operations. Downsampling is performed once in each convolutional combination operation, including but not limited to using backbone networks such as ResNet, VGG, and MobileNet, with a downsampling factor of R. The normalization layer includes, but is not limited to, instance normalization layer and adaptive instance normalization layer. The nonlinear activation layer includes, but is not limited to, ReLU and Leaky. Nonlinear activation functions such as ReLU are used. The feature aggregation network is used to aggregate high and low layer features extracted from different layers in the backbone network, providing more feature representations for subsequent key point detection. In convolutional neural networks, high-level semantic features are close to the output of the network but have low resolution, while high-resolution features are close to the input but have fewer semantic features. Therefore, aggregating features between different layers of the network can achieve the fusion of high and low layer features, thereby improving the detection accuracy of subsequent key point detection tasks.

[0044] For the keypoint prediction part of the main teacher network, the goal is to learn the center point of eight vehicle keypoints, as well as the offset of each vehicle keypoint from the center point.

[0045] The definition of the center point is: in, This represents the two-dimensional coordinates of the k-th key point of the i-th vehicle; the offset of each vehicle key point from the center point is defined as:

[0046] Furthermore, for the key point prediction part of the auxiliary teacher network, the goal is to learn the heatmap of eight vehicle key points. This is because the goal of the main teacher network is to learn the center point and offset, which is highly dependent on the center point. When the vehicle center point is not detected due to occlusion or other reasons, the vehicle's pose cannot be estimated. Therefore, an auxiliary teacher network is added to supplement the shortcomings of the main teacher network by learning the heatmap of key points.

[0047] 103. Perform data augmentation on the labeled normal lighting image data, and train the teacher network model based on the augmented normal lighting image data to generate the trained teacher network model.

[0048] In this embodiment of the invention, step 103 may specifically include: based on the loss function L main =L C +α main L O and L com =L H The teacher network model is trained to generate a trained teacher network model, where L C The training loss function for the center points of vehicle key points is represented by the MSE loss function; α main L is the coefficient parameter. O The loss function for the offset is the L1 loss function, L H The loss function for the key point heatmap is represented by the MSE loss function.

[0049] It should be noted that, regarding the training data, the data acquisition and labeling were described in step 101. Due to the difficulty in acquiring paired data and the challenge in labeling dark light data, only the data with normal lighting was labeled. In order to improve the generalization ability of the teacher model for dark light scene data, data augmentation was first performed on the labeled data with normal lighting. The purpose of the data augmentation is to process the data of the normal lighting image to simulate the characteristics of dark light scene data, including but not limited to data processing methods such as gamma correction, brightness adjustment, contrast adjustment, and adding Gaussian noise. Among them, the role of gamma correction and brightness adjustment is to increase the dark light characteristics of the image, the role of contrast adjustment is to reduce the contrast of the image, and the role of adding Gaussian noise is to introduce noise to simulate the noise characteristics of the dark light image.

[0050] Specifically, the data augmentation method described above is used to augment the labeled normal lighting images, and the proportion of data augmentation can be set to 0.5. Based on the augmented training data, a loss function is constructed, and the two teacher models are trained. For the primary teacher model, the loss function is defined as: L main =L C +α main L O , where L C The training loss function for the center points of vehicle key points is represented by the MSE loss function; α main L is the coefficient parameter. O The loss function for the offset is the L1 loss function; for the auxiliary loss function, the loss function is defined as: L com =L H Among them, L H The loss function for the keypoint heatmap is represented by the MSE loss function. Based on the above definitions of the teacher model's data, loss function, and network structure, the teacher model is trained to obtain a trained teacher model, which is then used for pseudo-label generation in the subsequent student model training process.

[0051] 104. Based on the trained teacher network model and the unlabeled dark light scene data, construct the student network and generate pseudo-labels corresponding to the dark light scene data.

[0052] In this embodiment of the invention, step 104 may specifically include: according to formula P all =NMS(Concat(P main [C main >s main ],P com [C com >S com Construct a student network and generate pseudo-labels corresponding to the low-light scene data, where P main P com s represents the prediction results of the primary teacher model and the auxiliary teacher model, respectively; main s com Let C represent the score thresholds of the two teacher models, respectively. main C com This represents the prediction scores of two teacher models. Concat() means concatenating and combining the predictions from the two models that are greater than a threshold, and then using NMS() to filter and remove duplicate predictions.

[0053] It should be noted that the network structure of the student model is the same as that of the main teacher model, including a backbone network, a feature aggregation network, and a keypoint prediction network. The keypoint prediction network learns the vehicle center point and keypoint offsets through model learning. To improve the student model's ability to learn vehicle pose estimation in low-light scenes and surpass the teacher model, the student model's learning is divided into two parts: supervised learning and unsupervised learning. For the supervised learning part, the process is similar to that of the main teacher model, except that the proportion of low-light data augmentation is increased in the training data to improve the student model's learning of low-light images. For the unsupervised learning part, the training data is unlabeled real low-light scene data. The annotation generation of this data is performed using the trained teacher model. The vehicle pose estimation results of the two teacher models on unlabeled real low-light scene images are selected and fused as pseudo-labels to guide the training of the student model.

[0054] Specifically, the selection and fusion process for obtaining the final pseudo-label by selecting and fusing the vehicle pose estimation results from two teacher models is defined as follows:

[0055] P all =NMS(Concat(P main [C main >s main ],P com [C com >s com ]))

[0056] Among them, P main P com s represents the prediction results of the primary teacher model and the auxiliary teacher model, respectively; main s com Let C represent the score thresholds of the two teacher models, respectively. main C com This represents the prediction scores of two teacher models. Concat() means concatenating and combining the predictions from the two models that are greater than a threshold, and then using NMS() to filter and remove duplicate predictions.

[0057] 105. Estimate the vehicle attitude under insufficient lighting conditions based on the pseudo-labels corresponding to the student network and the low-light scene data.

[0058] For embodiments of the present invention, step 105 may further include: according to formula L S =α sup L sup +α unsup L unsup Construct a loss function for the student network and train and update the student network, where αsup α unsup L represents the weight coefficients for supervised and unsupervised learning in the student network, respectively. sup L unsup These represent supervised and unsupervised loss functions in the student network, respectively, which are the same as the loss function used for training the main teacher model.

[0059] This invention provides a vehicle pose estimation method under low-light conditions. By constructing a set of complementary teacher network models to generate more reliable pseudo-labels, the student model can achieve good vehicle pose estimation even on images with extremely low light. It enables learning of vehicle pose estimation in low-light scenes without needing to obtain paired data of normal lighting and low-light scenes, only requiring images with normal lighting, which greatly reduces the difficulty of implementing vehicle pose estimation under low-light conditions.

[0060] As Figure 1 The specific implementation of the method shown in this embodiment of the invention provides a vehicle attitude estimation system under insufficient lighting conditions, such as... Figure 2 As shown, the system includes: a labeling module 21, used to label normally lit image data based on traffic scene video image data, to obtain two-dimensional detection boxes and three-dimensional bounding boxes of vehicles in the image;

[0061] Module 22 is used to train the model based on the data-enhanced normally lit image data and construct a teacher network model;

[0062] The generation module 23 is used to perform data augmentation on the labeled normal illumination image data, and to train the teacher network model based on the augmented normal illumination image data to generate the trained teacher network model.

[0063] The construction module 22 is also used to construct a student network and generate pseudo-labels corresponding to the dark light scene data based on the trained teacher network model and the unlabeled dark light scene data.

[0064] The estimation module 24 is used to estimate the vehicle attitude under insufficient lighting conditions based on the pseudo-labels corresponding to the student network and the low-light scene data.

[0065] Furthermore, the construction module 22 is specifically used to perform superposition operations of convolutional layers, normalization layers, and activation function layers on the backbone network model of the teacher network model based on the data-enhanced normal illumination image data; to perform data aggregation on features located between different layers in the feature aggregation network model of the teacher network model; and to learn the center points of eight vehicle key points, the offset of each vehicle key point from the center point, and the heatmap based on the two-dimensional detection box and three-dimensional bounding box annotation information of the vehicles in the image and the key point prediction network model of the teacher network model.

[0066] Furthermore, the generation module 23 is specifically used to generate according to the loss function L. main =L C +α main L O and L com =L H The teacher network model is trained to generate a trained teacher network model, where L C The training loss function for the center points of vehicle key points is represented by the MSE loss function; α main L is the coefficient parameter. O The loss function for the offset is the L1 loss function, L H The loss function for the key point heatmap is represented by the MSE loss function.

[0067] Furthermore, the construction module 22 is also used to calculate according to formula P. all =NMS(Concat(P main [C main >s main ],P com [C com >s com Construct a student network and generate pseudo-labels corresponding to the low-light scene data, where P main P com s represents the prediction results of the primary teacher model and the auxiliary teacher model, respectively; main s com Let C represent the score thresholds of the two teacher models, respectively. main C com This represents the prediction scores of two teacher models. Concat() means concatenating and combining the predictions from the two models that are greater than a threshold, and then using NMS() to filter and remove duplicate predictions.

[0068] Furthermore, the construction module 22 is also used to calculate according to formula L S =α sup L sup +αunsup L unsup Construct a loss function for the student network and train and update the student network, where α sup α unsup L represents the weight coefficients for supervised and unsupervised learning in the student network, respectively. sup L unsup These represent supervised and unsupervised loss functions in the student network, respectively, which are the same as the loss function used for training the main teacher model.

[0069] This invention provides a vehicle pose estimation system under low-light conditions. By constructing a set of complementary teacher network models to generate more reliable pseudo-labels, the student model can achieve good vehicle pose estimation even on images with extremely low light. It enables learning of vehicle pose estimation in low-light scenes without needing to acquire paired normal lighting and low-light scene data, only requiring normal lighting images, which greatly reduces the difficulty of implementing vehicle pose estimation under low-light conditions.

[0070] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0071] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0072] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0073] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0074] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0075] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0076] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0077] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while disks typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0078] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for vehicle pose estimation under low-light conditions, characterized in that, The method comprises: According to the traffic scene video image data, the image data under normal light is data labeled to obtain the two-dimensional detection frame and three-dimensional bounding box label information of the vehicle in the image; According to the data enhanced image data under normal light, the model is trained to build a teacher network model; The labeled image data under normal light is data enhanced, and the teacher network model is trained according to the enhanced image data under normal light to generate a trained teacher network model; According to the trained teacher network model and unlabeled dark light scene data, a student network is built and the corresponding pseudo label of the dark light scene data is generated; According to the student network and the corresponding pseudo label of the dark light scene data, the vehicle posture under insufficient light condition is estimated; The step of training the model according to the data enhanced image data under normal light to build a teacher network model comprises: According to the data enhanced image data under normal light, the convolution layer, the normalization layer and the activation function layer convolution combination layer of the backbone network model in the teacher network model are superimposed and operated; The features between different layers of the feature aggregation network model in the teacher network model are data aggregated; According to the two-dimensional detection frame and three-dimensional bounding box label information of the vehicle in the image and the key point prediction network model in the teacher network model, the center point of eight vehicle key points, the offset of each vehicle key point to the center point and the heat map are learned; The teacher network model comprises a main teacher network model and an auxiliary teacher network model, and the step of building a student network according to the trained teacher network model and unlabeled dark light scene data and generating the corresponding pseudo label of the dark light scene data comprises: According to the formula Construct a student network and generate pseudo labels corresponding to the dark light scene data, For the pseudo labels corresponding to the dark light scene data, Respectively represent the prediction results of the main teacher model and the auxiliary teacher model; Respectively represent the score thresholds of the two teacher models, Indicates the score of the prediction result of the two teacher models, Indicates that the prediction results greater than the threshold in the two model prediction results are spliced and combined, Indicates that the prediction results are filtered and removed.

2. The vehicle pose estimation method under the condition of insufficient illumination according to claim 1, characterized in that, The step of training the model according to the data enhanced image data under normal light to build a teacher network model comprises: According to the main loss function And the auxiliary loss function Train the teacher network model to generate a trained teacher network model, wherein Indicates the training loss function of the vehicle key point center point, Is a coefficient parameter, Is the loss function of the offset, Indicates the key point heat map loss function.

3. The vehicle pose estimation method under the condition of insufficient illumination according to claim 2, characterized in that, Before the step of estimating the vehicle posture under insufficient light condition according to the student network and the corresponding pseudo label of the dark light scene data, the method comprises: According to the formula constructing a loss function of the student network and training and updating the student network, wherein is the loss function of the student model, respectively represent the weight coefficients of supervised learning and unsupervised learning in the student network, respectively represent the supervised loss function and the unsupervised loss function in the student network, which are the same as the loss function of the main teacher model training.

4. A vehicle pose estimation system under low-light conditions, characterized by, The system comprises: The labeling module is configured to label the image data under normal light according to the traffic scene video image data to obtain the two-dimensional detection frame and three-dimensional bounding box label information of the vehicle in the image; The building module is configured to train the model according to the data enhanced image data under normal light to build a teacher network model; The generating module is configured to data enhance the labeled image data under normal light, and train the teacher network model according to the enhanced image data under normal light to generate a trained teacher network model; The building module is further configured to build a student network according to the trained teacher network model and unlabeled dark light scene data, and generate the corresponding pseudo label of the dark light scene data; The estimating module is configured to estimate the vehicle posture under insufficient light condition according to the student network and the corresponding pseudo label of the dark light scene data. The construction module is specifically used to perform superposition operations of convolutional layers, normalization layers, and activation function layers on the backbone network model of the teacher network model based on the data-enhanced normally lit image data; to perform data aggregation on features located between different layers in the feature aggregation network model of the teacher network model; and to learn the center points of eight vehicle key points, the offset of each vehicle key point relative to the center point, and a heatmap based on the two-dimensional detection boxes and three-dimensional bounding boxes of vehicles in the image and the key point prediction network model of the teacher network model. The construction module is also used to... Formula constructing a student network and generating pseudo labels corresponding to the dark-light scene data, wherein pseudo labels corresponding to the dark-light scene data, respectively represent the prediction results of the main teacher model and the auxiliary teacher model; respectively represent the score thresholds of the two teacher models, represent the scores of the prediction results of the two teacher models, represent the prediction results greater than the threshold in the prediction results of the two models, represent the prediction results after filtering and removing duplicates.

5. A vehicle attitude estimation system under insufficient lighting conditions according to claim 4, characterized in that, The generation module is specifically configured to train the teacher network model according to a main loss function and an auxiliary loss function to generate a trained teacher network model, wherein represents a training loss function for the center point of the vehicle key point, is a coefficient parameter, is a loss function of an offset, represents a key point heat map loss function.

6. The vehicle attitude estimation system under insufficient lighting conditions according to claim 5, characterized in that, The construction module is further configured to construct a loss function of the student network according to a formula construct a loss function of the student network and perform training update on the student network, wherein, is a loss function of the student model, respectively represent weight coefficients of supervised learning and unsupervised learning in the student network, respectively represent a supervised loss function and an unsupervised loss function in the student network, and are the same as the loss function of the main teacher model training.

Images