Vehicle lamp state detection method and device, electronic equipment and vehicle
By using BEV model and camera parameter conversion technology at high speed, the problem of inaccurate identification of vehicle light status with longer distances is solved, and higher prediction accuracy is achieved.
Patent Information
- Application Number
- CN202410211185.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-02-26
- Publication Date
- 2025-08-26
AI Technical Summary
In the case of high-speed driving, it is difficult for the prior art to accurately identify the light status of long-distance vehicles, resulting in poor prediction results.
By acquiring the acquired image, the 3D coordinates of the target vehicle are determined using the BEV model and a 3D frame is established. The first camera parameters are converted into the first 2D frame. If the recognition fails, the second camera parameters with a smaller field of view angle are switched to the second 2D frame, and the vehicle light status is determined in combination with image compensation.
It improves the accuracy of identifying the light status of long-distance vehicles and improves the accuracy of prediction of target vehicles during high-speed autonomous driving.
Smart Images
Figure CN120544150A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, electronic equipment and vehicle for detecting vehicle light status. Background Art
[0002] Current intelligent driving technologies require the collection of various information about the vehicle's driving environment, which is then used to inform vehicle control decisions. This includes obtaining information about the headlight status of other vehicles. The current method for obtaining this information is to input a vehicle image into a recognition model, which then outputs the headlight status information. During this process, the recognition model must combine the vehicle image to determine whether the lights in the image are headlights or taillights before obtaining the corresponding headlight status information. However, in certain circumstances, it can be difficult for a vehicle to identify the lighting status of vehicles at a distance using only a single camera. For example, on highways, it is impossible to identify the lighting status of vehicles more than 80 meters away. Furthermore, due to the high speeds of vehicles, predicting the target vehicle's trajectory is particularly important. Therefore, related technologies are prone to poor prediction results due to the inability to obtain the lighting status and thus the inability to further predict the vehicle's trajectory. Summary of the Invention
[0003] In view of this, the present application provides a vehicle light status detection method, device, electronic device and vehicle, which can improve the current problem of poor prediction results due to the inability to obtain vehicle light status at a long distance in high-speed conditions.
[0004] In a first aspect, the present application provides a method for detecting a vehicle light status, comprising:
[0005] Acquire a captured image; wherein the captured image contains a target vehicle;
[0006] Inputting the captured image into a BEV model to obtain 3D coordinates of each vertex of the target vehicle, and determining a 3D frame of the target vehicle using the 3D coordinates;
[0007] Based on first camera parameters, converting the 3D frame into a first 2D frame;
[0008] If the headlight state of the target vehicle cannot be identified using the headlight state result determined using the image corresponding to the first 2D frame, converting the 3D frame into a second 2D frame using second camera parameters; the field of view of the second camera parameters is smaller than that of the first camera parameters;
[0009] Determining the headlight status of the target vehicle using the image corresponding to the second 2D frame;
[0010] The vehicle light status includes left light on, right light on, both lights on, and off.
[0011] Optionally, after converting the 3D frame into a second 2D frame using the second camera parameters, the method further includes: if the image obtained based on the conversion of the second camera parameters exceeds the image boundary, switching to the first camera parameters; performing image compensation on the first 2D frame in combination with the captured image; and determining the headlight status of the target vehicle using the first 2D frame after image compensation.
[0012] Optionally, the 3D frame includes a front 3D frame and a rear 3D frame;
[0013] Using the 3D coordinates, determining the front 3D frame or the rear 3D frame of the target vehicle includes: determining the coordinates of the center point of the vehicle; determining two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining another two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the 3D frame of the target vehicle based on the determined four vertices.
[0014] Optionally, the image corresponding to the 2D frame is used to determine the headlight status of the target vehicle, including: using the driving direction vector of the target vehicle to determine the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame; the image corresponding to the 2D frame and the vehicle frame type are input into a headlight recognition model to determine the headlight status of the target vehicle.
[0015] Optionally, determining the vehicle frame type of the 2D frame by using the driving direction vector of the target vehicle includes: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the own vehicle; obtaining a second vector of the driving direction of the target vehicle; and determining the vehicle frame type of the 2D frame based on the first vector and the second vector.
[0016] The determining of the vehicle frame type of the 2D frame based on the first vector and the second vector includes: when the angle between the first vector and the second vector is less than 90 degrees, determining that the vehicle frame type is a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, determining that the vehicle frame type is a rear frame.
[0017] Optionally, the image corresponding to the 2D frame and the vehicle frame type are input into a vehicle light recognition model to determine the vehicle light status of the target vehicle, including: when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a tail frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a tail frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that both lights are on or off, then the vehicle light status is determined to be consistent with the output of the vehicle light recognition model.
[0018] In a second aspect, the present application provides a vehicle light status detection device, comprising:
[0019] An acquisition unit is configured to acquire an acquired image; the acquired image contains a target vehicle;
[0020] an input unit configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and determine a 3D frame of the target vehicle using the 3D coordinates;
[0021] a conversion unit configured to convert the 3D frame into a first 2D frame based on a first camera parameter;
[0022] a determination unit configured to, when the light state of the target vehicle cannot be identified by using the light state result determined using the image corresponding to the first 2D frame, convert the 3D frame into a second 2D frame using second camera parameters; the field of view of the second camera parameters being smaller than that of the first camera parameters;
[0023] a determining unit configured to determine a headlight status of the target vehicle using the image corresponding to the second 2D frame;
[0024] The vehicle light status includes left light on, right light on, both lights on, and off.
[0025] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle light status detection method described in the first aspect.
[0026] In a fourth aspect, the present application provides an electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle light status detection method described in the first aspect when executing the computer program.
[0027] In a fifth aspect, the present application provides a vehicle, comprising the vehicle light status detection device as mentioned in the second aspect or the electronic device as mentioned in the fourth aspect.
[0028] By utilizing the above-described technical solution, the present application provides a vehicle light status detection method, device, electronic device, and vehicle. The method first acquires a captured image containing a target vehicle, inputs the captured image into a BEV model, obtains the 3D coordinates of each vertex of the target vehicle, and uses the 3D coordinates to determine a 3D bounding box of the target vehicle. Based on first camera parameters, the 3D bounding box is converted into a first 2D bounding box. If the vehicle light status determined using the image corresponding to the first 2D bounding box cannot identify the target vehicle's light status, the 3D bounding box is converted into a second 2D bounding box using second camera parameters, and the vehicle light status of the target vehicle is further determined using the image corresponding to the second 2D bounding box. The second camera parameters have a smaller field of view than the first camera parameters. Compared with the solution in the related art that uses conventional camera parameters to project a vehicle at a longer distance, resulting in a small and blurred 2D frame, this application judges the headlight status result determined by the first 2D frame. If the headlight status cannot be identified, it uses the second camera parameters with a smaller field of view to project to obtain a second 2D frame, and determines the headlight status of the target vehicle based on the second 2D frame, thereby realizing the recognition of the headlight status at a longer distance and improving the accuracy of the target vehicle prediction during high-speed autonomous driving.
[0029] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0032] Figure 1 A schematic diagram showing a flow chart of a vehicle light status detection method provided in an embodiment of the present application is shown;
[0033] Figure 2 A schematic diagram showing the input and input results of the light detection model provided in an embodiment of the present application is shown;
[0034] Figure 3 A schematic diagram showing a flow chart of another vehicle light status detection method provided by an embodiment of the present application is shown;
[0035] Figure 4 A schematic structural diagram of a vehicle light status detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0036] In order to be able to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In addition, in order to be able to understand the characteristics and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through a plurality of details. However, in the absence of these details, one or more embodiments can still be implemented. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.
[0037] The vehicle light status detection method provided in this embodiment is applied to a vehicle light status detection device, which can be installed or integrated into an electronic control unit (ECU) in a vehicle, or installed in an independent system that communicates with the vehicle.
[0038] In order to improve the current problem of poor prediction results due to the inability to obtain the vehicle light status at a long distance under high-speed conditions, this embodiment proposes a vehicle light status detection method. Figure 1 As shown, the method includes:
[0039] S101, acquiring a captured image.
[0040] The captured image includes the target vehicle. This captured image refers to the image captured by the vehicle's camera of the area surrounding the vehicle. A vehicle can typically have one or more cameras. One captures a wide-angle image of the road ahead, while multiple cameras can be set facing different directions. The target vehicle is the vehicle whose lighting status is to be detected or whose movement is to be predicted based on its lighting status.
[0041] S102 , inputting the captured image into the BEV model, obtaining the 3D coordinates of each vertex of the target vehicle, and determining the 3D frame of the target vehicle using the 3D coordinates.
[0042] The captured image here can be a single image or a panoramic image of the surrounding area of the vehicle obtained by stitching multiple captured images. The image only needs to include the target vehicle to be predicted. The method proposed in this embodiment can also predict multiple target vehicles in the panoramic image.
[0043] The output of the BEV model includes the 3D attributes of the target vehicle. These 3D attributes include the coordinates of each vertex of the vehicle (generally three-dimensional coordinates established with the center point of the vehicle as the origin, which can also be converted into coordinates of other coordinate systems), the length, width, height and speed of the vehicle. Among them are the 3D coordinates of each vertex of the target vehicle, and the number of vertices is generally 8. Through the relationship between the 3D coordinates of the 8 vehicle vertices and the distance from the vehicle, a 3D frame in the width direction of the vehicle can be determined, but the 3D frame cannot temporarily distinguish whether it is the front or the rear of the vehicle, so the 3D frame here includes the front frame and the rear frame. S103, based on the first camera parameters, the 3D frame is converted into a first 2D frame.
[0044] The 3D frame is converted into a 2D frame by projection, and the 2D coordinates are obtained by matrix operation through internal parameters. The matrix parameters are composed of the camera intrinsic parameters.
[0045] S104 : When the headlight state of the target vehicle cannot be identified by using the image corresponding to the first 2D frame, the 3D frame is converted into a second 2D frame using the second camera parameters.
[0046] The 3D frame determined in S102 cannot be distinguished as the front or rear of the vehicle for the time being, so it needs to be converted to 2D processing first. Among them, the first camera parameters are the camera parameters of a single camera used to capture the front image under normal circumstances (the single here does not mean that there is only one camera installed on the vehicle, but that the foreground images captured by the vehicle are all captured by the camera based on this front camera, and multiple cameras are not used to capture one direction). For example, the first camera parameters select the front image captured through a 120-degree field of view. Based on the camera parameters, the 3D frame is matrix-projected to obtain a first 2D frame, and the first 2D frame is used to predict the distant vehicle and determine the headlight status result.
[0047] If the light status result determined in the first 2D frame cannot identify the light status of the target vehicle, it means that the target vehicle is far away. In this case, the second camera parameters are used, for example, a 30-degree field of view, to convert the 3D frame into a second 2D frame. The field of view of the second camera parameters is smaller than that of the first camera parameters, and the viewing distance of the second camera parameters with a smaller field of view is greater than that of the first camera parameters. This is used to predict the light status of the distant vehicle and determine the light status result. The first and second camera parameters can be achieved by switching camera parameters on the same vehicle camera, or by setting two cameras to use the first and second camera parameters respectively.
[0048] S105 : Determine the headlight status of the target vehicle using the image corresponding to the second 2D frame.
[0049] The headlight status includes left light on, right light on, both lights on, and off. The second 2D frame is used to determine the headlight status of the target vehicle, thereby realizing the movement prediction of the target vehicle. The image corresponding to the first 2D frame and the image corresponding to the second 2D frame are images reflecting the same position in space. The difference is that they are converted based on different camera parameters. In fact, the first 2D frame and the second 2D frame may essentially be the front line or the rear line of the vehicle, and this line segment needs to be reflected based on the image. Therefore, the image corresponding to the first 2D frame and the second 2D frame actually refers to the image converted by different camera parameters at the same position, and the image is used as the input of the subsequent headlight recognition model to determine the headlight status of the target vehicle.
[0050] In this embodiment, a captured image containing a target vehicle is first acquired and input into a BEV model to obtain the 3D coordinates of each vertex of the target vehicle. The 3D coordinates are then used to determine a 3D bounding box of the target vehicle. This 3D bounding box is then converted into a first 2D bounding box based on first camera parameters. If the light status of the target vehicle cannot be identified using the image corresponding to the first 2D bounding box, the 3D bounding box is converted into a second 2D bounding box using second camera parameters. The second camera parameters have a smaller field of view than the first camera parameters, and a longer viewing distance than the first camera parameters. Unlike conventional camera parameters in the related art, which project a relatively distant vehicle using conventional camera parameters, resulting in a small and blurry 2D bounding box, this embodiment evaluates the light status determined by the first 2D bounding box. If the light status cannot be identified, the second camera parameters, with a smaller field of view and a longer viewing distance, are used to project the second 2D bounding box. The target vehicle's light status is then determined based on the second 2D bounding box. This allows for identification of the light status of vehicles at greater distances, improving the accuracy of target vehicle prediction during high-speed autonomous driving.
[0051] Optionally, after converting the 3D frame into a second 2D frame using the second camera parameters, the method further includes: if the converted frame based on the second camera parameters exceeds the image boundary, switching to the first camera parameters; performing image compensation on the first 2D frame in combination with the captured image; and determining the headlight status of the target vehicle using the first 2D frame after image compensation.
[0052] In this embodiment, if the second camera parameters are used, the parameter changes may result in an inability to obtain a complete 2D frame (the second 2D frame) when converting the 3D frame to a 2D frame, i.e., the frame may exceed the image boundary and be truncation. This situation refers to an incomplete 2D frame, and the second camera parameters cannot be used. In this case, image compensation is performed on the first 2D frame determined based on the first camera parameters in combination with the captured image, so that the 3D coordinates and 3D frame formed by the target vehicle portion are clearer, thereby facilitating the determination of the target vehicle's headlight status based on the image-compensated first 2D frame.
[0053] BEV models are widely used in object detection technology, particularly in the field of vehicle driving. However, at high speeds, a vehicle's unidirectional camera struggles to identify the headlight status of vehicles at a distance ahead. This is because the 2D frames formed by converting the 3D coordinates of distant vehicles to 2D are too small or blurry, resulting in poor recognition and prediction results. The headlight status detection method proposed in this embodiment, when the first camera parameters are ineffective, uses the parameters of a second camera with a smaller field of view and longer viewing distance to perform 2D conversion, thereby making the resulting 2D frames clearer and improving recognition accuracy. Furthermore, if truncation occurs during the 2D conversion of the second camera parameters, image compensation is performed on the first 2D frame combined with the captured image, thereby making the 2D frame clearer and improving the quality of the subsequent 2D frames used as model input. This improves recognition accuracy even when the second camera parameters are not applicable.
[0054] Optionally, the 3D frame includes a front 3D frame and a rear 3D frame; using the 3D coordinates, determining the front 3D frame or the rear 3D frame of the target vehicle includes: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the 3D frame of the target vehicle based on the four determined vertices.
[0055] In this embodiment, normally, only two sides of a vehicle are visible from the current perspective: a longer side face and a shorter front / rear face. The principle is to first find the two vertices closest to the vehicle's center point among the target vehicle's eight vertices (generally the two vertices where the longer side face and the shorter face intersect). Then, from the remaining six vertices, find the two closest to these two vertices, as the vehicle's width is always smaller than its length. Thus, these four vertices can be used to determine the vehicle's front or rear face, and thus accurately create a 3D box for the front or rear face.
[0056] Optionally, the image corresponding to the 2D frame is used to determine the headlight status of the target vehicle, including: using the driving direction vector of the target vehicle to determine the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame; the image corresponding to the 2D frame and the vehicle frame type are input into the headlight recognition model to determine the headlight status of the target vehicle.
[0057] Furthermore, the vehicle frame type of the 2D frame is determined by using the driving direction vector of the target vehicle, including: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the own vehicle; obtaining a second vector of the driving direction of the target vehicle; and determining the vehicle frame type of the 2D frame based on the first vector and the second vector;
[0058] Based on the first vector and the second vector, the vehicle frame type of the 2D frame is determined, including: when the angle between the first vector and the second vector is less than 90 degrees, the vehicle frame type is determined to be a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, the vehicle frame type is determined to be a rear frame.
[0059] In this embodiment, the image is converted into a 2D frame, and the vehicle frame type is determined based on the 2D frame. First, the coordinates of the target vehicle's center are obtained. A first vector is formed from the target vehicle's center coordinates to the vehicle's center coordinates, and the target vehicle's driving direction is the second vector. Based on the first and second vectors, the 2D frame is determined to be the front or rear frame, facilitating subsequent headlight model recognition.
[0060] Furthermore, if the angle between the first and second vectors is less than 90 degrees, the vehicle frame type is determined to be a front frame; if the angle between the first and second vectors is greater than 90 degrees, the vehicle frame type is determined to be a rear frame. This facilitates subsequent input of the vehicle headlight model recognition.
[0061] Optionally, the image corresponding to the 2D frame and the vehicle frame type are input into a vehicle light recognition model to determine the vehicle light status of the target vehicle, including: when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is both on or off, then the vehicle light status is determined to be consistent with the output of the vehicle light recognition model.
[0062] In this embodiment, if Figure 2 As shown in the figure, the results of light input and output are shown. If the 2D frame is a rear frame, the target vehicle is traveling in the same direction as the vehicle itself, and the model output is consistent with the actual light state of the target vehicle. If the 2D frame is a front frame, the target vehicle is traveling in the opposite direction to the vehicle itself, and the model output is opposite to the actual light state of the target vehicle. The double-bright state is when both left and right lights are fully turned on (such as turning on the double flash). By inputting the vehicle frame type and 2D frame into the vehicle light detection model, the light state of the target vehicle can be determined, which is beneficial for the subsequent prediction of vehicle movement.
[0063] Optionally, before inputting the collected image into the BEV model, the method further includes: filtering the collected image.
[0064] In this embodiment, filtering refers to filtering out unimportant content in the captured image. For example, if the primary detection target is a vehicle, non-vehicle objects can be filtered out based on distance restrictions or vehicle number restrictions. This makes the target in the captured image clearer and more specific.
[0065] like Figure 3 FIG. 1 is a flow chart of another vehicle light status detection method provided by an embodiment of the present application, including:
[0066] First, a camera image of the target vehicle is fed into the BEV model to obtain the target vehicle's 3D attributes. These attributes include not only information such as the vehicle's length, width, height, and speed, but also the 3D coordinates of each vehicle's vertices. These coordinates are typically based on a 3D coordinate system established with the vehicle's center point as the origin, though other reference points can also be used. Furthermore, during this process, the camera image can be pre-processed, or filtered, to remove irrelevant objects from the image.
[0067] Furthermore, a 3D frame can be determined using the 3D coordinates. The steps for determining the 3D frame include: determining the coordinates of the center point of the ego vehicle; identifying the two vertices closest to the center point of the ego vehicle from the 3D coordinates of each vertex of the target vehicle; identifying the two other vertices closest to the two vertices from the 3D coordinates of each remaining vertices of the target vehicle; and establishing the 3D frame of the target vehicle based on the four determined vertices. This method can determine the 3D frame of the shorter side of the target vehicle (front or rear), but it cannot specifically determine whether it is the front or rear frame.
[0068] The 3D frame is first converted to a first 2D frame using the first camera parameters, i.e., those for a 120-degree field of view. If the first 2D frame cannot identify the headlight status, the 3D frame is converted to a second 2D frame using the second camera parameters, i.e., those for a 30-degree field of view and a longer viewing distance. Optionally, if the image exceeds the boundaries, image compensation is performed on the first 2D frame in combination with the original captured image to improve the quality of the first 2D frame, thereby ensuring the quality of the subsequent 2D frame input into the headlight recognition model.
[0069] After obtaining a relatively clear 2D frame, it is used as the input of the headlight recognition model. The headlight recognition model also includes the vehicle frame type of the 2D frame, which includes the front frame or the rear frame. The method of determining the vehicle frame type will not be repeated here.
[0070] Vehicle information merging refers to merging the headlight status information and other 3D attributes of the target vehicle output by the BEV, and then transmitting the information to the downstream to further predict the movement trajectory of the target vehicle.
[0071] In this embodiment, on the one hand, 2D conversion is performed by using the parameters of the second camera with a smaller field of view and a longer viewing distance, making the formed 2D frame clearer and improving recognition accuracy. On the other hand, if truncation occurs when the second camera parameters are converted to 2D, image compensation is performed on the first 2D frame combined with the captured image, making the 2D frame clearer and improving the quality of the subsequent 2D frame used for model input. This improves recognition accuracy even when the second camera parameters cannot be applied, thereby enabling recognition of vehicle headlight status at longer distances and improving the accuracy of target vehicle prediction during high-speed autonomous driving.
[0072] Further, as Figures 1 to 3 The specific implementation of the method shown in this embodiment provides a vehicle light status detection device, such as Figure 4 As shown, the device includes: an acquisition unit 401, an input unit 402, a conversion unit 403, a judgment unit 404 and a determination unit 405.
[0073] The acquisition unit 401 is configured to acquire a captured image; the captured image includes a target vehicle;
[0074] An input unit 402 is configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and determine a 3D frame of the target vehicle using the 3D coordinates;
[0075] a conversion unit 403 configured to convert the 3D frame into a first 2D frame based on a first camera parameter;
[0076] The determining unit 404 is configured to, if the headlight state of the target vehicle cannot be identified using the headlight state result determined using the image corresponding to the first 2D frame, convert the 3D frame into a second 2D frame using second camera parameters, wherein the field of view of the second camera parameters is smaller than that of the first camera parameters, and the viewing distance of the second camera parameters is greater than that of the first camera parameters;
[0077] a determining unit 405 configured to determine a headlight status of the target vehicle using the image corresponding to the second 2D frame;
[0078] The vehicle light status includes left light on, right light on, both lights on, and off.
[0079] In a specific application scenario, the conversion unit 403 is specifically configured to switch to the first camera parameters if the image exceeds the image boundary based on the conversion of the second camera parameters; perform image compensation on the first 2D frame in combination with the captured image; and use the first 2D frame after image compensation to determine the headlight status of the target vehicle.
[0080] In a specific application scenario, the input unit 402 is specifically configured to use the 3D coordinates to determine the front 3D frame or the rear 3D frame of the target vehicle, including: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the 3D frame of the target vehicle based on the determined four vertices.
[0081] In a specific application scenario, the determination unit 405 is further configured to use the driving direction vector of the target vehicle to determine the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame; the image corresponding to the 2D frame and the vehicle frame type are input into the headlight recognition model to determine the headlight status of the target vehicle.
[0082] In a specific application scenario, the determination unit 405 is further configured to obtain the center point coordinates of the target vehicle; establish a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; obtain a second vector of the driving direction of the target vehicle; and determine the vehicle frame type of the 2D frame based on the first vector and the second vector.
[0083] In a specific application scenario, the determination unit 405 is further configured to determine that the vehicle frame type is a front frame when the angle between the first vector and the second vector is less than 90 degrees; and to determine that the vehicle frame type is a rear frame when the angle between the first vector and the second vector is greater than 90 degrees.
[0084] In a specific application scenario, the determination unit 405 is further configured to determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the front frame; determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the front frame; and determine that the vehicle's headlight status is consistent with the output of the headlight recognition model when the output of the headlight recognition model is both on or off.
[0085] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle light status detection device provided in this embodiment, please refer to Figures 1 to 3 The corresponding description in will not be repeated here.
[0086] Based on the above Figures 1 to 3 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can realize the above-mentioned steps. Figures 1 to 3 The method shown.
[0087] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.
[0088] Based on the above Figures 1 to 3 The method shown, and Figure 4In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device that can be configured on a computer terminal side or a vehicle terminal side, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figures 1 to 3 The method shown.
[0089] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: Figure 4 The device shown or the electronic device as described above. The vehicle can be a new energy vehicle or a traditional vehicle.
[0090] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and may optionally include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.
[0091] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.
[0092] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.
[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. Applying the solution of this embodiment, first obtain a captured image containing the target vehicle, input the captured image into the BEV model, obtain the 3D coordinates of each vertex of the target vehicle, and use the 3D coordinates to determine the 3D frame of the target vehicle, and based on the first camera parameters, convert the 3D frame into a first 2D frame. In the case that the light status result determined by the image corresponding to the first 2D frame cannot identify the light status of the target vehicle, the 3D frame is converted into a second 2D frame using the second camera parameters, and then the light status of the target vehicle is determined using the image corresponding to the second 2D frame. Among them, the field of view angle of the second camera parameter is smaller than that of the first camera parameter. Compared to the related art solution where conventional camera parameters are used to project a vehicle at a distance, resulting in a small and blurry 2D frame, this embodiment determines the headlight status determined by the first 2D frame. If the headlight status cannot be identified, a second camera parameter with a smaller field of view is used to project a second 2D frame. The headlight status of the target vehicle is then determined based on the second 2D frame, thereby enabling identification of the headlight status at a greater distance and improving the accuracy of target vehicle prediction during high-speed autonomous driving.
[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.
[0095] The above description is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features of the present application.
[0096] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replaced with portions and features of other embodiments. As used in this application, the term "and / or" means including any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or device that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar parts between the embodiments can be referenced. For methods, devices, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be referenced to the description of the method part.
[0097] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0098] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0099] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.
Claims
1. A vehicle light status detection method, characterized in that: include: Acquire the collected image; The captured image contains a target vehicle; Inputting the captured image into a BEV model to obtain 3D coordinates of each vertex of the target vehicle, and determining a 3D frame of the target vehicle using the 3D coordinates; Based on first camera parameters, converting the 3D frame into a first 2D frame; If the headlight state of the target vehicle cannot be identified using the headlight state result determined using the image corresponding to the first 2D frame, converting the 3D frame into a second 2D frame using second camera parameters; the field of view of the second camera parameters is smaller than that of the first camera parameters; Determining the headlight status of the target vehicle using the image corresponding to the second 2D frame; The vehicle light status includes left light on, right light on, both lights on, and off.
2. The method according to claim 1, characterized in that After converting the 3D frame into a second 2D frame using the second camera parameters, the method further includes: If the second 2D frame converted based on the second camera parameters exceeds the image boundary, switching to the first camera parameters; performing image compensation on the first 2D frame in combination with the acquired image; The first 2D frame after image compensation is used to determine the headlight status of the target vehicle.
3. The method according to claim 1, characterized in that The 3D frame includes a front 3D frame and a rear 3D frame; Determining the front 3D frame or the rear 3D frame of the target vehicle using the 3D coordinates includes: Determine the coordinates of the vehicle center point; Among the 3D coordinates of the vertices of the target vehicle, determine the two vertices closest to the coordinates of the center point of the vehicle; Among the remaining 3D coordinates of the vertices of the target vehicle, determine the other two vertices that are closest to the two vertices; A 3D box of the target vehicle is established based on the determined four vertices.
4. The method according to claim 1, wherein Determining the headlight status of the target vehicle using the image corresponding to the 2D frame includes: Using the driving direction vector of the target vehicle, determining the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame; The image corresponding to the 2D frame and the vehicle frame type are input into a vehicle light recognition model to determine the vehicle light status of the target vehicle.
5. The method according to claim 4, characterized in that The determining the vehicle frame type of the 2D frame by using the driving direction vector of the target vehicle includes: Obtaining the center point coordinates of the target vehicle; Establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; Obtaining a second vector of the target vehicle's driving direction; Determining a vehicle frame type of the 2D frame based on the first vector and the second vector; The determining the vehicle frame type of the 2D frame based on the first vector and the second vector includes: When the angle between the first vector and the second vector is less than 90 degrees, it is determined that the vehicle frame type is a head frame; When the included angle between the first vector and the second vector is greater than 90 degrees, it is determined that the vehicle frame type is a tail frame.
6. The method according to claim 4, characterized in that Inputting the image corresponding to the 2D frame and the vehicle frame type into a vehicle light recognition model to determine the vehicle light state, including: When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a headlight frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, it is determined that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is double bright or off, it is determined that the vehicle light state is consistent with the output of the vehicle light recognition model.
7. A vehicle light status detection device, characterized in that: include: An acquisition unit is configured to acquire an image; The captured image contains a target vehicle; an input unit configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and determine a 3D frame of the target vehicle using the 3D coordinates; a conversion unit configured to convert the 3D frame into a first 2D frame based on a first camera parameter; a determination unit configured to, when the light state of the target vehicle cannot be identified by using the light state result determined using the image corresponding to the first 2D frame, convert the 3D frame into a second 2D frame using second camera parameters; the field of view of the second camera parameters being smaller than that of the first camera parameters; a determining unit configured to determine a headlight status of the target vehicle using the image corresponding to the second 2D frame; The vehicle light status includes left light on, right light on, both lights on, and off.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
10. A vehicle, characterized in that: include: The apparatus according to claim 7, or the electronic device according to claim 9.