Vehicle lamp state detection method and device, electronic equipment and vehicle

Abstract

The invention discloses a vehicle lamp state detection method and device, electronic equipment and a vehicle, and relates to the technical field of vehicles. The method comprises the steps of obtaining an acquisition image; inputting the acquired image into the BEV model to obtain 3D coordinates of each vertex of the target vehicle, and determining a 3D frame of the target vehicle by using the 3D coordinates; under the condition that the distance between the target vehicle and the vehicle is within a preset distance, converting the vehicle body part 3D frame according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter; calculating the proportion of each 2D frame to the 3D frame of the whole vehicle, and taking the 2D frame with the highest proportion as a target 2D frame; and determining the vehicle lamp state of the target vehicle based on the image corresponding to the target 2D frame. Compared with the prior art, the method has the advantages that the distance between the target vehicle and the vehicle is judged, if the distance is short, the 3D frame of the vehicle body part is converted into the 2D frame through the multiple camera parameters, the 2D frame with the highest ratio of the 2D frame to the 3D frame of the whole vehicle serves as the target 2D frame, the completeness and the clarity are guaranteed to the maximum degree, and therefore the prediction accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, electronic equipment and vehicle for detecting vehicle light status. Background Art

[0002] Current intelligent driving technologies require collecting various information about the vehicle's driving environment and using this information to inform vehicle control decisions. This includes obtaining information about the headlight status of other vehicles. The current method for obtaining this information involves inputting vehicle images into a recognition model, which then outputs the headlight status information. During this process, the recognition model must combine the vehicle image to determine whether the lights in the image are headlights or taillights before obtaining the corresponding headlight status information. However, in certain situations, vehicles closer to the ego vehicle may have blind spots, making it impossible to identify the target vehicle's lighting status when attempting to overtake from the side using a single camera. For example, on a highway, a target vehicle may be attempting to overtake from the left or rear of the ego vehicle. However, in this case, the forward-facing camera or a single side camera alone cannot clearly identify its lighting status. This results in a poor prediction of the target vehicle's trajectory, making it difficult to predict its trajectory. Summary of the Invention

[0003] In view of this, the present application provides a vehicle light status detection method, device, electronic device and vehicle, which can improve the current problem that at high speeds, vehicles on the side that are closer to the vehicle may have blind spots in the field of vision, and cannot identify the light status and thus cannot further predict their travel routes, resulting in poor prediction results.

[0004] In a first aspect, the present application provides a method for detecting a vehicle light status, comprising:

[0005] Acquire a captured image; wherein the captured image contains a target vehicle;

[0006] Inputting the captured image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle, and using the 3D coordinates to determine the 3D frame of the target vehicle; the 3D frame includes a body part 3D frame and a full vehicle 3D frame; the body part 3D frame includes a front 3D frame and / or a rear 3D frame;

[0007] When the distance between the target vehicle and the vehicle is within a preset distance, the 3D frame of the vehicle body is transformed according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter;

[0008] Calculate the ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as the target 2D frame;

[0009] Based on the image corresponding to the target 2D frame, the headlight status of the target vehicle is determined.

[0010] Optionally, after determining the 3D frame of the target vehicle using the 3D coordinates, the method further includes: when the distance between the target vehicle and the own vehicle is not within a preset distance, converting the 3D frame of the vehicle body part into a 2D frame based on default camera parameters; using the 2D frame corresponding to the default camera parameters as the target 2D frame, and determining the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0011] Optionally, the 3D coordinates are used to determine the front 3D frame or the rear 3D frame of the target vehicle, including: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the front 3D frame or the rear 3D frame of the target vehicle based on the determined four vertices.

[0012] Optionally, determining the headlight status of the target vehicle based on the image corresponding to the target 2D frame includes: using the driving direction vector of the target vehicle to judge the frame type of the target 2D frame; the frame type includes a front frame or a rear frame; inputting the image corresponding to the target 2D frame and the frame type into a headlight recognition model to determine the headlight status of the target vehicle.

[0013] Optionally, determining the vehicle frame type of the target 2D frame by using the driving direction vector of the target vehicle includes: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the own vehicle; obtaining a second vector of the driving direction of the target vehicle; and determining the vehicle frame type of the target 2D frame based on the first vector and the second vector.

[0014] The determining of the vehicle frame type of the target 2D frame based on the first vector and the second vector includes: when the angle between the first vector and the second vector is less than 90 degrees, determining that the vehicle frame type is a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, determining that the vehicle frame type is a rear frame.

[0015] Optionally, the image corresponding to the target 2D frame and the vehicle frame type are input into a headlight recognition model to determine the headlight status of the target vehicle, including: when the output of the headlight recognition model is that the left light is on and the input vehicle frame type is a tail frame, then the headlight status of the vehicle is determined to be the left light on; when the output of the headlight recognition model is that the right light is on and the input vehicle frame type is a tail frame, then the headlight status of the vehicle is determined to be the right light on; when the output of the headlight recognition model is that the left light is on and the input vehicle frame type is a front frame, then the headlight status of the vehicle is determined to be the right light on; when the output of the headlight recognition model is that the right light is on and the input vehicle frame type is a front frame, then the headlight status of the vehicle is determined to be the left light on; when the output of the headlight recognition model is both on or off, then the headlight status of the vehicle is determined to be consistent with the output of the headlight recognition model.

[0016] In a second aspect, the present application provides a vehicle light status detection device, comprising:

[0017] An acquisition unit is configured to acquire an acquired image; the acquired image contains a target vehicle;

[0018] An input unit is configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and determine a 3D frame of the target vehicle using the 3D coordinates; the 3D frame includes a body portion 3D frame and a full vehicle 3D frame; the body portion 3D frame includes a front 3D frame and / or a rear 3D frame;

[0019] a judgment unit configured to, when the distance between the target vehicle and the vehicle is within a preset distance, transform the 3D frame of the vehicle body according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter;

[0020] a calculation unit configured to calculate a ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as a target 2D frame;

[0021] The determining unit is configured to determine the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0022] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle light status detection method described in the first aspect.

[0023] In a fourth aspect, the present application provides an electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle light status detection method described in the first aspect when executing the computer program.

[0024] In a fifth aspect, the present application provides a vehicle, comprising the vehicle light status detection device as mentioned in the second aspect or the electronic device as mentioned in the fourth aspect.

[0025] By means of the above technical solution, the present application provides a method, device, electronic device and vehicle for detecting the status of vehicle lights. First, a captured image containing a target vehicle is acquired, and the image is input into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle, and then the 3D frame of the body part and the 3D frame of the entire vehicle of the target vehicle are determined. Here, the 3D frame of the body part includes the front frame and / or the rear frame. Then, the distance between the target vehicle and the own vehicle is determined. If the distance between the target vehicle and the own vehicle is not within the preset distance, the front frame or the rear frame is converted according to the preset multiple camera parameters to obtain a 2D frame corresponding to each camera parameter. The ratio of each 2D frame to the 3D frame of the entire vehicle is calculated, and the 2D frame with the highest proportion is used as the target 2D frame. Finally, based on the target 2D frame, the status of the headlights of the target vehicle is determined. Compared with related technologies, this application determines the distance between the target vehicle and the vehicle itself. If the distance is close, it is considered that there may be a blind spot in the field of view, making it impossible to determine the lighting status. Therefore, it converts the 3D frame of the vehicle body into 2D through multiple camera parameters, and uses the 2D frame with the highest ratio of 2D frame to the total 3D frame of the vehicle as the target 2D frame to maximize the integrity and clarity of the target 2D frame, thereby improving prediction accuracy. This improves the current problem of vehicles close to the vehicle at high speeds having blind spots in the field of view, making it impossible to identify the lighting status and thus unable to further predict their travel path, resulting in poor prediction results.

[0026] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 A schematic diagram showing a flow chart of a vehicle light status detection method provided in an embodiment of the present application is shown;

[0030] Figure 2A schematic diagram showing the input and input results of the light detection model provided in an embodiment of the present application is shown;

[0031] Figure 3 A schematic diagram showing a flow chart of another vehicle light status detection method provided by an embodiment of the present application is shown;

[0032] Figure 4 A schematic structural diagram of a vehicle light status detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] In order to be able to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In addition, in order to be able to understand the characteristics and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through a plurality of details. However, in the absence of these details, one or more embodiments can still be implemented. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0034] The vehicle light status detection method provided in this embodiment is applied to a vehicle light status detection device, which can be installed or integrated into an electronic control unit (ECU) in a vehicle, or installed in an independent system that communicates with the vehicle.

[0035] In order to improve the problem that at high speeds, vehicles close to the vehicle on the side may have blind spots, making it impossible to identify the light status and thus unable to further predict its route, resulting in poor prediction results, this embodiment proposes a vehicle light status detection method. Figure 1 As shown, the method includes:

[0036] S101, acquiring a captured image.

[0037] The captured image includes the target vehicle. This captured image refers to the image captured by the vehicle's camera of the area surrounding the vehicle. A vehicle can typically have one or more cameras. One captures a wide-angle image of the road ahead, while multiple cameras can be set facing different directions. The target vehicle is the vehicle whose lighting status is to be detected or whose movement is to be predicted based on its lighting status.

[0038] S102 , inputting the captured image into the BEV model, obtaining the 3D coordinates of each vertex of the target vehicle, and determining the 3D frame of the target vehicle using the 3D coordinates.

[0039] The captured image here can be a single image or a panoramic image of the surrounding area of ​​the vehicle obtained by stitching multiple captured images. The image only needs to include the target vehicle to be predicted. The method proposed in this embodiment can also predict multiple target vehicles in the panoramic image.

[0040] The output of the BEV model includes the 3D attributes of the target vehicle. These 3D attributes include the coordinates of each vertex of the vehicle (generally three-dimensional coordinates established with the center point of the vehicle as the origin, which can also be converted into coordinates of other coordinate systems), the length, width, height, speed and other information of the vehicle. Among them are the 3D coordinates of each vertex of the target vehicle, and the number of vertices is generally 8. The 3D frame of the target vehicle can be determined by combining these vertex coordinates with the distance from the vehicle. In this embodiment, the 3D frame includes a body part 3D frame and a full vehicle 3D frame. The body part 3D frame includes a front frame and a rear frame. The full vehicle 3D frame is the full vehicle frame of the target vehicle surrounded by the above 8 vertices.

[0041] S103 : When the distance between the target vehicle and the vehicle is within a preset distance, the 3D frame of the vehicle body is transformed according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter.

[0042] Next, the actual distance between the target vehicle and the ego vehicle is calculated and compared with a preset distance, typically between 5 and 10 meters. This allows for a better understanding of whether the ego vehicle's camera is prone to capturing images with a "small 2D frame, with all four headlights invisible." Other values ​​can also be selected based on actual conditions.

[0043] Taking the preset distance of 10 meters as an example, if the distance between the target vehicle and the vehicle itself does not exceed 10 meters, it means that the target vehicle is very close to the vehicle itself. In this case, the 3D frame of the vehicle body is converted according to the preset multiple camera parameters to obtain the corresponding 2D frame. The reason for converting the 3D frame of the vehicle body according to the preset multiple camera parameters is that the 3D frame is converted into a 2D frame by projection, and the 2D coordinates are obtained by matrix operation using the internal parameters of the camera. The matrix parameters are composed of the camera internal parameters. Different camera parameters will result in different generated 2D frames, and then the 2D frames corresponding to different camera parameters are obtained through S103.

[0044] It should be noted that the camera parameters here can be selected from camera parameters corresponding to different field of view angles such as 30 degrees, 60 degrees, and 120 degrees. It is worth distinguishing that the idea of ​​processing the 3D frame through different camera parameters is different from the idea of ​​collecting multiple images with cameras at different shooting angles and then processing them. In this embodiment, the 3D frame of the target vehicle in the same captured image is processed through different camera parameters. As for this captured image, it can be a single image or a stitching of images taken by multiple cameras. However, due to the shooting angle, stitching processing and distance, even if the multiple stitched images are very close, there will be a situation where the lights cannot be recognized. Therefore, different camera parameters are used to obtain 2D frames under different camera parameters, and the 2D frame with the highest ratio of 2D frame to the 3D frame of the entire vehicle is used as the target 2D frame, so as to maximize the quality of the 2D frame.

[0045] S104: Calculate the ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as the target 2D frame.

[0046] Then calculate the ratio of each 2D frame to the 3D frame of the whole vehicle. Specifically, the length of each 2D frame converted from the front or rear 3D frame processed by different parameters is compared with the length of the front or rear of the whole vehicle 3D frame, and the proportion is calculated. The 2D frame with the highest proportion indicates that the generated 2D frame is more obvious and clearer. Therefore, the 2D frame with the highest proportion is selected as the target 2D frame, so that the quality of the 2D frame input to the subsequent light recognition model is the best, and the results of light recognition and behavior prediction are improved. In the specific implementation method, the 3D frame can be first projected according to different camera parameters to obtain multiple 2D frames, and then the frame length of the front frame (or rear frame) of each 2D frame is compared with the corresponding frame length of the 3D full vehicle frame, and the 2D frame with the highest proportion is used as the target 2D frame.

[0047] S105 : Determine the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0048] The target 2D frame is used to determine the headlight status of the target vehicle, thereby realizing the movement prediction of the target vehicle. The image corresponding to the 2D frame here refers to the front image or parking space image obtained based on the 2D frame. For example, the 2D frame can actually be a rectangle surrounded by the four sides corresponding to the front frame (or rear frame), and it is necessary to cut out the corresponding front (or rear) part of the vehicle in the image based on the 2D frame of the rectangle, so as to obtain the image corresponding to the front or rear part. This part of the image is also the image corresponding to the 2D frame (front or rear), and then the image corresponding to the 2D frame and the vehicle frame type are used as the input of the model to obtain the headlight status.

[0049] In this embodiment, a captured image containing a target vehicle is first acquired and fed into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle. This is then used to determine the 3D body frame and the full vehicle 3D frame of the target vehicle. The distance between the target vehicle and the ego vehicle is then determined. If the distance between the target vehicle and the ego vehicle is not within a preset range, the 3D body frame is converted according to multiple preset camera parameters to obtain a 2D frame corresponding to each camera parameter. The ratio of each 2D frame to the full vehicle 3D frame is calculated, and the 2D frame with the highest ratio is selected as the target 2D frame. Finally, based on the target 2D frame, the target vehicle's headlight status is determined. Compared to related technologies, this embodiment determines the distance between the target vehicle and the ego vehicle. If the distance is close, it is assumed that there may be a blind spot in the field of view, making it impossible to determine the lighting status. Therefore, the 3D body frame is converted to 2D using multiple camera parameters. The 2D frame with the highest 2D frame-to-full vehicle 3D frame ratio is selected as the target 2D frame. This maximizes the integrity and clarity of the target 2D frame, thereby improving prediction accuracy.

[0050] It is also necessary to emphasize the application scenario of the technical problem to be solved by the embodiments of this application. In the case of highways, when the target vehicle is close to the ego vehicle, the vehicle captured by the camera may only see a corner of the headlights, and the 2D frame obtained after processing may not be able to identify the status of the headlights. (Here, no matter how many cameras are installed on the ego vehicle, each camera may have the problem of small and unclear 2D frames after processing due to the close distance of the target vehicle.) However, this embodiment converts the 3D frame of the vehicle body into 2D through multiple camera parameters, and uses the 2D frame with the highest ratio of 2D frame to the 3D frame of the entire vehicle as the target 2D frame, so as to maximize the integrity and clarity of the target 2D frame, thereby improving the accuracy of prediction.

[0051] Optionally, after determining the 3D frame of the target vehicle using 3D coordinates, the method further includes: when the distance between the target vehicle and the own vehicle is not within a preset distance, converting the 3D frame of the vehicle body part into a 2D frame based on default camera parameters; using the 2D frame corresponding to the default camera parameters as the target 2D frame, and determining the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0052] In this embodiment, if the target vehicle is far away from the vehicle, greater than the preset distance, the 3D frame of the vehicle body is converted into a target 2D frame based on the default camera parameters and the lighting status is determined.

[0053] Optionally, the 3D coordinates are used to determine the front 3D frame or the rear 3D frame of the target vehicle, including: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the front 3D frame or the rear 3D frame of the target vehicle based on the determined four vertices.

[0054] In this embodiment, normally, only two sides of a vehicle are visible from the current perspective: a longer side face and a shorter front / rear face. The principle is to first find the two vertices closest to the vehicle's center point among the target vehicle's eight vertices (generally the two vertices where the longer side face and the shorter face intersect). Then, from the remaining six vertices, find the two closest to these two vertices, as the vehicle's width is always smaller than its length. Thus, these four vertices can be used to determine the vehicle's front or rear face, and thus accurately create a 3D box for the front or rear face.

[0055] Optionally, based on the image corresponding to the target 2D frame, the headlight status of the target vehicle is determined, including: using the driving direction vector of the target vehicle to judge the frame type of the target 2D frame; the frame type includes the front frame or the rear frame; the image corresponding to the target 2D frame and the frame type are input into the headlight recognition model to determine the headlight status of the target vehicle; wherein the headlight status includes left light on, right light on, both lights on and off.

[0056] Furthermore, the vehicle frame type of the target 2D frame is determined by using the target vehicle's driving direction vector, including: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the own vehicle; obtaining a second vector of the target vehicle's driving direction; and determining the vehicle frame type of the target 2D frame based on the first vector and the second vector;

[0057] Based on the first vector and the second vector, the vehicle frame type of the target 2D frame is determined, including: when the angle between the first vector and the second vector is less than 90 degrees, the vehicle frame type is determined to be a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, the vehicle frame type is determined to be a rear frame.

[0058] In this embodiment, the target 2D frame is converted into a 2D frame, and the vehicle frame type is determined for the target 2D frame. First, the center coordinates of the target vehicle are obtained. A first vector is formed from the center coordinates of the target vehicle to the center coordinates of the ego vehicle, and the target vehicle's driving direction is the second vector. Based on the first and second vectors, the target 2D frame is determined to be the front or rear frame, facilitating subsequent headlight model recognition.

[0059] Furthermore, if the angle between the first and second vectors is less than 90 degrees, the vehicle frame type is determined to be a front frame; if the angle between the first and second vectors is greater than 90 degrees, the vehicle frame type is determined to be a rear frame. This facilitates subsequent input of the vehicle headlight model recognition.

[0060] Optionally, the image and vehicle frame type corresponding to the target 2D frame are input into a vehicle light recognition model to determine the vehicle light status of the target vehicle, including: when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is both on or off, then the vehicle light status is determined to be consistent with the output of the vehicle light recognition model.

[0061] In this embodiment, if Figure 2 As shown in the figure, the results of light input and output are shown. If the 2D frame is a rear frame, the target vehicle is traveling in the same direction as the vehicle itself, and the model output is consistent with the actual light state of the target vehicle. If the 2D frame is a front frame, the target vehicle is traveling in the opposite direction to the vehicle itself, and the model output is opposite to the actual light state of the target vehicle. The double-bright state is when both left and right lights are fully turned on (such as turning on the double flash). By inputting the vehicle frame type and 2D frame into the vehicle light detection model, the light state of the target vehicle can be determined, which is beneficial for the subsequent prediction of vehicle movement.

[0062] Optionally, before inputting the target 2D frame and the vehicle frame type into the vehicle light recognition model, the method further includes: performing image compensation on the target 2D frame in combination with the captured image.

[0063] In this embodiment, in actual situations, after converting a 3D frame to a 2D frame, the resulting 2D frame may have poor quality, appearing small or blurry, which is not conducive to subsequent recognition. Therefore, image compensation can be performed based on the original captured image to make the 2D frame wider and clearer.

[0064] Optionally, before inputting the collected image into the BEV model, the method further includes: filtering the collected image.

[0065] In this embodiment, filtering refers to filtering out unimportant content in the captured image. For example, if the primary detection target is a vehicle, non-vehicle objects can be filtered out based on distance restrictions or vehicle number restrictions. This makes the target in the captured image clearer and more specific.

[0066] like Figure 3 As shown, taking the preset distance of 10 meters as an example, a flow chart of another vehicle light status detection method provided by an embodiment of the present application is shown, including:

[0067] First, a camera image of the target vehicle is fed into the BEV model to obtain the target vehicle's 3D attributes. These attributes include not only information such as the vehicle's length, width, height, and speed, but also the 3D coordinates of each vehicle's vertices. These coordinates are typically based on a 3D coordinate system established with the vehicle's center point as the origin, though other reference points can also be used. Furthermore, during this process, the camera image can be pre-processed, or filtered, to remove irrelevant objects from the image.

[0068] Furthermore, the 3D coordinates can be used to determine the 3D frame of the body part and the 3D frame of the entire vehicle. The steps for confirming the 3D frame of the body part include: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the 3D frame of the target vehicle based on the four determined vertices. This method can confirm the 3D frame of the body part of the target vehicle on the shorter side (front or rear), but it cannot confirm whether it is the front frame or the rear frame. In addition, the coordinates of the eight vertices of the vehicle obtained above can be used to obtain the 3D frame of the entire vehicle of the target vehicle.

[0069] Then determine whether the distance between the target vehicle and the vehicle is within 10 meters. If not, directly use the default camera (usually the front-view camera) parameters for projection to obtain a 2D frame and then input it into the headlight recognition model.

[0070] If the distance between the target vehicle and the ego vehicle is within 10 meters, it means that the target vehicle is relatively close. At this time, based on different camera parameters (for example, the camera parameters of each camera set on the ego vehicle), the corresponding 2D frame is projected. Then, the length of each 2D frame is compared with the length of the front or rear of the vehicle in the 3D frame of the entire vehicle (since the headlights cannot be recognized in the front most of the time when overtaking, the rear of the vehicle is used as an example here). The 2D frame with the largest proportion is selected as the target 2D frame, thereby ensuring the quality of the 2D frame subsequently input into the headlight recognition model.

[0071] After obtaining a relatively clear 2D frame, it is used as the input of the headlight recognition model. The headlight recognition model also includes the vehicle frame type of the 2D frame, which includes the front frame or the rear frame. The method of determining the vehicle frame type will not be repeated here.

[0072] Vehicle information merging refers to merging the headlight status information and other 3D attributes of the target vehicle output by the BEV, and then transmitting the information to the downstream to further predict the movement trajectory of the target vehicle.

[0073] In this embodiment, by judging the distance between the target vehicle and the own vehicle, if the distance is close, it is considered that there may be a blind spot in the field of view, which makes it impossible to judge the lighting status. Therefore, the 3D frame of the vehicle body is converted into 2D through multiple camera parameters, and the 2D frame with the highest ratio of 2D frame to the 3D frame of the entire vehicle is used as the target 2D frame to maximize the integrity and clarity of the target 2D frame, thereby improving the prediction accuracy and solving the current problem that at high speeds, vehicles on the side that are close to the own vehicle may have blind spots in the field of view, cannot recognize the lighting status, and thus cannot further predict their travel routes, resulting in poor prediction results.

[0074] Further, as Figures 1 to 3 The specific implementation of the method shown in this embodiment provides a vehicle light status detection device, such as Figure 4 As shown, the device includes: an acquisition unit 401, an input unit 402, a judgment unit 403, a calculation unit 404 and a determination unit 405.

[0075] The acquisition unit 401 is configured to acquire a captured image; the captured image includes a target vehicle;

[0076] An input unit 402 is configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and use the 3D coordinates to determine a 3D frame of the target vehicle; the 3D frame includes a body 3D frame and a full vehicle 3D frame; the body 3D frame includes a front 3D frame and / or a rear 3D frame;

[0077] The judgment unit 403 is configured to, when the distance between the target vehicle and the vehicle is within a preset distance, transform the 3D frame of the vehicle body according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter;

[0078] The calculation unit 404 is configured to calculate the ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as the target 2D frame;

[0079] The determining unit 405 is configured to determine the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0080] In a specific application scenario, the judgment unit 403 is specifically configured to convert the 3D frame of the vehicle body part into a 2D frame based on the default camera parameters when the distance between the target vehicle and the own vehicle is not within the preset distance; use the 2D frame corresponding to the default camera parameters as the target 2D frame, and determine the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

[0081] In a specific application scenario, the input unit 402 is specifically configured to determine the coordinates of the center point of the vehicle; determine the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determine the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establish a 3D frame of the front or rear of the target vehicle based on the determined four vertices.

[0082] In a specific application scenario, the determination unit 405 is further configured to use the driving direction vector of the target vehicle to determine the vehicle frame type of the target 2D frame; the vehicle frame type includes a front frame or a rear frame; the image corresponding to the target 2D frame and the vehicle frame type are input into the headlight recognition model to determine the headlight status of the target vehicle; wherein, the headlight status includes left light on, right light on, both lights on, and off.

[0083] In a specific application scenario, the determination unit 405 is further configured to obtain the center point coordinates of the target vehicle; establish a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; obtain a second vector of the driving direction of the target vehicle; and determine the vehicle frame type of the target 2D frame based on the first vector and the second vector.

[0084] In a specific application scenario, the determination unit 405 is further configured to determine that the vehicle frame type is a front frame when the angle between the first vector and the second vector is less than 90 degrees; and to determine that the vehicle frame type is a rear frame when the angle between the first vector and the second vector is greater than 90 degrees.

[0085] In a specific application scenario, the determination unit 405 is further configured to determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the front frame; determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the front frame; and determine that the vehicle's headlight status is consistent with the output of the headlight recognition model when the output of the headlight recognition model is both on or off.

[0086] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle light status detection device provided in this embodiment, please refer to Figures 1 to 3 The corresponding description in will not be repeated here.

[0087] Based on the above Figures 1 to 3 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can realize the above-mentioned steps. Figures 1 to 3 The method shown.

[0088] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0089] Based on the above Figures 1 to 3 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device that can be configured on a computer terminal side or a vehicle terminal side, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figures 1 to 3 The method shown.

[0090] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: Figure 4 The device shown or the electronic device as described above. The vehicle can be a new energy vehicle or a traditional vehicle.

[0091] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and may optionally include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0092] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0093] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0094] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. Applying the solution of this embodiment, first obtain a captured image containing the target vehicle, and input the image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle, and then determine the body part 3D frame and the whole vehicle 3D frame of the target vehicle. Then judge the distance between the target vehicle and the own vehicle. If the distance of the target vehicle and the distance of the own vehicle are not within the preset distance, the body part 3D frame is converted according to the preset multiple camera parameters to obtain a 2D frame corresponding to each camera parameter. Calculate the ratio of each 2D frame to the whole vehicle 3D frame, and use the 2D frame with the highest proportion as the target 2D frame. Finally, based on the target 2D frame, determine the headlight status of the target vehicle. Compared with related technologies, this application judges the distance between the target vehicle and the vehicle itself. If the distance is close, it is considered that there may be a blind spot in the field of view, which makes it impossible to judge the lighting status. Therefore, the 3D frame of the vehicle body is converted into 2D through multiple camera parameters, and the 2D frame with the highest ratio of 2D frame to the entire vehicle 3D frame is used as the target 2D frame to maximize the integrity and clarity of the target 2D frame, thereby improving the prediction accuracy.

[0095] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0096] The above description is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features of the present application.

[0097] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replaced with portions and features of other embodiments. As used in this application, the term "and / or" means including any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or device that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar parts between the embodiments can be referenced. For methods, devices, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be referenced in the description of the method part.

[0098] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0099] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0100] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A vehicle light status detection method, characterized in that: include: Acquire the collected image; The captured image contains a target vehicle; Inputting the captured image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle, and using the 3D coordinates to determine the 3D frame of the target vehicle; the 3D frame includes a body part 3D frame and a full vehicle 3D frame; the body part 3D frame includes a front 3D frame and / or a rear 3D frame; When the distance between the target vehicle and the vehicle is within a preset distance, the 3D frame of the vehicle body is transformed according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter; Calculate the ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as the target 2D frame; Based on the image corresponding to the target 2D frame, the headlight status of the target vehicle is determined.

2. The method according to claim 1, characterized in that After determining the 3D frame of the target vehicle using the 3D coordinates, the method further includes: When the distance between the target vehicle and the vehicle is not within a preset distance, converting the 3D frame of the vehicle body into a 2D frame based on default camera parameters; The 2D frame corresponding to the default camera parameters is used as a target 2D frame, and the headlight status of the target vehicle is determined based on the image corresponding to the target 2D frame.

3. The method according to claim 1, characterized in that Determining the front 3D frame or the rear 3D frame of the target vehicle using the 3D coordinates includes: Determine the coordinates of the vehicle center point; Among the 3D coordinates of the vertices of the target vehicle, determine the two vertices closest to the coordinates of the center point of the vehicle; Among the remaining 3D coordinates of the vertices of the target vehicle, determine the other two vertices that are closest to the two vertices; Based on the determined four vertices, a front 3D box or a rear 3D box of the target vehicle is established.

4. The method according to claim 1, wherein The determining the headlight status of the target vehicle based on the image corresponding to the target 2D frame includes: Using the driving direction vector of the target vehicle, determining the vehicle frame type of the target 2D frame; the vehicle frame type includes a front frame or a rear frame; Inputting the image corresponding to the target 2D frame and the vehicle frame type into a headlight recognition model to determine the headlight status of the target vehicle; The vehicle light status includes left light on, right light on, both lights on, and off.

5. The method according to claim 4, characterized in that The determining the vehicle frame type of the target 2D frame by using the driving direction vector of the target vehicle includes: Obtaining the center point coordinates of the target vehicle; Establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; Obtaining a second vector of the target vehicle's driving direction; Determining a vehicle frame type of the target 2D frame based on the first vector and the second vector; The determining, based on the first vector and the second vector, a vehicle frame type of the target 2D frame includes: When the angle between the first vector and the second vector is less than 90 degrees, it is determined that the vehicle frame type is a head frame; When the included angle between the first vector and the second vector is greater than 90 degrees, it is determined that the vehicle frame type is a tail frame.

6. The method according to claim 4, characterized in that Inputting the image corresponding to the target 2D frame and the vehicle frame type into a vehicle light recognition model to determine the light state of the target vehicle, including: When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a headlight frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, it is determined that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is double bright or off, it is determined that the vehicle light state is consistent with the output of the vehicle light recognition model.

7. A vehicle light status detection device, characterized in that: include: An acquisition unit is configured to acquire an image; The captured image contains a target vehicle; An input unit is configured to input the captured image into a BEV model, obtain 3D coordinates of each vertex of the target vehicle, and determine a 3D frame of the target vehicle using the 3D coordinates; the 3D frame includes a body portion 3D frame and a full vehicle 3D frame; the body portion 3D frame includes a front 3D frame and / or a rear 3D frame; a judgment unit configured to, when the distance between the target vehicle and the vehicle is within a preset distance, transform the 3D frame of the vehicle body according to a plurality of preset camera parameters to obtain a 2D frame corresponding to each camera parameter; a calculation unit configured to calculate a ratio of each 2D frame to the entire vehicle 3D frame, and use the 2D frame with the highest ratio as a target 2D frame; The determining unit is configured to determine the headlight status of the target vehicle based on the image corresponding to the target 2D frame.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: The apparatus according to claim 7, or the electronic device according to claim 9.

Images