Vehicle lamp state detection method and device, electronic equipment and vehicle

Abstract

The invention discloses a vehicle lamp state detection method and device, electronic equipment and a vehicle, and relates to the technical field of vehicles. The method comprises the following steps: acquiring a collected image obtained by shooting a preset range by a vehicle camera; inputting the acquired image into the BEV model to obtain 3D coordinates of each vertex of the target vehicle; determining a vehicle head 3D frame or a vehicle tail 3D frame of the target vehicle by using the 3D coordinates, and converting the 3D frame into a 2D frame; judging the frame type of the 2D frame by using the driving direction vector of the 2D frame; and inputting the frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle lamp identification model, and determining the vehicle lamp state of the target vehicle. Compared with the prior art, the method has the advantages that the frame is determined through the 3D coordinates of the vehicle and converted into the 2D frame, and then the frame type is judged. Finally, the vehicle lamp state is determined through the vehicle frame type and the image corresponding to the 2D frame, the method for determining the vehicle light state based on the 3D attribute output by the BEV model is achieved, and the problem that vehicle lamp state information obtained through the model is not accurate at present is solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a method, device, electronic equipment and vehicle for detecting vehicle light status. Background Art

[0002] Current intelligent driving technology requires collecting various information about the vehicle's driving environment and using this information to inform vehicle control decisions. This includes obtaining information about the headlight status of other vehicles. The current method for obtaining this information is to input a vehicle image into a recognition model, which then outputs the headlight status information. During this process, the recognition model must combine the vehicle image to determine whether the lights in the image are front or rear lights before obtaining the corresponding headlight status information. However, due to the large number of front and rear vehicle styles, and the fact that some front and rear images are not very different, training data collection is difficult, model processing takes a long time, and the model output is inaccurate. Therefore, how to obtain headlight status information while avoiding these issues has become a pressing issue. Summary of the Invention

[0003] In view of this, the present application provides a vehicle light status detection method, device, electronic device and vehicle, which can improve the problem that the vehicle light status information currently obtained through the model is not accurate enough.

[0004] In a first aspect, the present application provides a method for detecting a vehicle light status, comprising:

[0005] Obtaining a captured image captured by the vehicle's camera within a preset range; wherein the captured image includes the target vehicle;

[0006] Inputting the collected image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle;

[0007] Determine the front 3D frame or the rear 3D frame of the target vehicle using the 3D coordinates, and convert the 3D frame into a 2D frame;

[0008] Using the driving direction vector of the target vehicle, determining the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame;

[0009] Inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light status of the target vehicle;

[0010] The vehicle light status includes left light on, right light on, both lights on, and off.

[0011] Optionally, using the 3D coordinates to determine the front 3D frame or the rear 3D frame of the target vehicle and converting the 3D frame into a 2D frame includes: determining the coordinates of the center point of the vehicle; determining two vertices closest to the coordinates of the center point of the vehicle from the 3D coordinates of each vertex of the target vehicle; determining two other vertices closest to the two vertices from the 3D coordinates of the remaining vertices of the target vehicle; and establishing the front 3D frame or the rear 3D frame of the target vehicle based on the determined four vertices;

[0012] The converting of the 3D frame into a 2D frame includes: performing coordinate transformation and dedistortion processing on vertex coordinates of the 3D frame based on internal parameters of the vehicle camera to obtain a 2D frame.

[0013] Optionally, using the driving direction vector of the target vehicle to determine the vehicle frame type of the 2D frame includes: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; obtaining a second vector of the driving direction of the target vehicle; and determining the vehicle frame type of the 2D frame based on the first vector and the second vector.

[0014] Optionally, determining the vehicle frame type of the 2D frame based on the first vector and the second vector includes: when the angle between the first vector and the second vector is less than 90 degrees, determining that the vehicle frame type is a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, determining that the vehicle frame type is a rear frame.

[0015] Optionally, the vehicle frame type of the 2D frame and the image corresponding to the 2D frame are input into a vehicle light recognition model to determine the vehicle light status of the target vehicle, including: when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a rear frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the right light on; when the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, then the vehicle light status is determined to be the left light on; when the output of the vehicle light recognition model is that both lights are on or off, then the vehicle light status is determined to be consistent with the output of the vehicle light recognition model.

[0016] Optionally, before inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into the vehicle light recognition model, the method further includes: performing image compensation on the 2D frame in combination with the collected image.

[0017] In a second aspect, the present application provides a vehicle light status detection device, comprising:

[0018] An acquisition unit is configured to acquire an image captured by a camera of the vehicle within a preset range; the image contains the target vehicle;

[0019] an input unit configured to input the collected image into a BEV model to obtain 3D coordinates of each vertex of the target vehicle;

[0020] a conversion unit configured to determine a front 3D frame or a rear 3D frame of the target vehicle using the 3D coordinates, and convert the 3D frame into a 2D frame;

[0021] a judgment unit configured to judge the vehicle frame type of the 2D frame using the driving direction vector of the target vehicle; the vehicle frame type includes a front frame or a rear frame;

[0022] a determination unit configured to input the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light state of the target vehicle;

[0023] The vehicle light status includes left light on, right light on, both lights on, and off.

[0024] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle light status detection method described in the first aspect.

[0025] In a fourth aspect, the present application provides an electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the vehicle light status detection method described in the first aspect when executing the computer program.

[0026] In a fifth aspect, the present application provides a vehicle, comprising the vehicle light status detection device as mentioned in the second aspect or the electronic device as mentioned in the fourth aspect.

[0027] By means of the above-mentioned technical solution, the present application provides a vehicle light status detection method, device, electronic device, and vehicle. The method first obtains a captured image obtained by capturing a preset range with the vehicle's camera. The captured image includes the target vehicle for behavior prediction or light detection. The captured image is then input into a BEV model to obtain the 3D coordinates of each vertex of the target vehicle. The 3D coordinates are used to determine the front 3D frame or rear 3D frame of the target vehicle and convert it into a 2D frame. The driving direction vector is then used to determine the vehicle frame type within the 2D frame: front or rear. Finally, the image corresponding to the obtained vehicle frame type and 2D frame is input into a vehicle light recognition model to determine whether the vehicle light status is left on, right on, both on, or off. Compared to related technologies, the present application first determines the front 3D frame or rear 3D frame using the 3D coordinates of the vehicle output by the BEV model, then converts the image to 2D processing and determines the vehicle frame type using the driving direction vector. Finally, the vehicle frame type and 2D frame image are input into the headlight recognition model to determine the headlight status. A method for determining the vehicle lighting status based on the 3D attributes output by the BEV model is proposed to improve the problem that the headlight status information currently obtained through the model is not accurate enough.

[0028] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0030] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0031] Figure 1 A schematic diagram showing a flow chart of a vehicle light status detection method provided in an embodiment of the present application is shown;

[0032] Figure 2 A schematic diagram showing the input and input results of the light detection model provided in an embodiment of the present application is shown;

[0033] Figure 3 A schematic diagram showing a flow chart of another vehicle light status detection method provided by an embodiment of the present application is shown;

[0034] Figure 4A schematic structural diagram of a vehicle light status detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] In order to be able to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In addition, in order to be able to understand the characteristics and technical content of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The accompanying drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the sake of convenience of explanation, a full understanding of the disclosed embodiments is provided through a plurality of details. However, in the absence of these details, one or more embodiments can still be implemented. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0036] The vehicle light status detection method provided in this embodiment is applied to a vehicle light status detection device, which can be installed or integrated into an electronic control unit (ECU) in a vehicle, or installed in an independent system that communicates with the vehicle.

[0037] In target detection technology, the BEV (Bird's-Eye-View) model is usually used to detect objects. Specifically, the internal parameters of the internal geometry and optical characteristics of sensors (such as LiDAR and cameras), and the external parameters obtained by the camera shooting in a certain direction or position, are used to perform an overall model transformation to obtain a 360-degree surrounding scene. However, in the current solution, the BEV model can only identify the center position and vertices of the surrounding target vehicles, but cannot determine the front and rear information of the target vehicle and the headlight status information. The headlight status information can reflect the turning intention of the vehicle in some road conditions (such as starting, turning, etc.), thereby achieving a more accurate prediction of its route. However, in reality, due to the large number of front and rear styles of vehicles, and the fact that some images of the front and rear of vehicles are not very different, it is difficult to collect training data, the model consumes a long time to process data, and the model output results are inaccurate. In order to solve the problem that the headlight status information currently obtained by the model is not accurate enough, this embodiment proposes a headlight status detection method. Figure 1 As shown, the method includes:

[0038] S101, obtaining a captured image obtained by shooting a preset range with a vehicle camera.

[0039] The captured image includes the target vehicle. The ego vehicle camera is typically a camera installed on the vehicle itself. There can be multiple cameras, distributed across the vehicle. Each camera captures an image of its direction, thereby providing a panoramic view of the area surrounding the ego vehicle. The preset range can typically include within 30 meters, 50 meters, or 100 meters of the vehicle, depending on the specific situation. The target vehicle is the vehicle in the desired behavior prediction or lighting state detection range.

[0040] S102: Input the captured image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle.

[0041] The captured image here can be a single image or a panoramic image of the surrounding area of ​​the vehicle obtained by stitching multiple captured images. The image only needs to include the target vehicle to be predicted. The method proposed in this embodiment can also predict multiple target vehicles in the panoramic image.

[0042] The BEV model output includes the target vehicle's 3D attributes. These 3D attributes include the coordinates of each vertex (generally based on the vehicle's center point, but can also be converted to coordinates in other coordinate systems), the vehicle's length, width, height, and speed. This includes the 3D coordinates of each vertex of the target vehicle, which typically has eight vertices.

[0043] S103 , using the 3D coordinates, determining the front 3D frame or the rear 3D frame of the target vehicle, and converting the 3D frame into a 2D frame.

[0044] The 3D coordinates of the eight vehicle vertices and their distance from the vehicle can be used to determine a 3D box across the width of the vehicle. However, this 3D box cannot yet be distinguished as the front or rear of the vehicle. Therefore, a 2D conversion is required.

[0045] S104: Using the driving direction vector of the target vehicle, determine the vehicle frame type of the 2D frame.

[0046] The driving direction vector is the current direction of the target vehicle, which can be obtained through the BEV model. For example, the current direction of the target vehicle can be determined by the relative position of the target vehicle in the current image and the previous image. The driving direction vector is then used to determine the vehicle frame type of the 2D frame, that is, whether it is a front frame or a rear frame.

[0047] S105 , inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light status of the target vehicle.

[0048] After obtaining the vehicle frame type, the vehicle frame type and the image corresponding to the 2D frame can be input into the vehicle light recognition model to determine the vehicle light status. Vehicle light status includes left light on, right light on, both lights on, and off, that is, one-side light on, both-side lights on, and both-side lights off. The image corresponding to the 2D frame here refers to the vehicle front image or parking space image obtained based on the 2D frame. For example, the 2D frame can actually be a rectangle surrounded by the four sides corresponding to the vehicle front frame (or vehicle rear frame), and the corresponding vehicle front (or vehicle rear) part needs to be cut out from the image based on the 2D frame of the rectangle, so as to obtain the image corresponding to the vehicle front or vehicle rear part. This part of the image is also the image corresponding to the 2D frame (vehicle front or vehicle rear), and then the image corresponding to the 2D frame and the vehicle frame type are used as input to the model to obtain the vehicle light status.

[0049] In this embodiment, a captured image is first captured by the vehicle's camera within a preset range. The captured image contains the target vehicle for behavior prediction or light detection. The captured image is then input into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle. The 3D coordinates are used to determine the front or rear 3D bounding box of the target vehicle and convert it into a 2D bounding box. The vehicle's frame type is then determined within the 2D bounding box using the vehicle's driving direction vector. Finally, the resulting frame type and the corresponding 2D bounding box image are input into a headlight recognition model to determine the vehicle's light status. Compared to related techniques, this embodiment uses the 3D coordinates of the vehicle output by the BEV model to first determine the front or rear 3D bounding box. This is then converted to 2D processing, where the vehicle frame type is determined using the driving direction vector. Finally, the frame type and 2D bounding box image are input into the headlight recognition model to determine the vehicle's light status. This presents a method for determining vehicle light status based on the 3D attributes output by the BEV model, addressing the inaccurate light status information currently obtained using models.

[0050] In related technologies, the BEV model is widely used in target detection technology, especially in the field of vehicle driving. However, the output of the current BEV model cannot detect the status of the headlights. For example, there is a target vehicle traveling at a 45° angle to the left of the current vehicle, but there may be a sudden turn (that is, the vehicle is not traveling in a straight line parallel to the vehicle itself, and may make a temporary turn at the current moment or the next moment). In this case, if the headlight status information can be obtained, the action of the target vehicle can be predicted earlier. In this embodiment, by determining the 3D frame and converting 3D to 2D and judging the vehicle frame type, the important information of the headlight status can be output, and actual vehicle deployment can be achieved, thereby providing favorable information for vehicle prediction.

[0051] Optionally, the 3D coordinates are used to determine the front 3D frame or the rear 3D frame of the target vehicle, and the 3D frame is converted into a 2D frame, including: determining the coordinates of the center point of the vehicle; determining the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determining the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establishing the front 3D frame or the rear 3D frame of the target vehicle based on the four determined vertices.

[0052] In this embodiment, normally, only two sides of a vehicle are visible from the current perspective: a longer side face and a shorter front / rear face. The principle is to first find the two vertices closest to the vehicle's center point among the target vehicle's eight vertices (generally the two vertices where the longer side face and the shorter face intersect). Then, from the remaining six vertices, find the two closest to these two vertices, as the vehicle's width is always smaller than its length. Thus, these four vertices can be used to determine the vehicle's front or rear face, and thus accurately create a 3D box for the front or rear face.

[0053] Optionally, converting the 3D frame into a 2D frame includes: performing coordinate transformation and dedistortion processing on vertex coordinates of the 3D frame based on internal parameters of the vehicle camera to obtain the 2D frame.

[0054] In this embodiment, the 3D frame is converted to 2D through projection, and 2D coordinates are obtained by performing matrix operations on internal parameters. The matrix parameters are composed of camera intrinsic parameters. Dedistortion processing is used to address the situation of fisheye cameras or multi-image stitching. Images obtained by fisheye cameras or multi-image stitching may have some locations with relatively curved images, so dedistortion processing is required to restore the image to a normal state and improve accuracy.

[0055] Optionally, the driving direction vector of the target vehicle is used to determine the vehicle frame type of the 2D frame, including: obtaining the center point coordinates of the target vehicle; establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the own vehicle; obtaining a second vector of the driving direction of the target vehicle; and determining the vehicle frame type of the 2D frame based on the first vector and the second vector.

[0056] In this embodiment, the image is converted into a 2D frame, which is then processed to determine the vehicle frame type. First, the coordinates of the target vehicle's center are obtained. A first vector is formed from the target vehicle's center coordinates to the vehicle's center coordinates, and the target vehicle's driving direction is the second vector. Based on the first and second vectors, the 2D frame is determined to be the front or rear frame, facilitating subsequent headlight model recognition.

[0057] Furthermore, based on the first vector and the second vector, the vehicle frame type of the 2D frame is determined, including: when the angle between the first vector and the second vector is less than 90 degrees, the vehicle frame type is determined to be a front frame; when the angle between the first vector and the second vector is greater than 90 degrees, the vehicle frame type is determined to be a rear frame.

[0058] In this embodiment, if the angle between the first vector and the second vector is less than 90 degrees, the vehicle frame type is determined to be a front frame; if the angle between the first vector and the second vector is greater than 90 degrees, the vehicle frame type is determined to be a rear frame. This facilitates subsequent vehicle light model recognition.

[0059] Optionally, the car frame type of the 2D frame and the image corresponding to the 2D frame are input into a car light recognition model to determine the car light status of the target vehicle, including: when the output of the car light recognition model is that the left light is on and the input car frame type is a rear frame, then the car light status of the vehicle is determined to be the left light on; when the output of the car light recognition model is that the right light is on and the input car frame type is a rear frame, then the car light status of the vehicle is determined to be the right light on; when the output of the car light recognition model is that the left light is on and the input car frame type is a front frame, then the car light status of the vehicle is determined to be the right light on; when the output of the car light recognition model is that the right light is on and the input car frame type is a front frame, then the car light status of the vehicle is determined to be the left light on; when the output of the car light recognition model is both on or off, then the car light status of the vehicle is determined to be consistent with the output of the car light recognition model.

[0060] In this embodiment, if Figure 2 As shown, the results of light input and output are shown. If the vehicle frame type of the 2D frame is a rear frame, the target vehicle is moving in the same direction as the vehicle itself, so the result of the model output is consistent with the actual light state of the target vehicle. If the 2D frame is a front frame, the target vehicle is moving in the opposite direction to the vehicle itself, so the result of the model output is opposite to the actual light state of the target vehicle. The double-bright state is when the left and right lights are fully turned on (such as turning on the double flash). By inputting the vehicle frame type and the corresponding image of the 2D frame into the vehicle light detection model, the light state of the target vehicle can be determined, which is beneficial to the subsequent prediction of the vehicle's movement.

[0061] It should also be noted here that the output result of the headlight recognition model in this embodiment is normal image-based recognition. After determining the vehicle frame type, a threshold inverter can be added (for example, when the vehicle frame type is a head frame, the final result is opposite to the model output result) to determine the final vehicle light state. In a feasible embodiment, the determination of the vehicle frame type can also be directly used as part of the model training, so that the final output result of the headlight recognition model can determine the vehicle light state. (For example, if the input image vehicle frame type is a head frame and the left light is on, the model directly outputs that the right light is on). This is just an explanation of the logic of determining the vehicle light state based on the vehicle frame type, and does not forcibly limit the specific implementation method.

[0062] Optionally, before inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into the vehicle light recognition model, the method further includes: performing image compensation on the 2D frame in combination with the captured image.

[0063] In this embodiment, in actual situations, after converting a 3D frame to a 2D frame, the resulting 2D frame may have poor quality, appearing small or blurry, which is not conducive to subsequent recognition. Therefore, image compensation can be performed based on the original captured image to make the 2D frame wider and clearer.

[0064] Optionally, before inputting the collected image into the BEV model, the method further includes: filtering the collected image.

[0065] In this embodiment, filtering refers to filtering out unimportant content in the captured image. For example, if the primary detection target is a vehicle, non-vehicle objects can be filtered out based on distance restrictions or vehicle number restrictions. This makes the target in the captured image clearer and more specific.

[0066] like Figure 3 FIG. 1 is a flow chart of another vehicle light status detection method provided by an embodiment of the present application, including:

[0067] Camera image input is the image of the surrounding environment captured by the vehicle's own camera, which contains the target vehicle.

[0068] The BEV model is then input to obtain the 3D attributes of the target vehicle. These attributes include not only information such as the length, width, height, and speed of the target vehicle, but also the 3D coordinates of each vehicle vertex. These coordinates are typically based on a 3D coordinate system with the center point of the vehicle as the origin, but other reference points can also be used to establish the coordinate system. Furthermore, the camera image can be pre-processed with the headlight model, i.e., filtered, to remove irrelevant objects from the image.

[0069] After the 3D attributes are output, they are preprocessed and then fed into the headlight model. Specifically, the preprocessing includes: creating a 3D frame of the front or rear of the vehicle based on the 3D coordinates, converting it into a 2D frame; determining whether the 2D frame is the front or rear frame based on the driving direction vector; and then feeding the frame type and 2D frame into the headlight model to obtain the headlight status information.

[0070] The 2D frame may be combined with the original captured image to perform image compensation, so as to make the effect of the 2D frame more accurate.

[0071] Vehicle information merging refers to merging the headlight status information and other 3D attributes of the target vehicle output by the BEV, and then transmitting the information to the downstream to further predict the movement trajectory of the target vehicle.

[0072] In this embodiment, the 3D coordinates of the vehicle output by the BEV model are first used to determine the front or rear 3D frame. This is then converted to 2D processing, and the vehicle frame type is determined based on the driving direction vector. Finally, the frame type and 2D frame are input into a headlight recognition model to determine the headlight status. This proposed method for determining vehicle headlight status based on the 3D attributes output by the BEV model addresses the current problem of being unable to determine headlight status information based solely on the BEV model, resulting in poor route prediction for the target vehicle.

[0073] Further, as Figures 1 to 3 The specific implementation of the method shown in this embodiment provides a vehicle light status detection device, such as Figure 4 As shown, the device includes: an acquisition unit 401, an input unit 402, a conversion unit 403, a judgment unit 404 and a determination unit 405.

[0074] The acquisition unit 401 is configured to acquire an image captured by the vehicle camera within a preset range; the image contains the target vehicle;

[0075] An input unit 402 is configured to input the captured image into a BEV model to obtain 3D coordinates of each vertex of the target vehicle;

[0076] The conversion unit 403 is configured to determine the front 3D frame or the rear 3D frame of the target vehicle using the 3D coordinates, and convert the 3D frame into a 2D frame;

[0077] The judging unit 404 is configured to judge the vehicle frame type of the 2D frame using the driving direction vector of the target vehicle; the vehicle frame type includes a front frame or a rear frame;

[0078] a determination unit 405 configured to input the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light status of the target vehicle;

[0079] The vehicle light status includes left light on, right light on, both lights on, and off.

[0080] In a specific application scenario, the conversion unit 403 is specifically configured to determine the coordinates of the center point of the vehicle; determine the two vertices closest to the coordinates of the center point of the vehicle among the 3D coordinates of each vertex of the target vehicle; determine the other two vertices closest to the two vertices among the 3D coordinates of the remaining vertices of the target vehicle; and establish a 3D frame of the front or rear of the target vehicle based on the determined four vertices.

[0081] In a specific application scenario, the conversion unit 403 is specifically configured to perform coordinate transformation and dedistortion processing on the vertex coordinates of the 3D frame based on the internal parameters of the vehicle camera to obtain a 2D frame.

[0082] In a specific application scenario, the judgment unit 404 is further configured to obtain the center point coordinates of the target vehicle; establish a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; obtain a second vector of the driving direction of the target vehicle; and determine the vehicle frame type of the 2D frame based on the first vector and the second vector.

[0083] In a specific application scenario, the judgment unit 404 is further configured to determine that the vehicle frame type is a front frame when the angle between the first vector and the second vector is less than 90 degrees; and to determine that the vehicle frame type is a rear frame when the angle between the first vector and the second vector is greater than 90 degrees.

[0084] In a specific application scenario, the determination unit 405 is further configured to determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the tail frame; determine that the vehicle's headlight status is the right light on when the output of the headlight recognition model is the left light on and the input vehicle frame type is the front frame; determine that the vehicle's headlight status is the left light on when the output of the headlight recognition model is the right light on and the input vehicle frame type is the front frame; and determine that the vehicle's headlight status is consistent with the output of the headlight recognition model when the output of the headlight recognition model is both on or off.

[0085] In a specific application scenario, the input unit 402 is further configured to perform image compensation by combining the 2D frame with the captured image.

[0086] It should be noted that for other corresponding descriptions of the functional units involved in the vehicle light status detection device provided in this embodiment, please refer to Figures 1 to 3 The corresponding description in will not be repeated here.

[0087] Based on the above Figures 1 to 3 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned steps are performed. Figures 1 to 3 The method shown.

[0088] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0089] Based on the above Figures 1 to 3 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device that can be configured on a computer terminal side or a vehicle terminal side, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figures 1 to 3 The method shown.

[0090] Based on the above electronic device, the embodiment of the present application further provides a vehicle, which may specifically include: Figure 4 The device shown or the electronic device as described above. The vehicle can be a new energy vehicle or a traditional vehicle.

[0091] Optionally, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and may optionally include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface), etc.

[0092] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0093] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0094] Through the above description of the embodiments, those skilled in the art will clearly understand that this application can be implemented using software plus the necessary general-purpose hardware platform, or it can be implemented using hardware. Using the solution of this embodiment, a captured image is first obtained by capturing a preset range using the vehicle's camera. The captured image includes the target vehicle for behavior prediction or light detection. The captured image is then input into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle. The 3D coordinates are used to determine the front or rear 3D frame of the target vehicle and convert it into a 2D frame. The driving direction vector is then used to determine the frame type within the 2D frame: front or rear. Finally, the obtained frame type and the 2D frame are input into a headlight recognition model to determine the headlight status. Compared to related technologies, this embodiment uses the 3D coordinates of the vehicle output by the BEV model to first determine the front or rear 3D frame, then converts the image into a 2D frame and determines the frame type using the driving direction vector. Finally, the vehicle frame type and 2D frame are input into the headlight recognition model to determine the headlight status. A method for determining the vehicle light status based on the 3D attributes output by the BEV model is proposed to improve the current problem that the headlight status information cannot be determined based on the BEV model alone, resulting in poor prediction of the target vehicle's route.

[0095] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0096] The above description is only a specific embodiment of the present application, which enables those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features of the present application.

[0097] The above description and accompanying drawings sufficiently illustrate the embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments represent only possible variations. Unless expressly required, individual components and functions are optional, and the order of operations may vary. Portions and features of some embodiments may be included in or replaced with portions and features of other embodiments. As used in this application, the term "and / or" means including any and all possible combinations of one or more associated listed items. In addition, when used in this application, the term "comprise" and its variations "comprises" and / or comprising refer to the presence of the stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or groups thereof. Without further limitation, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, or device that includes the element. In this document, each embodiment may focus on the differences from other embodiments, and similar parts between the embodiments can be referenced. For methods, devices, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can be referenced in the description of the method part.

[0098] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. The technicians will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0099] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units can be merely a logical functional division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, and can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. In addition, the functional units in the embodiments of the present disclosure may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0100] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to the embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A vehicle light status detection method, characterized in that: include: Obtaining the captured image obtained by the vehicle camera shooting a preset range; The captured image contains a target vehicle; Inputting the collected image into the BEV model to obtain the 3D coordinates of each vertex of the target vehicle; Determine the front 3D frame or the rear 3D frame of the target vehicle using the 3D coordinates, and convert the 3D frame into a 2D frame; Using the driving direction vector of the target vehicle, determining the vehicle frame type of the 2D frame; the vehicle frame type includes a front frame or a rear frame; Inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light status of the target vehicle; The vehicle light status includes left light on, right light on, both lights on, and off.

2. The method according to claim 1, characterized in that The method of determining the front 3D frame or the rear 3D frame of the target vehicle by using the 3D coordinates and converting the 3D frame into a 2D frame includes: Determine the coordinates of the vehicle center point; Among the 3D coordinates of the vertices of the target vehicle, determine the two vertices closest to the coordinates of the center point of the vehicle; Among the remaining 3D coordinates of the vertices of the target vehicle, determine the other two vertices that are closest to the two vertices; Based on the determined four vertices, a front 3D box or a rear 3D box of the target vehicle is established; The converting the 3D frame into a 2D frame includes: Based on the internal parameters of the vehicle camera, coordinate transformation and dedistortion processing are performed on the vertex coordinates of the 3D frame to obtain a 2D frame.

3. The method according to claim 1, characterized in that The determining the vehicle frame type of the 2D frame by using the driving direction vector of the target vehicle includes: Obtaining the center point coordinates of the target vehicle; Establishing a first vector from the center point coordinates of the target vehicle to the center point coordinates of the vehicle; Obtaining a second vector of the target vehicle's driving direction; A vehicle frame type of the 2D frame is determined based on the first vector and the second vector.

4. The method according to claim 3, characterized in that The determining the vehicle frame type of the 2D frame based on the first vector and the second vector includes: When the angle between the first vector and the second vector is less than 90 degrees, it is determined that the vehicle frame type is a head frame; When the included angle between the first vector and the second vector is greater than 90 degrees, it is determined that the vehicle frame type is a tail frame.

5. The method according to claim 4, characterized in that Inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light state of the target vehicle includes: When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a tail frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the left light is on and the input vehicle frame type is a headlight frame, determining that the vehicle light state is that the right light is on; When the output of the vehicle light recognition model is that the right light is on and the input vehicle frame type is a front frame, it is determined that the vehicle light state is that the left light is on; When the output of the vehicle light recognition model is double bright or off, it is determined that the vehicle light state is consistent with the output of the vehicle light recognition model.

6. The method according to claim 1, characterized in that Before inputting the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into the vehicle light recognition model, the method further includes: The 2D frame is combined with the acquired image to perform image compensation.

7. A vehicle light status detection device, characterized in that: include: An acquisition unit is configured to acquire an image captured by a camera of the vehicle within a preset range; The captured image contains a target vehicle; an input unit configured to input the collected image into a BEV model to obtain 3D coordinates of each vertex of the target vehicle; a conversion unit configured to determine a front 3D frame or a rear 3D frame of the target vehicle using the 3D coordinates, and convert the 3D frame into a 2D frame; a judgment unit configured to judge the vehicle frame type of the 2D frame by using the driving direction vector of the target vehicle; The vehicle frame type includes a front frame or a rear frame; a determination unit configured to input the vehicle frame type of the 2D frame and the image corresponding to the 2D frame into a vehicle light recognition model to determine the vehicle light state of the target vehicle; The vehicle light status includes left light on, right light on, both lights on, and off.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

10. A vehicle, characterized in that: include: The apparatus according to claim 7, or the electronic device according to claim 9.

Images