Method, device and electronic device for detecting lane change point of lever

Abstract

The present application provides a method, device and electronic device for detecting a lever change point. The present application directly binds the lever change point to the lane line, and directly outputs the lever change point through the target lane line model, thereby reducing complex post-processing logic and realizing the detection of the lever change point. Furthermore, the present embodiment randomly truncates the calibrated lane line whose line type category is the solid line category and randomly fills it with road paint pixels to show real and virtual changes or virtual and real changes, so as to obtain the calibrated lane line of the real and virtual change line category or the virtual and real change line category, which can quickly accumulate and supplement the calibration samples for the subsequent training of the target lane line model.

Description

Technical Field

[0001] The present application relates to intelligent driving technology, and in particular to a lever change point detection method, device and electronic equipment. Background Art

[0002] The lever change point refers to the point where the lane line changes from a dotted line to a solid line, or from a solid line to a dotted line. Currently, lane line detection is performed on the entire lane line instance (such as a dotted line or a solid line). Similarly, when classifying lane lines, the entire lane line area is extracted to classify the entire lane line (such as a solid line or a dotted line). However, this method of detecting or classifying the entire lane line cannot obtain the location information of the lever change point and cannot be used for lever change. Summary of the invention

[0003] The present application provides a lever change point detection method, device and electronic device to implement lever change point detection.

[0004] The present application provides a lever change point detection method, the method comprising:

[0005] Obtaining the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line; the line type category of any calibrated lane line is a solid line category, a dashed line category, a real-dummy change line category transformed from a solid line to a dashed line, or a dashed-to-solid change line category; the lane point type of any lane point is a dashed line point category or a solid line point category; at least one calibrated lane line whose line type category is the real-dummy change line category or the dashed-to-real change line category is obtained by performing the following processing on the calibrated lane line whose line type category is the solid line category: randomly truncating and randomly filling with road paint pixels to show real-dummy change or dashed-to-real change;

[0006] Obtain a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of a lane point on the calibrated lane line and a line type category of the lane line where the lane point is located, and the attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible;

[0007] A target lane line model is trained using the lane line type category loss, lane point visibility loss, and point set category loss and the modeling anchor point set; wherein, when calculating the point set category loss, a weight corresponding to the line type category being the real-virtual change line category or the virtual-real change line category is greater than a weight corresponding to the line type category being the solid line category or the dotted line category; and the target lane line model is used to detect a lever change point.

[0008] This embodiment provides a lever lane change point detection device, which includes:

[0009] A data unit for obtaining a line type category of each calibrated lane line and a lane point type of each lane point on each calibrated lane line; the line type category of any calibrated lane line is a solid line category, a dashed line category, a real-dummy change line category from a solid line to a dashed line, or a dashed-to-solid change line category; the lane point type of any lane point is a dashed line point category or a solid line point category; at least one calibrated lane line having a line type category of the real-dummy change line category or the dashed-to-real change line category is obtained by performing the following processing on the calibrated lane line having a line type category of the solid line category: random truncation and random filling with road paint pixels to show real-dummy change or dashed-to-real change; and,

[0010] Obtain a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of a lane point on the calibrated lane line and a line type category of the lane line where the lane point is located, and the attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible;

[0011] A training unit is used to train a target lane line model using a lane line type category loss, a lane point visibility loss, and a point set category loss and the modeling anchor point set; wherein, when calculating the point set category loss, a weight corresponding to a line type category being the real-virtual change line category or the virtual-real change line category is greater than a weight corresponding to a line type category being the solid line category or the dotted line category; and the target lane line model is used to detect a lever change point.

[0012] An embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a machine-readable storage medium;

[0013] The machine-readable storage medium stores machine-executable instructions that can be executed by the processor;

[0014] The processor is used to execute machine executable instructions to implement the above method.

[0015] It can be seen from the above technical solutions that in this application, the lever change point is directly bound to the lane line, and the lever change point is directly output through the target lane line model, which reduces the complex post-processing logic and realizes the lever change point detection.

[0016] Furthermore, this embodiment randomly cuts off the calibrated lane lines of the solid line category and randomly fills them with road paint pixels to show real and virtual changes or virtual and real changes, so as to obtain calibrated lane lines of the real and virtual change line category or the virtual and real change line category. This can quickly accumulate and supplement calibration samples for subsequent training of the target lane line model.

[0017] Furthermore, the present embodiment jointly trains the target lane line model based on the lane line type category, lane point visibility, and point set category, and when calculating the point set category loss, the corresponding weight when the line type category is a real virtual change line category, or a virtual real change line category is greater than the weight when the line type category is a solid line category or a dotted line category. This can enhance data and optimize model training, and effectively improve the accuracy of the target lane line model in detecting the lever change point. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Figure 1 A flow chart of the method provided in the embodiment of the present application;

[0020] Figure 2 A virtual-real changing effect diagram provided in the embodiment of the present application;

[0021] Figure 3 A schematic diagram of a modeling anchor point provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a model application provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the device structure provided in an embodiment of the present application;

[0024] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0026] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0027] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application and to make the above-mentioned purposes, features and advantages of the embodiments of the present application more obvious and understandable, the technical solutions in the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings.

[0028] The present application embodiment provides a method for detecting a lane change point of a lever. Figure 1 , Figure 1 A flowchart of a method provided in an embodiment of the present application. As an embodiment, the method can be applied to an electronic device. The electronic device here is, for example, a vehicle-mounted front-view camera, etc. The method can be applied to multiple road scenarios such as urban roads, highways, suburban roads, and parking lots.

[0029] like Figure 1 As shown, the process may include the following steps:

[0030] Step 101, obtaining the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line.

[0031] In this embodiment, the line type category of any calibrated lane line is a solid line category, a dotted line category, a real-dotted change line category from a solid line to a dotted line, or a dotted-dotted change line category from a dotted line to a solid line. Here, if the line type category of the lane line is a real-dotted change line category or a dotted-dotted change line category, the lane line here may be a complex line type, which will be involved below and will not be repeated here.

[0032] In this embodiment, the lane point type of any lane point is a dashed line point category or a solid line point category.

[0033] As an embodiment, at least one calibrated lane line of which line type category is a real-virtual change line category or a virtual-real change line category is obtained by performing the following processing on the calibrated lane line of which line type category is a solid line category: randomly truncating and randomly filling with road paint pixels to show real-virtual change or virtual-real change. An example will be given below and will not be repeated here.

[0034] Optionally, in this embodiment, the line type category of each calibrated lane line on the map can be determined based on the global marking information (including line type category) of each calibrated lane line. And based on the line type category of the calibrated lane line, and the starting point and the end point of the calibrated lane line, the lane point type of each lane point on the calibrated lane line can be assigned by difference. For example, if the line type category of the calibrated lane line is a real-virtual change line category or a virtual-real change line category, then according to the change from virtual to real or from real to virtual, the lane point type such as the dotted line point category or the solid line point category is assigned to each lane point on the calibrated lane line.

[0035] In the application, the line type category of most calibrated lane lines is the solid line category. In order to supplement the samples of the real-virtual change line category or the virtual-real change line category, this embodiment will also perform the following processing on at least one calibrated lane line of the solid line category: randomly truncate and randomly fill with road paint pixels to show the real-virtual change or virtual-real change, so as to obtain a calibrated lane line of the real-virtual change line category from solid line to dashed line, or obtain a calibrated lane line of the virtual-real change line category from dashed line to solid line; that is, a new sample (calibrated lane line of the real-virtual change line category or the virtual-real change line category) is obtained. Afterwards, based on the starting point and the end point of the calibrated lane line, and the line type category of the calibrated lane line, the lane point type of each lane point on the calibrated lane line is determined.

[0036] In this embodiment, the calibrated lane line of the solid line type is randomly cut off and randomly filled with the paint pixels of the inner side road surface to show the real-virtual change or the virtual-real change, for example Figure 2 The virtual-real change effect shown. This is equivalent to obtaining a calibrated lane line whose line type category is a virtual-real change line category. The position of the random cutoff is recorded, and the lane point type of at least one lane point on the calibrated lane line is updated based on the position, such as from the original solid line point category to the dotted line point category.

[0037] Step 102, obtaining a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, and the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of the lane point on the calibrated lane line and the line type category of the lane line where the lane point is located. The attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible.

[0038] In this embodiment, modeling anchor points in three-dimensional space, that is, pixel space coordinate system, are defined. The modeling anchor points here correspond to calibrated lane lines, which specifically refer to the mapping lines obtained by mapping the calibrated lane lines from the map coordinate system to the pixel space coordinate system. The modeling anchor point corresponding to any calibrated lane line is represented by a modeling anchor point representation set. The modeling anchor point representation set corresponding to any calibrated lane line includes the attribute information of the lane point on the calibrated lane line and the line type category of the lane line where the lane point is located. The attribute information of any lane point includes: coordinate position (represented by X, Z), visibility parameter (represented by V), lane point type (represented by C); Figure 3 The attribute information of each lane point is shown by way of example. In this embodiment, when the visibility parameter is a first value such as 1, it is used to indicate visibility, and when the visibility parameter is a second value such as 0, it is used to indicate non-visibility.

[0039] Step 103, using lane line type category loss, lane point visibility loss, and point set category loss and using the modeling anchor point set to train a target lane line model; wherein, when calculating the point set category loss, the corresponding weight when the line type category is the real virtual change line category, or the virtual real change line category is greater than the weight when the line type category is the solid line category or the dotted line category; the target lane line model is used to detect the lever change point.

[0040] In specific applications, in order to avoid misdetection of line type categories, for example, if the lane line of the dotted line category is painted, it is similar to the lane line of the solid line category. In order to avoid such misdetection, this embodiment combines the lane line type category loss, lane point visibility loss, and point set category loss to jointly train the target lane line model.

[0041] That is, in this embodiment, in the process of training the lane line model using the modeling anchor point set, if the lane line line type category loss, lane point visibility loss, and point set category loss determined based on the currently trained lane line model meet the corresponding loss requirements respectively, then the current lane line model is determined to be the target lane line model.

[0042] Optionally, the lane line type loss is calculated based on the calibrated line type of each calibrated lane line and the probability that the lane line model predicts that the line type of each calibrated lane line is the calibrated line type of the calibrated lane line. An example will be given below.

[0043] Optionally, the lane point visibility loss is calculated based on the visibility parameter of the lane point and the visibility result of the lane point predicted by the lane line model. An example will be given below.

[0044] Optionally, the point set category loss is determined by the category loss value weight corresponding to each calibrated lane line and the category loss value corresponding to each calibrated lane line; the category loss value corresponding to any calibrated lane line is determined by the calibrated line type category of the calibrated lane line and the line type category of the calibrated lane line predicted by the lane line model; the category loss value weight corresponding to the calibrated lane line whose line type category is the real-virtual change line category, or the virtual-real change line category is greater than the category loss value weight corresponding to the calibrated lane line whose line type category is the solid line category or the dotted line category.

[0045] For example, as an embodiment, the lane line type loss may be calculated using a focal loss function. For example, the focal loss function here may be expressed by the following formula: .in, represents the above probability, Indicates setting the balance factor to adjust the impact between positive and negative samples. Indicates setting the focus factor, which is used to adjust the weight of difficult and easy samples.

[0046] For example, as an embodiment, the visibility loss of a lane point can be determined based on the visibility parameters of the calibrated lane point, the visibility result of the lane point predicted by the lane line model, and the visibility truth mask (denoted as loc_mask).

[0047] For example, if a lane point is calibrated with a visibility parameter GT_Vis of the first value T1 (such as 1), indicating that it is visible, and GT_Vis is the second value T2, indicating that it is invisible, then the above loc_mask can be expressed as: loc_mask = [GT_Vis == T1]. That is, in this embodiment, the lane point visibility loss only focuses on the lane points with a visibility parameter of T1, which can prevent the invisible lane points from interfering with the lane lines of the dotted line category.

[0048] As an example, the lane point visibility loss may be determined using binary cross entropy loss (BCE Loss). For example, the lane point visibility loss (denoted as cates_loss) may be expressed as follows:

[0049] cates_loss = (BCEloss(pred_cates, gt_cates) * loc_mask).sum();

[0050] Among them, BCEloss represents the binary cross entropy loss, which can be expressed by the following formula: .

[0051] Here, y represents the visibility result of the lane point predicted by the lane line model (pred_cates), y^ represents the visibility parameter of the lane point (gt_cates); i represents the i-th lane point. sum() represents the total number of lane points.

[0052] For example, in this embodiment, the category loss value corresponding to each of the above-mentioned calibrated lane lines can be supervised based on the binary cross entropy loss BCE Loss. For example, the category loss value corresponding to a calibrated lane line is:

[0053] cates_loss = (BCEloss( , )).here, Indicates the line type category of the calibrated lane line predicted by the lane line model. Indicates the line type of the marked lane line.

[0054] As described above, the weight of the category loss value corresponding to the calibrated lane line whose line type category is the real virtual change line category or the virtual real change line category is greater than the weight of the category loss value corresponding to the calibrated lane line whose line type category is the solid line category or the dotted line category. For example, the weight of the category loss value corresponding to the calibrated lane line whose line type category is the real virtual change line category or the virtual real change line category is 3, and the weight of the category loss value corresponding to the calibrated lane line whose line type category is the solid line category or the dotted line category is 1. In this embodiment, when determining the point set category loss, for the calibrated lane line whose line type category is the real virtual change line category or the virtual real change line category, the weight of the category loss value corresponding to it is dynamically increased to ensure that the finally trained model accurately identifies the lever change point.

[0055] In the above manner, the target lane line model described above will eventually be trained.

[0056] Afterwards, when the target image to be detected is obtained, the target image is input into the target lane line model to obtain the coordinate position of each lane point on the lane line (for example, represented by three-dimensional coordinates x, y, z), lane point visibility parameters, and lane point type.

[0057] The coordinate position of each lane point on the same lane line, the visibility parameter of each lane point, and the lane point type of each lane point output by the target lane line model are used to determine the lever change point; wherein, when traversing each lane point on the same lane line in sequence, if a lane point whose lane point type changes is determined based on the lane point visibility parameter and the lane point type of the lane point, then the lane point is determined as the lever change point.

[0058] In this embodiment, the target lane line model further outputs the line type category of the lane line where the lane point is located; the line type category is used to verify the lever lane change point. Figure 4A schematic diagram of a target lane line model inputted into a target image is shown as an example.

[0059] Among them, checking the lever lane change point includes: when the line type category of a lane line is the real virtual change line category, or the virtual real change line category, it is determined that there is a lever lane change point on the lane line; when the line type category of the lane line is the solid line category or the dotted line category, it is determined that there is no lever lane change point on the lane line. At this time, even if the lever lane change point is determined when traversing the lane points on the same lane line in sequence, the lever lane change point is considered to be wrong.

[0060] The above describes the method provided in the embodiment of the present application. The following describes the device provided in the embodiment of the present application:

[0061] See also Figure 5 , Figure 5 This is a diagram of the device structure provided in the embodiment of the present application. Figure 5 As shown, the device may include:

[0062] A data unit for obtaining a line type category of each calibrated lane line and a lane point type of each lane point on each calibrated lane line; the line type category of any calibrated lane line is a solid line category, a dashed line category, a real-dummy change line category from a solid line to a dashed line, or a dashed-to-solid change line category; the lane point type of any lane point is a dashed line point category or a solid line point category; at least one calibrated lane line having a line type category of the real-dummy change line category or the dashed-to-real change line category is obtained by performing the following processing on the calibrated lane line having a line type category of the solid line category: random truncation and random filling with road paint pixels to show real-dummy change or dashed-to-real change; and,

[0063] Obtain a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of a lane point on the calibrated lane line and a line type category of the lane line where the lane point is located, and the attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible;

[0064] A training unit is used to train a target lane line model using a lane line type category loss, a lane point visibility loss, and a point set category loss and the modeling anchor point set; wherein, when calculating the point set category loss, a weight corresponding to a line type category being the real-virtual change line category or the virtual-real change line category is greater than a weight corresponding to a line type category being the solid line category or the dotted line category; and the target lane line model is used to detect a lever change point.

[0065] As an embodiment, obtaining the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line includes:

[0066] Obtaining the line type category of each marked lane line from the map; and determining the lane point type of each lane point on the marked lane line based on the starting point and the ending point of the marked lane line and the line type category of the marked lane line; and,

[0067] At least one calibrated lane line whose line type category is the solid line category is processed as follows: randomly truncated and randomly filled with road paint pixels to show real-virtual changes or virtual-real changes, so as to obtain a calibrated lane line whose line type category is a real-virtual change line category from a solid line to a dotted line, or to obtain a calibrated lane line whose line type category is a virtual-real change line category from a dotted line to a solid line; and based on the starting point and the ending point of the calibrated lane line, and the line type category of the calibrated lane line, determine the lane point type of each lane point on the calibrated lane line.

[0068] As an embodiment, the adopting lane line type loss, lane point visibility loss, and point set category loss and using the modeling anchor point set to train a target lane line model includes:

[0069] The lane line model is trained using the modeling anchor point set. If the lane line line type category loss, lane point visibility loss, and point set category loss determined based on the currently trained lane line model meet the corresponding loss requirements respectively, the current lane line model is determined to be the target lane line model.

[0070] The lane line type category loss is calculated based on the calibrated line type category of each calibrated lane line and the probability that the lane line model predicts that the line type category of each calibrated lane line is the calibrated line type category of the calibrated lane line.

[0071] The lane point visibility loss is calculated based on the calibrated visibility parameter of the lane point and the visibility result of the lane point predicted by the lane line model.

[0072] The point set category loss is determined by the category loss value weight corresponding to each calibrated lane line and the category loss value corresponding to each calibrated lane line; the category loss value corresponding to any calibrated lane line is determined by the calibrated line type category of the calibrated lane line and the line type category of the calibrated lane line predicted by the lane line model; the category loss value weight corresponding to the calibrated lane line whose line type category is the real-virtual change line category, or the real-virtual change line category is greater than the category loss value weight corresponding to the calibrated lane line whose line type category is the solid line category or the dotted line category.

[0073] As an embodiment, the target lane line model outputs the coordinate position of each lane point on the lane line, the lane point visibility parameter, and the lane point type after the target image to be detected is input;

[0074] The coordinate position of each lane point on the same lane line, the visibility parameter of each lane point, and the lane point type of each lane point output by the target lane line model are used to determine the lever lane change point; wherein, when traversing each lane point on the same lane line in sequence, if a lane point whose lane point type changes is determined based on the lane point visibility parameter and the lane point type of the lane point, then the lane point is determined as the lever lane change point.

[0075] As an embodiment, the target lane line model further outputs a line type category of the lane line where the lane point is located; the line type category is used to verify the lever lane change point; wherein, verifying the lever lane change point includes: when the line type category of a lane line is the real virtual change line category, or the virtual real change line category, it is determined that there is a lever lane change point on the lane line; when the line type category of the lane line is the solid line category or the dotted line category, it is determined that there is no lever lane change point on the lane line.

[0076] As an embodiment, the lane point visibility loss is determined based on the visibility parameter of the lane point calibrated, the visibility result of the lane point predicted by the lane line model, and the visibility truth mask loc_mask;

[0077] Wherein, if the visibility parameter GT_Vis of a lane point is calibrated to a first value T1, it indicates that it is visible, and if GT_Vis is a second value T2, it indicates that it is invisible, then the loc_mask is:

[0078] loc_mask = [GT_Vis == T1].

[0079] So far, completed Figure 5 Structural description of the device shown.

[0080] The present application also provides Figure 5 The hardware structure of the device shown is, for example, Figure 6 As shown, the hardware structure is Figure 6 The structure of the electronic device shown in FIG. Figure 6 As shown, the electronic device includes: a processor and a machine-readable storage medium. The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0081] Based on the same application concept as the above method, an embodiment of the present application also provides a machine-readable storage medium, on which a number of computer instructions are stored. When the computer instructions are executed by a processor, the method disclosed in the above example of the present application can be implemented.

[0082] Exemplarily, the above-mentioned machine-readable storage medium can be any electronic, magnetic, optical or other physical storage device, which can contain or store information, such as executable instructions, data, etc. For example, the machine-readable storage medium can be: RAM (Radom Access Memory), volatile memory, non-volatile memory, flash memory, storage drive (such as hard disk drive), solid state drive, any type of storage disk (such as optical disk, DVD, etc.), or similar storage medium, or a combination thereof.

[0083] The systems, devices, modules or units described in the above embodiments may be implemented by computers or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0084] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0085] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0086] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0087] Moreover, these computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0089] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for detecting a lane change point of a lever, characterized in that: The method includes: Obtaining the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line; the line type category of any calibrated lane line is a solid line category, a dashed line category, a real-dummy change line category transformed from a solid line to a dashed line, or a dashed-to-solid change line category; the lane point type of any lane point is a dashed line point category or a solid line point category; at least one calibrated lane line whose line type category is the real-dummy change line category or the dashed-to-real change line category is obtained by performing the following processing on the calibrated lane line whose line type category is the solid line category: randomly truncating and randomly filling with road paint pixels to show real-dummy change or dashed-to-real change; Obtain a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of a lane point on the calibrated lane line and a line type category of the lane line where the lane point is located, and the attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible; A target lane line model is trained using the lane line type category loss, lane point visibility loss, and point set category loss and the modeling anchor point set; wherein, when calculating the point set category loss, a weight corresponding to the line type category being the real-virtual change line category or the virtual-real change line category is greater than a weight corresponding to the line type category being the solid line category or the dotted line category; and the target lane line model is used to detect a lever change point.

2. The method according to claim 1, characterized in that The obtaining of the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line includes: Obtaining the line type category of each marked lane line from the map; and determining the lane point type of each lane point on the marked lane line based on the starting point and the ending point of the marked lane line and the line type category of the marked lane line; and, At least one calibrated lane line whose line type category is the solid line category is processed as follows: randomly truncated and randomly filled with road paint pixels to show real-virtual changes or virtual-real changes, so as to obtain a calibrated lane line whose line type category is a real-virtual change line category from a solid line to a dotted line, or to obtain a calibrated lane line whose line type category is a virtual-real change line category from a dotted line to a solid line; and based on the starting point and the ending point of the calibrated lane line, and the line type category of the calibrated lane line, determine the lane point type of each lane point on the calibrated lane line.

3. The method according to claim 1, characterized in that The method of adopting lane line type loss, lane point visibility loss, and point set category loss and using the modeling anchor point set to train a target lane line model includes: The lane line model is trained by using the modeling anchor point set. If the lane line line type loss, lane point visibility loss, and point set category loss determined based on the currently trained lane line model meet corresponding loss requirements, the current lane line model is determined to be the target lane line model; The lane line type loss is calculated based on the calibrated line type of each calibrated lane line and the probability that the lane line model predicts that the line type of each calibrated lane line is the calibrated line type of the calibrated lane line; The lane point visibility loss is calculated based on the calibrated visibility parameter of the lane point and the visibility result of the lane point predicted by the lane line model; The point set category loss is determined by the category loss value weight corresponding to each calibrated lane line and the category loss value corresponding to each calibrated lane line; the category loss value corresponding to any calibrated lane line is determined by the calibrated line type category of the calibrated lane line and the line type category of the calibrated lane line predicted by the lane line model; the category loss value weight corresponding to the calibrated lane line whose line type category is the real-virtual change line category, or the real-virtual change line category is greater than the category loss value weight corresponding to the calibrated lane line whose line type category is the solid line category or the dotted line category.

4. The method according to claim 1 or 3, characterized in that: The lane point visibility loss is determined based on the calibrated visibility parameters of the lane point, the visibility result of the lane point predicted by the lane line model, and the visibility truth mask loc_mask; Wherein, if the visibility parameter GT_Vis of a lane point is calibrated to a first value T1, it indicates that it is visible, and if GT_Vis is a second value T2, it indicates that it is invisible, then the loc_mask is: loc_mask = [GT_Vis == T1].

5. The method according to claim 1, characterized in that The target lane line model outputs the coordinate position of each lane point on the lane line, the lane point visibility parameter, and the lane point type after the target image to be detected is input; The coordinate position of each lane point on the same lane line, the visibility parameter of each lane point, and the lane point type of each lane point output by the target lane line model are used to determine the lever lane change point; wherein, when traversing each lane point on the same lane line in sequence, if a lane point whose lane point type changes is determined based on the lane point visibility parameter and the lane point type of the lane point, then the lane point is determined as the lever lane change point.

6. The method according to claim 5, characterized in that The target lane line model further outputs a line type category of the lane line where the lane point is located; the line type category is used to verify the lever lane change point; wherein, verifying the lever lane change point includes: when the line type category of a lane line is the real virtual change line category, or the virtual real change line category, it is determined that there is a lever lane change point on the lane line; when the line type category of the lane line is the solid line category or the dotted line category, it is determined that there is no lever lane change point on the lane line.

7. A lever change point detection device, characterized in that: The device includes: A data unit for obtaining a line type category of each calibrated lane line and a lane point type of each lane point on each calibrated lane line; the line type category of any calibrated lane line is a solid line category, a dashed line category, a real-dummy change line category from a solid line to a dashed line, or a dashed-to-solid change line category; the lane point type of any lane point is a dashed line point category or a solid line point category; at least one calibrated lane line having a line type category of the real-dummy change line category or the dashed-to-real change line category is obtained by performing the following processing on the calibrated lane line having a line type category of the solid line category: random truncation and random filling with road paint pixels to show real-dummy change or dashed-to-solid change; and, Obtain a modeling anchor point set; the modeling anchor point set includes a modeling anchor point representation set corresponding to each calibrated lane line, the modeling anchor point representation set corresponding to any calibrated lane line includes attribute information of a lane point on the calibrated lane line and a line type category of the lane line where the lane point is located, and the attribute information of any lane point includes: coordinate position, visibility parameter, lane point type; when the visibility parameter is a first value, it is used to indicate visible, and when the visibility parameter is a second value, it is used to indicate invisible; A training unit is used to train a target lane line model using a lane line type category loss, a lane point visibility loss, and a point set category loss and the modeling anchor point set; wherein, when calculating the point set category loss, a weight corresponding to a line type category being the real-virtual change line category or the virtual-real change line category is greater than a weight corresponding to a line type category being the solid line category or the dotted line category; and the target lane line model is used to detect a lever change point.

8. The device according to claim 7, characterized in that The obtaining of the line type category of each calibrated lane line and the lane point type of each lane point on each calibrated lane line includes: Obtaining the line type category of each marked lane line from the map; and determining the lane point type of each lane point on the marked lane line based on the starting point and the ending point of the marked lane line and the line type category of the marked lane line; and, The following processing is performed on at least one calibrated lane line whose line type category is the solid line category: randomly truncated and randomly filled with road paint pixels to show real-virtual change or virtual-real change, so as to obtain a calibrated lane line whose line type category is a real-virtual change line category from a solid line to a dashed line, or to obtain a calibrated lane line whose line type category is a virtual-real change line category from a dashed line to a solid line; and based on the starting point and the ending point of the calibrated lane line and the line type category of the calibrated lane line, the lane point type of each lane point on the calibrated lane line is determined; and / or, The method of adopting lane line type loss, lane point visibility loss, and point set category loss and using the modeling anchor point set to train a target lane line model includes: The lane line model is trained by using the modeling anchor point set. If the lane line line type loss, lane point visibility loss, and point set category loss determined based on the currently trained lane line model meet corresponding loss requirements, the current lane line model is determined to be the target lane line model; The lane line type loss is calculated based on the calibrated line type of each calibrated lane line and the probability that the lane line model predicts that the line type of each calibrated lane line is the calibrated line type of the calibrated lane line; The lane point visibility loss is calculated based on the calibrated visibility parameter of the lane point and the visibility result of the lane point predicted by the lane line model; The point set category loss is determined by the category loss value weight corresponding to each calibrated lane line and the category loss value corresponding to each calibrated lane line; the category loss value corresponding to any calibrated lane line is determined by the calibrated line type category of the calibrated lane line and the line type category of the calibrated lane line predicted by the lane line model; the category loss value weight corresponding to the calibrated lane line whose line type category is the real-virtual change line category or the virtual-real change line category is greater than the category loss value weight corresponding to the calibrated lane line whose line type category is the solid line category or the dashed line category; and / or, The target lane line model outputs the coordinate position of each lane point on the lane line, the lane point visibility parameter, and the lane point type after the target image to be detected is input; The coordinate position of each lane point on the same lane line, the visibility parameter of each lane point, and the lane point type of each lane point output by the target lane line model are used to determine the lever change point; wherein, when traversing the lane points on the same lane line in sequence, if a lane point whose lane point type changes is determined based on the lane point visibility parameter and the lane point type of the lane point, then the lane point is determined as the lever change point; and, The target lane line model further outputs a line type category of the lane line where the lane point is located; the line type category is used to verify the lever lane change point; wherein, verifying the lever lane change point includes: when the line type category of a lane line is the real virtual change line category, or the virtual real change line category, it is determined that there is a lever lane change point on the lane line; when the line type category of the lane line is the solid line category or the dotted line category, it is determined that there is no lever lane change point on the lane line.

9. The device according to claim 7 or 8, characterized in that The lane point visibility loss is determined based on the calibrated visibility parameters of the lane point, the visibility result of the lane point predicted by the lane line model, and the visibility truth mask loc_mask; Wherein, if the visibility parameter GT_Vis of a lane point is calibrated to a first value T1, it indicates that it is visible, and if GT_Vis is a second value T2, it indicates that it is invisible, then the loc_mask is: loc_mask = [GT_Vis == T1].

10. An electronic device, characterized in that: The electronic device includes: a processor and a machine-readable storage medium; The machine-readable storage medium stores machine-executable instructions that can be executed by the processor; The processor is used to execute machine executable instructions to implement any method of claims 1-6.

Images