A deep learning network training method, device, equipment and storage medium

Abstract

The application discloses a visual perception model training method and device, equipment and a storage medium, which can be used in the field of artificial intelligence. In the method, firstly, camera internal parameters, camera external parameters, a camera captured image and a labeled two-dimensional expression of a recognized target in the camera captured image are acquired; then, a predicted three-dimensional expression of the recognized target in the camera captured image is determined according to a preset visual perception model, the camera internal parameters and the camera external parameters; next, a network loss is obtained based on the predicted three-dimensional expression of the recognized target and the labeled two-dimensional expression of the recognized target in the camera captured image; and finally, the visual perception model is trained according to the network loss. Thus, the network loss can be obtained without acquiring the three-dimensional true value of the recognized target, so that the back propagation of the parameter gradient is completed in the training process, the true value-free training of the visual perception model is realized, and the visual perception model can be trained in a low-cost manner.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device and storage medium for training deep learning networks. Background Technology

[0002] With the development of artificial intelligence technology, the realization of autonomous driving functions has become possible. Lane detection, as an important part of autonomous driving perception system, has received increasing attention. Typically, a visual perception model is built through deep learning networks to directly realize three-dimensional lane visual perception.

[0003] Deep learning networks typically use the difference between ground truth and predicted values ​​as the loss function to complete the gradient backpropagation process. Currently, 3D lane line ground truth data is usually obtained by combining manually labeled 2D lane lines with radar information. This involves pre-labeling 2D lane lines in camera-captured images and then combining this with corresponding radar information to obtain the depth of points on the 2D lane lines, thus forming the 3D lane line ground truth data. However, obtaining highly accurate 3D lane line ground truth data requires extremely high-precision radar, making the cost of implementing 3D lane line visual perception high and hindering its widespread application.

[0004] Therefore, how to train visual perception models in a low-cost manner has become an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a deep learning network training method, apparatus, device, and storage medium that can train visual perception models in a low-cost manner.

[0006] The embodiments of this application disclose the following technical solutions:

[0007] In a first aspect, this application provides a method for training a visual perception model, the method comprising:

[0008] Obtain camera intrinsic parameters, camera extrinsic parameters, camera captured images, and labeled two-dimensional expressions of the targets identified in the camera captured images;

[0009] Based on the preset visual perception model, the camera intrinsic parameters, and the camera extrinsic parameters, a predicted three-dimensional expression for the target to be identified in the image captured by the camera is determined;

[0010] Based on the predicted three-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the camera-captured image, the network loss is obtained;

[0011] The visual perception model is trained based on the network loss.

[0012] Optionally, the process of obtaining the network loss based on the predicted three-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the camera-captured image includes:

[0013] Based on the predicted three-dimensional expression of the target and the preset coordinate system transformation rules, the predicted three-dimensional expression of the target is converted into a projected two-dimensional expression in the coordinate system of the image captured by the camera.

[0014] The loss function is obtained based on the difference between the projected two-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the image captured by the camera;

[0015] Calculate the network loss based on the loss function.

[0016] Optionally, the loss function obtained based on the difference between the projected two-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the camera-captured image includes:

[0017] A loss function is obtained based on the differences in coordinate values ​​and existence between the two-dimensional projection expression of the target and the two-dimensional annotation expression of the target in the camera-captured image, which constitute the target.

[0018] Optionally, the identified target includes: lane lines;

[0019] The network loss is obtained based on the predicted three-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the camera-captured image, including:

[0020] The first loss function is obtained based on the difference between the projection two-dimensional expression of the predicted three-dimensional expression of the identified target in the coordinate system of the camera-captured image and the labeled two-dimensional expression of the identified target in the camera-captured image.

[0021] Based on the predicted three-dimensional expression of the identified target, the lane width difference in the same lane is calculated to obtain the second loss function;

[0022] Based on the predicted three-dimensional expression of the identified target, the height difference between adjacent lane lines at the same distance is calculated to obtain the third loss function;

[0023] Calculate the network loss based on at least one of the first loss function, the second loss function, and the third loss function.

[0024] Optionally, the step of obtaining the lane width difference within the same lane based on the predicted three-dimensional coordinates of the identified target to obtain the second loss function includes:

[0025] Based on the predicted three-dimensional expression of the identified target, obtain any one of the standard deviation, variance, or root mean square deviation of the lane width difference in the same lane;

[0026] The second loss function is obtained based on any one of the standard deviation, variance, or root mean square deviation of the lane width difference in the same lane.

[0027] Optionally, the step of calculating the height difference between adjacent lane lines at the same distance based on the predicted three-dimensional expression of the identified target to obtain a third loss function includes:

[0028] Based on the predicted three-dimensional expression of the identified target, calculate any one of the absolute value, square, or power of the height difference between adjacent lane lines at the same distance;

[0029] The third loss function is obtained based on any one of the absolute value, square, or power of the height difference between adjacent lane lines at the same distance.

[0030] Optionally, training the visual perception model based on the network loss includes:

[0031] The parameters of the visual perception model are updated based on the network loss.

[0032] Secondly, this application provides a visual perception model training device, the device comprising: a data acquisition module, a coordinate prediction module, a loss calculation module, and a model training module;

[0033] The data acquisition module is used to acquire camera intrinsic parameters, camera extrinsic parameters, camera captured images, and labeled two-dimensional expressions of the targets identified in the camera captured images;

[0034] The coordinate prediction module is used to determine the predicted three-dimensional expression of the target in the image captured by the camera based on the preset visual perception model, the camera intrinsic parameters and the camera extrinsic parameters.

[0035] The loss calculation module is used to obtain the network loss based on the predicted three-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the image captured by the camera;

[0036] The model training module is used to train a visual perception model based on the network loss.

[0037] Thirdly, this application provides a visual perception model training device, the device comprising: a memory and a processor;

[0038] The memory is used to store program code and transmit the program code to the processor;

[0039] The processor is configured to execute the steps of the visual perception model training method described in any one of the first aspects according to the instructions in the program code.

[0040] Fourthly, this application provides a computer-readable storage medium storing computer instructions, wherein when the computer instructions are executed on a visual perception model training device, the visual perception model training device performs the steps of the visual perception model training method described in any one of the first aspects.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] This application provides a method for training a visual perception model. First, camera intrinsic parameters, camera extrinsic parameters, captured images, and labeled two-dimensional expressions of the target in the captured images are obtained. Then, based on a pre-defined visual perception model, camera intrinsic parameters, and camera extrinsic parameters, a predicted three-dimensional expression of the target in the captured images is determined. Next, based on the predicted three-dimensional expression of the target and the labeled two-dimensional expression of the target in the captured images, a network loss is obtained. Finally, the visual perception model is trained based on the network loss. Therefore, the network loss can be obtained without obtaining the ground truth value of the target, thus enabling backpropagation of parameter gradients during training and achieving ground truth-free training of the visual perception model. This allows for training of the visual perception model in a low-cost manner. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a visual perception model training method provided in this application embodiment;

[0045] Figure 2 A schematic diagram of a visual perception model training device provided in an embodiment of this application;

[0046] Figure 3 This is a structural diagram of a visual perception model training device provided in an embodiment of this application. Detailed Implementation

[0047] As described earlier, deep learning networks typically use the difference between ground truth and predicted values ​​as the loss function to complete the gradient backpropagation process. Currently, the common method to obtain 3D lane line ground truth data is to combine manually labeled 2D lane lines with radar information. This involves pre-labeling the 2D lane lines in the camera-captured image, then combining this with corresponding radar information to obtain the depth of points on the 2D lane lines, thus forming the 3D lane line ground truth data. However, obtaining relatively accurate 3D lane line ground truth data requires extremely high-precision radar, making the cost of implementing 3D lane line visual perception high and hindering its widespread application.

[0048] In view of this, this application provides a method for training a visual perception model. First, camera intrinsic parameters, camera extrinsic parameters, captured images, and labeled two-dimensional expressions of the target in the captured images are obtained. Then, based on a pre-defined visual perception model, camera intrinsic parameters, and camera extrinsic parameters, a predicted three-dimensional expression of the target in the captured images is determined. Next, based on the predicted three-dimensional expression of the target and the labeled two-dimensional expression of the target in the captured images, a network loss is obtained. Finally, the visual perception model is trained based on the network loss. Thus, the network loss can be obtained without obtaining the ground truth value of the target, thereby completing the backpropagation of parameter gradients during training and achieving ground truth-free training of the visual perception model, enabling training of the visual perception model in a low-cost manner.

[0049] The visual perception model training method, apparatus, device, and storage medium provided in this application can be used in the field of artificial intelligence. The above is only an example and does not limit the application field of the visual perception model training method, apparatus, device, and storage medium provided by this invention.

[0050] The terms “first,” “second,” “third,” and “fourth,” etc., used in this application specification, claims, and drawings are used to distinguish different objects, not to limit a specific order.

[0051] In the embodiments of this application, the terms "as an example" or "for example" are used to indicate that they are examples, illustrations, or explanations. Any embodiment or design that is described as "as an example" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of terms such as "as an example" or "for example" is intended to present the relevant concepts in a specific manner.

[0052] The terminology used in the implementation section of this application is for the purpose of explaining specific embodiments of this application only, and is not intended to limit this application.

[0053] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0054] See Figure 1 The figure illustrates a visual perception model training method provided in an embodiment of this application. The method includes:

[0055] S101: Obtain camera intrinsic parameters, camera extrinsic parameters, camera captured images, and labeled two-dimensional expressions of the targets identified in the camera captured images.

[0056] For example, camera extrinsic parameters can describe the orientation of the coordinate axes of the ground coordinate system relative to the camera coordinate axes and the position of the spatial origin in the camera coordinate system, that is, the position and orientation of the camera in the ground coordinate system; camera intrinsic parameters can describe the internal characteristics of the camera, such as the camera image center, camera focal length, and image distortion caused by the camera's own imaging characteristics; the two-dimensional expression of the target recognition in the camera-captured image can be pre-annotated in the camera-captured image, for example, a set of points in multiple two-dimensional coordinate systems can be used to represent the target recognition in the camera-captured image.

[0057] S102: Based on the preset visual perception model, camera intrinsic parameters, and camera extrinsic parameters, determine the predicted three-dimensional expression of the target to be identified in the image captured by the camera.

[0058] For example, the visual perception model can be a constructed deep learning network, which can take the form of a convolutional neural network. The parameters of each layer of the network can be predefined and initialized as a preset visual perception model. By inputting camera intrinsic parameters, camera extrinsic parameters, and camera-captured images into the visual perception model, the predicted 3D coordinates of points in the target in the camera-captured images in the ground coordinate system can be calculated through the forward inference of the network. The set of predicted 3D coordinates can be represented by a predicted 3D expression.

[0059] Optionally, the camera intrinsic parameters, camera extrinsic parameters, and camera-captured images can be preprocessed. Preprocessing may include operations such as image distortion correction, resizing, and normalization. Then, the preprocessed camera intrinsic parameters, camera extrinsic parameters, and camera-captured images are input into the visual perception model to obtain the predicted three-dimensional expression of the target in the ground coordinate system in the camera-captured images.

[0060] Taking lane lines as an example, the predicted 3D lane lines calculated by the visual perception model can be multiple. Each lane line can be represented as a set of points p on multiple lane lines. Each point p contains coordinate values ​​in three dimensions (x, y, z). The points in each set can be arranged in ascending order of their y-values. Thus, the predicted 3D expression for the target recognition in the camera-captured image can be represented as:

[0061]

[0062] Where i represents the number of predicted 3D lane lines, m i Let represent the number of points on the i-th predicted 3D lane line. Y represents the upper bound of the range of y, and a (i,y) This indicates the existence of a point with the corresponding y-value on the i-th predicted 3D lane line, with 1 indicating existence and 0 indicating non-existence. (i,y) z (i,y) Let x and z represent the x and z coordinates of the point corresponding to the y-value on the i-th predicted 3D lane line, respectively.

[0063] S103: The network loss is obtained based on the predicted 3D expression of the identified target and the labeled 2D expression of the identified target in the camera-captured image.

[0064] For example, the predicted three-dimensional expression of the target can be converted into a projected two-dimensional expression in the coordinate system of the camera-captured image using a preset coordinate system transformation rule. The preset coordinate system transformation rule can be a coordinate transformation matrix from the ground coordinate system to the coordinate system of the camera-captured image constructed using camera intrinsic and extrinsic parameters. Then, based on the difference between the projected two-dimensional expression of the target and the labeled two-dimensional expression of the target in the camera-captured image, a loss function is obtained.

[0065] Taking lane line identification as an example, the rotation matrix R, which represents the coordinate transformation matrix from the ground coordinate system to the camera coordinate system, can be obtained through methods such as Zhang Zhengyou's calibration. gc According to the rotation matrix R gc Given the camera height h, the translation matrix T is obtained using the following formula. gc :

[0066]

[0067] Therefore, any coordinate in the ground coordinate system becomes the following after transformation to the camera coordinate system:

[0068]

[0069] Among them, (x c ,y c ,z c (x) represents the coordinates in the camera coordinate system. g ,yg ,z g () represents the coordinates in the ground coordinate system.

[0070] The coordinates in the camera coordinate system multiplied by the camera intrinsic parameter K c And for z m Normalization allows the predicted 3D lane lines to be displayed in the ground coordinate system as coordinates (x, y, y). g ,y g ,z g The coordinates (u,v) are converted into projected two-dimensional coordinates (u,v) in the coordinate system where the camera captures the image, as shown below:

[0071]

[0072] By transforming the predicted 3D coordinates of all points on the predicted 3D lane line according to the above formula, the predicted 3D expression of the identified target can be converted into a projected 2D expression in the coordinate system of the camera-captured image, as shown below:

[0073]

[0074] Where h is the height of the projected two-dimensional lane line image, which is also the maximum value of the image's ordinate v, and a (i,v) This indicates the existence of a point with ordinate v on the i-th lane line in the projected two-dimensional lane lines; 1 indicates existence and 0 indicates non-existence. (i,v) This represents the x-coordinate value of the point with ordinate v on the i-th lane line in the projected two-dimensional lane line.

[0075] Therefore, the first loss function can be obtained by comparing the difference between the projected two-dimensional expression of the target and the labeled two-dimensional expression of the target in the camera-captured image. For example, the sum of the x-coordinate errors of all actual points in the projected two-dimensional expression and the labeled two-dimensional expression, as well as the sum of the existence differences of all points in the projected two-dimensional expression and the labeled two-dimensional expression, can be used as the first loss function LOSS. A It can be expressed by the following formula:

[0076]

[0077] Among them, A (i,v) This indicates the existence of a point with ordinate v on the i-th lane line in a two-dimensional lane marking system; 1 indicates existence, and 0 indicates non-existence. (i,v) This represents the x-coordinate value of the point with ordinate v on the i-th lane line in the two-dimensional lane marking.

[0078] For example, in addition to calculating the absolute value of the difference according to the above formula, the square or multiple powers of the difference can also be calculated to obtain the first loss function.

[0079] Alternatively, based on the fundamental principle that the lane width of the same lane remains basically constant, the difference in lane width within the same lane can be calculated to obtain the second loss function, LOSS. B It can be expressed by the following formula:

[0080]

[0081] Here, `std` represents the standard deviation of the elements in the set. In addition, the variance or mean squared error of the elements in the set can also be calculated to obtain a second loss function.

[0082] Alternatively, based on the basic principle that the heights of two lane lines in the same lane are roughly the same at the same distance, the height difference between adjacent lane lines at the same distance can be calculated to obtain the third loss function, LOSS. C It can be expressed by the following formula:

[0083]

[0084] For example, in addition to calculating the absolute value of the difference according to the above formula, the square or power of the difference can also be calculated to obtain the third loss function.

[0085] Optionally, at least one of the first loss function, the second loss function, or the third loss function can be used as the loss function to calculate the network loss. For example, the sum of the first loss function, the second loss function, and the third loss function can be used as the loss function LOSS, i.e.:

[0086] LOSS = LOSS A +LOSS B +LOSS C

[0087] By substituting the data into the loss function described above, the network loss can be calculated.

[0088] S104: Train the visual perception model based on the network loss.

[0089] As an example, the network loss can be backpropagated to calculate the gradients of each parameter of the visual perception model, and the parameters of the visual perception model can be updated according to the gradients, thereby achieving the training of the visual perception model.

[0090] Optionally, after updating the parameters of the visual perception model, the relationship between the actual number of parameter updates and the preset number of updates can be determined. If the actual number of updates reaches the preset number of updates, the visual perception model with updated parameters is output, and the training of the visual perception model is completed. If the actual number of updates is less than the preset number of updates, the steps S101-S104 above are repeated until the actual number of updates equals the preset number of updates, and then the visual perception model with updated parameters is output.

[0091] In this embodiment, firstly, camera intrinsic parameters, camera extrinsic parameters, camera-captured images, and labeled two-dimensional expressions of the target in the camera-captured images are obtained. Then, based on a preset visual perception model, camera intrinsic parameters, and camera extrinsic parameters, a predicted three-dimensional expression of the target in the camera-captured images is determined. Next, based on the predicted three-dimensional expression of the target and the labeled two-dimensional expression of the target in the camera-captured images, the network loss is obtained. Finally, the visual perception model is trained based on the network loss. Therefore, the network loss can be obtained without obtaining the ground truth value of the target, thus completing the backpropagation of parameter gradients during training and achieving ground truth-free training of the visual perception model, enabling training of the visual perception model in a low-cost manner.

[0092] See Figure 2 The figure is a schematic diagram of a visual perception model training device provided in an embodiment of this application. The device includes: a data acquisition module 201, a coordinate prediction module 202, a loss calculation module 203, and a model training module 204.

[0093] Data acquisition module 201: used to acquire camera intrinsic parameters, camera extrinsic parameters, camera captured images, and labeled two-dimensional expressions of the targets identified in the camera captured images.

[0094] Coordinate prediction module 202: used to determine the predicted three-dimensional expression of the target in the image captured by the camera based on the preset visual perception model, camera intrinsic parameters and camera extrinsic parameters.

[0095] Loss calculation module 203: used to obtain network loss based on the predicted three-dimensional expression of the identified target and the labeled two-dimensional expression of the identified target in the camera-captured image.

[0096] Optionally, the target to be identified can be a lane line. The loss calculation module 203 can obtain a first loss function based on the difference between the projection two-dimensional expression of the predicted three-dimensional expression of the target in the coordinate system of the camera-captured image and the labeled two-dimensional expression of the target in the camera-captured image; calculate the lane width difference of the same lane based on the predicted three-dimensional expression of the target to obtain a second loss function; calculate the height difference of adjacent lane lines at the same distance based on the predicted three-dimensional expression of the target to obtain a third loss function; and calculate the network loss according to at least one of the first loss function, the second loss function, and the third loss function.

[0097] Model training module 204: Used to train the visual perception model based on the network loss.

[0098] In this embodiment, firstly, the data acquisition module 201 acquires camera intrinsic parameters, camera extrinsic parameters, camera-captured images, and the labeled two-dimensional expression of the target in the camera-captured images. Then, the coordinate prediction module 202 determines the predicted three-dimensional expression of the target in the camera-captured images based on a preset visual perception model, camera intrinsic parameters, and camera extrinsic parameters. Next, the loss calculation module 203 obtains the network loss based on the predicted three-dimensional expression of the target and the labeled two-dimensional expression of the target in the camera-captured images. Finally, the model training module 204 trains the visual perception model based on the network loss. Thus, the network loss can be obtained without acquiring the three-dimensional ground truth of the target, thereby completing the backpropagation of parameter gradients during training and achieving ground truth-free training of the visual perception model, enabling training of the visual perception model in a low-cost manner.

[0099] See Figure 3 The figure is a structural diagram of a visual perception model training device provided in an embodiment of this application. The device includes a memory 301 and a processor 302.

[0100] Memory 301: Used to store program code and transfer program code to the processor.

[0101] Processor 302: Used to execute the steps of the above-described visual perception model training method according to the instructions in the program code.

[0102] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the visual perception model training method described in this application.

[0103] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and storage medium embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment solution according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0104] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for training a visual perception model, characterized in that, The method comprises: acquiring camera intrinsic parameters, camera extrinsic parameters, a camera captured image, and a labeled two-dimensional expression of a recognized target in the camera captured image; determining a predicted three-dimensional expression of the recognized target in the camera captured image according to a preset visual perception model, the camera intrinsic parameters, and the camera extrinsic parameters; obtaining a network loss based on the predicted three-dimensional expression of the recognized target and the labeled two-dimensional expression of the recognized target in the camera captured image; specifically, converting the predicted three-dimensional expression of the recognized target into a projected two-dimensional expression in a coordinate system of the camera captured image based on the predicted three-dimensional expression of the recognized target and a preset coordinate system conversion rule; obtaining a loss function based on coordinate value differences and existence differences of points constituting the recognized target in the projected two-dimensional expression of the recognized target and the labeled two-dimensional expression of the recognized target in the camera captured image; and calculating the network loss according to the loss function; training a visual perception model according to the network loss; the recognized target comprises a lane line; and obtaining the network loss based on the predicted three-dimensional expression of the recognized target and the labeled two-dimensional expression of the recognized target in the camera captured image comprises: obtaining a first loss function based on a difference between the projected two-dimensional expression of the predicted three-dimensional expression of the recognized target in the coordinate system of the camera captured image and the labeled two-dimensional expression of the recognized target in the camera captured image; obtaining a second loss function based on a lane width difference of a same lane calculated from the predicted three-dimensional expression of the recognized target; obtaining a third loss function based on a height difference of adjacent lane lines at a same distance calculated from the predicted three-dimensional expression of the recognized target; calculating the network loss according to the first loss function, the second loss function, and the third loss function.

2. The method of claim 1, wherein, the obtaining of the lane width difference of the same lane based on the predicted three-dimensional coordinate value of the recognized target to obtain the second loss function comprises: obtaining any one of a standard deviation, a variance, or a mean square deviation of the lane width difference of the same lane based on the predicted three-dimensional expression of the recognized target; obtaining the second loss function according to any one of the standard deviation, the variance, or the mean square deviation of the lane width difference of the same lane.

3. The method of claim 1, wherein, the obtaining of the height difference of the adjacent lane lines at the same distance based on the predicted three-dimensional expression of the recognized target to obtain the third loss function comprises: calculating any one of an absolute value, a square, or a multiple power of the height difference of the adjacent lane lines at the same distance based on the predicted three-dimensional expression of the recognized target; obtaining the third loss function according to any one of the absolute value, the square, or the multiple power of the height difference of the adjacent lane lines at the same distance.

4. The method of claim 1, wherein, the training of the visual perception model according to the network loss comprises: updating parameters of the visual perception model according to the network loss.

5. A visual perception model training apparatus, comprising: The device comprises a data acquisition module, a coordinate prediction module, a loss calculation module, and a model training module; the data acquisition module is configured to acquire camera intrinsic parameters, camera extrinsic parameters, a camera captured image, and a labeled two-dimensional expression of a recognized target in the camera captured image; The coordinate prediction module is configured to determine a predicted three-dimensional expression of a recognized target in the image captured by the camera according to a preset visual perception model, the camera intrinsic parameter, and the camera extrinsic parameter. The loss calculation module is configured to obtain a network loss based on the predicted three-dimensional expression of the recognized target and a labeled two-dimensional expression of the recognized target in the image captured by the camera. Specifically, the predicted three-dimensional expression of the recognized target is converted into a projected two-dimensional expression in a coordinate system of the image captured by the camera based on the predicted three-dimensional expression of the recognized target and a preset coordinate system conversion rule. A loss function is obtained based on a difference in coordinate values and a difference in existence of points constituting the recognized target between the projected two-dimensional expression of the recognized target and the labeled two-dimensional expression of the recognized target in the image captured by the camera. The network loss is calculated according to the loss function. The model training module is configured to train the visual perception model according to the network loss. The recognized target includes a lane line. The loss calculation module is specifically configured to: obtain a first loss function based on a difference between the projected two-dimensional expression of the predicted three-dimensional expression of the recognized target in the coordinate system of the image captured by the camera and the labeled two-dimensional expression of the recognized target in the image captured by the camera; obtain a second loss function based on a difference in lane width of the same lane calculated from the predicted three-dimensional expression of the recognized target; obtain a third loss function based on a difference in height of adjacent lane lines at the same distance calculated from the predicted three-dimensional expression of the recognized target; calculate the network loss according to the first loss function, the second loss function, and the third loss function.

6. A visual perception model training device, comprising: The device includes a memory and a processor. The memory is configured to store program code and transmit the program code to the processor. The processor is configured to execute the steps of the visual perception model training method according to the instructions in the program code.

7. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions. When the computer instructions run on the visual perception model training device, the visual perception model training device executes the steps of the visual perception model training method. The computer readable storage medium stores computer instructions. When the computer instructions run on the visual perception model training device, the visual perception model training device executes the steps of the visual perception model training method.

Images