Model training method, corner detection method and device

Abstract

The invention provides a model training method and device, and a corner detection method and device, and the method is applied to the field of machine learning, and comprises the steps: obtaining a sample image; obtaining predicted values of a plurality of angular points in the sample image predicted by the original angular point detection model; determining a true value of a target feature point on the object according to the true values of the plurality of angular points; determining a predicted value of a target feature point on the object according to the predicted values of the plurality of angular points; according to the true value of the target feature point and the predicted value of the target feature point, determining a target loss function used for describing the difference between the true value and the predicted value of the target feature point; and training according to the original loss function and the target loss function to generate a target corner detection model. By using the method, a target corner detection model with high detection accuracy can be trained, more accurate corners can be detected by using the model, and the corner detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and more specifically, to a model training method, a corner detection method and a device in the field of machine learning. Background Art

[0002] At present, when using the loss function to train the corner detection model, the L2 loss function is generally used for point-by-point regression optimization, and each point is independently regressed and optimized. There is no correlation between points. Even when the loss value of one point is high and the loss values ​​of the other points are low, the overall loss value will also become low, which will increase the difficulty of corner regression optimization. As a result, it is difficult to obtain accurate corner detection results when using the original corner detection model for corner detection. Therefore, how to improve the accuracy of corner detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a model training method, a corner point detection method and a device. The method can be used to train a target corner point detection model with high detection accuracy. The model can be used to detect more precise corner points, thereby improving the accuracy of corner point detection.

[0004] In a first aspect, a model training method is provided, the method comprising: obtaining a sample image; wherein the sample image comprises an object having an edge, the edge of the object comprises a plurality of corner points, and the corner points are marked with true values; obtaining predicted values ​​of a plurality of corner points in the sample image predicted by an original corner detection model; wherein the original corner detection model is trained based on an original loss function; the original loss function is used to describe the difference between the true value and the predicted value of the corner point; according to the true values ​​of the plurality of corner points, determining the true value of the target feature point on the object; according to the predicted values ​​of the plurality of corner points, determining the predicted value of the target feature point on the object; according to the true value of the target feature point and the predicted value of the target feature point, determining a target loss function used to describe the difference between the true value and the predicted value of the target feature point; training is performed according to the original loss function and the target loss function to generate a target corner detection model.

[0005] The above technical solution adds an additional loss function on the basis of the original loss function. The additional loss function is the target loss function. Compared with only considering the original loss function, adding the target loss function can increase the overall loss value of the target corner detection model. The increase in loss value means that the target corner detection model needs to achieve a higher prediction accuracy to meet the standard for ending training. This will directly lead to the continuous improvement of the target corner detection model during the training process, and continue to find a better combination of parameters to avoid the target corner detection model from converging to the local optimal solution too early, which is conducive to the target corner detection model eventually converging to the global optimal solution, thereby achieving higher detection accuracy. The original loss function is used to describe the difference between the true value and the predicted value of the corner point, which makes it possible to pay attention to the prediction accuracy of multiple corner points in the process of training the target corner detection model. The target loss function is used to describe the difference between the true value and the predicted value of the target feature point, which makes it possible to pay attention to the prediction accuracy of the target feature point in the process of training the target corner detection model. Therefore, by combining the original loss function and the target loss function, in the process of training the target corner detection model, we not only focus on the prediction accuracy of multiple corner points, but also focus on the prediction accuracy of target feature points. That is, in the process of training, not only can the prediction values ​​of multiple corner points gradually equal to or approach the true values ​​of multiple corner points, but also can make the prediction values ​​of feature points gradually equal to or approach the true values ​​of target feature points, thereby improving the detection accuracy of the trained target corner detection model. Finally, when using the target corner detection model for corner detection, more accurate corner points can be obtained, thereby improving the accuracy of corner detection.

[0006] In combination with the first aspect, in some possible implementations, the training according to the original loss function and the target loss function to generate the target corner detection model includes: calculating the original loss function plus the target loss function to obtain a corrected loss function; and training according to the corrected loss function to generate the target corner detection model.

[0007] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, determining the true value of the target feature point on the object based on the true values ​​of multiple corner points includes: determining the line between two non-adjacent corner points based on the true values ​​of multiple corner points to obtain the target true line; using the coordinates of the points on the target true line as the true value of the target feature point on the object; determining the predicted value of the target feature point on the object based on the predicted values ​​of multiple corner points includes: determining the line between two non-adjacent corner points based on the predicted values ​​of multiple corner points to obtain the target measured line; using the coordinates of the points on the target predicted line as the predicted value of the target feature point on the object; wherein the relative position of the true value of the target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line.

[0008] The above technical scheme determines the target real line according to the real values ​​of multiple corner points, determines the coordinates of the points of the target real line as the real values ​​of the target feature points, and determines the target predicted line according to the predicted values ​​of multiple corner points, determines the coordinates of the points of the target predicted line as the predicted values ​​of the target feature points, and the relative position of the real value of the target feature point on the target real line is consistent with the relative position of the predicted value of the target feature point on the target predicted line. In the process that the predicted value of the target feature point gradually approaches the real value of the target feature point until the predicted value of the target feature point is equal to the real value of the target feature point, the predicted values ​​of multiple corner points will also gradually approach the real values ​​of the multiple corner points until the predicted value of each corner point is equal to the real value, thereby significantly optimizing the corner point regression process.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the target real lines include multiple real lines that do not overlap with each other; the target predicted lines include multiple predicted lines that do not overlap with each other; the coordinates of the points on the target real lines are used as the real values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple real lines that do not overlap with each other as the real values ​​of the target feature points on the object; the coordinates of the points on the target predicted lines are used as the predicted values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple predicted lines that do not overlap with each other as the predicted values ​​of the target feature points on the object.

[0010] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the target loss function is determined by: calculating the absolute value of the difference between the true value of the target feature point and the predicted value of the target feature point, and obtaining the absolute value of the difference corresponding to the target feature point; and taking the square of the absolute value as the target loss function.

[0011] In combination with the first aspect and the above-mentioned implementation methods, in some possible implementation methods, the original loss function is determined by: calculating the absolute value of the difference between the true value and the predicted value of each of the corner points to obtain the absolute value of the difference corresponding to each of the multiple corner points; squaring the absolute value of the difference corresponding to each corner point, and adding the squared results of the absolute value of the difference corresponding to each corner point to obtain the sum of losses; dividing the sum of losses by the number of the multiple corner points as the original loss function.

[0012] In a second aspect, a corner point detection method is provided, which includes: acquiring an image to be detected; inputting the image to be detected into a target corner point detection model trained and generated by the model training method provided by the first aspect above, and outputting predicted values ​​of corner points in the image to be detected.

[0013] The above technical solution inputs the image to be detected into the target corner point detection model, which can detect more accurate corner points, thereby improving the accuracy of corner point detection.

[0014] In a third aspect, a model training device is provided, which includes: a first acquisition module, used to acquire a sample image; wherein the sample image includes an object with an edge, the edge of the object includes multiple corner points, and the corner points are marked with real values; a second acquisition module, used to acquire predicted values ​​of multiple corner points in the sample image predicted by an original corner detection model; wherein the original corner detection model is trained based on an original loss function; the original loss function is used to describe the difference between the real value and the predicted value of the corner point; a first determination module, used to determine the real value of the target feature point on the object according to the real value of the multiple corner points; a second determination module, used to determine the predicted value of the target feature point on the object according to the predicted value of the multiple corner points; a third determination module, used to determine a target loss function used to describe the difference between the real value and the predicted value of the target feature point according to the real value of the target feature point and the predicted value of the target feature point; a generation module, used to train according to the original loss function and the target loss function to generate a target corner detection model.

[0015] In combination with the third aspect, in certain implementations of the third aspect, the generation module is specifically used to: perform training according to the original loss function and the target loss function to generate a target corner detection model, including: calculating the corrected loss function obtained by adding the original loss function to the target loss function; and perform training according to the corrected loss function to generate the target corner detection model.

[0016] In combination with the third aspect and the above-mentioned implementation methods, in some implementation methods of the third aspect, the first determination module is specifically used to: determine the true value of the target feature point on the object based on the true values ​​of the multiple corner points, including: determine the line between two non-adjacent corner points based on the true values ​​of the multiple corner points to obtain the target true line; use the coordinates of the points on the target true line as the true value of the target feature point on the object; the second determination module is specifically used to: determine the predicted value of the target feature point on the object based on the predicted value of the multiple corner points, including: determine the line between two non-adjacent corner points based on the predicted values ​​of the multiple corner points to obtain the target predicted line; use the coordinates of the points on the target predicted line as the predicted value of the target feature point on the object; wherein the relative position of the true value of the target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line.

[0017] In combination with the third aspect and the above-mentioned implementation methods, in some implementation methods of the third aspect, the target real lines include multiple real lines that do not overlap with each other; the target predicted lines include multiple predicted lines that do not overlap with each other; the first determination module is specifically used for: using the coordinates of the points on the target real lines as the true values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple non-overlapping real lines as the true values ​​of the target feature points on the object; the second determination module is specifically used for: using the coordinates of the points on the target predicted lines as the predicted values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple non-overlapping predicted lines as the predicted values ​​of the target feature points on the object.

[0018] In combination with the third aspect and the above-mentioned implementation methods, in some implementation methods of the third aspect, the device also includes a fourth determination module, which is specifically used to: determine the target loss function in the following manner: calculate the absolute value of the difference between the true value of the target feature point and the predicted value of the target feature point, and obtain the absolute value of the difference corresponding to the target feature point; and use the square of the absolute value as the target loss function.

[0019] In combination with the third aspect and the above-mentioned implementation methods, in some implementation methods of the third aspect, the device also includes a fifth determination module, which is specifically used to: the original loss function is determined in the following manner: calculate the absolute value of the difference between the true value and the predicted value of each of the corner points to obtain the absolute value of the difference corresponding to each of the multiple corner points; square the absolute value of the difference corresponding to each corner point, and add the squared results of the absolute value of the difference corresponding to each corner point to obtain the loss sum; divide the loss sum by the number of the multiple corner points as the original loss function.

[0020] In a fourth aspect, a corner point detection device is provided, which includes: an acquisition module for acquiring an image to be detected; an output module for inputting the image to be detected into a target corner point detection model trained and generated by the model training method provided by the third aspect, and outputting the predicted values ​​of the corner points in the image to be detected.

[0021] In a fifth aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the method in the first aspect or any possible implementation of the first aspect, or the electronic device executes the method in the second aspect.

[0022] In a sixth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect, or enables the computer to execute the method in the second aspect.

[0023] In the seventh aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the first aspect or any possible implementation of the first aspect, or the computer executes the method in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of a calibration workshop provided in an embodiment of the present application.

[0025] Figure 2 It is a schematic diagram of a calibration block provided in an embodiment of the present application.

[0026] Figure 3 It is a schematic diagram of corner point detection provided in an embodiment of the present application.

[0027] Figure 4 It is a schematic diagram of a corner point motion path provided in an embodiment of the present application.

[0028] Figure 5 It is a schematic flowchart of a model training method provided in an embodiment of the present application.

[0029] Figure 6 It is a schematic diagram of a method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application.

[0030] Figure 7It is a schematic diagram of another method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application.

[0031] Figure 8 It is a schematic diagram of a third method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application.

[0032] Fig. 9 It is a schematic diagram of a fourth method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application.

[0033] Fig.10 It is a schematic flow chart of a corner point detection method provided in an embodiment of the present application.

[0034] Fig.11 It is a structural schematic diagram of a model training device provided in an embodiment of the present application.

[0035] Fig.12 It is a structural schematic diagram of a corner point detection device provided in an embodiment of the present application.

[0036] Fig.13 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0038] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0039] At present, the calibration of vehicle cameras is a necessary process before a vehicle leaves the factory, which is directly related to the accuracy of functions such as AVM (Around View Monitor) and intelligent driving that rely on camera calibration accuracy. The calibration of vehicle cameras is usually carried out in a dedicated calibration workshop. Figure 1 As shown, Figure 1This is a schematic diagram of a calibration workshop provided in an embodiment of the present application. The diagram is taken from a bird's eye view, so the diagram can also be called a BEV (Bird's Eye View) view. Figure 1 There are four calibration blocks, namely calibration block A, calibration block B, calibration block C and calibration block D. At the same time, the car can be equipped with four cameras, namely front-view camera, side-view camera, rear-view camera and built-in camera. The four cameras can respectively shoot the above four different calibration blocks to obtain four different calibration block pictures.

[0040] Figure 2 Schematic diagram of a calibration block provided in an embodiment of the present application. Figure 2 As shown in the figure, any of the calibration block images taken by the above camera and not processed by the calibration algorithm is shown, and the calibration block image includes 4 corner points, namely corner point 1, corner point 2, corner point 3 and corner point 4. The calibration algorithm refers to the method of detecting the above corner points. Figure 2 The four corner points marked with black circles in the figure are used to calculate the positions of the above corner points mapped in a unified three-dimensional coordinate system using a series of computer imaging methods such as corner point coordinates, site prior information, and camera intrinsic parameters, thereby completing the distortion correction and spatial position transformation of the camera imaging image.

[0041] The above corner detection methods can be divided into traditional methods and methods using CNN (Convolutional Neural Network). The above traditional methods are methods related to computer graphics.

[0042] Figure 3 is a schematic diagram of a corner point detection provided by an embodiment of the present application. Figure 3 As shown in the figure, G1 to G4 are the true value points manually marked in the above calibration block (the location of the real corner points), and P1 to P4 are the predicted points detected by CNN (the location of the predicted corner points). In the process of training the original corner point detection model using CNN, regression optimization is performed on each corner point based on the original loss function, that is, the optimization between points is independent and unrelated. The purpose of regression optimization is to make Figure 3 In the example, P1 and G1 overlap, P2 and G2 overlap, P3 and G3 overlap, and P4 and G4 overlap. The original loss function may include an L2 loss function.

[0043] For example, the expression of L2 loss function can be as follows:

[0044]

[0045] In the process of training the original corner detection model using the L2 loss function, the training goal is: Figure 3 In the figure, P1 approaches G1, P2 approaches G2, P3 approaches G3, and P4 approaches G4. In the process of P1 to P4 approaching G1 to G4 respectively, the loss value of a certain point may no longer change. The specific performance is as follows: Figure 4 As shown, Figure 4 It is a schematic diagram of the motion path of a corner point P1 provided in an embodiment of the present application. In the figure, P1 and G1 are taken as examples. In the figure, P1 moves on a dotted circle with G1 as the center, which causes the original corner point detection model to converge to a local optimal solution and stop training, which in turn causes the corner points detected by the trained original corner point detection model to be inaccurate. At the same time, it can be seen from the calculation formula of the above L2 loss function that the four points are independently regressed and optimized, and there is no correlation between the points. Even when the loss value of one of the points is higher and the loss values ​​of the other three points are lower, the overall loss value Loss-Point will also become lower. The overall loss value becomes lower, which will cause the training of the original corner point detection model to end prematurely, making it impossible to continue to optimize the original corner point detection model.

[0046] For example, when three predicted points all overlap with their corresponding true value points, and only one predicted point remains that does not overlap with the true value point, it means that the loss values ​​of the three true value points are all equal to 0, and only the remaining true value point has a higher loss value. Assuming that the loss value is equal to 0.5, according to the calculation formula of the above L2 loss function, the overall loss value at this time needs to be divided by 4 on the basis of calculating the sum of the above loss values, so that the overall loss value is reduced to 0.125. However, due to the low overall loss value, the difficulty of corner regression is increased, resulting in the inability to better optimize the original corner detection model. Finally, when using the original corner detection model for corner detection, sufficiently accurate corners cannot be detected. Therefore, how to improve the accuracy of corner detection has become a technical problem that needs to be solved urgently.

[0047] In order to at least solve the above-mentioned technical problems, an embodiment of the present application provides a method for training a target corner point detection model. This method can be used to train a target corner point detection model with high detection accuracy. Using this target corner point detection model, more precise corner points can be detected, thereby improving the accuracy of corner point detection.

[0048] Figure 5 It is a schematic flowchart of a model training method provided in an embodiment of the present application.

[0049] For example, Figure 5 As shown, the method 500 includes:

[0050] Step 501, obtaining a sample image; wherein the sample image includes an object with an edge, the edge of the object includes a plurality of corner points, and the corner points are marked with real values.

[0051] Step 502, obtaining predicted values ​​of multiple corner points in the sample image predicted by the original corner detection model; wherein the original corner detection model is trained based on the original loss function; the original loss function is used to describe the difference between the true value and the predicted value of the corner point.

[0052] Step 503: Determine the true value of the target feature point on the object according to the true values ​​of the multiple corner points.

[0053] Step 504: Determine the predicted value of the target feature point on the object according to the predicted values ​​of the multiple corner points.

[0054] Step 505 , determining a target loss function for describing the difference between the real value and the predicted value of the target feature point according to the real value of the target feature point and the predicted value of the target feature point.

[0055] Step 506: Perform training according to the original loss function and the target loss function to generate a target corner point detection model.

[0056] In an embodiment of the present application, an additional loss function is added on the basis of the original loss function, and the additional loss function is the target loss function. Compared with only considering the original loss function, adding the target loss function can increase the overall loss value of the target corner detection model. The increase in loss value means that the target corner detection model needs to achieve a higher prediction accuracy to meet the standard for ending training, which will directly lead to the continuous improvement of the target corner detection model during the training process, and continue to find a better combination of parameters to avoid the target corner detection model from converging to the local optimal solution too early, which is conducive to the target corner detection model eventually converging to the global optimal solution, thereby achieving higher detection accuracy. The original loss function is used to describe the difference between the true value and the predicted value of the corner point, which makes it possible to pay attention to the prediction accuracy of multiple corner points in the process of training the target corner detection model. The target loss function is used to describe the difference between the true value and the predicted value of the target feature point, which makes it possible to pay attention to the prediction accuracy of the target feature point in the process of training the target corner detection model. Therefore, by combining the original loss function and the target loss function, in the process of training the target corner detection model, we not only focus on the prediction accuracy of multiple corner points, but also focus on the prediction accuracy of target feature points. That is, in the process of training, not only can the prediction values ​​of multiple corner points gradually equal to or approach the true values ​​of multiple corner points, but also can make the prediction values ​​of feature points gradually equal to or approach the true values ​​of target feature points, thereby improving the detection accuracy of the trained target corner detection model. Finally, when using the target corner detection model for corner detection, more accurate corner points can be obtained, thereby improving the accuracy of corner detection.

[0057] Below Figure 5 The implementation method of each step in the embodiment shown is specifically described:

[0058] As for 501, it can be understood that the sample image can be an object with an edge, the edge of the object can include multiple corner points, and the multiple corner points are all marked with real values. For example, the object with an edge can be a calibration block, the calibration block can have four corner points, and the four corner points can be four vertices on the black surface of the calibration block, and the real values ​​of the four corner points are all obtained by manual marking.

[0059] In a possible implementation, in a scenario where a vehicle-mounted camera is calibrated in a calibration workshop, the sample image may be obtained by taking a photo with a device having a camera function (such as a vehicle-mounted camera) on the vehicle.

[0060] For 502, it can be understood that the predicted values ​​of the above-mentioned multiple corner points are the coordinates of the corner points predicted using the original corner point prediction model, the above-mentioned original corner point prediction model is trained based on the original loss function, and the original loss function is used to describe the difference between the true value of the corner point and the predicted value of the corner point, and the above-mentioned original loss function can be an L2 loss function.

[0061] As mentioned above, the original loss function is used to describe the difference between the true value of the corner point and the predicted value of the corner point. In order to more reasonably describe the difference through the original loss function, a smooth error distribution method can be used. For example, the difference between the true value of the corner point and the predicted value of the corner point is squared to amplify the impact of large errors, so that the original corner prediction model pays more attention to samples with large prediction deviations, which helps to correct significant errors faster during training. The following is a detailed introduction to this method:

[0062] In some embodiments, the original loss function is determined as follows: the absolute value of the difference between the true value and the predicted value of each corner point is calculated to obtain the absolute value of the difference corresponding to each of the multiple corner points; the absolute value of the difference corresponding to each corner point is squared, and the squared absolute value of the difference corresponding to each corner point is added to obtain the sum of the losses; the sum of the losses is divided by the number of multiple corner points as the original loss function.

[0063] For example, Figure 3 As shown, for the calibration block with four corner points and the L2 loss function, the calculation method of the original loss function is: first, calculate the differences of G1-P1, G2-P2, G3-P3 and G4-P4 respectively, then take the absolute values ​​of the above four groups of differences respectively to obtain |G1-P1|, |G2-P2|, |G3-P3| and |G4-P4|, and then square the above four groups of absolute values ​​respectively to obtain |G1-P1|2 、|G2-P2| 2 、|G3-P3| 2 and |G4-P4| 2 , then add the above four sets of square values ​​to get the loss sum, and finally calculate the quotient of the above loss sum divided by the number of corner points to get the original loss function. Therefore, the calculation formula of the above original loss function can be expressed as

[0064] As for 503, it can be understood that the multiple corner points can be multiple corner points on the calibration block, the true values ​​of the multiple corner points are manually marked coordinates, and the true value of the target feature point is determined according to the true values ​​of the multiple corner points. The true value of the target feature point is used to link the true values ​​of the multiple independent corner points.

[0065] The target feature point is a feature point on an object with an edge on the sample image. The feature point can be a point on the line between two non-adjacent corner points of the object with an edge, or an intersection between multiple lines of the object with an edge. In other words, the true value of the target feature value is the coordinate of the point on the target true line.

[0066] The true value of the target feature point will affect the loss value of the target loss function. Therefore, the true value of the target feature point also affects the detection accuracy of the target corner point detection model to a certain extent. In order to link the true values ​​of multiple corner points through the true value of the target feature point, the true value of the target feature point can be determined in the following way.

[0067] In some embodiments, the implementation method of the above step 503 includes: determining the connection line between two non-adjacent corner points based on the true values ​​of multiple corner points to obtain the target true connection line; and using the coordinates of the points on the target true connection line as the true values ​​of the target feature points on the object.

[0068] For ease of description, the true values ​​of multiple corner points are referred to as true corner points, and the predicted values ​​of multiple corner points are referred to as predicted corner points. In the accompanying drawings, black circles are used to represent true corner points, and gray circles are used to represent predicted corner points.

[0069] It can be understood that any two non-adjacent real corner points among the above-mentioned multiple real corner points are selected to connect to obtain the target real line, and any point on the target real line is selected, and the coordinates of the point are determined as the true value of the target feature point on the calibration block.

[0070] For example, Figure 6 Schematic diagram of a method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application. Figure 6As shown in the figure, there are two groups of non-adjacent real corner points, namely G1 and G3 and G2 and G4. Figure 6 As shown in (a), connecting G1 and G3 can obtain a straight line LG1 = F Line (G1,G3), such as Figure 6 As shown in (b), connecting G2 and G4 can obtain the straight line LG2 = F Line (G2, G4), therefore, the above target real line can be LG1 or LG2. Take any point at 1 / n on LG1 or LG2, and use the coordinates of the point at 1 / n as the real value of the target feature point on the calibration block, where n is greater than 1. For example, Figure 6 As shown in (a), n = 3, select point G5 at 1 / 3 on LG1, and use the coordinates of G5 as the true value of the target feature point. Figure 6 As shown in (b), n=3, select point G6 at 1 / 3 on LG2, and use the coordinates of G6 as the true value of the target feature point.

[0071] For example, Figure 7 Schematic diagram of another method for determining the true value and predicted value of a target feature point provided in an embodiment of the present application. Figure 7 As shown in the figure, there are nine groups of non-adjacent real corner points. Take G1ˋ and G3ˋ as an example. Figure 7 As shown, connecting G1ˋ and G3ˋ can obtain the straight line LG1ˋ=F Line (G1ˋ,G3ˋ), therefore, the above target real line can be LG1ˋ. Take any point at 1 / n on LG1ˋ, and take the coordinate of the point at 1 / n as the real value of the target feature point on the calibration block, where n is greater than 1. For example, Figure 7 As shown, n = 3, select point G7ˋ at 1 / 3 on LG1ˋ, and use the coordinates of G7ˋ as the true value of the target feature point.

[0072] It should be noted that Figure 6 For example, if the shape of the object with edges is a quadrilateral, Figure 7 In the example, the shape of the object with edges is a hexagon. However, in this embodiment, the shape of the object with edges is not specifically limited.

[0073] As for 504, it can be understood that the prediction value of the target feature point is determined according to the prediction values ​​of the multiple corner points. The prediction value of the target feature point is used to link the prediction values ​​of the multiple independent corner points.

[0074] The predicted value of the target feature point will affect the loss value of the target loss function. Therefore, the predicted value of the target feature point also affects the detection accuracy of the target corner point detection model to a certain extent. In order to link the predicted values ​​of multiple corner points through the predicted value of the target feature point, the predicted value of the target feature point can be determined in the following way.

[0075] In some embodiments, the implementation method of the above-mentioned step 504 includes: determining the connecting line between two non-adjacent corner points based on the predicted values ​​of multiple corner points to obtain the target predicted connecting line; using the coordinates of the points on the target predicted connecting line as the predicted values ​​of the target feature points on the object; wherein the relative position of the true value of the target feature point on the target true connecting line is consistent with the relative position of the predicted value of the target feature point on the target predicted connecting line.

[0076] It can be understood that any two non-adjacent prediction corner points among the above-mentioned multiple prediction corner points are selected to connect to obtain a target prediction line, and any point on the target prediction line is selected, and the coordinates of the point are determined as the prediction value of the target feature point on the calibration block.

[0077] In a possible implementation, after obtaining the predicted values ​​of the above-mentioned multiple corner points, the dedistortion function F of the above-mentioned sample image can be obtained by using the preset distortion coefficients. disort , and the dedistortion function F disort The dedistortion function can be used to perform dedistortion operations on the predicted values ​​of the above multiple corner points, and the predicted values ​​of the multiple corner points after dedistortion can be obtained. Then, any two non-adjacent predicted corner points from the four dedistorted corner points are selected to connect, and the target predicted connection line is obtained.

[0078] For example, Figure 6 As shown in the figure, there are two groups of non-adjacent predicted corner points, namely P1 and P3 and P2 and P4. Figure 6 As shown in (a), connecting P1 and P3 can obtain a straight line LP1 = F Line (P1,P3), such as Figure 6 As shown in (b), connecting P2 and P4 can obtain a straight line LP2 = F Line (P2, P4), therefore, the target prediction line can be LP1 or LP2. Take any point at 1 / n on LP1 or LP2, and use the coordinates of the point at 1 / n as the prediction value of the target feature point on the calibration block, where n is greater than 1. For example, Figure 6 As shown in (a), n = 3, select point P5 at 1 / 3 on LP1, and use the coordinates of P5 as the predicted value of the target feature point. Figure 6 As shown in (b), n=3, point P6 at 1 / 3 is selected on LP2, and the coordinates of P6 are used as the predicted values ​​of the target feature points.

[0079] For example, Figure 7 As shown in the figure, there are nine groups of non-adjacent predicted corner points. Taking P1ˋ and P3ˋ as an example, Figure 7 As shown, connecting P1ˋ and P3ˋ can obtain the straight line LP1ˋ=F Line (P1ˋ,P3ˋ), therefore, the above target real line can be LP1ˋ. Take any point at 1 / n on LP1ˋ, and take the coordinate of the point at 1 / n as the real value of the target feature point on the calibration block, n is greater than 1. For example, Figure 7 As shown, n = 3, select point P7ˋ at 1 / 3 on LP1ˋ, and use the coordinates of P7ˋ as the true value of the target feature point.

[0080] The relative position of the true value of the above-mentioned target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line, which can be understood as: taking a point at 1 / n on the target true line, and taking the coordinates of the point at 1 / n as the true value of the target feature point, taking a point at the same position 1 / n on the target predicted line, and taking the coordinates of the point at 1 / n as the predicted value of the target feature point, n is greater than 1, and the n of the two must be the same, that is, taking a point at the same 1 / n on the target true line and the target predicted line respectively, at this time, the relative position of the true value of the target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line.

[0081] The relative position of the true value of the above target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line, which can also be understood as:

[0082] For example, Figure 6 As shown in (a), the distance between P5 and P1 is obtained, and the distance is recorded as the first predicted distance P15, the distance between P5 and P3 is obtained, and the distance is recorded as the second predicted distance P35, the distance between G5 and G1 is obtained, and the distance is recorded as the first true distance G15, the distance between G3 and G5 is obtained, and the distance is recorded as the second true distance G35, and the ratios of P15 to P35 and G15 to G35 are calculated respectively. If the ratio of P15 to P35 is equal to the ratio of G15 to G35, it means that the relative position of the predicted value of the target feature point on the target predicted line is consistent with the relative position of the true value of the target feature point on the target true line.

[0083] For example, Figure 7As shown, the distance between P7ˋ and P1ˋ is obtained, and the distance is recorded as the third predicted distance P17ˋ, the distance between P7ˋ and P3ˋ is obtained, and the distance is recorded as the fourth predicted distance P37, the distance between G7ˋ and G1ˋ is obtained, and the distance is recorded as the third true distance G17ˋ, the distance between G3ˋ and G7ˋ is obtained, and the distance is recorded as the second true distance G37, and the ratios of P17 to P37 and G17 to G37 are calculated respectively. If the ratio of P17 to P37 is equal to the ratio of G17 to G37, it means that the relative position of the predicted value of the target feature point on the target prediction line is consistent with the relative position of the true value of the target feature point on the target true line.

[0084] In the above technical scheme, during the process of training the target corner detection model according to the modified loss function, when P5 gradually approaches G5, P1 will also gradually approach G1, P3 will also gradually approach G3, and when P7ˋ gradually approaches G7ˋ, P1ˋ will also gradually approach G1ˋ, and P3ˋ will also gradually approach G3ˋ, eventually making the predicted values ​​of multiple corner points equal to the true values ​​of multiple corner points, and the predicted values ​​of the target feature points equal to the true values ​​of the target feature points, thereby obtaining a target corner detection model with higher detection accuracy, thereby improving the accuracy of corner detection.

[0085] It is understandable that in the above embodiment, the coordinates of the points on the target real line are used as the real values ​​of the target feature points. However, the target real line includes many points, and selecting different points as the real values ​​of the target feature points may produce different optimization effects for the training of the target corner point detection model. Based on this, in order to produce a better optimization effect, so that more predicted corner points gradually approach their corresponding real corner points, the real values ​​of the target feature points can be determined in the following manner.

[0086] In some embodiments, the target real line includes multiple non-overlapping real lines; the coordinates of the points on the target real line are used as the real values ​​of the target feature points on the object, including: using the coordinates of the intersection of multiple non-overlapping real lines as the real values ​​of the target feature points on the object.

[0087] For example, taking the number of corner points equal to 4 as an example, assuming that the above target real line includes a first real line and a second real line, and the first real line is LG1, the second real line is LG2, and LG1 and LG2 do not overlap. Figure 8 As shown, Figure 8 It is a schematic diagram of the third method for determining the true value and predicted value of the target feature point provided in an embodiment of the present application. The intersection point of LG1 and LG2 is G7. G7 is also the center point of an object with an edge (such as a calibration block). At this time, the coordinates of G7 are the true values ​​of the target feature points on the calibration block.

[0088] For example, taking the number of corner points equal to 6 as an example, assuming that the above-mentioned target real connection line includes the third real connection line and the fourth real connection line, and the third real connection line is LG1ˋ, and the fourth real connection line is LG2ˋ (where LG2ˋ is the connection line between the real corner point G2ˋ and the real corner point G6ˋ, that is, LG2ˋ=F Line (G2ˋ,G6ˋ)), LG1ˋ and LG2ˋ do not overlap. Fig. 9 As shown in (a), Fig. 9 It is a schematic diagram of the fourth method for determining the true value and predicted value of the target feature point provided in an embodiment of the present application. The intersection point of LG1ˋ and LG2ˋ is G8ˋ. At this time, the coordinates of G8ˋ are the true values ​​of the target feature points on the calibration block.

[0089] It is understandable that in the above embodiment, the coordinates of the points on the target prediction line are used as the predicted values ​​of the target feature points. However, the target prediction line includes many points, and selecting different points as the true values ​​of the target feature points may produce different optimization effects for the training of the target corner point detection model. Based on this, in order to produce a better optimization effect and make more predicted corner points gradually approach their corresponding true corner points, the true values ​​of the target feature points can be determined in the following manner.

[0090] In some embodiments, the target prediction line includes multiple non-overlapping prediction lines; using the coordinates of the points on the target prediction line as the prediction values ​​of the target feature points on the object includes: using the coordinates of the intersection of the multiple non-overlapping prediction lines as the prediction values ​​of the target feature points on the object.

[0091] For example, taking the number of corner points equal to 4 as an example, assuming that the target prediction line includes a first prediction line and a second prediction line, and the first prediction line is LP1, the second prediction line is LP2, and LP1 and LP2 do not overlap. Figure 8 As shown, the intersection point of LP1 and LP2 is P7. At this time, the coordinates of P7 are the predicted values ​​of the target feature points on the calibration block.

[0092] For example, taking the number of corner points equal to 6 as an example, assuming that the target prediction line includes the third prediction line and the fourth prediction line, and the third prediction line is LP1, and the fourth prediction line is LP2 (where LP2 is the line between the real corner point P2 and the real corner point P6, that is, LP2 = F Line (P2ˋ,P6ˋ)), LP1ˋ and LP2ˋ do not overlap. Fig. 9 As shown in (a) in the figure, the intersection point of LP1ˋ and LP2ˋ is P8ˋ. At this time, the coordinates of P8ˋ can be used as the predicted value of the target feature point on the calibration block.

[0093] In the above technical solution taking the number of corner points equal to 4 as an example, in the process of training the target corner point detection model according to the modified loss function, the coordinates of the intersection point G7 of the first real line (the line between non-adjacent real corner points G1 and G3) and the second real line (the line between non-adjacent real corner points G2 and G4) are selected as the real value of the target feature point. And the coordinates of the intersection point P7 of the first predicted line (the line between non-adjacent predicted corner points P1 and P3) and the second predicted line (the line between non-adjacent predicted corner points P2 and P4) are selected as the predicted value of the target feature point. By adding the pair of points P7 and G7, while optimizing P7 to gradually approach G7, P1 to P4 can be simultaneously optimized to gradually approach G1 to G4, ensuring that the predicted values ​​of multiple corner points can be close to the true values ​​of multiple corner points at the same time. Ultimately, the predicted values ​​of multiple corner points are equal to the true values ​​of multiple corner points and the predicted values ​​of the target feature points are equal to the true values ​​of the target feature points. This enables the target corner detection model to converge more quickly during the training process, thereby more quickly obtaining a target corner detection model with higher detection accuracy.

[0094] In the above technical solution taking the number of corner points equal to 6 as an example, in the process of training the target corner point detection model according to the modified loss function, the coordinates of the intersection point G8' of the third real line (the line between non-adjacent real corner points G1' and G3') and the fourth real line (the line between non-adjacent real corner points G2' and G6') are selected as the true value of the target feature point. And the coordinates of the intersection point P8' of the third predicted line (the line between non-adjacent predicted corner points P1' and P3') and the fourth predicted line (the line between non-adjacent predicted corner points P2' and P6') are selected as the predicted value of the target feature point. By adding the pair of points P8ˋ and G8ˋ, while optimizing P8ˋ to gradually approach G8ˋ, P1ˋ, P2ˋ, P3ˋ and P6ˋ can be simultaneously optimized to gradually approach G1ˋ, G2ˋ, G3ˋ and G6ˋ respectively. This also ensures that the predicted values ​​of multiple corner points can be close to the true values ​​of multiple corner points at the same time. Ultimately, the predicted values ​​of multiple corner points are equal to the true values ​​of multiple corner points and the predicted values ​​of the target feature points are equal to the true values ​​of the target feature points. This enables the target corner detection model to converge more quickly during the training process, thereby more quickly obtaining a target corner detection model with higher detection accuracy.

[0095] It can be understood that, taking the number of corner points as 6 as an example, Fig. 9 As shown in (a) in , any two non-adjacent predicted corner points can be connected into a predicted line, and any two predicted lines can produce an intersection, so there may be multiple intersections. Selecting different intersections as the predicted values ​​of the target feature points may produce different optimization effects for the training of the target corner point detection model. Fig. 9As shown in (a) in the figure, taking the coordinates of the target feature point whose predicted value is P8ˋ as an example, in the process of P8ˋ gradually regressing to G8ˋ, P1ˋ, P2ˋ, P3ˋ and P6ˋ can only be gradually approached to G1ˋ, G2ˋ, G3ˋ and G6ˋ respectively. Taking the coordinates of the target feature point whose predicted value is P9ˋ as an example, in the process of P9ˋ gradually regressing to G9ˋ, P3ˋ, P4ˋ, P5ˋ and P6ˋ can only be gradually approached to G3ˋ, G4ˋ, G5ˋ and G6ˋ respectively. In order to obtain the best optimization effect and make all the predicted corner points gradually approach their corresponding real corner points, the following method can be used to determine the real value of the target feature point and the predicted value of the target feature point.

[0096] Select the target predicted intersection point from the multiple intersection points generated by any two predicted lines, and the target predicted intersection point passes through the largest number of predicted lines. Select the target real intersection point from the multiple intersection points generated by any two real lines, and the target real intersection point passes through the largest number of real lines. For ease of understanding, the following is combined with Fig. 9 To explain:

[0097] For example, Fig. 9 As shown in (b), it is assumed that the target real line includes the fifth real line, the sixth real line and the seventh real line, and the fifth real line is LG3ˋ (where LG3ˋ is the line between the real corner point G1ˋ and the real corner point G4ˋ, that is, LG3ˋ=F Line (G1ˋ,G4ˋ)), the sixth real line is LG4ˋ (where LG4ˋ is the line between the real corner point G2ˋ and the real corner point G5ˋ, that is, LG4ˋ=F Line (G2ˋ,G5ˋ)), the seventh real line LG5ˋ (where LG5ˋ is the line between the real corner point G3ˋ and the real corner point G6ˋ, that is, LG5ˋ=F Line (G3ˋ,G6ˋ)), LG3ˋ, LG4ˋ and LG5ˋ do not overlap. Fig. 9 As shown in (b) of , the intersection point of LG3ˋ, LG4ˋ and LG5ˋ is G10ˋ, and the number of real lines passing through the intersection point G10ˋ is 3. Fig. 9 As shown in (a) in the figure, the intersection point of LG1ˋ and LG2ˋ is G8ˋ, and the number of real lines passing through the intersection point G8ˋ is 2. Therefore, G10ˋ is the target real intersection point with the largest number of real lines passing through it. At this time, the coordinates of G10ˋ are the true values ​​of the target feature points on the calibration block.

[0098] Assume that the target real line includes the fifth predicted line, the sixth predicted line and the seventh predicted line, and the fifth predicted line is LP3ˋ (where LP3ˋ is the line between the predicted corner point P1ˋ and the predicted corner point P4ˋ, that is, LP3ˋ=FLine (P1ˋ,P4ˋ)), the sixth predicted line is LP4ˋ (where LP4ˋ is the line between the predicted corner point P2ˋ and the predicted corner point P5ˋ, that is, LP4ˋ=F Line (P2ˋ,P5ˋ)), the seventh predicted line LP5ˋ (where LP5ˋ is the line between the predicted corner point P3ˋ and the predicted corner point P6ˋ, that is, LP5ˋ=F Line (P3ˋ,P6ˋ)), LP3ˋ, LP4ˋ and LP5ˋ do not overlap. Fig. 9 As shown in (b) of , the intersection point of LP3ˋ, LP4ˋ and LP5ˋ is P10ˋ, and the number of predicted links passing through the intersection point P10ˋ is 3. Fig. 9 As shown in (a) in the figure, the intersection point of LP1ˋ and LP2ˋ is P8ˋ, and the number of predicted lines passing through the intersection point P8ˋ is 2. Therefore, P10ˋ is the target predicted intersection point with the largest number of predicted lines passing through it. At this time, the coordinates of P10ˋ are the predicted values ​​of the target feature points on the calibration block.

[0099] In the above technical solution, in the process of training the target corner detection model according to the modified loss function, the coordinates of the intersection point G10ˋ of the fifth real line, the sixth real line and the seventh real line are selected as the real value of the target feature point. And the coordinates of the intersection point P10ˋ of the fifth predicted line, the sixth predicted line and the seventh predicted line are selected as the predicted value of the target feature point. By adding the pair of points P10ˋ and G10ˋ, in the process of optimizing P10ˋ to gradually approach G10ˋ, P1ˋ to P6ˋ can be simultaneously optimized to gradually approach G1ˋ to G6ˋ, respectively, to ensure that the predicted values ​​of all corner points can simultaneously approach the real values ​​of their corresponding corner points, and finally the predicted values ​​of all corner points are equal to the real values ​​of all corner points and the predicted values ​​of the target feature points are equal to the real values ​​of the target feature points, which enables the target corner detection model to converge at the fastest speed during the training process, so that the target corner detection model with higher detection accuracy can be obtained as quickly as possible.

[0100] For 505, it can be understood that the target loss function is determined according to the true value of the target feature point and the predicted value of the target feature point, for example, the target loss function is determined according to the coordinates of G5 and P5. The target loss function is a target loss function for describing the difference between the true value of the target feature point and the predicted value of the target feature point.

[0101] As mentioned above, the target loss function is used to describe the difference between the true value and the predicted value of the target feature point. In order to more reasonably describe the difference through the target loss function, a smooth error distribution method can be used. For example, the difference between the true value and the predicted value of the target feature point is squared to amplify the impact of large errors, so that the target corner prediction model pays more attention to the target feature point, which helps to correct significant errors faster during the training process. The following is a detailed introduction to this method:

[0102] In some embodiments, the target loss function is determined by: calculating the absolute value of the difference between the true value of the target feature point and the predicted value of the target feature point to obtain the absolute value of the difference corresponding to the target feature point; and taking the square of the absolute value as the target loss function.

[0103] Exemplarily, for the above G5 and P5, the calculation method of the target loss function is: first, calculate the difference between G5 and P5, then take the absolute value of the difference to obtain |G5-P5|, and finally square the absolute value |G5-P5| 2 , we get the target loss function, so the calculation formula of the above target loss function can be expressed as Loss-Center = |G5-P5| 2 .

[0104] In the above technical solution, the calculation principle of the target loss function is consistent with the calculation principle of the original loss function, so that the power of the target loss function and the original loss function are consistent, which facilitates the direct addition of the target loss function and the original loss function to obtain the corrected loss function, thereby making the training process of the target corner detection model more stable and efficient.

[0105] For 506, it can be understood that, according to the original loss function and the target loss function determined above, multiple corner points are trained and a target corner point detection model is generated, so that the predicted values ​​of the multiple corner points are equal to the true values ​​of the multiple corner points, and the predicted values ​​of the target feature points are equal to the true values ​​of the target feature points, thereby obtaining a target corner point detection model with higher detection accuracy, thereby improving the accuracy of corner point detection.

[0106] The original loss function and the target loss function are combined to increase the overall loss value of the target corner detection model. In order to achieve the purpose of increasing the overall loss value of the target corner detection model and simplify the calculation method of the overall loss value, the original loss function is added to the target loss function. The following is a detailed introduction to this method:

[0107] In some embodiments, the implementation of the above 506 includes: calculating the original loss function plus the target loss function to obtain a modified loss function; and training according to the modified loss function to generate a target corner detection model.

[0108] It can be understood that the corrected loss function is obtained by calculating the original loss function plus the target loss function. After obtaining the corrected loss function, the gradient descent algorithm is used to continuously reduce the loss value of the corrected loss function, so as to achieve the purpose that the predicted values ​​of multiple corner points are equal to the true values ​​of multiple corner points, and the predicted value of the target feature point is equal to the true value of the target feature point, and finally generate a target corner point detection model.

[0109] Exemplarily, the original loss function plus the target loss function is calculated, that is, Loss-Center is added to Loss-Point to obtain the corrected loss function, that is, the calculation formula of the corrected loss function Loss is: Loss = Loss-Center + Loss-Point, and the target corner point detection model is generated according to the corrected loss function Loss.

[0110] In summary, on the basis of the original loss function, an additional target loss function is added to increase the overall loss value of the target corner detection model, so that the model can be further optimized. The above target loss function can connect the four independent corner points, avoiding the difficulty of corner regression due to the original loss function's loss value being too small, thus failing to better optimize the original corner detection model; after adding Loss-Center, the overall loss value will increase, and when the loss value of the corrected loss function is higher than that of the original loss function, the isolated gray point originally moving on the arc will continue to move closer to the black point, thereby obtaining a more accurate target corner detection model, and then when using the target corner detection model for corner detection, the accuracy of corner detection can be improved.

[0111] Fig.10 It is a schematic flow chart of a corner point detection method provided in an embodiment of the present application.

[0112] For example, Fig.10 As shown, the method 1000 includes:

[0113] Step 1001: Acquire an image to be detected.

[0114] Step 1002: input the image to be detected into the target corner point detection model trained and generated by the model training method in the above embodiment, and output the predicted values ​​of the corner points in the image to be detected.

[0115] exist Figure 5 In the embodiment shown, after obtaining the target corner detection model, a device with a camera function on the vehicle can be used to capture the image to be detected, and the image to be detected can be input into the target corner detection model. The device with a camera function can be a vehicle-mounted camera.

[0116] It can be understood that after obtaining the target corner detection model, the target corner detection model can be deployed in an electronic device on the vehicle that is equipped with an image processing module, such as the vehicle's central control screen and HUD (Head-Up Display), etc., and the embodiments of the present application are not limited to this.

[0117] In summary, by using the target corner point detection model trained and generated in the above embodiment, more accurate corner points can be detected, thereby improving the accuracy of corner point detection, improving the overall effect of vehicle camera calibration, and further improving the accuracy of functions such as AVM and intelligent driving that rely on camera calibration accuracy.

[0118] Fig.11 It is a structural schematic diagram of a model training device provided in an embodiment of the present application.

[0119] For example, Fig.11 As shown, the device 1100 includes:

[0120] The first acquisition module 1101 is used to acquire a sample image; wherein the sample image includes an object with an edge, the edge of the object includes a plurality of corner points, and the corner points are marked with real values;

[0121] The second acquisition module 1102 is used to obtain the predicted values ​​of multiple corner points in the sample image predicted by the original corner detection model; wherein the original corner detection model is trained based on the original loss function; the original loss function is used to describe the difference between the true value and the predicted value of the corner point;

[0122] A first determination module 1103 is used to determine the true value of the target feature point on the object according to the true values ​​of the multiple corner points;

[0123] A second determination module 1104 is used to determine the prediction value of the target feature point on the object according to the prediction values ​​of the multiple corner points;

[0124] A third determination module 1105 is used to determine a target loss function for describing the difference between the real value and the predicted value of the target feature point according to the real value of the target feature point and the predicted value of the target feature point;

[0125] The generation module 1106 is used to perform training according to the original loss function and the target loss function to generate a target corner point detection model.

[0126] In one possible implementation, the generation module is specifically used to: perform training based on the original loss function and the target loss function to generate a target corner detection model, including: calculating the original loss function plus the target loss function to obtain a corrected loss function; and perform training based on the corrected loss function to generate a target corner detection model.

[0127] In a possible implementation, the first determination module is specifically used to: determine the true value of the target feature point on the object according to the true values ​​of multiple corner points, including: determining the line between two non-adjacent corner points according to the true values ​​of the multiple corner points to obtain the target true line; using the coordinates of the points on the target true line as the true value of the target feature point on the object; the second determination module is specifically used to: determine the predicted value of the target feature point on the object according to the predicted values ​​of the multiple corner points, including: determining the line between two non-adjacent corner points according to the predicted values ​​of the multiple corner points to obtain the target predicted line; using the coordinates of the points on the target predicted line as the predicted value of the target feature point on the object; wherein the relative position of the true value of the target feature point on the target true line is consistent with the relative position of the predicted value of the target feature point on the target predicted line.

[0128] In one possible implementation, the target real lines include multiple real lines that do not overlap with each other; the target predicted lines include multiple predicted lines that do not overlap with each other; the first determination module is specifically used to: use the coordinates of the points on the target real lines as the real values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple real lines that do not overlap with each other as the real values ​​of the target feature points on the object; the second determination module is specifically used to: use the coordinates of the points on the target predicted lines as the predicted values ​​of the target feature points on the object, including: using the coordinates of the intersection points of the multiple predicted lines that do not overlap with each other as the predicted values ​​of the target feature points on the object.

[0129] In one possible implementation, the device also includes a fourth determination module, which is specifically used to: determine the target loss function in the following manner: calculate the absolute value of the difference between the true value of the target feature point and the predicted value of the target feature point, and obtain the absolute value of the difference corresponding to the target feature point; and use the square of the absolute value as the target loss function.

[0130] In a possible implementation, the device also includes a fifth determination module, which is specifically used to: determine the original loss function in the following manner: calculate the absolute value of the difference between the true value and the predicted value of each corner point to obtain the absolute value of the difference corresponding to each of the multiple corner points; square the absolute value of the difference corresponding to each corner point, and add the squared results of the absolute value of the difference corresponding to each corner point to obtain the sum of losses; divide the sum of losses by the number of multiple corner points as the original loss function.

[0131] Fig.12 It is a structural schematic diagram of a corner point detection device provided in an embodiment of the present application.

[0132] For example, Fig.12 As shown, the device 1200 includes:

[0133] The acquisition module 1201 is used to acquire the image to be detected;

[0134] The output module 1202 is used to input the image to be detected into the target corner point detection model pre-trained and generated by the device in the above embodiment, and output the predicted values ​​of the corner points in the image to be detected.

[0135] Fig.13 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0136] For example, Fig.13 As shown, the electronic device 1300 includes: a memory 1301 and a processor 1302, wherein the memory 1301 stores an executable program code 13011, and the processor 1302 is used to call and execute the executable program code 13011 to perform a training method for a target corner detection model or a target corner detection method.

[0137] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a training method for a target corner detection model or a target corner detection method provided in an embodiment of the present application.

[0138] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0139] In the case of dividing each functional module according to each function, the training device of the target corner detection model may further include a first acquisition module, a second acquisition module, a first determination module, a second determination module, a third determination module and a generation module, etc., and the device for detecting the target corner may further include a collection module and an output module. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0140] It should be understood that the device provided in this embodiment is used to execute the above-mentioned training method of a target corner detection model or a method of target corner detection, and thus can achieve the same effect as the above-mentioned implementation method.

[0141] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0142] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0143] In addition, the device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a training method for a target corner detection model or a target corner detection method provided in the above embodiments.

[0144] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a training method for a target corner detection model or a target corner detection method provided in the above embodiment.

[0145] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a training method for a target corner detection model or a method for detecting target corner points provided in the above-mentioned embodiment.

[0146] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0147] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0148] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a sample image; wherein the sample image includes an object with an edge, the edge of the object includes a plurality of corner points, and the corner points are marked with real values; Obtain predicted values ​​of multiple corner points in the sample image predicted by an original corner point detection model; wherein the original corner point detection model is trained based on an original loss function; and the original loss function is used to describe the difference between the true value and the predicted value of the corner point; Determining the true value of the target feature point on the object according to the true values ​​of the plurality of corner points; Determining a predicted value of a target feature point on the object according to the predicted values ​​of the plurality of corner points; Determine, according to the true value of the target feature point and the predicted value of the target feature point, a target loss function for describing the difference between the true value and the predicted value of the target feature point; Training is performed according to the original loss function and the target loss function to generate a target corner point detection model.

2. The method according to claim 1, characterized in that The training is performed according to the original loss function and the target loss function to generate a target corner point detection model, including: Calculate the original loss function plus the target loss function to obtain a modified loss function; Training is performed according to the modified loss function to generate the target corner point detection model.

3. The method according to claim 1, characterized in that: Determining the true value of the target feature point on the object according to the true values ​​of the plurality of corner points comprises: Determine a connection line between two non-adjacent corner points according to the true values ​​of the plurality of corner points to obtain a target true connection line; Taking the coordinates of the points on the target real line as the real values ​​of the target feature points on the object; Determining the predicted value of the target feature point on the object according to the predicted values ​​of the plurality of corner points comprises: Determine a connection line between two non-adjacent corner points according to the predicted values ​​of the plurality of corner points to obtain a target predicted connection line; The coordinates of the points on the target prediction line are used as the predicted values ​​of the target feature points on the object; wherein the relative position of the real value of the target feature point on the target real line is consistent with the relative position of the predicted value of the target feature point on the target prediction line.

4. The method according to claim 3, characterized in that The target real connection line includes a plurality of real connection lines that do not overlap with each other; the target predicted connection line includes a plurality of predicted connection lines that do not overlap with each other; and the coordinates of points on the target real connection line are used as the true values ​​of the target feature points on the object, including: Taking the coordinates of the intersections of the plurality of non-overlapping real connecting lines as the real values ​​of the target feature points on the object; The step of using the coordinates of the points on the target prediction line as the predicted values ​​of the target feature points on the object includes: The coordinates of the intersections of the multiple non-overlapping prediction lines are used as the prediction values ​​of the target feature points on the object.

5. The method according to any one of claims 1 to 4, characterized in that: The objective loss function is determined as follows: Calculating the absolute value of the difference between the true value of the target feature point and the predicted value of the target feature point; The square of the absolute value is used as the target loss function.

6. The method according to any one of claims 1 to 4, characterized in that: The original loss function is determined as follows: Calculate the absolute value of the difference between the true value and the predicted value of each corner point to obtain the absolute value of the difference corresponding to each of the multiple corner points; The absolute value of the difference corresponding to each corner point is squared, and the squared results of the absolute value of the difference corresponding to each corner point are added together to obtain a loss sum; The loss sum is divided by the number of the multiple corner points to obtain the original loss function.

7. A corner point detection method, characterized in that: The method comprises: Collecting images to be detected; The image to be detected is input into a target corner point detection model trained and generated by the model training method described in any one of claims 1 to 6, and the predicted values ​​of the corner points in the image to be detected are output.

8. A model training device, characterized in that: The device comprises: A first acquisition module is used to acquire a sample image; wherein the sample image includes an object with an edge, the edge of the object includes a plurality of corner points, and the corner points are marked with real values; A second acquisition module is used to obtain predicted values ​​of multiple corner points in the sample image predicted by an original corner detection model; wherein the original corner detection model is trained based on an original loss function; and the original loss function is used to describe the difference between the true value and the predicted value of the corner point; A first determination module, used to determine the true value of the target feature point on the object according to the true values ​​of the plurality of corner points; A second determination module, used to determine the predicted value of the target feature point on the object according to the predicted values ​​of the plurality of corner points; A third determination module is used to determine a target loss function for describing the difference between the real value and the predicted value of the target feature point according to the real value of the target feature point and the predicted value of the target feature point; A generation module is used to perform training according to the original loss function and the target loss function to generate a target corner point detection model.

9. A corner point detection device, characterized in that: The device comprises: An acquisition module, used for acquiring images to be detected; The output module is used to input the image to be detected into the target corner point detection model trained and generated by the model training method described in any one of claims 1 to 6, and output the predicted values ​​of the corner points in the image to be detected.

10. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images