Parking space calibration model training method and device, equipment and storage medium

Abstract

The invention discloses a training method, device and equipment of a parking space calibration model and a storage medium. The training method of the parking space calibration model comprises the following steps: acquiring world coordinates corresponding to multiple groups of parking space angles acquired by a laser radar and pixel coordinates corresponding to multiple groups of parking space angles acquired by a camera; if the world coordinate and the pixel coordinate meet a preset precision requirement, determining model training data based on the world coordinate and the pixel coordinate; training a parking space calibration model based on the model training data to obtain a trained target parking space calibration model; and determining a reference coordinate corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinate of the target parking space. According to the technical scheme, the parking space line of the target parking space and the accurate reference coordinates corresponding to the obstacles in the target parking space can be output, the calibration time is shortened, and the camera calibration efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle intelligent driving technology, and specifically to a training method, device, equipment and storage medium for a parking space calibration model. Background Art

[0002] At present, traditional parking space calibration usually uses direct camera calibration. However, traditional camera calibration usually requires manual setting of initial values ​​and has high requirements for the precise manufacture and placement of calibration plates, which limits its degree of automation. In addition, traditional methods often rely on specific scene assumptions and are difficult to flexibly expand to different camera models and calibration scenes. There are also problems such as computational complexity, high time and resource consumption, limitations in distortion processing, and the possibility of not being able to guarantee a global optimal solution. Therefore, traditional calibration methods may not be able to achieve ideal accuracy, especially when the camera lens is a wide-angle lens, and the ideal model cannot accurately describe the geometric relationship of the imaging. Summary of the invention

[0003] In view of the above problems, the present application provides a training method, device, equipment and storage medium for a parking space calibration model, which can not only output the precise reference coordinates corresponding to the parking space line of the target parking space and the internal obstacles of the target parking space, but also shorten the calibration time and improve the camera calibration efficiency.

[0004] According to one aspect of an embodiment of the present application, a training method for a parking space calibration model is provided, the method comprising: obtaining world coordinates corresponding to multiple groups of parking space angles collected by a laser radar and pixel coordinates corresponding to multiple groups of parking space angles collected by a camera; if the world coordinates and the pixel coordinates meet preset accuracy requirements, determining model training data based on the world coordinates and the pixel coordinates; training the parking space calibration model based on the model training data to obtain a trained target parking space calibration model; determining the reference coordinates corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0005] In an optional manner, the method steps of training the parking space calibration model based on the model training data to obtain the trained target parking space calibration model include: forward propagating the pixel coordinates as the input layer signal to obtain the output layer world coordinates of the parking space calibration model; back propagating the output layer world coordinates to the input layer to obtain the model error value of the parking space calibration model; and correcting the parking space calibration model based on the model error value to obtain the target parking space calibration model.

[0006] In an optional manner, the method steps of using the pixel coordinates as input layer signals for forward propagation to obtain the output layer world coordinates of the parking space calibration model include: using the pixel coordinates as input layer signals for weighted processing according to a preset weighting strategy to obtain weighted pixel coordinates; processing the weighted pixel coordinates through a hidden layer to obtain processed pixel coordinates; and propagating the processed pixel coordinates to the output layer of the parking space calibration model to obtain the output layer world coordinates of the parking space calibration model.

[0007] In an optional manner, the method steps of back-propagating the output layer world coordinates to the input layer to obtain the model error value of the parking space calibration model include: calculating the model error value between the output layer world coordinates and the actual world coordinates, wherein the actual world coordinates are the world coordinates in the model training data; and adjusting the weight value corresponding to each layer in the parking space calibration model based on the model error value to obtain a corrected parking space calibration model.

[0008] In an optional manner, the method steps for adjusting the weight value corresponding to each layer in the parking space calibration model based on the model error value to obtain a corrected parking space calibration model include: obtaining a preset weight value adjustment strategy, the preset weight value adjustment strategy including a weight value gradient descent adjustment model; calculating the weight corresponding to each layer in the input layer from the output layer world coordinates back-propagated to the input layer according to the weight value gradient descent adjustment model.

[0009] In an optional manner, the method further includes: after detecting that the vehicle is parked at the target parking space, obtaining the real-time world coordinates corresponding to the target parking space collected by the laser radar and the real-time pixel coordinates corresponding to the target parking space collected by the camera; if the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space do not match the target parking space calibration model, updating the target parking space calibration model based on the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space.

[0010] In an optional manner, the method steps for updating the target parking space calibration model based on the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space include: if the actual error value is greater than the preset error value, and greater than or equal to the error threshold, obtaining the sensor data of the vehicle and the environmental data of the vehicle; determining error characterization information based on the sensor data and the environmental data; and updating the target parking space calibration model based on the error characterization information and the actual error value corresponding to the error characterization information.

[0011] According to another aspect of an embodiment of the present application, a training device for a parking space calibration model is provided, the device comprising: an acquisition module for acquiring world coordinates and pixel coordinates corresponding to multiple sets of parking space angles; a first determination module for determining model training data based on the world coordinates and the pixel coordinates if the world coordinates and the pixel coordinates meet preset accuracy requirements; a training module for training the parking space calibration model based on the model training data to obtain a trained target parking space calibration model; and a second determination module for determining the reference coordinates corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0012] According to another aspect of an embodiment of the present application, a computing device is provided, including: a controller; and a memory for storing one or more programs, wherein when the one or more programs are executed by the controller, the controller implements the above-mentioned training method of the parking space calibration model.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed on a training device / computing equipment of a parking space calibration model, the training device / computing equipment of the parking space calibration model executes the training method of the parking space calibration model as described above.

[0014] In the parking space calibration model training method of the present application, the parking space calibration model training method first selects world coordinates and pixel coordinates that meet preset accuracy requirements from multiple sets of world coordinates and pixel coordinates corresponding to the acquired parking space angles as model training data; then the parking space calibration model is trained with the model training data to obtain the target parking space calibration model; finally, the reference coordinates corresponding to the target parking space are determined based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0015] Therefore, the calibration model obtained by the training method of the parking space calibration model of the present application can not only obtain the precise reference coordinates corresponding to the parking space line of the target parking space and the internal obstacles of the target parking space, but also compared with the traditional camera, when obtaining the calibration model, it is necessary to complete the calibration in two steps (camera internal parameter calibration and external parameter calibration), which shortens the calibration time and improves the camera calibration efficiency. It does not rely on special calibration rooms and calibration stations, does not need to place a camera calibration plate, and avoids the calibration plate placement size model error value and cumbersome calibration process.

[0016] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0018] Figure 1 It is a schematic diagram of an implementation environment involved in a training method for a parking space calibration model provided in an embodiment of the present application.

[0019] Figure 2 A schematic flow chart of steps S210-S240 of a method for training a parking space calibration model provided in an embodiment of the present application is shown.

[0020] Figure 3 A schematic flowchart of steps S211-S213 of a method for training a parking space calibration model provided in an embodiment of the present application is shown.

[0021] Figure 4 A schematic flowchart of steps S231-S233 of a method for training a parking space calibration model provided in an embodiment of the present application is shown.

[0022] Figure 5 A schematic block diagram of the process of steps S2311-S2313 of a parking space calibration model training method provided in an embodiment of the present application is shown.

[0023] Figure 6 A schematic block diagram of the process of steps S2321-S2322 of a parking space calibration model training method provided in an embodiment of the present application is shown.

[0024] Figure 7 A schematic structural diagram of a parking space calibration model training device provided in an embodiment of the present application is shown.

[0025] Figure 8 A schematic diagram of the structure of an embodiment of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0026] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0027] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0028] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0029] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0030] See also Figure 1 , Figure 1 1 is a schematic diagram of an implementation environment involved in a method for training a parking space calibration model provided in an embodiment of the present application. The implementation environment includes a vehicle-mounted terminal 110 and a server 120, wherein the terminal 110 and the server 120 communicate via wired or wireless means.

[0031] Specifically, the vehicle is allowed to drive a certain distance in a garage with parking space lines to collect multiple sets of world coordinates and pixel coordinates corresponding to parking space angles, and then the world coordinates and pixel coordinates that meet the preset accuracy requirements can be directly trained as a target parking space calibration model through the vehicle terminal 110, and finally the target parking space calibration model can be transmitted to the server 120 so that other vehicles can directly obtain the target parking space calibration model; or, the collected world coordinates and pixel coordinates that meet the preset accuracy requirements can be directly uploaded to the server 120 and trained as a target parking space calibration model through the server 120; finally, when the vehicle determines the precise reference coordinates corresponding to the parking space lines and internal obstacles of the target parking space, it can be determined by combining the target parking space calibration model of the server 120.

[0032] in. Figure 1 The server 120 shown can be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and is not limited here. The smart terminal 110 can communicate with the server 120 through wireless networks such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), and 5G (fifth generation mobile information technology), and is not limited here.

[0033] First of all, it should be noted that during the parking process, the coordinates of the parking space and the coordinates of the obstacles in the parking space need to be accurately identified so that the vehicle can be parked safely in the parking space. At present, most vehicles identify parking spaces and obstacles in parking spaces by first collecting image information corresponding to the parking space through a camera, and then determining the pixel coordinates of the parking space line and the pixel coordinates of the obstacles in the parking space based on the image information, and finally converting the pixel coordinates into world coordinates to obtain the coordinates of the parking space line and the obstacles in the parking space required by the vehicle.

[0034] Therefore, in the process of converting pixel coordinates into world coordinates, an accurate algorithm is needed to obtain the coordinates of the parking space lines and obstacles in the parking space required by the vehicle.

[0035] To solve these problems, the embodiments of the present application respectively propose a parking space calibration model training method, a parking space calibration model training device, an electronic device, a computer-readable storage medium and a computer program product, which will be described in detail below.

[0036] See also Figure 2 , Figure 2 The schematic flow diagram of steps S210-S240 of a method for training a parking space calibration model provided in an embodiment of the present application is shown. The method is jointly executed by the vehicle terminal 110 and the server 120, and the method includes steps S210 to S240, which are described in detail as follows:

[0037] S210: Obtain world coordinates corresponding to multiple groups of parking angles collected by the laser radar and pixel coordinates corresponding to multiple groups of parking angles collected by the camera.

[0038] Optionally, at least 10 groups or more of world coordinates and pixel coordinates corresponding to parking angles are obtained.

[0039] It should be understood that in a standard parking space, each set of parking space angles corresponds to four corners on each parking space line. If the parking space is a non-standard parking space, that is, a parking space with a non-rectangular shape, each set of parking space angles includes all corners of the parking space.

[0040] In an exemplary embodiment of the present application, Figure 3 The flowchart of steps S211-S213 of a method for training a parking space calibration model provided in an embodiment of the present application is shown. Figure 3 The method for obtaining the world coordinates corresponding to the multiple groups of parking angles collected by the laser radar includes steps S211 to S213, which are described in detail as follows:

[0041] Step S211: Receive the laser radar data returned by the preset parking space.

[0042] For example, after driving the vehicle into the garage, if there are vacant parking spaces on both sides of the vehicle (for example, there are five vacant parking spaces on either side), and the parking space line and the four parking space angles on the parking space line are clear, it can be defined as a preset parking space. Then, if the vehicle and the sensors involved in collecting LiDAR data are in normal and fault-free status and meet the calibration conditions, laser pulses can be emitted toward the area where the parking space is located and the reflected light can be received, thereby collecting LiDAR data of the surrounding environment.

[0043] Step S212: Obtain three-dimensional point cloud data corresponding to the preset parking space based on the laser radar data.

[0044] For example, the laser radar data includes the laser reflectivity of the garage floor and the parking space line. Therefore, the three-dimensional point cloud data corresponding to the preset parking space can be extracted according to the laser reflectivity of the garage floor and the parking space line.

[0045] Step S213: obtaining the parking space angle world coordinates corresponding to the parking space angle of the preset parking space based on the three-dimensional point cloud data.

[0046] In an exemplary embodiment of the present application, obtaining pixel coordinates corresponding to multiple groups of parking angles collected by the camera includes: identifying the same parking space through the camera, identifying the parking space line and four parking angles on the parking space line, and outputting the pixel coordinates of the four parking angles.

[0047] It should be understood that "the same parking space" means that after obtaining the lidar data of a parking space through the lidar, the image information of the parking space is obtained through the camera, and then the parking space line and the four parking space angles on the parking space line are identified based on the image information, and the pixel coordinates of the four parking space angles are output.

[0048] S220: If the world coordinates and the pixel coordinates meet the preset accuracy requirements, the model training data is determined based on the world coordinates and the pixel coordinates. The vehicle coordinate system is based on a right-handed Cartesian coordinate system, and the origin of the coordinate system is the ground projection of the center of the rear axle, which is also the origin of the world coordinate system.

[0049] In an exemplary embodiment of the present application, after the world coordinates and the pixel coordinates are acquired, the world coordinates and the pixel coordinates need to be preprocessed to eliminate the world coordinates and the pixel coordinates that do not meet the preset accuracy requirements.

[0050] For example, the raw point cloud data collected by the vehicle for determining the world coordinates may contain noise and irrelevant reflection points. Therefore, the world coordinates need to be preprocessed to obtain world coordinates that meet the preset accuracy requirements. Specifically: filtering, denoising and ground segmentation can be used to remove points of non-ground objects such as vehicles and pedestrians. Then, the geometric features in the preprocessed point cloud data are used to detect the parking space line and the four parking space angles on the parking space line through the algorithm. In this process, edge detection, corner point detection or model-based matching is involved. Among them, the lidar can measure the size and shape of the parking space, determine the location of the parking space by analyzing the gaps and spatial layout in the point cloud data, and then train the deep learning model to identify the characteristics of the parking space, such as parking space lines, parking space angles and idle status.

[0051] For example, in the process of preprocessing pixel coordinates, the image collected by the camera is grayed, filtered (noise removed), and edge detected to enhance the features of the parking space lines in the image. Then, an edge detection algorithm (such as the Canny algorithm) is used to identify the parking space lines. Among them, the parking space lines usually have the characteristics of straight lines or curves, which can be identified by methods such as Hough transform; further, based on the parking space lines, a corner detection algorithm (such as Harris corner detection or Shi-Tomasi corner detection) is used to identify the four parking corners of the parking space. The parking space outline is identified and the four parking space corners are extracted. The pixel coordinates of each parking space corner are read by the camera CCD / COMES, and finally the pixel coordinates of the four parking space corners that meet the preset accuracy requirements can be output.

[0052] S230: Training the parking space calibration model based on the model training data to obtain a trained target parking space calibration model.

[0053] In an exemplary embodiment of the present application, the parking space calibration model is trained by a BP neural network based on the model training data to obtain a trained target parking space calibration model. The training process of the parking space calibration model by the BP neural network is mainly divided into two stages: forward propagation and back propagation.

[0054] For example, the BP neural network structure includes three layers: input layer, intermediate hidden layer and output layer; the input layer contains 2 neurons, the hidden layer contains 6 neurons and 6 biases, and the output layer contains 3 neurons.

[0055] Further, Figure 4 The schematic flow chart of steps S231-S233 of a method for training a parking space calibration model provided in an embodiment of the present application is shown. Figure 4 The method of training the parking space calibration model based on the model training data to obtain the trained target parking space calibration model includes steps S231 to S233, which are described in detail as follows:

[0056] S231: Use the pixel coordinates as the input layer signal for forward propagation to obtain the output layer world coordinates of the parking space calibration model.

[0057] It should be understood that forward propagation is the process of signal propagation from the input layer to the output layer in the network. Specifically, the signal of the input layer is passed to the hidden layer after weighting and operation. The neurons in the hidden layer receive the signal from the previous layer, which is processed by the activation function and then passed to the next layer until it finally reaches the output layer. The output of each layer is the source of the input of the next layer. Among them, the threshold of the hidden layer uses the tangent function (such as the hyperbolic tangent function). This is because the BP neural network has only one hidden layer, the normalized output range is between (-1, 1), and the convergence speed of the tanh function is greater than that of the sigmoid function. The specific formula is as follows:

[0058]

[0059] Further, Figure 5 The flowchart of steps S2311-S2313 of the training method of a parking space calibration model provided in an embodiment of the present application is shown. Figure 5 , the method of forward propagating the pixel coordinates as the input layer signal to obtain the output layer world coordinates of the parking space calibration model includes steps S2311 to S2313, which are described in detail as follows:

[0060] S2311: The pixel coordinates are used as input layer signals and weighted according to a preset weighting strategy to obtain weighted pixel coordinates.

[0061] For example, now let the weight between input layer node i and hidden layer node j be W ij , the bias term of node j is b j , the output value of each hidden layer node is x j, and the output value of each node is realized based on the output values ​​of all nodes in the upper layer, the weights of the current node and all nodes in the upper layer, the bias term of the current node, and the activation function. The specific calculation method is as follows:

[0062]

[0063] Where: w ij is the weight between hidden layer node i and node j; b j is the bias term of node j. m is the number of neurons in the input layer; x i is the input parameter of the input layer; S j is the output of the input layer and also the input of the hidden layer.

[0064] X j =f(S j )

[0065] Where: X j is the output of the hidden layer and also the input of the output layer.

[0066] Furthermore, the neuron output of the output layer is:

[0067]

[0068] Where: y n is the output of the output layer, and n is the number of neurons in the output layer.

[0069] S2312: Process the weighted pixel coordinates through a hidden layer to obtain processed pixel coordinates.

[0070] S2313: Propagate the processed pixel coordinates to the output layer of the parking space calibration model to obtain the output layer world coordinates of the parking space calibration model.

[0071] S232: Back-propagate the output layer world coordinates to the input layer to obtain a model error value of the parking space calibration model.

[0072] It should be understood that back propagation is the process of backpropagating the model error value from the output layer to the input layer, which is used to adjust the connection weights and bias terms in the network to reduce the model error value between the network output and the expected output.

[0073] For example, backpropagation uses the delta learning algorithm to adjust the weights of each layer.

[0074] Further, Figure 6 The flowchart of steps S2321-S2322 of a method for training a parking space calibration model provided in an embodiment of the present application is shown. Figure 6, the method of back-propagating the output layer world coordinates to the input layer to obtain the model error value of the parking space calibration model includes steps S2321 to S2322, which are described in detail as follows:

[0075] S2321: Calculate the model error value between the output layer world coordinates and the actual world coordinates, where the actual world coordinates are the world coordinates in the model training data.

[0076] For example, the formula for calculating the model error between the network output and the ideal output is as follows:

[0077] e(k)=y(k)-y n (k)

[0078] Among them: y(k) is the world coordinate of the collected parking space corner point.

[0079] Furthermore, the model error value performance index function is:

[0080]

[0081] S2322: Adjust the weight value corresponding to each layer in the parking space calibration model based on the model error value to obtain a corrected parking space calibration model.

[0082] For example, the method of adjusting the weight value corresponding to each layer in the parking space calibration model based on the model error value to obtain the modified parking space calibration model includes: first obtaining a preset weight value adjustment strategy, the preset weight value adjustment strategy includes a weight value gradient descent adjustment model; then calculating the weight corresponding to each layer in the input layer from the output layer world coordinates back propagated according to the weight value gradient descent adjustment model. The specific weight learning algorithm is as follows:

[0083]

[0084] Where: η is the learning rate, is the gradient of the loss function with respect to the weights.

[0085] Furthermore, the network weight at time (k+1) is:

[0086] W jk (k+1)=W jk (k)+Δw jk

[0087] Furthermore, the connection weight learning algorithm of the hidden layer and the input layer is as follows:

[0088]

[0089] Furthermore, the network weight at time K+1 is:

[0090] wij (k+1)=w ij (k)+Δw ij

[0091] S233: Correcting the parking space calibration model based on the model error value to obtain a target parking space calibration model.

[0092] In an exemplary embodiment of the present application, the method steps of correcting the parking space calibration model based on the model error value to obtain the target parking space calibration model include: if the model error value is greater than a preset error threshold, adjusting the weight value corresponding to each layer in the parking space calibration model based on the model error value until the model error value is not greater than the preset error threshold. Then, the parking space calibration model is corrected based on the model error value to obtain the target parking space calibration model.

[0093] It should be understood that the preset error threshold can be determined according to the user's accuracy selection when the user calibrates the parking space, so as to achieve the user's personalized setting.

[0094] S240: Determine the reference coordinates corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0095] For example, after the target parking space calibration model is determined, the image information about the target parking space can be collected through the camera on the vehicle, and then the parking space angle pixel coordinates of the target parking space in the image information can be input into the trained model to obtain the reference coordinates corresponding to the target parking space. Finally, the reference coordinates of each obstacle in the parking space can be accurately calculated.

[0096] Furthermore, the training method of the parking space calibration model also includes: after detecting that the vehicle is parked at the target parking space, obtaining the real-time world coordinates corresponding to the target parking space collected by the laser radar and the real-time pixel coordinates corresponding to the target parking space collected by the camera, and judging whether the real-time world coordinates and the real-time pixel coordinates corresponding to the target parking space match the target parking space calibration model, so as to update the target parking space calibration model when the real-time world coordinates and the real-time pixel coordinates corresponding to the target parking space do not match the target parking space calibration model.

[0097] In an exemplary embodiment of the present application, the method steps for determining whether the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space match the target parking space calibration model include: obtaining the actual error value between the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space, and determining whether the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space match the target parking space calibration model based on the magnitude relationship between the actual error value and the preset error value. If the actual error value is greater than the preset error value, it is determined that the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space do not match the target parking space calibration model, otherwise they match.

[0098] Furthermore, the training method of the parking space calibration model also includes: if the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space do not match the target parking space calibration model, then updating the target parking space calibration model based on the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space.

[0099] Furthermore, the method steps of updating the target parking space calibration model based on the real-time world coordinates and the real-time pixel coordinates corresponding to the target parking space include: updating the target parking space calibration model based on the real-time world coordinates and the real-time pixel coordinates.

[0100] For example, the method steps for updating the target parking space calibration model based on the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space include: if the actual error value between the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space is greater than the preset error value and less than the error threshold, and the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space meet the preset accuracy requirements, then the real-time world coordinates and real-time pixel coordinates corresponding to the target parking space are determined as new model training data, and the parking space calibration model is updated based on the new model training data to obtain an updated target parking space calibration model. Among them, the preset error value and the error threshold can be adjusted according to factors such as the sensor accuracy of the vehicle.

[0101] It should be understood that the target parking space calibration model can continuously update the target parking space calibration model through the real-time world coordinates and real-time pixel coordinates obtained by the vehicle after parking, so as to improve the accuracy of the target parking space calibration model, thereby gradually improving the target parking space calibration model's ability to adapt to working conditions.

[0102] In an exemplary embodiment of the present application, the training method of the parking space calibration model also includes: if the actual error value between the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space is greater than the preset error value, and greater than or equal to the error threshold, then obtaining the sensor data of the vehicle and the environmental data of the vehicle, and analyzing the reasons why the actual error value is greater than the preset error value and greater than or equal to the error threshold based on the sensor data of the vehicle and the environmental data of the vehicle, so as to obtain the corresponding error characterization information; finally, updating the target parking space calibration model based on the error characterization information and the actual error value corresponding to the error characterization information. Among them, the sensor includes at least a laser radar for obtaining real-time world coordinates and a camera for obtaining real-time pixel coordinates; the environmental data of the vehicle includes at least the light brightness information of the vehicle's environment and the obstacle information of the vehicle's environment.

[0103] It should be understood that when the vehicle drives into some dimly lit garages or the lens of the vehicle camera has stains such as water droplets, the real-time pixel coordinates corresponding to the target parking space collected by the camera or the real-time world coordinates corresponding to the target parking space collected by the lidar may be incorrect, resulting in a large actual error value between the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space.

[0104] At this time, the error characterization information and the actual error value corresponding to the error characterization information can be determined as abnormality exclusion data, and the abnormality exclusion data can be updated in a separate database in the target parking space calibration model to be used to troubleshoot the cause of the vehicle's sensor failure or the cause of special environment when the actual error value is greater than the preset error value and greater than or equal to the error threshold.

[0105] For example, after obtaining the sensor data of the vehicle, if it is determined based on the sensor data that the actual error value is greater than the preset error value due to the contamination of the vehicle's camera lens, then the error characterization information at this time can be determined as camera lens contamination, and the corresponding actual error value (for example, the actual error value at this time is a) is recorded, and then the error characterization information and its corresponding actual error value are updated in a separate database in the target parking space calibration model. When the actual error value of the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space is also a next time, it can be determined that the camera lens is contaminated at this time, and it is necessary to clean the contamination and re-collect the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space and then update the target parking space calibration model.

[0106] It should be understood that if the vehicle's sensor is in an abnormal state for a long time or the environment in which the vehicle is located is abnormal for a long time, and the user does not find it, the vehicle may collect data that does not conform to the updated target parking space calibration model to update the target parking space calibration model, which may eventually lead to the accuracy of the target parking space calibration model becoming worse and worse. Therefore, updating the target parking space calibration model through error characterization information and the actual error value corresponding to the error characterization information can avoid this problem and further improve the accuracy of the target parking space calibration model.

[0107] In the present invention, the training method of the parking space calibration model first selects world coordinates and pixel coordinates that meet preset accuracy requirements from multiple sets of world coordinates and pixel coordinates corresponding to the acquired parking space angles as model training data; then the parking space calibration model is trained with the model training data to obtain the target parking space calibration model, and finally the reference coordinates corresponding to the parking space are determined based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0108] Therefore, the calibration model obtained by the training method of the parking space calibration model of the present application can not only obtain the precise reference coordinates corresponding to the parking space line of the target parking space and the internal obstacles of the target parking space, but also compared with the traditional camera, when obtaining the calibration model, it is necessary to complete the calibration in two steps (camera internal parameter calibration and external parameter calibration), which shortens the calibration time and improves the camera calibration efficiency. It does not rely on special calibration rooms and calibration stations, does not need to place a camera calibration plate, and avoids the calibration plate placement size model error value and cumbersome calibration process.

[0109] In summary, the parking space calibration model obtained by the training method of the parking space calibration model of the present invention can provide a high-precision calibration method. Among them, the use of BP neural network for camera calibration can reduce human intervention, reduce errors caused by improper operation, and improve the repeatability and consistency of the calibration process; and the BP neural network can handle challenging scenes, such as large distortion, occlusion and uneven lighting, etc., solving the problem that traditional methods may be difficult to accurately calibrate camera parameters in these cases.

[0110] Figure 7 FIG. 1 is a schematic diagram showing a structure of a training device for a parking space calibration model provided in an embodiment of the present application. Figure 7 As shown, in this embodiment, a training device for a parking space calibration model is also provided, which is used to execute the training method for the parking space calibration model in the above embodiment.

[0111] Furthermore, the training device 300 of the parking space calibration model includes an acquisition module 310, which is used to obtain world coordinates corresponding to multiple groups of parking space angles collected by the laser radar and pixel coordinates corresponding to multiple groups of parking space angles collected by the camera.

[0112] Furthermore, the training device 300 for the parking space calibration model also includes a first determination module 320, and the first determination module 320 is used to determine the model training data based on the world coordinates and the pixel coordinates if the world coordinates and the pixel coordinates meet the preset accuracy requirements.

[0113] Furthermore, the training device 300 for the parking space calibration model further includes a training module 330, and the training module 330 is used to train the parking space calibration model based on the model training data to obtain a trained target parking space calibration model.

[0114] Furthermore, the training device 300 of the parking space calibration model also includes a second determination module 340, which is used to determine the reference coordinates corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

[0115] It should be noted that the training device for the parking space calibration model provided in the above embodiment and the training method for the parking space calibration model provided in the above embodiment belong to the same concept, and the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here.

[0116] Figure 8 A structural schematic diagram of an embodiment of a computing device of the present application is shown, which shows a structural schematic diagram of a computer system of a computing device suitable for implementing an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the computing device.

[0117] See also Figure 8 As shown, the computing device includes: a controller; a memory for storing one or more programs, and when the one or more programs are executed by the controller, the above-mentioned parking space calibration model training method is executed.

[0118] Please continue reading Figure 8 As shown, the computer system of the computing device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 to the random access memory (RAM) 503, such as executing the method in the above embodiment. In the RAM 503, various programs and data required for system operation are also stored. The CPU 501, ROM 502 and RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0119] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0120] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 509, and / or installed from a removable medium 511. When the computer program is executed by a central processing unit (CPU) 501, various functions defined in the system of the present application are executed.

[0121] Another aspect of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the above parking space calibration model training method when the computer program is executed by a processor. The computer-readable storage medium may be included in the computing device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0122] Another aspect of the present application further provides a computer program product or a computer program, which includes at least one executable instruction. When the executable instruction is executed on a computing device / equipment, the computing device / equipment executes the above-mentioned training method for the parking space calibration model.

[0123] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as a part of a carrier wave, wherein a computer-readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0124] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the apparatus, methods and computer program products according to various embodiments of the present application. Among them, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0126] According to one aspect of an embodiment of the present application, a computer system is also provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM), such as executing the method in the above embodiment. In RAM, various programs and data required for system operation are also stored. CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0127] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed.

[0128] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0129] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0130] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0132] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A method for training a parking space calibration model, characterized in that: The method comprises: Obtain the world coordinates corresponding to the multiple groups of parking angles collected by the laser radar and the pixel coordinates corresponding to the multiple groups of parking angles collected by the camera; If the world coordinates and the pixel coordinates meet the preset accuracy requirement, determining the model training data based on the world coordinates and the pixel coordinates; Training the parking space calibration model based on the model training data to obtain a trained target parking space calibration model; The reference coordinates corresponding to the target parking space are determined based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

2. The method according to claim 1, characterized in that The method steps of training the parking space calibration model based on the model training data to obtain the trained target parking space calibration model include: The pixel coordinates are used as input layer signals for forward propagation to obtain output layer world coordinates of the parking space calibration model; Back-propagating the output layer world coordinates to the input layer to obtain a model error value of the parking space calibration model; The parking space calibration model is corrected based on the model error value to obtain a target parking space calibration model.

3. The method according to claim 2, characterized in that The method steps of using the pixel coordinates as input layer signals for forward propagation to obtain the output layer world coordinates of the parking space calibration model include: The pixel coordinates are used as input layer signals and weighted according to a preset weighting strategy to obtain weighted pixel coordinates; Processing the weighted pixel coordinates through a hidden layer to obtain processed pixel coordinates; The processed pixel coordinates are propagated to the output layer of the parking space calibration model to obtain the output layer world coordinates of the parking space calibration model.

4. The method according to claim 2, characterized in that The method step of back-propagating the output layer world coordinates to the input layer to obtain the model error value of the parking space calibration model includes: Calculating a model error value between the output layer world coordinates and the actual world coordinates, wherein the actual world coordinates are the world coordinates in the model training data; The weight value corresponding to each layer in the parking space calibration model is adjusted based on the model error value to obtain a corrected parking space calibration model.

5. The method according to claim 4, characterized in that The method step of adjusting the weight value corresponding to each layer in the parking space calibration model based on the model error value to obtain a corrected parking space calibration model includes: Obtaining a preset weight value adjustment strategy, wherein the preset weight value adjustment strategy includes a weight value gradient descent adjustment model; The weight corresponding to each layer in the input layer is calculated by back-propagating the output layer world coordinates according to the weight value gradient descent adjustment model.

6. The method according to claim 1, characterized in that The method further comprises: After detecting that the vehicle is parked at the target parking space, obtaining the real-time world coordinates corresponding to the target parking space collected by the laser radar and the real-time pixel coordinates corresponding to the target parking space collected by the camera; Obtaining an actual error value between the real-time world coordinates corresponding to the target parking space and the reference coordinates corresponding to the target parking space; If the actual error value is greater than the preset error value, the target parking space calibration model is updated based on the real-time world coordinates and the real-time pixel coordinates corresponding to the target parking space.

7. The method according to claim 6, characterized in that The method further comprises: If the actual error value is greater than the preset error value and greater than or equal to the error threshold, acquiring sensor data of the vehicle and environmental data of the vehicle; determining error characterization information based on the sensor data and the environmental data; The target parking space calibration model is updated based on the error characterization information and the actual error value corresponding to the error characterization information.

8. A training device for a parking space calibration model, characterized in that: The device comprises: An acquisition module is used to obtain world coordinates and pixel coordinates corresponding to multiple sets of parking angles; A first determination module, configured to determine model training data based on the world coordinates and the pixel coordinates if the world coordinates and the pixel coordinates meet a preset accuracy requirement; A training module, used for training the parking space calibration model based on the model training data to obtain a trained target parking space calibration model; The second determination module is used to determine the reference coordinates corresponding to the target parking space based on the target parking space calibration model and the real-time parking space angle pixel coordinates of the target parking space.

9. A computing device, characterized in that include: Controller; A memory is used to store one or more programs. When the one or more programs are executed by the controller, the controller implements the training method of the parking space calibration model described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, which includes executable instructions. When the executable instructions are executed on the training device / computing equipment of the parking space calibration model, the training device / computing equipment of the parking space calibration model executes the training method of the parking space calibration model as described in any one of claims 1 to 7.

Images