Automatic parking method, device and equipment

Abstract

The invention provides an automatic parking method, device and equipment. A target image of a front area of a vehicle is collected through a vehicle-mounted camera of the vehicle; if a preset road surface identifier is detected from the target image, obtaining a target stop line of a parking space identified by the road surface identifier; acquiring the current position of the vehicle; according to the position relation between the target stop line and the current position of the vehicle, the parking track of the vehicle is obtained; and controlling the vehicle to park according to the parking track. According to the method, the vehicle can be controlled to realize automatic parking through the image acquired by the vehicle-mounted camera, so that the automatic parking efficiency of the vehicle is improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to an automatic parking method, device and equipment. Background Art

[0002] With the development of information technology and computer technology, the research of autonomous driving technology has been pushed to a new level. Among them, autonomous precision parking technology is an important part of unmanned vehicles and also a difficult point. In the special operating environment of the port, higher requirements are placed on precision parking.

[0003] The existing automatic parking control system for self-driving cars mainly uses fuzzy logic models, combines the information received by multiple sensors, fuses the data of multiple sensors and puts them into the model for calculation, so as to control the parking of self-driving cars. However, this method requires obtaining information from multiple sensors and performing a large number of nonlinear equation calculations. The calculations are complex and take a long time, resulting in low efficiency of automatic parking of vehicles. Summary of the invention

[0004] The present application provides an automatic parking method, device and equipment, which can improve the efficiency of automatic parking of vehicles.

[0005] In a first aspect, an embodiment of the present application provides an automatic parking method, the method comprising:

[0006] The target image of the area in front of the vehicle is collected by using the vehicle's onboard camera;

[0007] If a preset road surface mark is detected from the target image, a target stop line of the parking space marked by the road surface mark is obtained;

[0008] Get the current position of the vehicle;

[0009] According to the positional relationship between the target stop line and the current position of the vehicle, the parking trajectory of the vehicle is obtained;

[0010] Control the vehicle to park according to the parking trajectory.

[0011] In a second aspect, the present application provides an automatic parking device, the device comprising:

[0012] A collection module, used for collecting a target image of the area in front of the vehicle through an on-board camera of the vehicle;

[0013] A first acquisition module is used to acquire a target stop line of a parking space identified by a preset road surface mark if a preset road surface mark is detected from a target image;

[0014] The second acquisition module is used to acquire the current position of the vehicle;

[0015] A determination module, for obtaining a parking trajectory of the vehicle according to a positional relationship between a target stop line and a current position of the vehicle;

[0016] The control module is used to control the vehicle to park according to the parking trajectory.

[0017] In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;

[0018] When the processor executes the computer program instructions, the automatic parking method as in any one of the embodiments of the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, an automatic parking method as in any one of the embodiments in the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product. When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes an automatic parking method as in any one of the embodiments in the first aspect above.

[0021] In an automatic parking method, device and apparatus provided in an embodiment of the present application, a target image of the area in front of the vehicle is collected by the vehicle's on-board camera; if a preset road surface mark is detected from the target image, the target stop line of the parking space marked by the road surface mark is obtained; the current position of the vehicle is obtained; according to the positional relationship between the target stop line and the current position of the vehicle, the parking trajectory of the vehicle is obtained; and the vehicle is controlled to park according to the parking trajectory. In the above manner, the image collected by the vehicle's on-board camera is obtained, and the target stop line is identified according to the image, and the parking trajectory of the vehicle is generated based on the positional relationship between the current position of the vehicle and the target stop line, and the vehicle is controlled to park according to the parking trajectory. The image collected by the on-board camera can be used to control the vehicle to achieve automatic parking. On the one hand, the whole process does not require human participation. On the other hand, the present application does not need to combine the parameters of multiple sensors to control the vehicle to park automatically. The image collected by the on-board camera can achieve automatic parking of the vehicle, which reduces the computational complexity of the automatic parking of the vehicle, thereby improving the response speed of the automatic parking of the vehicle and the efficiency of the automatic parking of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 is a flow chart of an automatic parking method provided by an embodiment of the present application;

[0024] Figure 2 is a schematic diagram of a road surface marking provided by an embodiment of the present application;

[0025] Figure 3 is a schematic diagram of a vehicle coordinate system and a camera coordinate system provided by an embodiment of the present application;

[0026] Figure 4 It is a structural schematic diagram of an automatic parking device provided in an embodiment of the present application;

[0027] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0030] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0031] In order to solve the problems of the prior art, the embodiments of the present application provide an automatic parking method, device and equipment. The automatic parking method provided by the embodiments of the present application is first introduced below.

[0032] Figure 1 FIG. 1 is a flow chart of an automatic parking method provided by an embodiment of the present application. Figure 1As shown, the method may specifically include the following steps:

[0033] S100, collecting a target image of the area in front of the vehicle through a vehicle-mounted camera.

[0034] Optionally, in the embodiment of the present application, the vehicle-mounted camera may be a vehicle-mounted front-view camera. Specifically, the image data collected by the vehicle-mounted front-view camera may be acquired by calling the interface of the vehicle-mounted front-view camera, and then image processing and recognition may be performed to obtain a target image. Optionally, the image processing and recognition methods include but are not limited to: color space conversion, edge detection, image segmentation, feature extraction, target detection, etc.

[0035] Optionally, in the embodiment of the present application, the position and angle of the camera can be adjusted so that the camera can capture the required area; or the target image of the required area in front of the vehicle can be captured by interception. Specifically, the front area can be the ground area in front of the vehicle.

[0036] Of course, in the embodiment of the present application, acquisition parameters such as resolution, exposure time, white balance, etc. can also be set to ensure that clear and accurate images are acquired. Images can also be acquired once or continuously as needed. In general, the acquired images can be observed through a device such as an onboard computer or a display screen on the vehicle to ensure whether the required target area and target are captured, thereby improving the accuracy of acquisition by the onboard camera.

[0037] S200: If a preset road surface mark is detected from the target image, a target stop line of the parking space marked by the road surface mark is obtained.

[0038] Optionally, in the embodiment of the present application, the target image can be preprocessed, such as denoising, image enhancement, distortion removal, contrast enhancement, etc., to improve the accuracy of subsequent processing and better detect the road surface markings. In the preprocessed target image, image processing algorithms, such as Canny edge detection, Hough transform, etc., can be used to detect the position of the road surface markings in the target image. And according to the position and geometric features of the road surface markings, it can be determined whether the road surface markings identified by the target image are the markings of the preset road surface. If it is determined that the road surface markings identified by the target image are the markings of the preset road surface, the above-mentioned method similar to the road surface marking detection can be used to detect the target stop line of the parking space corresponding to the preset road surface marking. Through the detected target stop line, the position and direction of the stop line can be determined, thereby providing a reference for vehicle parking. It should be noted that in the embodiment of the present application, after the target stop line of the parking space is detected, it is also necessary to verify and calibrate it to ensure the precision and accuracy of the stop line. In addition, since the position and direction of the parking space may be affected by different factors, such as light, weather, etc., real-time detection and calibration are required.

[0039] Alternatively, in a possible implementation of the present application, it is assumed that the two-dimensional code (Hamming code) in the visual tag system (AprilTag) is used as a road surface mark, and then after the corresponding two-dimensional code is detected in the target image, the target image can be converted into a binary image to do Hough transform recognition straight line, and the corresponding filter conditions are set to filter the stop line in front of the vehicle, and the pixel coordinates of the stop line are obtained by Hough change, and the position of the stop line is derived. Or, in another possible implementation of the present application, the pixel mark stop line of a fixed color can also be used, and then the pixel of the color is detected in the image using a color recognition algorithm, and the stop line position is calculated according to the pixel position. Or a specific mode or pattern is used as the stop line, and then the matching mode or pattern is found in the image using an image matching algorithm, and the stop line position is calculated according to the matching result. These methods all need to set corresponding marks or features in the preset road surface mark, and then detect these marks or features by an image processing algorithm, and calculate the position of the stop line according to the detection result.

[0040] Optionally, in an embodiment of the present application, AprilTag is used as a road sign, and the vehicle's front-view camera recognizes AprilTag information to determine whether the vehicle is parked there for operations such as work or charging. Therefore, after determining that the vehicle is located in a preset area, the parking preparation stage can be started, the position of the stop line for parking can be obtained, and automatic parking of the vehicle can be achieved.

[0041] S300, obtaining the current position of the vehicle.

[0042] Optionally, in an embodiment of the present application, the data captured by the vehicle-mounted front-view camera in step 100 can be input based on visual positioning and visual tracking technology to deduce the current position of the vehicle and locate the position of the vehicle. Of course, in another possible implementation of the present application, the current position of the vehicle can also be obtained by other methods, for example: using the Global Positioning System (GPS) to receive satellite signals and determine the position coordinates of the vehicle. The acceleration and angular velocity of the vehicle are sensed by on-board inertial sensors (such as accelerometers, gyroscopes, etc.), and the position and posture of the vehicle are calculated by integration. The surrounding environment images are collected by the on-board camera, and the landmarks or feature points are identified and matched by computer vision algorithms to determine the position and posture of the vehicle. The surrounding environment is scanned by the on-board laser radar to obtain point cloud data, and the position and posture of the vehicle are determined by matching the point cloud and the map. Of course, it is easy to understand that the above methods can also be used in combination to improve the accuracy and robustness of vehicle position estimation.

[0043] S400: Obtain a parking trajectory of the vehicle according to a positional relationship between the target stop line and the current position of the vehicle.

[0044] Optionally, in the embodiment of the present application, after obtaining the current position of the vehicle and the position of the target stop line, the distance and direction that the vehicle needs to move can be calculated, thereby obtaining the parking trajectory. It is easy to understand that when calculating the positional relationship between the vehicle and the target stop line, the coordinates of the current position of the vehicle and the position coordinates of the target stop line need to be converted to the same coordinate system, thereby improving the efficiency and accuracy of calculating the positional relationship between the current position of the vehicle and the target stop line.

[0045] Optionally, in a possible implementation of the present application, a Kalman filter can be used to predict the speed value of the vehicle at the next moment, thereby obtaining a trajectory curve of the vehicle at the current time. Then, the distance and angle that the vehicle needs to move can be calculated based on the positional relationship between the current position of the vehicle and the target stop line. For example, if the current position of the vehicle is on the left side of the target stop line, the parking trajectory is to move a certain distance to the right to reach the parking space, and adjust the angle so that the vehicle faces the parking space.

[0046] Optionally, in another possible implementation of the present application, vector subtraction can also be used to calculate the distance and direction that the vehicle needs to move. And the angle of the vector and the current orientation of the vehicle and the direction to be moved are used to calculate the angle that the vehicle needs to rotate. After calculating the distance and rotation angle that the vehicle needs to move, the parking trajectory of the vehicle can be obtained. Optionally, in an embodiment of the present application, the parking trajectory can be divided into two stages: the first stage is before the vehicle reaches the target stop line along a straight line, and the second stage is that the vehicle is rotated and adjusted so that the vehicle can accurately stop on the target stop line. It should be noted that in the process of calculating the parking trajectory, the motion constraints of the vehicle, such as maximum speed, maximum acceleration, maximum steering angle, etc., need to be taken into account. The motion trajectory of the vehicle can be adjusted according to these motion constraints to ensure that the vehicle can safely stop on the target stop line. At the same time, the vehicle and the camera need to be accurately calibrated and positioned, so that the measurement error can be compensated and corrected, and the accuracy of the automatic parking of the vehicle can be further improved.

[0047] S500: Control the vehicle to park according to the parking trajectory.

[0048] Optionally, in an embodiment of the present application, the parking trajectory can be converted into a specific control instruction, and the control instruction is sent to the control system of the vehicle to control the vehicle to park according to the parking trajectory.

[0049] Specifically, corresponding control commands, such as speed, steering angle, braking and other control signals, can be generated according to the parking trajectory. The control signal is sent to the control system of the vehicle so that the vehicle performs corresponding actions according to the parking trajectory. In specific implementation, the generation and sending of control commands can be achieved through the vehicle's electronic control system or automatic driving system. The specific content and form of the control command will vary according to different control systems, and generally need to be designed and debugged according to actual conditions. For example, for electric vehicles, the output torque and speed of the motor can be controlled by the electronic control system to achieve acceleration and deceleration. At the same time, the steering angle of the wheel can be controlled by the steering system so that the vehicle travels along a predetermined trajectory. When parking, the braking force of the vehicle also needs to be controlled by the braking system so that the vehicle stops at a predetermined position (i.e., before the target stop line). For automatic driving systems, the generation and sending of control commands are generally achieved through algorithms. For example, autonomous vehicle control can be achieved based on the vehicle's sensor data and predetermined trajectory based on machine learning or deep learning methods.

[0050] In an automatic parking method provided in an embodiment of the present application, a target image of the area in front of the vehicle is captured by the vehicle's on-board camera; if a preset road surface mark is detected from the target image, the target stop line of the parking space identified by the road surface mark is obtained; the current position of the vehicle is obtained; according to the positional relationship between the target stop line and the current position of the vehicle, the parking trajectory of the vehicle is obtained; and the vehicle is controlled to park according to the parking trajectory. In the above manner, the image captured by the vehicle's on-board camera is obtained, and the target stop line is identified according to the image, and the parking trajectory of the vehicle is generated based on the positional relationship between the current position of the vehicle and the target stop line, and the vehicle is controlled to park according to the parking trajectory. The image captured by the on-board camera can be used to control the vehicle to achieve automatic parking. On the one hand, the whole process does not require human participation. On the other hand, the present application does not need to combine the parameters of multiple sensors to control the vehicle to automatically park. The image captured by the on-board camera can achieve automatic parking of the vehicle, which reduces the computational complexity of the automatic parking of the vehicle, thereby improving the response speed of the automatic parking of the vehicle and the efficiency of the automatic parking of the vehicle.

[0051] In one embodiment, the above step 100 may specifically perform the following steps:

[0052] S110, collecting an initial image of the area in front of the vehicle through a vehicle-mounted camera;

[0053] S120, taking a screenshot of the initial image to obtain a target image, where the target image includes an area of ​​interest on the ground in front of the vehicle in the initial image.

[0054] Optionally, in an embodiment of the present application, an initial image of the area in front of the vehicle can be captured by the vehicle's front-view camera, and the area on the ground in front of the vehicle in the image can be captured as a region of interest (ROI), and the image corresponding to the region of interest (i.e., the target image) can be subjected to preprocessing operations such as dedistortion to improve the accuracy of image processing.

[0055] Specifically, in a possible implementation of the present application, the vehicle-mounted camera can be turned on and the camera parameters, such as image resolution, exposure time, etc., can be set. The video stream is obtained through the camera, which can be implemented using a common image processing library (such as the Open Source Computer Vision Library (OpenCV)). A frame is extracted from the video stream as the initial image, and image preprocessing (such as denoising, enhancement, dedistortion, etc.) can be performed as needed. It should be noted that calibration is required to correct image distortion, such as distortion correction of fisheye lenses. Subsequently, the collected initial image is saved for subsequent image processing and algorithm analysis. The area on the ground in front of the vehicle in the initial image is then intercepted as the area of ​​interest.

[0056] Optionally, in another possible implementation of the present application, the video capture (VideoCapture) function in the OpenCV library can also be used to obtain the real-time video stream of the vehicle camera, and then the crop function is used to take the area of ​​the ground in front of the vehicle as the area of ​​interest, and cut it down to obtain the target image. Alternatively, with the help of a target detection algorithm, the ground in the area in front of the vehicle and the vehicle are identified, and then the target image is cropped out using image processing technology according to the position and size of the ground. Deep learning models such as a fully convolutional network (FCN), a semantic segmentation network (SegNet), etc. can also be used to perform semantic segmentation on the image in front of the vehicle, segment the area of ​​interest, and obtain the target image. The present application does not limit the specific interception method, as long as the target image can be obtained.

[0057] In these optional embodiments, by intercepting the image of the ground area in front of the vehicle as the target image, the amount of data for subsequent image processing can be saved, and the amount of parking trajectory calculation can be saved, thereby improving the parking efficiency of the vehicle's automatic parking.

[0058] In one embodiment, after step 100 and before step 200, the method may specifically perform the following steps:

[0059] S101, performing edge detection on a target image to obtain multiple graphics.

[0060] Optionally, in the embodiment of the present application, some common edge detection algorithms may be used, such as Canny edge detection, Sobel edge detection, etc. These algorithms may detect the edges in the image and form a series of edge points. Then, these edge points may be clustered into multiple graphics to obtain multiple graphics.

[0061] Specifically, the target image can be grayed to convert the color image into a gray image. The gray image can be Gaussian filtered to remove the noise in the image. The Canny edge detection algorithm can be used to detect the edges in the image. The edges can be Hough transformed to detect the straight lines and circles in the image. These straight lines and circles can be formed into multiple graphics.

[0062] S102, acquiring a target graphic that matches the graphic of the road surface marking from a plurality of graphics.

[0063] Optionally, taking AprilTag as a road sign for illustration, in the embodiment of the present application, a quadrilateral pattern matching the AprilTag road sign pattern can be found in the edge image (i.e., the multiple graphics formed as described above). Specifically, the non-straight edges in the graphics can be first removed, and then the adjacent edges of the straight edges can be searched. Finally, if these straight edges form a closed loop, it is indicated that a quadrilateral is detected, and the graphics with the quadrilateral pattern are used as the target graphics.

[0064] Furthermore, the specific steps for finding a quadrilateral pattern that matches the graphics of the AprilTag road sign are as follows. First, polygon analysis can be performed on the binary edge image obtained (i.e., multiple graphics obtained by edge detection), and polygons can be found using edge structure analysis (Topological structural analysis of digitized binary images). This method is used to determine the surrounding relationship of the binary image edge, that is, to determine the outer edge, the inner edge, and the nested relationship between them. The determined edges have a corresponding relationship with the original image (the edge is the graphics obtained by edge detection), so the original image can be completely represented by the edge. Specifically, the idea of ​​encoding can be adopted. First, different edges are assigned different encoding values, which makes it easy to confirm the hierarchical relationship of polygons. Starting from the starting point, edit the edge pixels and find the edge points of the same type as the starting point. When scanning to the starting point, the polygon is closed. Switch to the next starting point and repeat the operation until all binary points are traversed. Then exclude the edges with less than 4 edges, and further process those that meet the conditions.

[0065] Then, we use the polygon convex hull search algorithm (Finding the convex hull of a simple polygon) to calculate the convex hull of each polygon itself, and find the area of ​​the convex hull and its polygon. The two areas are compared. When the polygon area is larger than the convex hull, the polygon is excluded. This can effectively exclude non-convex polygons and polygons that do not meet the ratio of polygon area to convex hull area greater than 0.8, and retain the remaining polygons that meet the conditions. Finally, for the polygons that finally meet the conditions, the Douglas-Peucker algorithm is used for quadrilateral approximation to obtain a quadrilateral pattern that matches the graphics of the AprilTag road sign.

[0066] Optionally, in another possible implementation of the present application, some shape matching algorithms may be used, such as a shape context-based matching algorithm, a scale-invariant feature transform (SIFT)-based matching algorithm, etc. These algorithms can compare the similarity between two graphics to find the graphics with the highest similarity. The present application does not limit the specific matching method, as long as the target graphics that matches the graphics corresponding to the image identifier can be found.

[0067] S103: Determine whether the target image includes a road sign based on the image area surrounded by the target graphic.

[0068] Optionally, in a possible implementation of the present application, as Figure 2 As shown, the QR code of AprilTag (i.e. Figure 2 Taking the black and white pixel image in the quadrilateral box in the figure as an example of road surface marking, it is possible to determine whether there is a QR code number corresponding to AprilTag in the image area surrounded by the target graphic by encoding and decoding the image area surrounded by the target graphic.

[0069] Optionally, in another possible implementation of the present application, some target detection algorithms can also be used to determine whether there are road markings in the image area surrounded by the target graphic. For example, the target detection algorithm Single Shot MultiBox Detector (SSD) based on deep learning, etc. These algorithms can detect the position and category information of the target object in the image, so as to determine whether the target image includes road markings. In addition, it is also possible to combine image segmentation algorithms, such as FCN, SegNet, etc., to divide the target image into different areas to further determine whether the target image includes road markings. Specifically, color segmentation, shape analysis and other techniques can be used to search for road markings in the image area surrounded by the target graphic; if a road marking is found in the image area, it is considered that the target image contains the road marking, otherwise it is not included.

[0070] In these optional embodiments, by identifying road signs, it is possible to accurately identify whether the vehicle needs to perform parking operations such as parking or charging, thereby improving the accuracy of identifying the vehicle's location.

[0071] In one embodiment, the above step 103 may specifically perform the following steps:

[0072] S1031, generating a point array in the detected target graphic, wherein the point array is used to calculate the value of each color block in the image area.

[0073] Optionally, in the embodiment of the present application, the black-border color block length may be set to 6 color block lengths, and a point array may be generated within the detected quadrilateral to calculate the value of each color block.

[0074] Optionally, in a possible implementation of the present application, OpenCV can be used to create a matrix of a specified size and data type, and then a loop structure is used to traverse the pixels in the target graphic, calculate the value of each pixel, and store it in the corresponding matrix position.

[0075] Optionally, in another possible implementation of the present application, a grid division method can be used to generate small square areas in the target image, called color blocks, and then a point is generated at the center of each color block. The value of each color block can then be calculated using the average value, maximum value, or other statistical methods of all pixel values ​​in the color block. Through the above steps, a point array can be generated in the detected target image to calculate the value of each color block in the image area.

[0076] S1032, constructing a classifier according to the local binary pattern, classifying each color block in the image area surrounded by the target graphic, and obtaining a classification result for each color block, wherein the classification result includes a positive example color block and a negative example color block.

[0077] Optionally, in an embodiment of the present application, after calculating the value of each color block through the point array, a simple classifier is constructed according to local binary patterns (LBP) to classify the color blocks within the quadrilateral to obtain a classification result for each color block.

[0078] Optionally, in a possible implementation of the present application, a classifier can be constructed using OpenCV. It is necessary to first use images of known positive and negative color blocks, and obtain the feature vector of each color block by calculating the local binary pattern. Then, these feature vectors are input into the classifier for training to obtain a classifier model. For each color block in the image area surrounded by the target graphic, the local binary pattern is also calculated to obtain its feature vector, and then the classifier model is used for classification to obtain the classification result of each color block.

[0079] Optionally, a machine learning algorithm, such as a support vector machine or a random forest, may be used to construct a classifier based on the local binary pattern.

[0080] S1033, encoding the positive example color block in the classification result as 1, and encoding the negative example color block as 0, to obtain encoding information of the image area.

[0081] Optionally, in the embodiment of the present application, a positive color block in the classification result may be encoded as 1, and a negative color block may be encoded as 0. In this way, the encoding information of the image region may be obtained.

[0082] Optionally, in the embodiment of the present application, a binary number may be used to represent the coded information, wherein each binary bit corresponds to a color block in the image area, and a binary bit value of 1 indicates that the color block is a positive example, and a binary bit value of 0 indicates that the color block is a negative example. For example, if the image area contains 16 color blocks, of which there are 5 positive example color blocks, a 5-bit binary number may be used to represent the coded information, for example, 00011 indicates that the first three color blocks are negative examples, and the last two color blocks are positive examples.

[0083] S1034: According to a preset correspondence relationship between different encoding information and region numbers, the encoding information of the image region is decoded to obtain a region number corresponding to the image region.

[0084] Optionally, the AprilTag QR code is used as an example of a road sign. In the implementation of this application, different coded information can be numbered in advance corresponding to the area where the vehicle is located, so that the QR code can be decoded to obtain the number, and the area where the vehicle is located can be determined by the number, so as to determine whether the vehicle is parked there for operation or charging. If the automatic driving sends an instruction for the vehicle to park in the area corresponding to the number, the vehicle enters the parking stage.

[0085] Optionally, in a possible implementation of the present application, the decoding method needs to correspond to the encoding method. For example, if the encoding method encodes the classification result of each color block into a binary number, the decoding method needs to convert the binary number into a corresponding decimal number as the area number corresponding to the image area.

[0086] S1035: If the area number corresponding to the image area is the area number corresponding to the road surface sign, determine that the target image includes the road surface sign.

[0087] Optionally, in an embodiment of the present application, by comparing the area number corresponding to the image area and the area number corresponding to the road surface marking, if the two are equal, it can be determined that the target image includes the road surface marking, thereby judging that the vehicle is parked there for operations such as operation or charging, and the vehicle enters the parking stage.

[0088] In these optional embodiments, by identifying road signs, it is possible to accurately identify whether the vehicle needs to perform parking operations such as parking or charging, thereby improving the accuracy of identifying the vehicle's location.

[0089] In one embodiment, the above step 200 may specifically perform the following steps:

[0090] S210: If a preset road surface mark is detected from the target image, the target image is converted into a black and white image.

[0091] Optionally, in an embodiment of the present application, when a preset road surface marking is detected in the target image, it means that the vehicle has entered the parking area. At this time, it is necessary to detect the stop line in the parking area, wherein the stop line is used to instruct the vehicle to park, and the vehicle should stop in front of the stop line when parking.

[0092] Optionally, in a possible implementation of the present application, the target image can be converted into a black and white image using image binarization. The image can be divided into black and white parts according to color, grayscale value or other specific image attributes. Specifically, the image can be binarized using a threshold method, that is, pixels with grayscale values ​​higher than a certain threshold are set to white, and pixels below the threshold are set to black. In this way, a black and white binary image can be obtained.

[0093] Furthermore, in the embodiment of the present application, the color image of the target image can be first converted into a grayscale image; then Gaussian denoising is performed on the grayscale image; a threshold is set according to the image brightness, and the grayscale image is binarized. In this case, the grayscale threshold is set to 80, and pixels with grayscale values ​​greater than 80 are converted to white, and pixels with grayscale values ​​less than 80 are converted to black, so as to obtain a black and white image.

[0094] S220, obtaining a plurality of straight lines in the black-and-white image.

[0095] Optionally, in an embodiment of the present application, to obtain multiple straight lines in a black-and-white image, edge detection can be performed on the binarized black-and-white image to obtain all edges in the image. The Hough Transform algorithm is applied to the edge image to obtain the polar coordinate parameters (i.e., straight line length, angle, and endpoint coordinates) of all straight lines in the image. The straight line parameters are clustered to classify similar straight lines into the same category. The straight lines in each category are merged to obtain a final set of all straight lines.

[0096] Furthermore, in the embodiment of the present application, the binary image may be subjected to edge extraction, and in this case, canny edge extraction is adopted; and the Hough transform function in the OpenCV library is called to obtain the coordinates of all the straight line endpoints.

[0097] S230, determining a target stop line according to line information of each of the plurality of lines, the line information including the length and angle of the line, the target stop line being a stop line among the plurality of lines that matches the preset stop line.

[0098] Optionally, in an embodiment of the present application, the preset stop line is a stop line corresponding to a parking space for the vehicle.

[0099] Optionally, in the embodiment of the present application, since the Hough transform can detect all straight lines, it is necessary to further screen the detection results, for example, filtering according to the characteristics of the straight line length, slope, etc., to retain the straight line that best meets the requirements (i.e., the target stop line). That is, according to the characteristics of the preset stop line such as the length and angle, the interfering straight lines are filtered out to obtain the endpoint coordinates of the target stop line.

[0100] Furthermore, the length and angle range of the stop line can be determined according to the preset stop line information. Each straight line in the straight line set is traversed, the length and angle of the straight line are calculated, and compared with the preset stop line range. If the length and angle of the straight line are within the preset range, the straight line is regarded as the target stop line. If there are multiple straight lines that meet the conditions, the straight line closest to the preset stop line is selected as the target stop line.

[0101] In these optional embodiments, converting a color image into a black and white image can reduce the complexity of image processing, making subsequent image processing and straight line recognition more efficient and accurate; obtaining straight lines in the image can detect and segment the road, determine key information such as lane lines and stop lines, and provide necessary data for autonomous driving decisions; by analyzing the straight line information, a stop line that matches the preset stop line can be found, thereby ensuring that the vehicle complies with traffic regulations during the automatic parking process and improving driving safety and reliability.

[0102] In one embodiment, after the above step 300 and before the above step 400, the method may specifically perform the following steps:

[0103] S301, constructing a vehicle coordinate system, wherein the vehicle coordinate system takes the current position of the vehicle as the coordinate origin.

[0104] Optionally, in the embodiment of the present application, the vehicle coordinate system needs to determine the current position of the vehicle and the direction of the vehicle. Usually, sensors such as GPS and inertial navigation systems can be used to obtain the position and direction information of the vehicle, so as to establish a vehicle coordinate system, with the current position of the vehicle as the coordinate origin and the direction of the vehicle as the positive direction of the X axis. The vehicle coordinate system is a world coordinate system with the origin on the vehicle.

[0105] S302, mapping the target stop line to the vehicle coordinate system to obtain the stop line coordinates.

[0106] Optionally, in an embodiment of the present application, a coordinate transformation is required to map the target stop line to the vehicle coordinate system. First, the position of the stop line in the image coordinate system needs to be determined, and image processing technology can be used to obtain the position information of the stop line. Then, according to the corresponding relationship between the vehicle coordinate system and the image coordinate system, the position of the stop line in the image coordinate system is mapped to the position in the vehicle coordinate system, thereby obtaining the coordinates of the stop line in the vehicle coordinate system.

[0107] In these optional embodiments, constructing a vehicle coordinate system and mapping the target stop line into the vehicle coordinate system can facilitate the subsequent calculation of key information such as the vehicle position and the stop line distance. By mapping the target stop line into the vehicle coordinate system, information such as the distance and angle from the vehicle to the stop line can be easily calculated, thereby obtaining the parking trajectory of the vehicle. At the same time, the establishment of the vehicle coordinate system can also facilitate subsequent vehicle path planning and control, helping the system to more accurately control the vehicle's driving direction and speed. Improve the accuracy of automatic vehicle parking.

[0108] In one embodiment, the above step 302 may specifically perform the following steps:

[0109] S3021, obtain the internal and external parameters of the vehicle camera.

[0110] Optionally, in the embodiment of the present application, if you want to obtain the intrinsic and extrinsic parameters of the vehicle-mounted camera, you usually need to calibrate the camera. Generally, you can use a special pattern such as a chessboard to take multiple images, and process these images to obtain the intrinsic and extrinsic parameters of the camera. Specifically, you can prepare a chessboard pattern with a known chessboard size and fix it on a plane; aim the camera at the chessboard and take multiple images at different angles; use the camera calibration software to process these images to obtain the intrinsic and extrinsic parameters of the camera, including the camera's focal length, principal point coordinates, distortion coefficients and other intrinsic parameter information, as well as the camera's rotation matrix and translation matrix and other extrinsic parameter information.

[0111] Optionally, commonly used camera calibration software includes OpenCV, MATLAB, etc. It should be noted that when performing camera calibration, the diversity of the posture and position of the calibration board should be ensured as much as possible to obtain more accurate intrinsic and extrinsic parameters.

[0112] S3022, determining a first transformation relationship based on external parameters and coordinates of multiple reference objects in the vehicle coordinate system, where the first transformation relationship includes a transformation relationship for transforming the coordinates of the multiple reference objects in the vehicle coordinate system into the coordinates of the multiple reference objects in the camera coordinate system.

[0113] Optionally, in the embodiment of the present application, based on the external parameters and the coordinates of multiple reference objects in the vehicle coordinate system, the coordinates of the reference objects in the vehicle coordinate system can be converted into the coordinates of the reference objects in the camera coordinate system by means of matrix transformation. The camera coordinate system is a three-dimensional rectangular coordinate system formed with the optical center (pinhole) of the camera as the origin, the X-axis as the horizontal direction, the Y-axis as the vertical direction, and the Z-axis pointing to the direction observed by the camera (that is, perpendicular to the imaging plane). It changes with the movement of the camera, that is, it is a relative coordinate system. Figure 3 As shown, O c is the origin of the camera coordinate system, X c , Y c , Z c Indicates the direction of each coordinate axis of the camera coordinate system; v is the origin of the vehicle system, X v , Y v , Z v Indicates the direction of each coordinate axis of the vehicle coordinate system.

[0114] Furthermore, the first conversion relationship is determined by first expressing the coordinates of the reference object in the vehicle coordinate system as a column vector; converting the coordinates of the reference object in the vehicle coordinate system into the coordinates of the reference object in the camera coordinate system according to the rotation matrix and translation matrix of the camera; and expressing the coordinates of the reference object in the camera coordinate system as a column vector. In this way, a conversion matrix from the vehicle coordinate system to the camera coordinate system can be obtained, which is the first conversion relationship.

[0115] Optionally, in another possible implementation of the present application, the coordinates of the reference object in the vehicle coordinate system can be expressed as homogeneous coordinates; the origin of the camera coordinate system can be expressed as homogeneous coordinates; the coordinates of the reference object in the vehicle coordinate system can be converted to coordinates in the camera coordinate system through the camera extrinsic matrix; the coordinates in the camera coordinate system can be converted to homogeneous coordinates; the coordinates in the camera coordinate system can be converted to the coordinates of the reference object in the camera coordinate system; the coordinates of the reference object in the camera coordinate system can be expressed as homogeneous coordinates; according to the coordinates of the reference object in the vehicle coordinate system and the coordinates of the reference object in the camera coordinate system, the first conversion relationship can be obtained by solving. After obtaining the first conversion relationship, the coordinates in the vehicle coordinate system can be converted to coordinates in the camera coordinate system.

[0116] S3023, determining a second transformation relationship based on the internal reference and the coordinates of the multiple reference objects in the camera coordinate system, the second transformation relationship including a transformation relationship for transforming the coordinates of the multiple reference objects in the camera coordinate system into the coordinates of the multiple reference objects in the pixel coordinate system.

[0117] Optionally, in an embodiment of the present application, according to the coordinates of the internal reference and multiple reference objects in the camera coordinate system, the coordinates of the reference object in the camera coordinate system can be converted into the coordinates of the reference object in the pixel coordinate system by a matrix transformation method. Among them, the pixel coordinate system is based on pixels, and the origin of the coordinates is in the upper left corner. The plane rectangular coordinate system with pixels as units fixed on the image has its origin located in the upper left corner of the image, and the X-axis and Y-axis are parallel to the X and Y-axes of the image coordinate system. It relies on the image coordinate system and is also a relative coordinate system. Among them, the image coordinate system is also called a plane coordinate system, which is introduced in transition to introduce the pixel coordinate system. The origin is the intersection of the lens optical axis and the imaging plane, and the X-axis and Y-axis are respectively parallel to the Xc and Yc axes of the camera coordinate system. It is a two-dimensional plane rectangular coordinate system with units of millimeters (mm). It relies on the camera coordinate system and is a relative coordinate system.

[0118] Furthermore, to determine the second transformation relationship, the coordinates of the reference object in the camera coordinate system can be represented as a column vector; according to the intrinsic parameters of the camera, the coordinates of the reference object in the camera coordinate system are converted into the coordinates of the reference object in the pixel coordinate system; the coordinates of the reference object in the pixel coordinate system are represented as a column vector. In this way, a coordinate system from the camera coordinate system to the pixel coordinate system can be obtained. In the process of determining the second transformation relationship, it is necessary to obtain the intrinsic parameter matrix of the camera. The intrinsic parameter matrix contains parameters such as the focal length of the camera, the center point of the image, and the pixel scaling factor, and can map points in the camera coordinate system to the pixel coordinate system. Alternatively, the present application can also solve the transformation matrix of the second transformation relationship by linear regression or other methods.

[0119] In these optional embodiments, by acquiring the intrinsic and extrinsic parameters of the vehicle camera, as well as the coordinate information of multiple reference objects in the vehicle coordinate system, the camera coordinate system and the pixel coordinate system, the conversion and mapping between different coordinate systems can be realized, thereby realizing the precise correspondence between the vehicle coordinate system, the camera coordinate system and the pixel coordinate system. And it is possible to achieve high precision and high reliability of visual perception tasks.

[0120] S3024: Generate a mapping relationship according to the first conversion relationship and the second conversion relationship.

[0121] Optionally, in an embodiment of the present application, the coordinates of the reference object in the vehicle coordinate system can be first converted into the coordinates of the reference object in the camera coordinate system according to a first conversion relationship; then, according to a second conversion relationship, the coordinates of the reference object in the camera coordinate system can be converted into the coordinates of the reference object in the pixel coordinate system; next, for each point in the pixel coordinate system, according to its corresponding point in the camera coordinate system, its corresponding point in the vehicle coordinate system is obtained through the first conversion relationship; thereby, a mapping relationship between the pixel coordinate system and the vehicle coordinate system can be generated.

[0122] In these optional embodiments, by generating a mapping relationship, information in different coordinate systems can be mapped to the vehicle coordinate system, so that the position and motion state of the target and other information can be accurately acquired and judged, thereby improving the accuracy of acquiring the target stop line coordinates.

[0123] S3025, mapping the target stop line to the vehicle coordinate system according to the mapping relationship to obtain the stop line coordinates.

[0124] Optionally, in an embodiment of the present application, after obtaining the target stop line through steps 210-230, the coordinates of the target stop line in the pixel coordinate system can be directly obtained from the Hough transform, and then its coordinate points in the vehicle coordinate system can be obtained through a mapping relationship; further, the slope and intercept of the stop line can be calculated based on the coordinate points in the vehicle coordinate system; finally, based on the slope and intercept of the stop line, the equation of the stop line is calculated with the origin of the vehicle coordinate system as the starting point to obtain the coordinates of the stop line.

[0125] Optionally, in a possible implementation of the present application, a visual positioning algorithm can be studied, and a mapping relationship between a vehicle coordinate system and a pixel coordinate system can be obtained based on the coordinate transformation principle. The coordinate systems involved include a vehicle coordinate system (i.e., a world coordinate system with a coordinate origin on the vehicle), a camera coordinate system, and an image coordinate system. A camera imaging model is constructed, and a mapping relationship is established from the coordinates of an object space point in the vehicle coordinate system to the coordinates of the corresponding image pixel coordinate system, as shown in formula (1).

[0126]

[0127] Among them, Z in formula (1) c is the Z coordinate of the camera coordinate system; u and v are the horizontal and vertical coordinates in the pixel coordinate system; d x d y is the pixel size; u0 and v0 are the image centers; f is the camera focal length; R is the rotation matrix; T is the translation vector; X W , Y W , Z W is the coordinate value in the world coordinate system; f x =f / dx,f y =f / dy,f x 、f y are the normalized focal lengths on the x-axis and y-axis, respectively.

[0128] It can be seen from formula (1) that there is a one-to-one correspondence between the points in the world coordinate system and the points in the pixel coordinate system in the image. When the internal and external parameters of the camera are known, the world coordinates corresponding to the pixel coordinates can be obtained. However, the world coordinates are three-dimensional information, and the pixel coordinates are two-dimensional information. The two-dimensional information lacks one dimension when seeking three-dimensional information. Therefore, the world coordinates obtained by using the pixel coordinates can only obtain the coordinates of a point in a known plane of the world coordinates. The stop line, QR code and other information calculated in this case are all drawn on the ground. Therefore, the information in the world coordinates that needs to be calculated in this case is the coordinates of the point Z=0. Therefore, the position of the vehicle in the world coordinate system can be obtained by using the coordinate conversion relationship through the image information. Therefore, according to the mapping relationship between the coordinates in the pixel coordinate system and the corresponding coordinates in the vehicle coordinate system, the position information of the stop line in the vehicle coordinate system can be derived using the straight line result detected by the filtered Hough transform, that is, the pixel coordinates of the straight line, thereby obtaining the coordinates of the stop line in the vehicle coordinate system.

[0129] In these optional embodiments, by constructing a mapping relationship, an accurate spatial relationship mapping between the vehicle and the surrounding environment is achieved, which enables the autonomous vehicle to more accurately perceive and understand the surrounding environment. By mapping the target stop line into the vehicle coordinate system to obtain the stop line coordinates, the automatically parked vehicle can achieve accurate vehicle control and path planning, thereby achieving safer and more efficient parking.

[0130] In one embodiment, the above step 400 may specifically perform the following steps:

[0131] S410, obtaining vehicle driving information.

[0132] Optionally, in an embodiment of the present application, the vehicle form information may include information such as the vehicle's acceleration value, vehicle speed, angle, etc.

[0133] Optionally, in the embodiments of the present application, the driving information of the vehicle can be obtained in a variety of ways, such as obtaining the driving information of the vehicle through vehicle-mounted sensors. Modern cars are usually equipped with a variety of sensors, such as accelerometers, gyroscopes, speed sensors, etc., and the driving information of the vehicle, such as speed, acceleration, steering angle, etc., can be obtained by reading the data of these sensors. It can also be obtained through the GPS positioning system: by reading the GPS positioning system installed on the vehicle, the vehicle's location, speed, driving direction and other information can be obtained. Or through the Internet of Vehicles technology: Internet of Vehicles technology allows vehicles to obtain real-time road and traffic information through the Internet, such as traffic congestion, construction and other information, so as to optimize the driving route and improve driving efficiency.

[0134] In these optional embodiments, obtaining the vehicle's driving information can understand the vehicle's current state, including speed, direction, acceleration, etc., and can provide data basis for subsequent control and prediction.

[0135] S420, predicting the driving trajectory of the vehicle based on the driving information and the current position of the vehicle.

[0136] Optionally, in the embodiments of the present application, the vehicle's driving trajectory prediction is usually achieved by using machine learning and computer vision technology. Specifically, a trajectory prediction model can be established by using machine learning technology to predict the future driving trajectory based on historical data and the current position of the vehicle. The current position and historical data are then input into the trajectory prediction model to predict the future driving trajectory of the vehicle. The trajectory prediction results are continuously optimized by updating the vehicle's position and driving information in real time to improve the accuracy of the prediction.

[0137] Optionally, the prediction algorithm can use a Bayesian filter-based method for prediction, such as a Kalman filter, an extended Kalman filter, or an unscented Kalman filter. These algorithms can use the current vehicle state (position, speed, etc.) as the initial state, and then recursively and update the vehicle state based on the kinematic model and measurement data to obtain a trajectory prediction for a period of time in the future.

[0138] Optionally, in another possible implementation of the present application, it is also possible to consider using machine learning methods for trajectory prediction, such as neural network-based methods or decision tree methods, etc. These methods can establish a vehicle driving model by learning historical data and make predictions based on current driving information and vehicle status.

[0139] In these optional embodiments, based on the driving information and the current position of the vehicle, the vehicle's driving trajectory can be predicted to pre-determine the vehicle's movement trend and path, which can provide data reference for the next parking plan and improve the accuracy of the vehicle's automatic parking.

[0140] S430, determining parking information of the vehicle according to the positional relationship between the stop line and the current position of the vehicle, the parking information including a distance to be moved by the vehicle and an angle to be moved by the vehicle.

[0141] Optionally, in the embodiment of the present application, the pixel coordinates of the target stop line can be converted from the pixel coordinate system to the vehicle coordinate system through the mapping relationship mentioned above. Then, the Euclidean distance and direction angle between the current position of the vehicle and the stop line can be calculated to obtain the distance and angle information to be moved by the vehicle.

[0142] In these optional embodiments, parking information for the vehicle is determined based on the positional relationship between the stop line and the current position of the vehicle, and can be adjusted based on the predicted vehicle trajectory to ensure that the vehicle stops at the designated location, further improving the accuracy and reliability of automatic parking of the vehicle.

[0143] S440: Determine a parking trajectory of the vehicle according to the parking information and the driving trajectory.

[0144] Optionally, in an embodiment of the present application, after the driving trajectory and parking information of the vehicle are determined, a mathematical model can be used to determine the parking trajectory of the vehicle. For example, a proportional-integral-differential controller (PID) is used. Specifically, the driving trajectory of the vehicle can be used as a reference input, and the parking trajectory of the vehicle can be used as a control output. Then, the PID controller is used to calculate the direction and speed required by the vehicle to achieve the desired parking position and direction.

[0145] Furthermore, the parking position and direction of the vehicle can be considered as the target point, and the current position and direction of the vehicle can be considered as feedback. The PID controller can calculate the control output based on the error signal, which is the difference between the target point and the current position. By appropriately scaling and transforming the control output of the vehicle, it can be converted into the direction and speed of the vehicle. These instructions can then be sent to the vehicle's control system to achieve parking trajectory control.

[0146] Optionally, in another possible implementation of the present application, the position information of the vehicle at the current time is obtained. And according to the Kalman filter prediction, the speed value of the vehicle at the next moment is obtained, so as to obtain the trajectory curve of the vehicle at the current time. The Kalman filter process is an iterative process. The two-dimensional position information (x, y) of each frame of the vehicle is known. After Kalman filtering, the updated car position information (x, y) and its decomposed speed information (Vx, Vy) are output. The positional relationship between the stop line and the current position of the vehicle is used to calculate the distance and angle to be moved by the vehicle, and accurate parking is achieved according to the trajectory curve of the vehicle.

[0147] Optionally, in the embodiment of the present application, AprilTag is first used as a road sign, and the vehicle's front-view camera recognizes the AprilTag information to determine whether the vehicle is parked there for operation or charging; in the process of the autonomous driving vehicle preparing to park, based on visual positioning and visual tracking technology, the data captured by the front-view camera is input to deduce the current position of the vehicle, locate the vehicle position, and obtain the position information of the vehicle at the current time. The speed value of the vehicle at the next moment is predicted and obtained according to the Kalman filter, so as to obtain the trajectory curve of the vehicle at the current time; the position relationship between the step stop line and the current position of the vehicle is used to calculate the distance and angle to be moved by the vehicle, and accurate parking is achieved according to the trajectory curve of the vehicle.

[0148] In these optional embodiments, the parking trajectory of the vehicle is determined based on the parking information and the driving trajectory, so that the parking process of the vehicle can be controlled to prevent the vehicle from deviating from the designated parking position, thereby ensuring the accuracy and safety of parking.

[0149] Figure 4 A schematic structural diagram of an automatic parking device provided in another embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0150] Reference Figure 4 , the automatic parking device may include:

[0151] The acquisition module 401 is used to acquire a target image of the area in front of the vehicle through the vehicle-mounted camera;

[0152] A first acquisition module 402 is used to acquire a target stop line of a parking space identified by a preset road surface mark if a preset road surface mark is detected from a target image;

[0153] The second acquisition module 403 is used to acquire the current position of the vehicle;

[0154] A determination module 404, for obtaining a parking trajectory of the vehicle according to a positional relationship between a target stop line and a current position of the vehicle;

[0155] The control module 405 is used to control the vehicle to park according to the parking trajectory.

[0156] In one embodiment, the acquisition module 401 may include:

[0157] The acquisition submodule is used to acquire an initial image of the area in front of the vehicle through the vehicle's onboard camera;

[0158] The screenshot submodule is used to take a screenshot of the initial image to obtain a target image, where the target image includes an area of ​​interest on the ground in front of the vehicle in the initial image.

[0159] In one embodiment, the automatic parking device may further include:

[0160] A detection module, used for performing edge detection on a target image to obtain multiple graphics;

[0161] A third acquisition module is used to acquire a target graphic that matches the graphic of the road surface mark from the multiple graphics;

[0162] The fourth acquisition module is used to determine whether the target image includes a road surface mark based on the image area surrounded by the target graphic.

[0163] In one embodiment, the fourth acquisition module may include:

[0164] A first generating submodule is used to generate a point array in the detected target graphic, and the point array is used to calculate the value of each color block in the image area;

[0165] The classification submodule is used to construct a classifier according to the local binary pattern, classify each color block in the image area surrounded by the target graphic, and obtain the classification result of each color block, which includes positive color blocks and negative color blocks;

[0166] The first determination submodule is used to encode the positive color block in the classification result as 1 and the negative color block as 0 to obtain the encoding information of the image area;

[0167] A decoding submodule, used to decode the encoding information of the image area according to the preset correspondence between different encoding information and area numbers, and obtain the area number corresponding to the image area;

[0168] The second determination submodule is configured to determine that the target image includes a road sign if the area number corresponding to the image area is the area number corresponding to the road sign.

[0169] In one embodiment, the first acquisition module 402 may include:

[0170] A conversion submodule, for converting the target image into a black and white image if a preset road surface mark is detected from the target image;

[0171] A first acquisition submodule is used to acquire a plurality of straight lines in a black and white image;

[0172] The third determination submodule is used to determine the target stop line according to the line information of each of the multiple straight lines, the line information includes the length and angle of the straight line, and the target stop line is the stop line among the multiple straight lines that matches the preset stop line.

[0173] In one embodiment, the automatic parking device may further include:

[0174] A construction module is used to construct a vehicle coordinate system, where the vehicle's current position is used as the coordinate origin;

[0175] The mapping module is used to map the target stop line into the vehicle coordinate system to obtain the stop line coordinates.

[0176] In one embodiment, the mapping module may include:

[0177] The second acquisition submodule is used to obtain the internal and external parameters of the vehicle-mounted camera;

[0178] A fourth determination submodule is used to determine a first conversion relationship according to the external parameter and the coordinates of the multiple reference objects in the vehicle coordinate system, wherein the first conversion relationship includes a conversion relationship of converting the coordinates of the multiple reference objects in the vehicle coordinate system into the coordinates of the multiple reference objects in the camera coordinate system;

[0179] A fifth determination submodule, configured to determine a second conversion relationship according to the internal reference and the coordinates of the multiple reference objects in the camera coordinate system, wherein the second conversion relationship includes a conversion relationship of converting the coordinates of the multiple reference objects in the camera coordinate system into the coordinates of the multiple reference objects in the pixel coordinate system;

[0180] A second generating submodule, used to generate a mapping relationship according to the first conversion relationship and the second conversion relationship;

[0181] The mapping submodule is used to map the target stop line to the vehicle coordinate system according to the mapping relationship to obtain the stop line coordinates.

[0182] In one embodiment, the determination module 404 may include:

[0183] The third acquisition submodule is used to acquire the driving information of the vehicle;

[0184] A prediction submodule, used to predict the vehicle's driving trajectory based on driving information and the vehicle's current position;

[0185] a sixth determination submodule, configured to determine parking information of the vehicle according to a positional relationship between the stop line and the current position of the vehicle, the parking information including a distance to be moved by the vehicle and an angle to be moved by the vehicle;

[0186] The seventh determination submodule is used to determine the parking trajectory of the vehicle according to the parking information and the driving trajectory.

[0187] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application, and are devices corresponding to the above-mentioned battery thermal runaway warning method. All implementation methods in the above-mentioned method embodiment are applicable to the embodiments of the device. Its specific functions and technical effects can be found in the method embodiment part, which will not be repeated here.

[0188] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0189] Figure 5 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown.

[0190] The device may include a processor 501 and a memory 502 storing program instructions.

[0191] When the processor 501 executes the program, the steps in any of the above method embodiments are implemented.

[0192] Exemplarily, the program may be divided into one or more modules / units, one or more modules / units are stored in the memory 502, and executed by the processor 501 to complete the present application. One or more modules / units may be a series of program instruction segments capable of completing a specific function, and the instruction segments are used to describe the execution process of the program in the device.

[0193] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0194] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 502 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 502 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 502 is a non-volatile solid-state memory.

[0195] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0196] The processor 501 implements any one of the methods in the above embodiments by reading and executing program instructions stored in the memory 502 .

[0197] In one example, the electronic device may further include a communication interface 503 and a bus 510. The processor 501, the memory 502, and the communication interface 503 are connected via the bus 510 and communicate with each other.

[0198] The communication interface 503 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0199] Bus 510 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0200] In addition, in combination with the method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores program instructions; when the program instructions are executed by a processor, any one of the methods in the above embodiment is implemented.

[0201] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0202] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0203] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0204] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0205] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware or their combination. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium that can store or transmit information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0206] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0207] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0208] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. An automatic parking method, characterized in that: The method comprises: Capturing a target image of the area in front of the vehicle by using an on-board camera of the vehicle; If a preset road surface mark is detected from the target image, a target stop line of the parking space marked by the road surface mark is obtained; Obtaining the current position of the vehicle; Obtaining a parking trajectory of the vehicle according to a positional relationship between the target stop line and the current position of the vehicle; The vehicle is controlled to park according to the parking trajectory.

2. The method according to claim 1, characterized in that The target image of the area in front of the vehicle is collected by using the vehicle's onboard camera, including: Collecting an initial image of the area in front of the vehicle by using a vehicle-mounted camera of the vehicle; The initial image is captured to obtain the target image, where the target image includes an area of ​​interest on the ground in front of the vehicle in the initial image.

3. The method according to claim 1, characterized in that: After the target image of the area in front of the vehicle is captured by the vehicle-mounted camera, if a preset road surface mark is detected from the target image, before the target stop line of the parking space marked by the road surface mark is obtained, the method further includes: Performing edge detection on the target image to obtain multiple graphics; Acquire a target graphic that matches the graphic of the road surface mark from the multiple graphics; Based on the image area surrounded by the target graphic, it is determined whether the target image includes the road surface marking.

4. The method according to claim 3, characterized in that The determining, based on the image area surrounded by the target graphic, whether the target image includes the road surface marking comprises: Generating a point array in the detected target graphic, wherein the point array is used to calculate the value of each color block in the image area; Constructing a classifier according to the local binary pattern, classifying each color block in the image area surrounded by the target graphic, and obtaining a classification result for each color block, wherein the classification result includes a positive example color block and a negative example color block; Encoding the positive color block in the classification result as 1 and the negative color block as 0 to obtain the encoding information of the image area; According to the preset correspondence between different coding information and area numbers, the coding information of the image area is decoded to obtain the area number corresponding to the image area; If the area number corresponding to the image area is the area number corresponding to the road surface sign, it is determined that the target image includes the road surface sign.

5. The method according to claim 1, characterized in that If a preset road surface mark is detected from the target image, obtaining a target stop line of the parking space marked by the road surface mark includes: If a preset road surface mark is detected from the target image, converting the target image into a black and white image; Acquire a plurality of straight lines in the black and white image; The target stop line is determined according to the line information of each of the plurality of lines, wherein the line information includes the length and angle of the line, and the target stop line is the stop line among the plurality of lines that matches the preset stop line.

6. The method according to claim 1, characterized in that After acquiring the current position of the vehicle and before obtaining the parking trajectory of the vehicle according to the positional relationship between the target stop line and the current position of the vehicle, the method further includes: Constructing a vehicle coordinate system, wherein the vehicle coordinate system takes the current position of the vehicle as a coordinate origin; The target stop line is mapped into the vehicle coordinate system to obtain stop line coordinates.

7. The method according to claim 6, characterized in that Mapping the target stop line to the vehicle coordinate system to obtain the stop line coordinates includes: Obtaining the internal and external parameters of the vehicle-mounted camera; Determine a first conversion relationship according to the external reference and the coordinates of the multiple reference objects in the vehicle coordinate system, wherein the first conversion relationship includes a conversion relationship for converting the coordinates of the multiple reference objects in the vehicle coordinate system into the coordinates of the multiple reference objects in the camera coordinate system; Determine a second transformation relationship according to the intrinsic reference and the coordinates of the multiple reference objects in the camera coordinate system, wherein the second transformation relationship includes a transformation relationship for transforming the coordinates of the multiple reference objects in the camera coordinate system into the coordinates of the multiple reference objects in the pixel coordinate system; Generate a mapping relationship according to the first conversion relationship and the second conversion relationship; According to the mapping relationship, the target stop line is mapped into the vehicle coordinate system to obtain stop line coordinates.

8. The method according to claim 1, characterized in that The obtaining of the parking trajectory of the vehicle according to the positional relationship between the stop line and the current position of the vehicle comprises: Acquiring driving information of the vehicle; Predicting a driving trajectory of the vehicle based on the driving information and the current position of the vehicle; Determining parking information of the vehicle according to a positional relationship between the stop line and the current position of the vehicle, the parking information including a distance to be moved by the vehicle and an angle to be moved by the vehicle; A parking trajectory of the vehicle is determined according to the parking information and the driving trajectory.

9. An automatic parking device, characterized in that: The device comprises: A collection module, used for collecting a target image of the area in front of the vehicle through a vehicle-mounted camera; A first acquisition module, configured to acquire a target stop line of a parking space identified by a preset road surface mark if a preset road surface mark is detected from the target image; A second acquisition module is used to acquire the current position of the vehicle; A determination module, configured to obtain a parking trajectory of the vehicle according to a positional relationship between the target stop line and the current position of the vehicle; A control module is used to control the vehicle to park according to the parking trajectory.

10. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the automatic parking method according to any one of claims 1 to 8 is implemented.

Images