Parking space tracking method, device, electronic equipment and vehicle

Abstract

The present application provides a parking space tracking method, device, electronic device and vehicle. Continuous video frame images are extracted based on a parking space recognition video, and the same parking space identity number is marked on the same parking space in different video frame images to achieve association between different video frame images; the parking space corner point of each parking space in the video frame image is identified, and since the parking space corner point in the video frame image will not change with the movement of the vehicle, the parking space corner point is selected as the identification for parking space tracking; the corner point is predicted every other frame based on the state information and the parking space corner point to obtain the predicted parking space corner point, and the parking space is tracked based on the predicted parking space corner point and the parking space identity number, and the parking space corner point is tracked between consecutive frames by predicting the position of the parking space corner point in the next video frame image, and the same parking space in the consecutive frame images is associated through the parking space identity number to achieve associated tracking of the parking space, so that the parking spaces between consecutive frames correspond one to one.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a parking space tracking method, device, electronic equipment and vehicle. Background Art

[0002] Today, with the explosion of artificial intelligence, more and more automobile manufacturers are empowering cars with artificial intelligence. Autonomous driving, advanced driver assistance systems, driver monitoring systems, smart cockpits, 5G+AI, etc. Automobile manufacturers are divided into two camps, namely, pure vision solutions and vision + radar solutions. Pure vision is the most direct, without multi-sensor fusion, and low cost. In the field of automatic parking, the pure vision solution includes the most critical parking space detection. Only with parking space detection can there be automatic parking. In parking space detection, an algorithm similar to target detection is often used, which is executed for a single image. When the car is driving to the garage, continuous frames are input, and there is no necessary relationship between the detection results of the frames, resulting in the inability to associate the parking spaces between continuous frames, which brings many inconveniences to the actual automatic parking project. In the actual automatic parking project, the corresponding parking spaces between continuous frames need to be unique. It cannot be that because parking space A is detected in this frame, parking space A is detected again in the next frame and is assigned the label parking space B, resulting in the inability to associate the parking spaces between continuous frames one by one, which is not conducive to the subsequent automatic parking project. Summary of the invention

[0003] In view of this, the purpose of the present application is to propose a parking space tracking method, device, electronic equipment and vehicle, which are used to solve the problem that parking spaces cannot correspond one to one between consecutive frames.

[0004] Based on the above purpose, the first aspect of the present application provides a parking space tracking method, comprising:

[0005] Acquire a parking space recognition video in real time, and extract each preset interval frame of the parking space recognition video to obtain a video frame image;

[0006] Identify all parking spaces in the video frame image, and mark the parking spaces to obtain a parking space identity number corresponding to each parking space;

[0007] Identify the parking space corner points of each parking space in the video frame image;

[0008] Acquire vehicle status information, and perform corner point prediction every other frame based on the status information and the parking space corner point to obtain a predicted parking space corner point;

[0009] Parking space tracking is performed according to the predicted parking space corner point and the parking space identity number.

[0010] Optionally, the video frame image includes a continuous first video frame image and a second video frame image; and the parking space tracking according to the predicted parking space corner point and the parking space identity number includes:

[0011] Acquire the predicted parking space corner point predicted according to the first video frame image;

[0012] Identify the actual parking space corner point of each parking space in the second video frame image;

[0013] Parking space tracking is performed according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point.

[0014] Optionally, the performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point includes:

[0015] Determine a target actual parking space corner point corresponding to the predicted parking space corner point pair from among all the actual corner point coordinates according to any one of the predicted corner point coordinates;

[0016] Using the parking space identification number corresponding to the predicted parking space corner point as the parking space identification number of the target parking space corresponding to the target actual parking space corner point;

[0017] The parking space tracking is stopped until the parking space tracking of all the predicted corner point coordinates is completed.

[0018] Optionally, the performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point further includes:

[0019] For any one of the predicted parking space corner points, if there is no target actual parking space corner point corresponding to the predicted parking space corner point among all the actual parking space corner points, the predicted parking space corner point is determined as a faulty parking space corner point;

[0020] Performing validity judgment on the corner point of the faulty parking space;

[0021] In response to the faulty parking space corner point being invalid, stopping tracking the predicted parking space corner point;

[0022] In response to the faulty parking space corner point being valid, corner point missing is filled according to the predicted corner point coordinates corresponding to the faulty parking space corner point.

[0023] Optionally, the performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point further includes:

[0024] For any actual parking space corner point, if there is no predicted parking space corner point corresponding to the actual parking space corner point among all the predicted parking space corner points, the actual parking space corner point is determined as a new starting corner point for parking space tracking, and a new parking space identity number is marked for the new starting corner point.

[0025] Optionally, the determining the validity of the corner point of the faulty parking space includes:

[0026] Determining a field of view boundary of the second video frame image;

[0027] Determining whether the corner point of the faulty parking space exceeds the boundary of the field of view according to the predicted corner point coordinates of the corner point of the faulty parking space;

[0028] In response to the faulty parking space corner point exceeding the boundary of the field of view, determining that the faulty parking space corner point is invalid;

[0029] In response to the faulty parking space corner point not exceeding the boundary of the field of view, it is determined that the faulty parking space corner point is valid.

[0030] Optionally, performing corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point includes:

[0031] Determine the corner point coordinates of the parking space corner point;

[0032] Determining a lateral movement distance and a longitudinal movement distance according to the state information based on a pre-constructed lateral dynamics model of the vehicle;

[0033] Determine a predicted horizontal coordinate according to the horizontal coordinate of the corner point coordinate and the horizontal movement distance;

[0034] Determine the predicted ordinate according to the ordinate of the corner point coordinate and the longitudinal movement distance;

[0035] The predicted corner point coordinates are determined according to the predicted horizontal coordinates and the predicted vertical coordinates, and the predicted parking space corner point is obtained in the video frame image.

[0036] A second aspect of the present application provides a parking space tracking device, characterized in that it includes:

[0037] The image extraction module is configured to: acquire a parking space recognition video in real time, and extract frames at preset intervals from the parking space recognition video to obtain a video frame image;

[0038] The identification module is configured to: identify all parking spaces in the video frame image, and identify the parking spaces to obtain a parking space identification number corresponding to each parking space;

[0039] A corner point recognition module is configured to: recognize a parking space corner point of each parking space in the video frame image;

[0040] The corner point prediction module is configured to: obtain the state information of the vehicle, and perform corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point;

[0041] The corner point tracking module is configured to track the parking space according to the predicted parking space corner point and the parking space identity number.

[0042] The third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method provided in the first aspect of the present application is implemented.

[0043] A fourth aspect of the present application provides a vehicle, comprising the electronic device as described in the third aspect of the present application.

[0044] As can be seen from the above, the parking space tracking method, device, electronic device and vehicle provided by the present application obtain the parking space recognition video in real time, and extract the parking space recognition video at each preset interval frame to obtain a video frame image. In order to meet the input requirements of automatic parking, it is necessary to extract the parking space recognition video into continuous video frame images to provide a basis for subsequent parking space tracking; identify all parking spaces in the video frame image, mark the parking spaces, and obtain a parking space identity number corresponding to each parking space, and realize the association between different video frame images by marking the same parking space in different video frame images with the same parking space identity number; identify the video frame image The parking corner points of each parking space in the video frame image are selected as the identification of parking space tracking because the parking corner points in the video frame image will not change with the movement of the vehicle; the vehicle status information is obtained, and the corner points are predicted every other frame according to the status information and the parking corner points to obtain the predicted parking corner points, and the parking space is tracked according to the predicted parking corner points and the parking space identity number, and the parking corner points are tracked between consecutive frames by predicting the positions of the parking corner points in the next video frame image, and the same parking space in consecutive frame images is associated through the parking space identity number to realize the associated tracking of the parking space, so that the parking spaces between consecutive frames correspond one to one. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0046] Figure 1This is a schematic diagram of parking space detection without parking space tracking between parking spaces in an embodiment of the present application;

[0047] Figure 1 a is a schematic diagram of the first video frame image in which parking space tracking is not performed between parking spaces in an embodiment of the present application;

[0048] Figure 1 b is a schematic diagram of a second video frame image in which parking space tracking is not performed between parking spaces in an embodiment of the present application;

[0049] Figure 1 c is a schematic diagram of the third video frame image without parking space tracking between parking spaces in the embodiment of the present application;

[0050] Figure 2 This is a flow chart of a method for tracking parking spaces according to an embodiment of the present application;

[0051] Figure 3 A schematic diagram of parking space detection and identification according to an embodiment of the present application;

[0052] Figure 4 A schematic diagram of parking space detection for tracking parking spaces between parking spaces according to an embodiment of the present application;

[0053] Figure 4 a is a schematic diagram of a first video frame image for parking space tracking between parking spaces in an embodiment of the present application;

[0054] Figure 4 b is a schematic diagram of a second video frame image for performing parking space tracking between parking spaces in an embodiment of the present application;

[0055] Figure 4 c is a schematic diagram of a third video frame image for performing parking space tracking between parking spaces in an embodiment of the present application;

[0056] Figure 5 This is a flow chart of an embodiment of the present application for tracking a parking space according to predicted parking space corner points and parking space identity numbers;

[0057] Figure 6 A schematic diagram of the allowable error range of the embodiment of the present application;

[0058] Figure 7 This is a flow chart of an embodiment of the present application for performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point;

[0059] Figure 8 This is a flowchart of an embodiment of the present application for performing corner point prediction every other frame based on state information and parking space corner points to obtain predicted parking space corner points;

[0060] Fig. 9A schematic diagram of a vehicle lateral dynamics model according to an embodiment of the present application;

[0061] Fig.10 A schematic diagram of the structure of the parking space tracking device of the embodiment of the present application

[0062] Fig.11 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0064] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0065] As shown in the background technology, if the traditional visual method is used to track and match parking spaces between consecutive frames, the same parking space in adjacent video frames is matched according to the features obtained by parking space detection, so that the parking spaces in different frames are associated. However, due to the limitation of the panoramic monitoring imaging system (Around View Monitor, AVM), the imaging must have enough features for matching. When there are cars or obstacles such as pillars on the parking space, the imaging of the obstacles will be distorted. The same obstacle between each frame will have different distorted images, and the feature changes are large, making it difficult to match. Among them, Figure 1 The three consecutive video frame images shown are as follows: Figure 1 a. Figure 1 b. Figure 1c, that is, when the vehicle is driving into the garage, the continuous frame images input by the automatic parking system detect a total of 8 parking spaces through 3 video frame images of parking space detection, but there is no correlation between the parking spaces. The lack of correlation will bring great inconvenience to automatic parking. For example, after the automatic parking system finds a parking space in a single video frame image, it determines that it is an available parking space and plans parking. It needs to drive the vehicle to reverse into the garage. At this time, the video frame images input by the parking system obviously change with the movement of the car, and the detected parking space also changes. If the parking spaces between continuous frames are not associated, the parking space decided to be parked and the actual parking space that is constantly changing cannot be matched one by one, and automatic parking cannot be performed. The pixel distortion caused by AVM imaging in the automatic parking system is because the imaging is generated by AVM in the automatic parking system. The generated AVM video frame images are bird's-eye views, which belong to the God's perspective, that is, looking from top to bottom, which causes the video frame images generated by the vehicle's built-in AVM imaging system to be distorted. The distorted pixels seriously block the adjacent parking spaces in the AVM video frame images, and it is difficult to see the adjacent parking space lines between parking spaces. In addition, some parking spaces themselves are not a complete rectangle, with only spacing lines on both sides, and no entry lines drawn at one end of the parking space. This non-existent entry line can only be seen by the human eye, so in the absence of an entry line, the image features available for parking space detection itself become fewer. If you want to associate the parking spaces detected between consecutive frames, it will be more difficult because there is correspondingly less information available for pairing and association.

[0066] In the related technology, other pure vision-based methods can be used for parking space tracking, such as directly using a multi-target tracking algorithm based on deep learning (referred to as the deep tracking algorithm). However, the parking space deep tracking algorithm requires a lot of computing power and video memory. However, the current car computers simply cannot carry such a large computing power. Even if other functions are sacrificed to barely implement the deep tracking algorithm, since automatic parking must be carried out in real time, it cannot tolerate delayed and slow processing. Therefore, even if the deep tracking algorithm can be implemented offline, it is difficult to apply it in online automatic parking.

[0067] Optical character recognition (OCR) can also be performed on the video frame images detected by AVM imaging to identify the numbers of ground parking spaces, and use the same parking space numbers between frames for matching. However, this will also encounter certain problems, because the AVM imaging itself is blurred at the edge of the AVM image. When the parking space appears at the edge of the AVM imaging, it is difficult to detect it, and it will be even more difficult to detect the parking space number. Only when the parking space number is in the middle of the AVM imaging will there be a better recognition result. Such inevitable factors will lead to missed detection of parking space numbers. In addition, during the use of the parking space, the numbers in the parking space will gradually become blurred as the use time increases, or some parking spaces will set the numbers on the wall at the corresponding position of the parking space. In this case, the numbers in the parking space will not be recognized, resulting in missed detection and tracking failure.

[0068] The parking space tracking method provided in the embodiment of the present application extracts the parking space recognition video (such as AVM imaging video) obtained by parking space detection at each preset interval frame to obtain a video frame image. In order to meet the input requirements of automatic parking, it is necessary to extract the parking space recognition video into continuous video frame images to provide a basis for subsequent parking space tracking; identify all parking spaces in the video frame images, mark the parking spaces, and obtain a parking space identity number corresponding to each parking space. The association between different video frame images is achieved by marking the same parking space in different video frame images with the same parking space identity number, without the need to identify the parking space number that comes with the parking space, thereby avoiding the situation where the parking space number cannot be recognized. Then identify the parking corner points of each parking space in the video frame image. Since the parking corner points in the video frame image will not change with the movement of the vehicle, the parking corner points are selected as the identification of the parking space tracking, which can improve the accuracy of the parking space tracking; obtain the vehicle status information, and predict the corner points every other frame based on the status information and the parking corner points to obtain the predicted parking corner points, and track the parking space based on the predicted parking corner points and the parking space identity number, and track the parking corner points between consecutive frames by predicting the position of the parking corner points in the next video frame image, and associate the same parking space in the consecutive frame images through the parking space identity number to achieve the associated tracking of the parking space, so that the parking spaces between consecutive frames correspond one to one. It should be noted that the embodiment of the present application is mainly used in the use scenario of automatic parking. During automatic parking, the vehicle speed is generally low, so that enough time can be left for the tracking of the parking space. Therefore, the parking space tracking method provided in the embodiment of the present application is mainly applicable to low-speed scenarios, which will be explained in conjunction with the following embodiments.

[0069] In some embodiments, Figure 2 As shown, a method for tracking a parking space includes:

[0070] Step 100: Acquire the parking space recognition video in real time, and extract the parking space recognition video at each preset interval frame to obtain a video frame image.

[0071] In this step, it is first necessary to obtain a parking space recognition video. For example, an Around View Monitor (AVM) imaging video can be used as a parking space recognition video. Since the embodiment of the present application is based on continuous frames for parking space tracking, it is necessary to extract each preset interval frame of the parking space recognition video to obtain a video frame image. Among them, in the process of extracting the parking space recognition video, the number of preset interval frames needs to be determined according to the video frame rate of the parking space recognition video obtained by the video recognition device. When the frame rate is large, the number of interval frames can be set to be relatively large, and when the frame rate is small, the number of interval frames can be set to be relatively small, because in this way, the extraction time interval between two consecutive video frame images can be kept the same, because the distance moved by the vehicle is determined by the speed and time, and the extraction time interval is relatively stable. When the speed is relatively stable, it can be ensured that the distance moved by the vehicle between any two consecutive video frame images is similar, and the parking space tracking will not be difficult due to the excessive distance moved by the vehicle, nor will the tracking calculation amount be too large due to the small distance moved by the vehicle.

[0072] Step 200: Identify all parking spaces in the video frame image, and mark the parking spaces to obtain a parking space identity number corresponding to each parking space.

[0073] In this step, when detecting the parking space, if Figure 3 As shown, it is necessary to detect the corner points, entry lines and dividing lines of the parking spaces. Since not all parking spaces are complete rectangular frames, for example, an Anchor Free detection algorithm without an anchor frame can be used, or a Hungarian algorithm based on a DETR (DETEction TRansformer) query can be used. Alternatively, an anchor-free version of a target detection model based on the YOLO (You Only Look Once) series can be used. The parking space detection algorithm for corner point detection detects the information of the corner points and edges (edges corresponding to the entry lines and dividing lines) of the parking space from the distorted AVM video frame image, and needs to avoid the particularity of the parking space, that is, some parking spaces have no entry lines, and some parking spaces have incomplete entry lines and dividing lines. Therefore, in the implementation of this application, only one dividing line and one entry line intersecting at the corner points of the parking space need to be detected, and each parking space corner point corresponds to a parking space, and each parking space is marked with a unique parking space identity document (ID), so that the parking space corresponds to the parking space corner point and the parking space identity number respectively, and after the automatic parking is completed, all parking space identity numbers are cleared and re-numbered when used again.

[0074] Step 300: Identify the parking space corner points of each parking space in the video frame image.

[0075] In this step, according to the AVM imaging characteristics, the point at the upper left corner of the parking space can be preferentially selected as the parking space corner point, and the parking space has a one-to-one correspondence with the parking space corner point and the parking space identity number. It should be noted that the parking space corner point can be selected according to actual conditions, and the selection of the parking space corner point is not limited here.

[0076] For example, Figure 3 In the identification video frame image shown, a geometric representation method is used. The vertical marking line 1 represents the entry line of the ID2 parking space. The longitudinal marking line 2 located on the upper side of the figure represents the dividing line between the ID2 parking space and the ID3 parking space above. The longitudinal marking line 3 located on the lower side of the figure represents the dividing line between the ID2 parking space and the ID1 parking space below. The dot at the intersection of the marking line 1 and the marking line 2 represents the parking space corner point of the ID2 parking space, that is, the key point of this parking space. The marking line 2 and the marking line 3 represent the parking space lines blocked by the distorted vehicles. The marking line 1 represents the parking space entry line that is not drawn on the ground. The parking space corner point is used to associate the parking space between consecutive frames. The corner point of the parking space is selected as the marking point to avoid the situation where the dividing line is blocked by the distorted vehicle, and also to avoid the situation where there is no explicit white marking line on the entry line. For a parking space, the parking space detection algorithm detects this parking space, and then the parking space color corner point corresponding to this parking space will not change due to the driving of the car. However, the dividing line and the entry line will be affected and will become incomplete as the vehicle moves. When the parking space corner point disappears, the parking space identification number corresponding to the disappeared parking space corner point can be directly deleted.

[0077] Step 400: Acquire the state information of the vehicle, and perform frame-by-frame prediction of the corner points based on the state information and the corner points of the parking space to obtain the predicted corner points of the parking space.

[0078] In this step, it is relatively easy for the vehicle to obtain the vehicle's status information. At the same time, according to the video frame image generated by the AVM imaging system, the ground physical distance corresponding to each pixel in the actual space is known in advance. Moreover, in the automatic parking scene, in order to ensure that the automatic parking function is normally enabled, the vehicle is generally driven at a speed of 30km / h or less. Therefore, after detecting the parking space and the parking corner point, some information in the current scene can be calculated, such as the length of the parking space entry line and the corner point coordinates of the parking space corner point in the current garage scene. Based on this information, the range of the length of the parking space separation line can be estimated, and then the lateral dynamics model can be used to predict the corner point every frame based on the status information and the parking space corner point to obtain the predicted parking space corner point, which provides support for the tracking of the parking space.

[0079] Step 500: Tracking the parking space according to the predicted parking space corner point and the parking space identification number.

[0080] In this step, if Figure 1 and Figure 4 As shown, Figure 1 Indicates three consecutive video frames without associated tracking. Figure 4 Represents three consecutive video frame images obtained after tracking the parking space based on the predicted parking space corner points and parking space identification numbers. Figure 1 It can be seen from the identity identification numbers in that a total of 8 parking spaces are detected in three consecutive video frame images without associated tracking. However, it can be seen that in fact, ID4 and ID7 are the same parking space, ID1, ID5 and ID8 are the same parking space, ID6 and ID2 are the same parking space, and ID3 is a separate parking space. There is no association between the parking spaces in different video frame images. The lack of association will bring great inconvenience to automatic parking. For example, after the automatic parking system finds the parking space in the first video frame image, it determines that ID1 is an available target parking space and plans to park. It is necessary to drive the vehicle to reverse into the garage and enter the ID1 parking space. At this time, the video frame image input by the parking system obviously changes with the movement of the car. The detected ID1 parking space becomes ID5 in the second video frame image, and becomes ID8 in the third video frame image, resulting in the target parking space for parking decision cannot be one-to-one correspondence in different video frame images, and automatic parking cannot be performed. According to Figure 4It can be seen from the identification number in that a total of 4 parking spaces were detected in three consecutive video frames after the associated tracking. It can be seen that the ID1 parking space appears in the three video frames. Although the imaging screen changes as the vehicle travels, the parking space shifts in the image, but it is still associated and tracked, which is consistent with the actual situation, because the ID1 parking space in the three video frames is the same parking space. Among them, it can be seen that the ID3 parking space was not detected in the second video frame because the parking space was not fully displayed, so this ID3 parking space was cleared in the second video frame, and the ID4 parking space appeared in the lower middle of the second video frame, and the ID4 parking space also appeared in the third video frame. In the third video frame, the ID2 parking space was not detected by the vehicle, so it was cleared. Illustratively, after the automatic parking system finds a parking space in the first video frame image, it determines that ID1 is an available parking space and plans parking. The vehicle needs to be driven to reverse into the parking space ID1. At this time, the video frame image input by the parking system obviously changes with the movement of the car. The detected ID1 parking space is still the ID1 parking space in the second video frame image and the third video frame image, realizing the association tracking of the same parking space between consecutive video frame images, so that the target parking space for parking decision is one-to-one correspondence in different video frame images, ensuring the smooth progress of automatic parking.

[0081] In summary, the parking space tracking method provided in the embodiment of the present application meets the input requirements of automatic parking. After obtaining continuous video frame images, all parking spaces in the video frame images are identified and marked to obtain parking space identification numbers corresponding to each parking space. The same parking space identification numbers are marked on the same parking spaces in different video frame images to achieve association between different video frame images, without the need to identify the parking space numbers that come with the parking spaces, thereby avoiding the situation where the parking space numbers cannot be identified. Then, the parking space corner points of each parking space in the video frame images are identified. Since the parking space corner points in the video frame images will not change with the change of the parking space number, the parking space corner points in the video frame images will not change with the change of the parking space number. The position of the parking space corner point in the next video frame is predicted to track the parking space corner point between consecutive frames, and the same parking space in consecutive frame images is associated with the parking space identity number to realize the associated tracking of the parking space, so that the parking space between consecutive frames corresponds to each other, thereby ensuring the smooth progress of automatic parking.

[0082] In some embodiments, the video frame image includes a first video frame image and a second video frame image that are continuous; Figure 5 As shown, the parking space tracking is performed according to the predicted parking space corner point and the parking space identity number, including:

[0083] Step 510: Obtain predicted parking space corner points predicted based on the first video frame image.

[0084] In this step, the parking space tracking is explained by taking the adjacent first video frame image and second video frame image as an example. Since the parking space recognition video is obtained by real-time monitoring, the second video frame image has not been generated when the first video frame image is obtained. Therefore, the vehicle status information is obtained while obtaining the first video frame image, and the corner points are predicted every other frame based on the status information and the parking space corner points to obtain the predicted parking space corner points. Then, when the second video frame image is extracted, the predicted parking space corner points predicted based on the first video frame image are obtained, and the actual parking space corner points of each parking space in the second video frame image are identified. The parking space is tracked between consecutive video frame images by comparing the coordinates of the predicted parking space corner points and the actual parking space corner points.

[0085] Step 520: Identify the actual parking space corner point of each parking space in the second video frame image.

[0086] In this step, if you want to track the parking space between consecutive video frame images, after the predicted parking space corner points of the previous video frame image are known, you need to identify the actual parking space corner points of each parking space in the second video frame image. For example, if the predicted parking space corner points are obtained based on the ID1 parking space and status information, you need to determine the actual parking space corner points that are the same as the ID1 parking space in the first video frame image in the second video frame image based on the predicted parking space corner points, and then determine the parking space that is the same as the ID1 parking space in the second video frame image based on the one-to-one correspondence between the parking space corner points and the parking space.

[0087] Step 530: Parking space tracking is performed according to the predicted corner point coordinates of each predicted parking space corner point and the actual corner point coordinates of each actual parking space corner point in the first video frame image.

[0088] In this step, the predicted parking space corner point has a corresponding predicted corner point coordinate, and the actual parking space corner point has a corresponding actual corner point coordinate, so it is necessary to determine the actual parking space corner point that is closest to the predicted parking space corner point in the second video frame image based on the predicted corner point coordinate and the actual corner point coordinate, and then determine the parking space that is the same as the ID1 parking space in the second video frame image based on the one-to-one correspondence between the parking space corner point and the parking space, and mark the ID of the parking space as 1 in the second video frame image to achieve tracking of the parking space between consecutive video frame images. Among them, the predicted corner point coordinates of the predicted parking space corner point will produce a certain error with the actual corner point coordinates, and a certain error range can be set in advance. When the error between the predicted corner point coordinates and the actual corner point coordinates is within the error range, it is determined that the predicted corner point coordinates match the actual corner point coordinates, and it is determined that they are the corner points of the same parking space. For example, Figure 6As shown, according to the preset error distance, a circular error allowable range is determined with the predicted corner point coordinates as the center and the error distance as the radius. When the actual corner point coordinates detected are within the range, the predicted corner point coordinates are determined to match the actual corner point coordinates, and they are determined to be the corner points of the same parking space.

[0089] In some embodiments, Figure 7 As shown, parking space tracking is performed according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point, including:

[0090] Step 531: Determine the target actual parking space corner point corresponding to the predicted parking space corner point pair from among all the actual corner point coordinates according to any predicted corner point coordinate.

[0091] In this step, illustratively, Figure 4 Taking the first video frame image as the first video frame image and the second video frame image as the second video frame image as an example, parking space ID1 is taken as an example for explanation, prediction is made based on the parking corner points of parking space ID1 in the first video frame image, and the predicted corner point coordinates of parking space ID1 in the second video frame image are obtained. When the second video frame image is extracted, the actual corner point coordinates of all actual parking corner points are identified, and the actual parking corner points whose actual corner point coordinates are within the error range of the predicted corner point coordinates are determined as the target actual parking corner points.

[0092] Step 532: The parking space identification number corresponding to the predicted parking space corner point is used as the parking space identification number of the target parking space corresponding to the target actual parking space corner point.

[0093] In this step, exemplarily, if the parking space identification number corresponding to the predicted parking space corner point is determined to be ID1, and the parking space corresponding to the target actual parking space corner point is determined in the second video frame image to be the second parking space from top to bottom of the image, the second parking space is marked with ID1, and the tracking of the ID1 parking space is completed.

[0094] Step 533: Stop the parking space tracking until all the predicted corner point coordinates are tracked.

[0095] In this step, when the parking corner point tracking of the same parking space is completed, the identity identification number of the disappeared parking space is cleared, and the parking identity identification number marking of the newly added parking space is completed, it means that the parking space tracking of all the predicted corner point coordinates in the first video frame image is completed, and the parking space tracking according to the first video frame image is stopped, and the second video frame image is used as the new first video frame image to continue the new parking space tracking until the automatic parking is completed. It should be noted that in order to reduce the continuous accumulation of errors in the subsequent parking space tracking process, the target actual parking space corner point coordinates of the parking space in the second video frame image are replaced with the corresponding predicted corner point coordinates. Because the prediction accuracy is greater than the recognition accuracy, the predicted corner point coordinates are used as the actual corner point coordinates of the parking space in the second video frame image, which can improve the tracking accuracy and precision.

[0096] In some embodiments, performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point and the actual corner point coordinates of each actual parking space corner point in the first video frame image further includes:

[0097] Step 501: For any predicted parking space corner point, if there is no target actual parking space corner point corresponding to the predicted parking space corner point among all actual parking space corner points, the predicted parking space corner point is determined as a faulty parking space corner point.

[0098] In this step, in this step, exemplary, with Figure 4 Taking the first video frame image as the first video frame image and the second video frame image as the second video frame image as an example, for the ID3 parking space, in the first video frame image, the ID3 parking space is close to the edge of the image. As the vehicle moves, the real-time parking corner point corresponding to the ID3 parking space cannot be identified in the extracted second video frame image. In the second video frame image, there will be no target actual parking corner point corresponding to the predicted corner point coordinates obtained by predicting the parking corner point of the ID3 parking space in the first video frame image. At this time, the predicted parking corner point is used as the faulty parking corner point for subsequent fault analysis.

[0099] Step 502: Determine the validity of the corner point of the faulty parking space.

[0100] In this step, the occurrence of a faulty parking space corner point includes two situations. One is that the parking space corner point of the parking space corresponding to the predicted parking space corner point has actually moved out of the picture in the second video frame image and cannot be identified. The other is that the parking space corner point recognition fails due to the existence of obstacles or unclear parking space lines, and the target actual parking space corner point corresponding to the predicted parking space corner point is not identified. Therefore, when a faulty parking space corner point appears, it is necessary to judge the validity of the faulty parking space corner point to determine the cause of the faulty parking space corner point and the solution.

[0101] In some optional embodiments, determining the validity of the corner point of the faulty parking space includes:

[0102] Determine a field of view boundary of the second video frame image;

[0103] Determine whether the corner point of the faulty parking space exceeds the boundary of the visual range according to the predicted corner point coordinates of the corner point of the faulty parking space;

[0104] In response to the corner point of the faulty parking space exceeding the boundary of the field of view, determining that the corner point of the faulty parking space is invalid;

[0105] In response to the faulty parking space corner point not exceeding the boundary of the field of view, it is determined that the faulty parking space corner point is valid.

[0106] Among them, the second video frame image has its own field of view range boundary in the coordinate system. The fault situation of the fault corner point is judged by comparing whether the predicted corner point coordinates of the faulty parking space corner point exceed the field of view range boundary. If the faulty parking space corner point exceeds the field of view range boundary, it means that the parking space corner point of the parking space corresponding to the faulty parking space corner point has really moved out of the picture in the second video frame image and cannot be identified, and it is determined that the faulty parking space corner point is invalid; if the faulty parking space corner point does not exceed the field of view range boundary, it means that the parking space corner point recognition has failed, and the target actual parking space corner point corresponding to the predicted parking space corner point is not identified, and it is determined that the faulty parking space corner point is valid.

[0107] Step 503: In response to the corner point of the obstructed parking space being invalid, stop tracking the predicted corner point of the parking space.

[0108] In this step, if it is determined that the faulty parking space corner point is invalid, it means that the parking space corner point of the parking space corresponding to the faulty parking space corner point has really moved out of the picture in the second video frame image and cannot be identified, and the tracking of the predicted parking space corner point is stopped.

[0109] Step 504: In response to the corner point of the blocked parking space being valid, performing corner point filling according to the predicted corner point coordinates corresponding to the corner point of the blocked parking space.

[0110] In this step, if it is determined that the faulty parking space corner point is valid, it means that the parking space corner point recognition has failed, and the target actual parking space corner point corresponding to the predicted parking space corner point has not been recognized. Corner point leaks are filled according to the predicted corner point coordinates corresponding to the faulty parking space corner point, and the predicted corner point coordinates corresponding to the faulty parking space corner point are used as the actual corner point coordinate table in the second video frame image, and subsequent parking space tracking is performed according to the actual corner point coordinates after leak filling. Fault analysis of the faulty parking space corner point is performed through validity judgment to avoid the influence of corner point detection failure on parking space tracking, making parking space tracking more accurate.

[0111] In some embodiments, performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point and the actual corner point coordinates of each actual parking space corner point in the first video frame image further includes:

[0112] For any actual parking space corner point, if there is no predicted parking space corner point corresponding to the actual parking space corner point among all the predicted parking space corner points, the actual parking space corner point is determined as a new starting corner point for parking space tracking, and a new parking space identity number is marked for the new starting corner point.

[0113] Among them, for any actual parking space corner point, if there is no predicted parking space corner point corresponding to the actual parking space corner point among all the predicted parking space corner points, it means that the parking space corner point is the parking space corner point corresponding to the newly added parking space, and for the newly appeared parking space, it is necessary to track the newly appeared parking space based on the newly added starting corner point of the newly added parking space as the starting point, and for the newly appeared parking space, a new parking space identity identification number will be marked for the new parking space (which can be different from the existing parking space identification number, which is not limited here). By marking the new parking space identification number to carry out the associated tracking of the newly appeared parking space, the timeliness of parking space tracking is ensured.

[0114] In some optional embodiments, such as Figure 8 As shown, the corner points are predicted every other frame according to the state information and the parking space corner points to obtain the predicted parking space corner points, including:

[0115] Step 410: Determine the corner point coordinates of the parking space corner points.

[0116] In this step, for the continuous video frame images, it is necessary to determine the corner point coordinates (x f ,y f ).

[0117] Step 420: According to the pre-built vehicle lateral dynamics model, the lateral movement distance and the longitudinal movement distance are determined according to the state information.

[0118] In this step, Fig. 9 In the vehicle lateral dynamics model shown, the front wheels are combined into one wheel, that is, the center of the front wheels is taken for analysis, and the same is true for the rear wheels. In the actual automatic parking process, because it is a low-speed state, for front-wheel drive vehicles, the actual movement of the vehicle can be assumed to be a model in which the rear wheels have no deflection angle and the front wheels have deflection. Since in the AVM system, the vehicle is always in the center of the screen on the car display, the body of the vehicle is always positive, that is, the wheelbase line connecting the center points of the front and rear wheels always coincides with the Y axis in the model and is perpendicular to the X axis, that is, the value of α is always 90°, and the lateral movement distance and the longitudinal movement distance are determined according to the following calculation formula:

[0119] x=vcos(δ+θ+α)

[0120] y=vsin(δ+θ+α)

[0121]

[0122] Among them, x represents the lateral moving distance, represents the longitudinal moving distance, θ represents the yaw angle, v represents the current wheel speed, L represents the wheelbase, α represents the horizontal angle, and δ represents the front wheel steering angle.

[0123] Step 430: Determine the predicted horizontal coordinate according to the horizontal coordinate of the corner point coordinate and the horizontal movement distance.

[0124] In this step, the sum of the calculated lateral movement distance and the value of the abscissa of the corner point coordinates is determined as the predicted abscissa.

[0125] Step 440: Determine the predicted vertical coordinate according to the vertical coordinate of the corner point coordinate and the vertical movement distance.

[0126] In this step, the sum of the value of the calculated longitudinal movement distance and the value of the longitudinal coordinate of the corner point coordinate is determined as the predicted longitudinal coordinate.

[0127] Step 450: Determine the predicted corner point coordinates according to the predicted horizontal coordinate and the predicted vertical coordinate, and obtain the predicted parking space corner point in the video frame image.

[0128] In this step, the new coordinate point composed of the predicted horizontal coordinate and the predicted vertical coordinate is determined as the predicted parking corner point of the vehicle in the next video frame image. By predicting the parking corner point, the tracking of the same parking corner point in two adjacent video frame images is achieved.

[0129] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0130] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0131] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a parking space tracking device.

[0132] refer to Fig.10 , a parking space tracking device, characterized in that it includes:

[0133] The image extraction module 10 is configured to: acquire the parking space recognition video in real time, and extract the parking space recognition video at each preset interval frame to obtain a video frame image;

[0134] The identification module 20 is configured to: identify all parking spaces in the video frame image, identify the parking spaces, and obtain a parking space identification number corresponding to each parking space;

[0135] The corner point recognition module 30 is configured to: recognize the parking space corner point of each parking space in the video frame image;

[0136] The corner point prediction module 40 is configured to: obtain the state information of the vehicle, and perform corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point;

[0137] The corner point tracking module 50 is configured to track the parking space according to the predicted parking space corner points and the parking space identity number.

[0138] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0139] The device of the above embodiment is used to implement the corresponding parking space tracking method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0140] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the parking space tracking method described in any of the above embodiments is implemented.

[0141] Fig.11 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0142] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0143] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0144] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0145] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0146] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0147] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0148] The electronic device of the above embodiment is used to implement the corresponding parking space tracking method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0149] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the parking space tracking method described in any of the above embodiments.

[0150] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0151] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the parking space tracking method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0152] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0153] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0154] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0155] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A parking space tracking method, characterized in that: include: Acquire a parking space recognition video in real time, and extract each preset interval frame of the parking space recognition video to obtain a video frame image; The video frame images include a continuous first video frame image and a second video frame image; Identify all parking spaces in the video frame image, and mark the parking spaces to obtain a parking space identity number corresponding to each parking space; Identify the parking space corner points of each parking space in the video frame image; Acquire the state information of the vehicle, and perform corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point, and determine whether the predicted parking space corner point is a faulty parking space corner point: for any predicted parking space corner point, if there is no target actual parking space corner point corresponding to the predicted parking space corner point among all the actual parking space corner points, determine the predicted parking space corner point as a faulty parking space corner point; In response to the predicted parking space corner point being a faulty parking space corner point, the validity of the faulty parking space corner point is judged; wherein, the validity of the faulty parking space corner point is judged, including: determining the field of view boundary of the second video frame image; judging whether the faulty parking space corner point exceeds the field of view boundary according to the predicted corner point coordinates of the faulty parking space corner point; in response to the faulty parking space corner point exceeding the field of view boundary, determining that the faulty parking space corner point is invalid; in response to the faulty parking space corner point not exceeding the field of view boundary, determining that the faulty parking space corner point is valid; If the faulty parking space corner point is invalid, stop tracking the predicted parking space corner point; if the faulty parking space corner point is valid, perform corner point missing according to the predicted corner point coordinates corresponding to the faulty parking space corner point: if the faulty parking space corner point has been determined to be valid, it is determined that the parking space corner point recognition has failed, and the target actual parking space corner point corresponding to the predicted parking space corner point has not been identified, perform corner point missing according to the predicted corner point coordinates corresponding to the faulty parking space corner point, use the predicted corner point coordinates corresponding to the faulty parking space corner point as the actual corner point coordinates in the second video frame image, and perform subsequent parking space tracking according to the actual corner point coordinates after missing. In response to the predicted parking space corner point not being a faulty parking space corner point, parking space tracking is performed according to the predicted parking space corner point and the parking space identity number.

2. The method according to claim 1, characterized in that The performing parking space tracking according to the predicted parking space corner point and the parking space identity number comprises: Acquire the predicted parking space corner point predicted according to the first video frame image; Identify the actual parking space corner point of each parking space in the second video frame image; Parking space tracking is performed according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point.

3. The method according to claim 2, characterized in that The performing parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point comprises: Determine a target actual parking space corner point corresponding to the predicted parking space corner point pair from among all the actual corner point coordinates according to any one of the predicted corner point coordinates; Using the parking space identification number corresponding to the predicted parking space corner point as the parking space identification number of the target parking space corresponding to the target actual parking space corner point; The parking space tracking is stopped until the parking space tracking of all the predicted corner point coordinates is completed.

4. The method according to claim 3, characterized in that The performing of parking space tracking according to the predicted corner point coordinates of each predicted parking space corner point in the first video frame image and the actual corner point coordinates of each actual parking space corner point also includes: For any actual parking space corner point, if there is no predicted parking space corner point corresponding to the actual parking space corner point among all the predicted parking space corner points, the actual parking space corner point is determined as a new starting corner point for parking space tracking, and a new parking space identity number is marked for the new starting corner point.

5. The method according to claim 1, characterized in that The performing corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point includes: Determine the corner point coordinates of the parking space corner point; Determining a lateral movement distance and a longitudinal movement distance according to the state information based on a pre-constructed lateral dynamics model of the vehicle; Determine a predicted horizontal coordinate according to the horizontal coordinate of the corner point coordinate and the horizontal movement distance; Determine the predicted ordinate according to the ordinate of the corner point coordinate and the longitudinal movement distance; The predicted corner point coordinates are determined according to the predicted horizontal coordinates and the predicted vertical coordinates, and the predicted parking space corner point is obtained in the video frame image.

6. A parking space tracking device, characterized in that: include: The image extraction module is configured to: acquire a parking space recognition video in real time, and extract frames at each preset interval from the parking space recognition video to obtain a video frame image; the video frame image includes a continuous first video frame image and a second video frame image; The identification module is configured to: identify all parking spaces in the video frame image, and identify the parking spaces to obtain a parking space identification number corresponding to each parking space; A corner point recognition module is configured to: recognize a parking space corner point of each parking space in the video frame image; The corner point prediction module is configured to: obtain the state information of the vehicle, and perform corner point prediction every other frame according to the state information and the parking space corner point to obtain the predicted parking space corner point, and determine whether the predicted parking space corner point is a faulty parking space corner point: for any predicted parking space corner point, if there is no target actual parking space corner point corresponding to the predicted parking space corner point among all the actual parking space corner points, the predicted parking space corner point is determined as a faulty parking space corner point; In response to the predicted parking space corner point being a faulty parking space corner point, the validity of the faulty parking space corner point is judged; wherein, the validity of the faulty parking space corner point is judged, including: determining the field of view boundary of the second video frame image; judging whether the faulty parking space corner point exceeds the field of view boundary according to the predicted corner point coordinates of the faulty parking space corner point; in response to the faulty parking space corner point exceeding the field of view boundary, determining that the faulty parking space corner point is invalid; in response to the faulty parking space corner point not exceeding the field of view boundary, determining that the faulty parking space corner point is valid; If the faulty parking space corner point is invalid, stop tracking the predicted parking space corner point; if the faulty parking space corner point is valid, perform corner point missing according to the predicted corner point coordinates corresponding to the faulty parking space corner point: if the faulty parking space corner point has been determined to be valid, it is determined that the parking space corner point recognition has failed, and the target actual parking space corner point corresponding to the predicted parking space corner point has not been identified, perform corner point missing according to the predicted corner point coordinates corresponding to the faulty parking space corner point, use the predicted corner point coordinates corresponding to the faulty parking space corner point as the actual corner point coordinates in the second video frame image, and perform subsequent parking space tracking according to the actual corner point coordinates after missing. The corner point tracking module is configured to: in response to the predicted parking space corner point not being a faulty parking space corner point, track the parking space according to the predicted parking space corner point and the parking space identity number.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 7.

Images