Image-based parking space information processing method, device and equipment, and storage medium

Abstract

The present disclosure provides an image-based parking space information processing method and device, equipment and storage medium, relates to the technical field of image processing, and particularly relates to the fields of computer vision, intelligent transportation, smart parking and the like. The specific implementation scheme is as follows: N auxiliary lines of a to-be-labeled line of a parking space in an image are acquired, the N auxiliary lines are parallel to the to-be-labeled line, N is an integer greater than or equal to 3; a target blanking point of the N auxiliary lines is determined; a line connecting any point on the to-be-labeled line and the target blanking point is constructed to obtain position information of the to-be-labeled line in the image; and a parking space key point is acquired based on the position information of the to-be-labeled line in the image. In the present disclosure, the parking space information is determined based on the blanking point, which can reduce the work of manually configuring a parking line and reduce the complexity of manually labeling a parking position.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to the fields of computer vision, intelligent transportation, and smart parking. Background Technology

[0002] With the continuous improvement of the market economy and the acceleration of people's pace of life, the demand for automobiles is showing a rapid growth trend. As the number of cars increases rapidly, more and more on-street parking spaces are appearing in the public eye.

[0003] To make full use of urban road infrastructure, on-street parking spaces should be set up where traffic capacity permits. On-street parking not only meets the needs of users requiring short-term parking, but also helps to adjust the layout of urban parking lots and compensate for the inadequacy of existing parking facilities.

[0004] Since gates cannot be used to manage vehicle entry and exit, on-street parking requires monitoring and image collection, followed by image analysis to manage on-street parking. Image analysis relies on manually marked parking space locations. However, manually marking parking space locations is not only complex but also highly subjective. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, and storage medium for image-based parking space information processing.

[0006] According to one aspect of this disclosure, an image-based parking space information processing method is provided, comprising:

[0007] Obtain N auxiliary lines from the parking space in the image that are to be labeled. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3.

[0008] Determine the target hidden point for N auxiliary lines;

[0009] Construct a line connecting any point on the line to be labeled to the target hidden point to obtain the position information of the line to be labeled in the image;

[0010] Based on the positional information of the line to be labeled in the image, the key points of the parking space are obtained.

[0011] According to another aspect of this disclosure, an image-based parking space information processing apparatus is provided, comprising:

[0012] The auxiliary line determination module is used to obtain N auxiliary lines for the parking space to be labeled in the image. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3.

[0013] The hidden point determination module is used to determine the target hidden point of N auxiliary lines;

[0014] The construction module is used to construct a line connecting any point on the line to be labeled to the target hidden point, thereby obtaining the position information of the line to be labeled in the image;

[0015] The key point determination module is used to obtain the key points of the parking space based on the position information of the line to be labeled in the image.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0017] At least one processor; and

[0018] The memory is communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0020] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0021] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0022] In this embodiment, the target hidden point of the line to be labeled can be obtained based on auxiliary lines. After obtaining the target hidden point, only any point on the line to be labeled is needed to determine the position of the line in the image, thus obtaining the key points of the parking space. This method can reduce the complexity of manually labeling parking space lines. When the parking space is obscured, the position of the labeling line can be determined based on the hidden point and any point on the line to be labeled, which is more objective than the labeling line determined subjectively by humans.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of an image-based parking space information processing method according to an embodiment of the present disclosure;

[0026] Figure 2This is a flowchart illustrating an image-based parking space information processing method according to another embodiment of the present disclosure;

[0027] Figure 3(a) is a schematic diagram of a balanced parking space according to another embodiment of the present disclosure;

[0028] Figure 3(b) is a schematic diagram of a tilted parking space according to another embodiment of the present disclosure;

[0029] Figure 3(c) is a schematic diagram of a perpendicular parking space according to another embodiment of the present disclosure;

[0030] Figure 3(d) is a schematic diagram of the position of the line to be marked according to another embodiment of the present disclosure;

[0031] Figure 4 This is a schematic diagram illustrating the determination of the target blanking point according to another embodiment of this disclosure;

[0032] Figure 5(a) is a schematic diagram of the parking space sub-line markings according to another embodiment of the present disclosure;

[0033] Figure 5(b) is a schematic diagram of key points of a parking space according to another embodiment of the present disclosure;

[0034] Figure 6 This is a schematic diagram illustrating the acquisition of overlap and relative angle according to another embodiment of this disclosure;

[0035] Figure 7(a) is a schematic diagram of abnormal parking according to another embodiment of the present disclosure;

[0036] Figure 7(b) is a schematic diagram of angled parking according to another embodiment of the present disclosure;

[0037] Figure 7(c) is a schematic diagram of cross-space parking according to another embodiment of the present disclosure;

[0038] Figure 7(d) is a schematic diagram of reverse parking according to another embodiment of the present disclosure;

[0039] Figure 8 This is a schematic diagram illustrating the acquisition of vehicle height information according to another embodiment of this disclosure;

[0040] Figure 9 This is a schematic diagram of the overall process of an image-based parking space information processing method according to another embodiment of the present disclosure;

[0041] Figure 10 This is a schematic diagram of the overall process of an image-based parking space information processing method according to another embodiment of the present disclosure;

[0042] Figure 11 This is a schematic diagram of the overall process for determining parking type based on overlap and relative angle according to another embodiment of this disclosure;

[0043] Figure 12 This is a schematic diagram of the overall process for obtaining vehicle height information according to another embodiment of this disclosure;

[0044] Figure 13 This is a schematic diagram of the structure of an image-based parking space information processing device according to another embodiment of the present disclosure;

[0045] Figure 14 This is a block diagram of an electronic device used to implement the image-based parking space information processing method of the embodiments of this disclosure. Detailed Implementation

[0046] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0047] With economic development, the number of households owning cars will continue to increase, leading to parking problems. To alleviate these problems, on-street parking spaces are set up where traffic capacity permits. Correspondingly, on-street parking systems have emerged and rapidly been adopted to facilitate the management of on-street parking. Currently, on-street parking systems generally use high-position video recording, marking parking space areas in the captured images to capture and identify vehicles entering and leaving parking spaces.

[0048] However, on-street parking systems face the following difficulties when marking parking space locations: Often, when these systems are configured, parking spaces are already occupied, obscuring the parking lines and making it impossible to accurately mark key points. In such cases, markers may rely on subjective judgment to mark parking spaces on the image. In conclusion, regardless of whether parking spaces are obscured, manually marking parking space locations is not only complex but also highly subjective, hindering the assessment of whether vehicles are properly parked and thus impacting on-street parking management.

[0049] To address the above issues, this disclosure proposes an image-based parking space information processing method. This method is applicable not only to on-street parking lots but also to open parking lots without gates.

[0050] like Figure 1 The diagram shown illustrates an application scenario applicable to the embodiments of this disclosure. This application scenario includes multiple monitoring units 11, a server 12, a parking platform 13, and a parking violation processing platform 14.

[0051] The monitoring system 11 and server 12 are connected via a wireless or wired network. The monitoring system 11 can use bullet cameras, dome cameras, or a combined bullet-dome camera system. The bullet camera monitors the parking space status in real time, while the dome camera captures the license plate information of the corresponding parking space based on the bullet camera's commands. The combined bullet-dome camera system integrates both bullet and dome cameras. The panoramic camera in the bullet camera monitors the entire target area, while the dome camera adjusts the magnification to track and capture detailed images of the target area, thus providing clearer monitoring of the target area.

[0052] Server 12 can be a single server, a server cluster consisting of several servers, or a cloud computing center. Server 12 can be an independent physical server, a server cluster consisting of multiple physical servers, or a distributed system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0053] Figure 1 As shown, server 12 can support the implementation of parking platform 13 and illegal parking processing platform 14. Parking platform 13 is used to record the parking status of vehicles and form parking records; illegal parking processing platform 14 is used to process and record vehicles that seriously affect traffic.

[0054] like Figure 2 The diagram shown is a flowchart illustrating an image-based parking space information processing method provided in this embodiment of the present disclosure, including:

[0055] S201, obtain N auxiliary lines for the parking space to be labeled in the image. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3.

[0056] The lines to be marked are the main parking space lines or secondary parking space lines. A main parking space line can be understood as a line connecting different parking space lines. Examples of main parking space lines for the three types of on-street parking are provided. On-street parking space types include: balanced parking spaces, angled parking spaces, and perpendicular parking spaces. A balanced parking space, as shown in Figure 3(a), is where the vehicle is parked in the same direction as the road's travel direction. An angled parking space, as shown in Figure 3(b), is where the vehicle is parked at an angle to the road's travel direction, which can be 45°. A perpendicular parking space, as shown in Figure 3(c), is where the vehicle is parked perpendicular to the road's travel direction. Secondary parking space lines refer to parking space lines that can be shared with adjacent parking spaces. Examples of secondary parking space lines for the three types of parking spaces are provided in Figures 3(a), 3(b), and 3(c).

[0057] S202, determine the target hidden point of N auxiliary lines.

[0058] After performing two-dimensional imaging on a three-dimensional space, parallel line segments in the three-dimensional space will intersect at infinitely extended lines in the two-dimensional image; these intersection points are called hidden line points. In this embodiment, the auxiliary lines and the lines to be labeled are multiple line segments that should be parallel in three-dimensional space. After performing two-dimensional imaging on them, intersection points exist between the extensions of these line segments in the two-dimensional image space, thus obtaining the target hidden line point for the line to be labeled. When the line to be labeled is a main parking space line, the corresponding target hidden line point is the hidden line point of the main parking space line. When the line to be labeled is a secondary parking space line, the corresponding target hidden line point is the hidden line point of the secondary parking space line.

[0059] S203, construct a line connecting any point on the line to be labeled with the target hidden point to obtain the position information of the line to be labeled in the image.

[0060] S204. Based on the position information of the line to be labeled in the image, obtain the key points of the parking space.

[0061] In this embodiment, the target hidden point of the line to be labeled can be obtained based on auxiliary lines. After obtaining the target hidden point, only any point on the line to be labeled is needed to determine the position of the line in the image, thus obtaining the key points of the parking space. This method can reduce the work of manually configuring parking space lines and reduce the complexity of manually labeling parking space positions. When the parking space is obscured, the position of the labeling line can be determined based on the hidden point and any point on the line to be labeled, which is more objective than the labeling line determined subjectively by humans. Moreover, this method can realize batch automatic labeling of parking space positions, has no restrictions on the application environment, can improve the adaptability of the on-street parking system to the environment, further alleviate urban parking problems, and promote the development of smart transportation and smart cities.

[0062] In some embodiments, obtaining N auxiliary lines for the parking space to be labeled in an image can be implemented as follows: in response to the labeling operation of line segments in the image, N auxiliary lines parallel to the line to be labeled are obtained.

[0063] The main parking space line and the secondary parking space line can be used as lines to be labeled. As shown in Figure 3(d), the black solid line represents the main parking space line, and the gray solid line represents the secondary parking space line. Taking the main parking space line as an example, since the lane lines are parallel to the main parking space line, the lane lines in the image can be labeled manually. The lane lines shown in Figure 3(d) represent auxiliary lines of the main parking space line, and the curb line can also be set as auxiliary lines of the main parking space line. Any line segment parallel to the main parking space line can be set as an auxiliary line of the main parking space line, and this embodiment does not limit this.

[0064] The method for setting auxiliary lines for parking spaces is similar to that for setting auxiliary lines for main parking spaces; that is, any line segment in the image that is parallel to the auxiliary line of the parking space can be set as an auxiliary line for the auxiliary line of the parking space.

[0065] In this embodiment of the disclosure, the auxiliary lines are marked manually, and line segments parallel to the lines to be marked can be flexibly determined as auxiliary lines based on prior knowledge.

[0066] In other embodiments, auxiliary lines can also be determined by line detection. For example, it can be implemented as follows: in response to the labeling operation of the parking space to be labeled line in the image, the line to be labeled is obtained; based on the line detection method, candidate line segments in the image are obtained; among the candidate line segments, N line segments parallel to the line to be labeled are selected to obtain N auxiliary lines.

[0067] Line detection can be performed using a line detection algorithm. This algorithm can generate multiple candidate line segments. These candidate segments are not necessarily parallel to the line to be labeled; therefore, it is necessary to select N line segments from the candidate segments that are parallel to the line to be labeled, and use these N segments as auxiliary lines.

[0068] In this embodiment of the disclosure, auxiliary lines can be automatically extracted from the image based on the line detection method, further reducing the operation of manually marking auxiliary lines and improving the efficiency of obtaining auxiliary lines for parking spaces.

[0069] In some embodiments, the purpose of obtaining auxiliary lines is to obtain the target hidden point of the line to be labeled. To obtain accurate target hidden points, this embodiment of the disclosure may be implemented as follows:

[0070] Step A1: Obtain the intersection point between every two auxiliary lines in the N auxiliary lines.

[0071] Step A2: Perform data analysis on the intersection points between each pair of auxiliary lines to obtain the initial hidden point.

[0072] In some embodiments, the mean value of the intersection points between every two auxiliary lines can be determined to obtain the initial hidden point.

[0073] For example, if there are N auxiliary lines, and each line intersects the other two lines, resulting in M ​​intersection points, the average of these intersection points can be calculated as shown in expression (1):

[0074]

[0075] Where (x0, y0) represents the initial hidden point, (x1, y1), (x2, y2), ..., (x M y M ) represents the coordinates of M intersection points, where M indicates that there are M intersection points and M is a positive integer greater than or equal to 3.

[0076] In this embodiment of the disclosure, the initial hidden point is determined based on the mean value. This method can effectively eliminate the influence of errors and thus obtain a more accurate initial hidden point.

[0077] In other embodiments, the median of multiple intersection points can be determined as the initial hidden point.

[0078] For example, the coordinates of the M intersection points can be represented as (x1, y1), (x2, y2), ..., (x... M y M The coordinates of the initial hidden point on the x-axis can be determined based on the median of the x-axis coordinates of the M intersection points; the coordinates of the initial hidden point on the y-axis can be determined based on the median of the y-axis coordinates of the M intersection points.

[0079] In some embodiments, the initial hidden point can also be obtained based on the mode of the intersection points between every two auxiliary lines.

[0080] In this embodiment, the initial hidden point position is determined based on the mode and median. This method is simple to operate, requires no large amount of calculation, and saves computing resources.

[0081] Since the accuracy of the initial hidden point cannot be guaranteed, it is necessary to optimize the initial hidden point to obtain a more accurate hidden point.

[0082] Step A3: Adjust the position of the initial hidden point based on the preset constraints to obtain the target hidden point of N auxiliary lines; wherein, the preset constraints are used to make the line connecting the adjusted initial hidden point and the target point on the auxiliary line parallel to the N auxiliary lines.

[0083] In this embodiment of the disclosure, since the initial hidden point may have deviations, the initial hidden point is optimized based on preset constraints, which can obtain a more accurate target hidden point, thereby improving the accuracy of parking space labeling.

[0084] In some embodiments, the position of the initial hidden point is adjusted according to preset constraints to obtain the target hidden points of N auxiliary lines. For each of the N auxiliary lines, the following steps can be performed:

[0085] Step B1: Connect the initial hidden point and the midpoint of the auxiliary line to obtain the reference line; where the midpoint is the target point.

[0086] Step B2: Obtain the distances from the endpoints of the auxiliary lines to the reference line.

[0087] Step B3: Obtain the sum of the distances from the 2N endpoints of the N auxiliary lines to the corresponding reference lines. Using minimizing the sum of distances as a preset constraint, adjust the position of the initial hidden point to obtain the target hidden point of the N auxiliary lines.

[0088] Taking one of the auxiliary lines as an example, the method for determining the target hidden line disappearance point is as follows: Figure 4 As shown, the initial hidden line point is connected to the midpoint of the auxiliary line to obtain the reference line. Since the straight-line distance from the two endpoints of the auxiliary line to the reference line is the shortest, for ease of subsequent calculations, the two endpoints of the i-th auxiliary line are set as the head point H. i Foot point F i The reference line L corresponding to the i-th auxiliary line i Then, the distances from the endpoints of the auxiliary lines to the reference lines are calculated, and the sum of the distances from the 2N endpoints of the N auxiliary lines to the corresponding reference lines is obtained. The preset constraint is to obtain the minimum sum of distances, which can be expressed as the objective function form shown in expression (2):

[0089]

[0090] Wherein d(H i ,L i ) indicates the head point H i (One endpoint of the auxiliary line) to the reference line L i The distance, d(F) i ,L i ) indicates foot point F i (The other endpoint of the auxiliary line) to the reference line L i The distance is defined as minf(x,y), where (x,y) represents the minimum sum of distances, (x,y) represents the location information of the hidden point, and N represents the total number of auxiliary lines, where N is a positive integer greater than or equal to 3. It should be noted that the use of the midpoint of the auxiliary line as the target point in this embodiment is merely illustrative; the target point can also be other points on the auxiliary line, and this embodiment does not limit this selection.

[0091] In some embodiments, the gradient descent algorithm can be used to adjust the position of the initial hidden point. The position of the initial hidden point is adjusted to find the direction of the fastest gradient change, and then the initial hidden point is adjusted in that direction. If the sum of the aforementioned distances is less than a preset threshold, the position of the initial hidden point is set as the target hidden point. The method for adjusting the position of the initial hidden point is not limited to the gradient descent algorithm; convex optimization algorithms or Newton's iteration method can also be used, and this disclosure does not limit this approach.

[0092] Since the closer the line to be labeled is to the auxiliary line, the smaller the distance from the endpoints of the auxiliary line to the reference line. Based on this, minimizing the sum of the distances from the endpoints of the auxiliary line to the reference line makes the line connecting the hidden point and the midpoint of the auxiliary line as parallel as possible to the auxiliary line. Using the minimized distance as a preset constraint condition can obtain an accurate target hidden point.

[0093] In some embodiments, to improve the efficiency of parking space labeling, the concept of a parking space group is proposed in this disclosure. In practice, parking space lines within the same parking space group can be labeled. For example, in normal parking conditions, the main parking space line that is easily obscured is called the inner edge main parking space line, and the main parking space line that is not easily obscured is called the outer edge main parking space line. Multiple parking spaces whose outer edge main parking space line and any one of the inner edge main parking space lines are on the same straight line constitute a parking space group. For each parking space group, the positions of the main parking space line and the secondary parking space line in the image can be obtained separately. The following describes different situations:

[0094] 1) When the line to be labeled is the main line of a parking space group, construct a line connecting any point on the line to be labeled with the target hidden point to obtain the position information of the label line in the image. This can be implemented as follows:

[0095] In response to the selection operation on any point on the outer edge of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the outer edge of the parking space, yielding the equation of the straight line on the outer edge of the parking space in the image. This equation represents the position of the outer edge of the parking space in the image; and...

[0096] In response to the selection operation of any point on the inner edge of the parking space in the image, a line is constructed connecting the target hidden point and any point on the inner edge of the parking space to obtain the linear equation of the inner edge of the parking space in the image. This linear equation represents the position of the inner edge of the parking space in the image.

[0097] It should be noted that there is no restriction on the execution order of determining the equation of the straight line of the outer edge of the parking space in the image and determining the equation of the straight line of the inner edge of the parking space in the image.

[0098] Since there may be partial obstruction on the parking space line, any point on the visible main parking space line is selected in the image. Based on this point and the target hidden point in the direction of the main parking space line, the straight line equation of the main parking space line in the image is constructed. The straight line equation can be expressed by a linear equation, thereby obtaining the position information of the main parking space line in the image.

[0099] During implementation, the straight line equation of the resulting parking space main line can be displayed in the image for easy viewing.

[0100] In this embodiment of the disclosure, based on obtaining the target hidden point, only any point on the parking space main line is needed to obtain the parking space main line. Based on this method, the excessive reliance on the visibility of the parking space line can be reduced, the workload of manual annotation can be reduced, and the parking space main line can be annotated in batches.

[0101] 2) When the line to be labeled is a secondary parking space line of a parking space group, constructing a line connecting any point on the line to be labeled with the target hidden point to obtain the position information of the labeled line in the image can be implemented as follows:

[0102] For any parking space sub-line in the parking space group, in response to the selection operation of any point on the parking space sub-line in the image, a straight line is constructed between the target hidden point and the vertex of the parking space, and the straight line equation of the parking space sub-line in the image is obtained, that is, the position of the parking space sub-line in the image is obtained.

[0103] This method, even when parking space auxiliary lines are occluded, only one point on the auxiliary line is needed to construct the position information of the auxiliary line in the image based on the target hidden point of the auxiliary line. This method solves the problem that parking space auxiliary lines are easily occluded and difficult to manually annotate.

[0104] In this embodiment of the disclosure, based on obtaining the target hidden point, only any point on the parking space sub-line is needed to obtain the parking space sub-line. Based on this method, the excessive reliance on the visibility of the parking space sub-line can be reduced, the workload of manual annotation can be reduced, and the configuration efficiency of parking space lines can be improved.

[0105] The construction method for secondary parking spaces is similar to that for main parking spaces, and will not be elaborated here. For example, as shown in Figure 5(a), when most of the secondary parking space line is obscured by vehicles, a straight line can be constructed between the target hidden point and any point on the secondary parking space line, thus obtaining the equation of the secondary parking space line in the image. When the secondary parking space line is completely obscured, since the length, width, and type of parking spaces within the same parking space group are the same, the secondary parking space line of the obscured parking space can be inferred from the reference parking spaces within the same group.

[0106] In some embodiments, obtaining parking space key points based on the position information of the line to be labeled in the image can be implemented as follows: obtaining two intersection points based on the straight line equation of the outer edge main line of the parking space in the image and the straight line equations of the two secondary parking lines in the image; and obtaining two intersection points based on the straight line equation of the inner edge main line of the parking space in the image and the straight line equations of the two secondary parking lines in the image; and determining the obtained intersection points as parking space key points.

[0107] The intersection point is obtained by combining the equations of the main line along the outer edge of the parking space with the equations of the two secondary parking lines, where the two secondary lines must be adjacent. This method yields the parking space vertex on the main line along the outer edge, which is the key point of the parking space. Similarly, the intersection point is obtained by combining the equations of the main line along the inner edge of the parking space with the equations of the two secondary parking lines. This method also yields the parking space vertex on the main line along the inner edge, which is also a key point of the parking space.

[0108] In this embodiment of the disclosure, the method of obtaining intersection points based on the equation of a straight line can quickly determine the key points of the parking space.

[0109] After obtaining the key points of each parking space in the parking space group, the parking space information within the group can be summarized and then saved. This parking space information includes not only the key points but also information such as width, length, type, and orientation. Once all parking space information for the group is obtained, the group's parking space information is saved.

[0110] The width and length information of parking spaces can be based on manual measurement or derived from relevant national standards for parking spaces and planning information during the construction phase. The orientation of a parking space can be determined based on the flow of vehicles in adjacent lanes or from orientation signs. There are three types of parking spaces: balanced parking spaces, angled parking spaces, and perpendicular parking spaces.

[0111] In some embodiments, after obtaining accurate parking space information as described above, the parking type of the vehicle can also be determined based on at least one piece of information in the parking space information, which can be implemented as follows:

[0112] Step C1: Perform target detection on the acquired image to be processed to obtain the target vehicle.

[0113] The acquired images to be processed can be video streams. Based on the video streams, targets within the scene are analyzed in real time. The real-time analysis can use target detection and target tracking to detect vehicles that do not move within a parking area for a short period of time, which are the target vehicles.

[0114] Step C2: Based on the key points of the parking space and the positional relationship between the target vehicle and the target vehicle in the image, determine the parking space where the target vehicle is located as the target parking space.

[0115] This involves filtering key points near the target vehicle's location, selecting the key point closest to the target vehicle, and then identifying the parking space corresponding to that key point as the target parking space.

[0116] In other embodiments, the target parking space can be selected based on the position of the target vehicle in the image to be processed, with the highest overlap with the target vehicle.

[0117] Of course, the methods for determining the target parking space are not limited in the embodiments disclosed herein.

[0118] Step C3: Obtain the overlap between the target vehicle and the target parking space, as well as the relative angle between the orientation of the target vehicle and the orientation of the target parking space.

[0119] In some embodiments, obtaining the overlap between the target vehicle and the target parking space can be implemented as follows: obtaining the parking space rectangle in the top plane of the world coordinate system based on the parking space key points of the target parking space; determining the vehicle rectangle in the top plane of the world coordinate system based on the vehicle key points of the target vehicle; and determining the overlap between the target vehicle and the target parking space based on the overlap area between the parking space rectangle and the vehicle rectangle.

[0120] In other words, during implementation, the vehicle key points of the target vehicle are extracted from the image to be processed. For example, as shown in Figure 5(b), when the front of the vehicle is facing the camera, the key points of the parking space that can be extracted include the windows, headlights, body, and some key points on the vehicle. These vehicle key points are in the image coordinate system of the image to be processed, and need to be transformed to the world coordinate system to obtain the vehicle bounding box of the target vehicle in the top plane.

[0121] Similarly, the key points of the target parking space are located in the image coordinate system of the image to be processed, and need to be transformed to the world coordinate system to obtain the parking space rectangle in the top plane.

[0122] In some embodiments, the overlap degree O between the target parking space and the target vehicle can be calculated based on the overlap area between the parking space rectangle A and the vehicle rectangle B of the target vehicle. Figure 6 As shown, the ratio of the intersection of parking space rectangle A and vehicle rectangle B to the union of parking space rectangle A and vehicle rectangle B is calculated as shown in expression (3):

[0123] O=(A∩B) / (A∪B) (3)

[0124] Where O represents the overlap between the target parking space and the corresponding target vehicle, A∩B represents the intersection of the target parking space frame A and the target vehicle frame B, and A∪B represents the union of the target parking space frame A and the target vehicle frame B.

[0125] In this embodiment of the disclosure, the degree of overlap between the target vehicle and the target parking space can be effectively determined based on the overlapping area, thereby accurately obtaining the parking type of the target vehicle.

[0126] In some embodiments, the relative angle between the orientation of the target vehicle and the orientation of the target parking space is calculated, such as... Figure 6As shown, the relative angle can be determined by selecting a line segment parallel to the target vehicle frame and a line segment parallel to the main line of the parking space.

[0127] Step C4: Determine the parking type of the target vehicle based on the degree of overlap and the relative angle.

[0128] In this embodiment, the positional relationship between the target vehicle and the target parking space is determined based on the degree of overlap and relative angle, thereby identifying the parking type of the target vehicle. The parking type provides a data foundation for parking management. For example, subsequent processing based on the parking type can regulate parking behavior and reduce traffic congestion.

[0129] In some embodiments, since there are various parking methods, in order to accurately determine the positional relationship between the vehicle and the parking space, so as to manage the vehicle in a standardized manner based on different parking types, the parking type of the target vehicle is one of the parking type sets, which includes at least one of the following: reverse parking, abnormal parking, angled parking, cross-space parking, normal parking, etc.

[0130] Reverse parking refers to parking in a roadside parking space with the vehicle facing the opposite direction to the parking space.

[0131] Abnormal parking refers to parking in an on-street parking space where the vehicle's orientation is too large relative to the parking space, thereby affecting the passage of vehicles in the adjacent lane.

[0132] Angled parking refers to a situation where the vehicle's orientation is at a relative angle to the orientation of the parking space, and the vehicle body is not fully inside the parking space, but the relatively abnormal parking has little impact on road traffic.

[0133] Cross-space parking refers to a situation where a vehicle occupies multiple parking spaces.

[0134] During implementation, the parking type set can be configured according to actual needs. For example, if the focus is on identifying reverse parking, the parking type set can include both reverse parking and normal parking. Abnormal parking and angled parking can also be combined into one category. Flexible configuration of the parking type set can meet the application needs of different open parking lots.

[0135] In some embodiments, determining the parking type of the target vehicle based on overlap and relative angle can be implemented as follows:

[0136] If the overlap is greater than a first set threshold and less than a second set threshold, and the relative angle is greater than a first angle threshold, the parking type is determined to include abnormal parking.

[0137] If the parking type is not abnormal parking, the overlap is greater than the first set threshold and less than the third set threshold, and the relative angle is greater than the second angle threshold, then the parking type is determined to include angled parking; the third set threshold is greater than the second set threshold, and the second angle threshold is less than the first angle threshold.

[0138] If the overlap is greater than the first set value but less than the fourth set threshold, the parking type is determined to include cross-space parking; the fourth set threshold is less than the second set threshold but greater than the first set threshold.

[0139] In this embodiment, abnormal parking needs to be identified first. If it is not abnormal parking, then angled parking is identified. The execution order of cross-space parking is not restricted. If cross-space parking is more important than abnormal parking, it is identified first. If cross-space parking is less important than abnormal parking, it can be identified later.

[0140] If the parking type is not angled parking, abnormal parking, or cross-space parking, and the relative angle is greater than the third angle threshold, the parking type is determined to include reverse parking, and the third angle threshold is greater than the first angle threshold.

[0141] For example, the first threshold is set to 0.3, the second to 0.65, the third to 0.75, and the fourth to 0.55. The first angle threshold is 45°, the second to 15°, and the third to 270°. If the overlap is greater than 0.3 and less than 0.65, and the relative angle is greater than 45°, the target vehicle is identified as abnormally parked. Figure 7(a) shows an example of abnormal parking, where abnormal parking refers to parking at an angle of more than 45 degrees. This type of parking severely affects on-site parking order and road traffic, causing serious traffic congestion. Therefore, abnormal parking can be prioritized for correction.

[0142] If the parking type is not abnormal, the target vehicle is identified as angled parking if the overlap is greater than 0.3 and less than 0.75, and the relative angle is greater than 15°. Figure 7(b) shows an example of angled parking, which may affect the traffic flow on the road.

[0143] If the parking type is not angled parking, and the overlap is greater than 0.3 and less than 0.55, the target vehicle is determined to be a cross-space parking vehicle. As shown in Figure 7(c), an example of cross-space parking is illustrated, in which the target vehicle occupies two parking spaces, thus affecting the parking of other vehicles.

[0144] When the parking type is not angled parking, not abnormal parking, and not cross-space parking, and the relative angle is greater than 270°, the target vehicle is determined to be parked in reverse. Figure 7(d) shows an example of reverse parking.

[0145] It should be noted that multiple parking types, such as abnormal parking, reverse parking, and cross-space parking, may appear in one vehicle, as shown in vehicle 1 in Figure 7(d). This vehicle has multiple parking types, including abnormal parking, reverse parking, and cross-space parking.

[0146] In other embodiments, reverse parking can be identified first, followed by abnormal parking. For example, if the parking space faces the opposite direction to the target vehicle, it is determined to be reverse parking.

[0147] In this embodiment, parking types are identified in the order of abnormal parking and angled parking, and cross-space parking is also identified. Based on this identification order, serious violations or those affecting traffic can be screened out first, and then parking types that do not seriously affect road traffic can be screened out so that parking lot managers can be notified in a timely manner. This allows parking lot managers to handle vehicles based on the urgency level, thereby ensuring the normal passage of road traffic.

[0148] In some embodiments, it can also be determined whether the target vehicle is parked in reverse based on the key points at the front and rear of the vehicle. This can be implemented by: extracting the key points at the front and rear of the target vehicle; determining the orientation of the target vehicle based on the key points at the front and rear of the target vehicle; and determining the parking type as reverse parking if the orientation of the target vehicle and the orientation of the target parking space are inconsistent.

[0149] In some embodiments, key points at the front and rear of the target vehicle can be extracted based on target segmentation technology. When the front of the vehicle faces the camera, a series of front key points can be extracted; when the rear of the vehicle faces the camera, a series of rear key points can be extracted. The number of key points extracted from the target vehicle is counted. If the number of front key points exceeds a preset threshold, the target vehicle is determined to be facing the camera; if the number of rear key points exceeds a preset threshold, the target vehicle is determined to be facing the camera. Finally, combined with the parking space orientation information, the parking type can be determined. That is, if the orientation of the target vehicle and the orientation of the target parking space are inconsistent, as shown in Figure 7(d), the parking type is determined to be reverse parking.

[0150] In this embodiment of the disclosure, the vehicle's orientation information is obtained by statistically analyzing key points at the front and rear of the vehicle. This information is then combined with the parking space orientation information to determine the parking type of the vehicle. This method can effectively determine whether the target vehicle is parked in the wrong direction.

[0151] In addition to the aforementioned priority-based parking type determination, parallel parking type determination can also be performed. For example, when a target vehicle is detected, it can be matched against the criteria for each parking type in parallel based on its overlap and relative angle, thus obtaining the matched parking type. For example:

[0152] If the overlap is greater than a first threshold and less than a second threshold, and the relative angle is greater than a first angle threshold, the parking type is determined to be abnormal parking, where the first threshold is less than the second threshold.

[0153] If the overlap is greater than the third threshold and less than the fourth threshold, and the relative angle is less than the first angle threshold and greater than the second angle threshold, the parking type is determined to be angled parking, where the third threshold is less than the fourth threshold and greater than the second threshold, and the first angle threshold is greater than the second angle threshold.

[0154] If the overlap is greater than the fifth threshold and less than the sixth threshold, the parking type is determined to be cross-space parking, where the fifth threshold is less than the sixth threshold and less than the first threshold.

[0155] If the orientation of the target vehicle and the orientation of the target parking space are inconsistent, the parking type is determined to be reverse parking.

[0156] When the criteria for multiple parking types are matched, the parking type of greatest interest is selected as the final parking type to be reported and recorded.

[0157] The aforementioned thresholds can be set based on actual needs, and this disclosure does not limit this.

[0158] In one embodiment, parking types can be divided into three categories and processed separately. For vehicles parked normally, the vehicle information is sent to the parking platform for recording. For vehicles parked in reverse, angled, or across parking spaces, the vehicle information and parking type are sent to the parking platform, and an alarm message is sent to the vehicle owner to urge them to park properly. For vehicles parked abnormally, the vehicle information and parking type are sent to the illegal parking processing platform for manual processing before the vehicle owner's violation or irregular behavior is dealt with.

[0159] In some embodiments, in order to further understand the height information of the target vehicle and to quantify the vehicle height, the hidden point in the vertical direction can be calculated based on the auxiliary line in the vertical direction; based on the hidden point in the vertical direction and the height information of the benchmark, the transformation relationship from the image coordinate system to the world coordinate system can be determined.

[0160] The marker is an object perpendicular to the ground and of known height, such as a pedestrian or a utility pole. In this embodiment, a wheel can be used as the marker.

[0161] In some embodiments, the height of the target vehicle in the image to be processed can be determined based on the transformation relationship from the image coordinate system to the world coordinate system.

[0162] Taking a wheel as an example, such as Figure 8 As shown, auxiliary lines in the vertical direction can be obtained. Based on the aforementioned method of determining the initial and target hidden point, the hidden point in the vertical direction is obtained. The center point of the wheel is selected in the image, and the center point of the wheel is connected to the hidden point in the vertical direction. The height information of the wheel is obtained as reference information, and the transformation relationship from the image coordinate system to the world coordinate system can be obtained.

[0163] For example, in Figure 5(b), the user selects key point 1 to estimate the height of the target vehicle. The line connecting the hidden point in the vertical direction and key point 1 is the straight line in the vertical direction, which is L1. Based on the key point at the road tangent, as shown in Figure 5(b), a straight line on the ground can be obtained from key point 10 and any point on the parking line, which is L2. The intersection of L1 and L2 is determined. The value of the z-direction of this intersection in the height direction of the world coordinate system is 0, and its value in the world coordinate system is denoted as (x1, y1, 0). The value of key point 1 in the world coordinate system is denoted as (x1, y1, z). z is the reference height of the target vehicle, which can be obtained according to the transformation relationship from the image coordinate system to the world coordinate system and (x1, y1). The final height of the target vehicle is shown in expression (4):

[0164] H = a * z (4)

[0165] In the expression, H represents the height of the target vehicle, and 'a' is a transformation coefficient derived from statistical information. For example, when determining the vehicle height using the key point of vehicle number 9 in Figure 5(b), the value of 'a' corresponding to key point number 9 is used; when determining the vehicle height using the key point of vehicle number 3 in Figure 5(b), the value of 'a' corresponding to key point number 3 is used.

[0166] In this embodiment of the disclosure, by quantizing the height direction, the transformation relationship from the image coordinate system to the world coordinate system is obtained, which can accurately identify the height of the vehicle and provide basic data for parking management.

[0167] In some embodiments, when a benchmark is used to determine camera parameters, the conversion relationship can be determined based on the camera parameters if the camera parameters are available.

[0168] In this embodiment of the disclosure, the height quantization information in the vertical direction of the ground is determined based on the method of hidden point elimination, which provides reference information for determining the height of objects in the image and enables a more comprehensive description of the objects in the image.

[0169] In summary, the embodiments of this disclosure provide a method for automatically marking parking lines based on hidden point elimination. This method is applicable even when the parking lines are unobstructed. When the parking lines are obstructed, it reduces manual operation and improves marking efficiency compared to manual marking.

[0170] Of course, when the parking space is unobstructed, key points can also be obtained through manual marking. In some embodiments, to improve marking efficiency, parking space information can also be obtained through a simple two-point line method when the parking space is unobstructed. The overall process is as follows: Figure 9 As shown:

[0171] S901, group parking spaces.

[0172] That is, parking spaces that are on a straight line along any parking space main line are identified as the same group of parking spaces.

[0173] S902, in response to user operation, obtains the outer and inner main lines of the parking space group marked by the user.

[0174] S903, in response to user actions, determines the target parking space.

[0175] S904, in response to user operation, obtains the labeling result of the secondary parking line of the target parking space.

[0176] S905, obtain the key points of the target parking space.

[0177] Similarly, after obtaining the main parking space line and the secondary parking space line, determine the intersection of the main parking space line and the secondary parking space line, thereby obtaining the vertex of the target parking space, which serves as the key point.

[0178] S906, summarizes parking space information.

[0179] Similarly, in this embodiment, the parking space information includes key points of the parking space, such as parking space length, parking space width, parking space type, and parking space orientation.

[0180] S907, saves the parking information of each parking space in the parking space group.

[0181] In this system, multiple parking spaces whose main lines lie on the same straight line and are parallel to each other are grouped into a parking space group. Both the outer and inner main lines of a parking space in the scene can be referred to as parking space main lines. As shown by the solid black lines on the parking spaces in Figure 3(d), these are parking space main lines. Parking space groups simplify annotation; only one annotation is needed on either the inner or outer main line of a parking space, eliminating the need for repeated annotation for each parking space. Taking the outer main line as an example, two points can be selected on the outer main line of the same parking space group to construct a straight line equation, thus obtaining the outer main line of the same parking space group. After completing the annotation of the parking space main lines, the target parking space to be annotated is selected, and two points are selected on the secondary line of the target parking space in the image to complete the annotation of the secondary line. The secondary line is shown by the solid gray lines on the parking spaces in Figure 3(d). If the parking spaces are adjacent, only one annotation is needed; that is, one secondary line can be used by two adjacent parking spaces, thus reducing the number of secondary line annotations. Key points of parking spaces are extracted based on the secondary parking space line and the main parking space line.

[0182] When parking lines are obscured, they can be marked based on the hidden point. The overall process is illustrated in the diagram below. Figure 10 As shown, it includes:

[0183] S1001, grouping multiple parking spaces into the same parking space group by having any one of the outer or inner main lines of the parking space that is on the same straight line.

[0184] The main parking space line and the secondary parking space line can be used as lines to be marked. For any line to be marked, execute S1002-S1003.

[0185] S1002, Obtain N auxiliary lines for the parking spaces to be labeled in the image.

[0186] S1003, determine the target hidden point of N auxiliary lines.

[0187] The method for determining the target hidden point has been explained above and will not be repeated here.

[0188] S1004, Annotation of the lines to be annotated based on the target hidden point, including:

[0189] When the line to be labeled is the main line of a parking space group, in response to the selection operation of any point on the outer edge of the main line of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the outer edge of the main line of the parking space to obtain the straight line equation of the outer edge of the main line of the parking space in the image; and, in response to the selection operation of any point on the inner edge of the main line of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the inner edge of the main line of the parking space to obtain the straight line equation of the inner edge of the main line of the parking space in the image.

[0190] When the line to be labeled is a secondary parking line of a parking space group, for any secondary parking line in the parking space group, in response to the selection operation of any point on the secondary parking line in the image, a straight line is constructed between the target hidden point and the vertex of the parking space, and the straight line equation of the secondary parking line in the image is obtained.

[0191] S1005, Select target parking space.

[0192] S1006, Obtain the key points of the target parking space.

[0193] That is, based on the straight line equations of the secondary parking line and the main parking line of the target parking space, the intersection point of the straight line equations is determined, and the key points of the target parking space are obtained.

[0194] S1007, summarizes parking space information.

[0195] S1008, saves the parking information of each parking space in the parking space group.

[0196] In summary, parking space information can be obtained under both obstructed and unobstructed conditions, thereby enabling the modeling of open parking lots.

[0197] Based on this, parking lot management can be automatically achieved through monitoring.

[0198] As explained above, the automatic identification of parking types follows the overall process as follows: Figure 11 As shown, it includes:

[0199] S1101, acquire real-time video stream information to obtain the image to be processed.

[0200] S1102, Based on video stream information, target detection and target tracking of vehicles are performed to obtain the target vehicle in the image to be processed.

[0201] S1103, Based on the key points of the parking space and the positional relationship of the target vehicle in the image, determine the parking space where the target vehicle is located as the target parking space.

[0202] S1104, extract the key points at the front and rear of the target vehicle.

[0203] S1105, based on the number of key points at the front of the vehicle and the number of key points at the rear of the vehicle, determines the orientation of the target vehicle.

[0204] For example, if the number of key points at the front of the vehicle exceeds the first key point threshold, then the front of the vehicle is determined to be facing the camera. If the number of key points at the rear of the vehicle exceeds the second key point threshold, then the rear of the vehicle is determined to be facing the camera.

[0205] The orientation of the target parking space can also be recorded as facing the camera or away from the camera, thus determining whether the vehicle's orientation and the parking space's orientation are consistent.

[0206] S1106, if the vehicle orientation of the target vehicle and the parking space orientation of the target parking space are inconsistent, determine that the parking type includes reverse parking, and continue to execute S1106.

[0207] S1107, Identify whether the target vehicle is parked abnormally. If the target vehicle is determined to be parked abnormally, determine that the parking type includes abnormal parking and execute S1111; if it is determined that it does not include abnormal parking, execute S1108.

[0208] S1108, identify whether the target vehicle is parked at an angle. If the target vehicle is determined to be parked at an angle, determine that the parking type includes angled parking, and then execute S1109.

[0209] S1109, identify whether the target vehicle is parked across spaces. If the target vehicle is determined to be parked at an angle, determine that the parking type includes parking across spaces, and then execute S1110.

[0210] S1110, if the overlap between the target vehicle and the target parking space is less than a first preset threshold, it is determined that the target vehicle is not in the parking space. At this time, the parking event is not reported.

[0211] S1111, if the overlap between the target vehicle and the target parking space is greater than the first set threshold, and the parking type is not reverse parking, abnormal parking, angled parking, or cross-space parking, the parking type of the target vehicle is determined to be normal parking. At this time, the parking type is normal parking.

[0212] S1112 pushes parking type information to the relevant platform.

[0213] For example, the identified parking type can be reported to the parking platform for recording. If the parking type includes violations or irregularities, it can also be reported to the illegal parking processing platform. The car owner can also be notified to correct the wrong parking behavior in a timely manner.

[0214] In some embodiments, the vehicle height can also be determined to achieve 3D modeling of the vehicle. The overall process can be implemented as follows: Figure 12 As shown:

[0215] S1201, label the auxiliary lines in the direction perpendicular to the ground.

[0216] S1202, determine the hidden point in the vertical direction based on the auxiliary line in the vertical direction.

[0217] S1203, choosing the wheel as the benchmark.

[0218] S1204, connecting the center point of the wheel and the disappearing point in the direction perpendicular to the ground.

[0219] S1205, in response to user operation, obtains wheel height information.

[0220] S1206, based on wheel height information and the hidden point in the vertical direction, determines the transformation relationship from the image coordinate system to the world coordinate system.

[0221] S1207, based on the transformation relationship from the image coordinate system to the world coordinate system, determine the height of the target vehicle in the image to be processed.

[0222] Therefore, in this embodiment of the disclosure, the height of a vehicle can be accurately identified by the transformation relationship from the image coordinate system to the world coordinate system, providing basic data for parking management.

[0223] Based on the same technical concept, this disclosure provides an image-based parking space information processing device, such as... Figure 13 The following are included:

[0224] The auxiliary line determination module 1301 is used to obtain N auxiliary lines of the parking space to be labeled in the image. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3.

[0225] The hidden point determination module 1302 is used to determine the target hidden point of N auxiliary lines;

[0226] The construction module 1303 is used to construct a line connecting any point on the line to be labeled to the target hidden point, so as to obtain the position information of the line to be labeled in the image;

[0227] The key point determination module 1304 is used to obtain parking space key points based on the position information of the line to be labeled in the image.

[0228] In some embodiments, the blanking point determination module includes:

[0229] The intersection point determination unit is used to obtain the intersection point between every two auxiliary lines out of N auxiliary lines;

[0230] The initial hidden point determination unit is used to perform data analysis on the intersection points between each pair of auxiliary lines to obtain the initial hidden point.

[0231] The hidden point optimization unit is used to adjust the position of the initial hidden point based on preset constraints to obtain the target hidden points of N auxiliary lines; wherein, the preset constraints are used to make the line connecting the adjusted initial hidden point and the target point on the auxiliary line parallel to the N auxiliary lines.

[0232] In some embodiments, the blanking point optimization unit is further configured to:

[0233] For each of the N auxiliary lines, perform the following steps: connect the initial hidden line point to the midpoint of the auxiliary line to obtain the reference line; the midpoint is the target point.

[0234] Obtain the distances from the endpoints of the auxiliary line to the reference line;

[0235] Obtain the sum of the distances from the 2N endpoints of the N auxiliary lines to the corresponding reference lines. Using minimizing the sum of distances as a preset constraint, adjust the position of the initial hidden point to obtain the target hidden point of the N auxiliary lines.

[0236] In some embodiments, the initial blanking point determination unit is configured to:

[0237] Determine the mean of the intersection points between every two auxiliary lines to obtain the initial hidden point.

[0238] In some embodiments, the auxiliary line determination module is configured to:

[0239] In response to the annotation operation on the line segments in the image, N auxiliary lines parallel to the line to be annotated are obtained.

[0240] In some embodiments, the auxiliary line determination module is configured to:

[0241] In response to the annotation operation on the parking space line to be annotated in the image, the line to be annotated is obtained;

[0242] Candidate line segments in the image are obtained based on line detection.

[0243] From the candidate line segments, select N line segments that are parallel to the line to be labeled to obtain N auxiliary lines.

[0244] In some embodiments, it also includes:

[0245] The vehicle detection module is used to perform target detection on the acquired images to be processed, and to obtain the target vehicles;

[0246] The parking space determination module is used to determine the parking space where the target vehicle is located as the target parking space based on the key points of the parking space and the positional relationship of the target vehicle in the image;

[0247] The acquisition module is used to acquire the overlap between the target vehicle and the target parking space, as well as the relative angle between the orientation of the target vehicle and the orientation of the target parking space.

[0248] The determination module is used to determine the parking type of the target vehicle based on the degree of overlap and the relative angle.

[0249] In some embodiments, the acquisition module is configured to:

[0250] Obtain the rectangular bounding box of the target parking space in the top plane of the world coordinate system based on the key points of the target parking space.

[0251] Determine the vehicle bounding box of the target vehicle in the top plane of the world coordinate system based on the vehicle key points of the target vehicle.

[0252] The overlap between the target vehicle and the target parking space is determined based on the overlapping area between the parking space rectangle and the vehicle rectangle.

[0253] In some embodiments, the determining module is configured to:

[0254] If the overlap is greater than a first set threshold and less than a second set threshold, and the relative angle is greater than a first angle threshold, the parking type is determined to include abnormal parking.

[0255] If the parking type is not abnormal parking, the overlap is greater than the first set threshold and less than the third set threshold, and the relative angle is greater than the second angle threshold, then the parking type is determined to include angled parking; the third set threshold is greater than the second set threshold, and the second angle threshold is less than the first angle threshold.

[0256] If the overlap is greater than the first set threshold but less than the fourth set threshold, the parking type is determined to include cross-space parking; the fourth set threshold is less than the second set threshold but greater than the first set threshold.

[0257] In some embodiments, the determining module is further configured to:

[0258] Extract key points from the front and rear of the target vehicle;

[0259] The orientation of the target vehicle is determined based on the key points at the front and rear of the vehicle.

[0260] If the orientation of the target vehicle and the orientation of the target parking space are inconsistent, the parking type is determined to be reverse parking.

[0261] In some embodiments, a height quantization module is also included, for:

[0262] Calculate the hidden point in the vertical direction based on the auxiliary line in the vertical direction;

[0263] Based on the hidden point and the height information of the benchmark in the vertical direction, the transformation relationship from the image coordinate system to the world coordinate system is determined.

[0264] In some embodiments, the high quantization module is further configured to:

[0265] Based on the transformation relationship from the image coordinate system to the world coordinate system, the height of the target vehicle in the image to be processed is determined.

[0266] In some embodiments, multiple parking spaces whose outer or inner main lines are on the same straight line constitute a parking space group. When the line to be labeled is the main line of a parking space group, a line is constructed connecting any point on the line to be labeled to the target hidden point to obtain the position information of the labeled line in the image. The construction module is used for:

[0267] In response to the selection operation on any point on the outer edge of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the outer edge of the parking space, thus obtaining the equation of the straight line on the outer edge of the parking space in the image; and,

[0268] In response to the selection operation of any point on the inner edge of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the inner edge of the parking space to obtain the straight line equation of the inner edge of the parking space in the image.

[0269] In some embodiments, multiple parking spaces whose outer and inner main lines are on the same straight line constitute a parking space group. Where the line to be labeled is a secondary line of a parking space group, a line is constructed connecting any point on the line to be labeled to the target hidden point to obtain the position information of the labeled line in the image. The construction module is further configured to:

[0270] For any parking space sub-line in the parking space group, in response to the selection operation of any point on the parking space sub-line in the image, construct the straight line between the target hidden point and the parking space vertex, and obtain the straight line equation of the parking space sub-line in the image.

[0271] In some embodiments, the key point determination module is further configured to:

[0272] Two intersection points are obtained based on the equations of the main line along the outer edge of the parking space and the equations of the two secondary lines along the parking space in the image; and...

[0273] Two intersection points are obtained based on the equations of the main line within the parking space and the two secondary lines within the parking space in the image.

[0274] The obtained intersection points are determined as key points for parking spaces.

[0275] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0276] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0277] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0278] Figure 14 A schematic block diagram of an example electronic device 1400 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0279] like Figure 14 As shown, device 1400 includes a computing unit 1401, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1402 or a computer program loaded from storage unit 1408 into random access memory (RAM) 1403. The RAM 1403 may also store various programs and data required for the operation of device 1400. The computing unit 1401, ROM 1402, and RAM 1403 are interconnected via bus 1404. Input / output (I / O) interface 1405 is also connected to bus 1404.

[0280] Multiple components in device 1400 are connected to I / O interface 1405, including: input unit 1406, such as a keyboard, mouse, etc.; output unit 1407, such as various types of displays, speakers, etc.; storage unit 1408, such as a disk, optical disk, etc.; and communication unit 1409, such as a network card, modem, wireless transceiver, etc. Communication unit 1409 allows device 1400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0281] The computing unit 1401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1401 performs the various methods and processes described above, such as an image-based parking information processing method. For example, in some embodiments, the image-based parking information processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1408. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1400 via ROM 1402 and / or communication unit 1409. When the computer program is loaded into RAM 1403 and executed by the computing unit 1401, one or more steps of the image-based parking information processing method described above may be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform an image-based parking information processing method by any other suitable means (e.g., by means of firmware).

[0282] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0283] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0284] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0285] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0286] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0287] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0288] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0289] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image-based parking space information processing method, comprising: Obtain N auxiliary lines for the parking space to be labeled in the image. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3. Obtain the intersection point between every two auxiliary lines among the N auxiliary lines; Data analysis is performed on the intersection points between each pair of auxiliary lines to obtain the initial hidden point; The position of the initial hidden point is adjusted based on preset constraints to obtain the target hidden points of the N auxiliary lines; wherein, the preset constraints are used to make the line connecting the adjusted initial hidden point and the target point on the auxiliary line parallel to the N auxiliary lines. Construct a line connecting any point on the line to be labeled to the target hidden point to obtain the position information of the line to be labeled in the image; Based on the position information of the line to be labeled in the image, the key points of the parking space are obtained.

2. The method according to claim 1, wherein, Adjusting the position of the initial hidden line disappearance point according to preset constraints yields the target hidden line disappearance points for the N auxiliary lines, including: Perform the following for each of the N auxiliary lines: Connect the initial hidden point and the midpoint of the auxiliary line to obtain a reference straight line; the midpoint is the target point. Obtain the distances from the endpoints of the auxiliary line to the reference line; The sum of the distances from the 2N endpoints of the N auxiliary lines to the corresponding reference lines is obtained. The minimum of the sum of the distances is used as the preset constraint condition. The position of the initial hidden point is adjusted to obtain the target hidden point of the N auxiliary lines.

3. The method according to claim 1, wherein, Data analysis is performed on the intersection points between each pair of auxiliary lines to obtain the initial hidden point, including: The initial hidden point is obtained by determining the mean value of the intersection points between every two auxiliary lines.

4. The method according to any one of claims 1-3, wherein, The acquisition of N auxiliary lines for the parking space to be labeled in the image includes: In response to the annotation operation on the line segments in the image, N auxiliary lines parallel to the line to be annotated are obtained.

5. The method according to claim 1, wherein, The acquisition of N auxiliary lines for the parking space to be labeled in the image includes: In response to the annotation operation on the parking space line to be annotated in the image, the line to be annotated is obtained; Candidate line segments in the image are obtained based on line detection. Among the candidate line segments, N line segments parallel to the line to be labeled are selected to obtain the N auxiliary lines.

6. The method according to any one of claims 1-3, further comprising: Target detection is performed on the acquired images to be processed to obtain the target vehicles; Based on the key points of the parking space and the positional relationship of the target vehicle in the image, the parking space where the target vehicle is located is determined as the target parking space; Obtain the overlap between the target vehicle and the target parking space, as well as the relative angle between the orientation of the target vehicle and the orientation of the target parking space; The parking type of the target vehicle is determined based on the degree of overlap and the relative angle.

7. The method according to claim 6, wherein, Obtaining the overlap between the target vehicle and the target parking space includes: Based on the key points of the target parking space, obtain the rectangular frame of the target parking space in the top plane of the world coordinate system. The vehicle bounding box of the target vehicle in the top plane of the world coordinate system is determined based on the vehicle key points of the target vehicle. The degree of overlap between the target vehicle and the target parking space is determined based on the overlapping area between the parking space rectangle and the vehicle rectangle.

8. The method according to claim 6, wherein, Determining the parking type of the target vehicle based on the overlap and the relative angle includes: If the overlap is greater than a first set threshold and less than a second set threshold, and the relative angle is greater than a first angle threshold, then the parking type is determined to include abnormal parking. If the parking type is not abnormal parking, the overlap is greater than the first set threshold and less than the third set threshold, and the relative angle is greater than the second angle threshold, then the parking type is determined to include angled parking; the third set threshold is greater than the second set threshold, and the second angle threshold is less than the first angle threshold. If the overlap is greater than the first set threshold and less than the fourth set threshold, the parking type is determined to include cross-space parking; the fourth set threshold is less than the second set threshold and greater than the first set threshold.

9. The method according to claim 6, further comprising: Extract the key points at the front and rear of the target vehicle; The orientation of the target vehicle is determined based on the key points at the front and rear of the target vehicle. If the orientation of the target vehicle and the orientation of the target parking space are inconsistent, the parking type is determined to be reverse parking.

10. The method according to any one of claims 1-3, further comprising: Calculate the hidden point in the vertical direction based on the auxiliary line in the vertical direction; Based on the hidden point and the height information of the marker in the vertical direction, the transformation relationship from the image coordinate system to the world coordinate system is determined.

11. The method of claim 10, further comprising: Based on the transformation relationship from the image coordinate system to the world coordinate system, the height of the target vehicle in the image to be processed is determined.

12. The method according to any one of claims 1-3, wherein multiple parking spaces whose outer edge main line and inner edge main line are on the same straight line constitute a parking space group, wherein, When the line to be labeled is the main line of the parking space group, the step of constructing a line connecting any point on the line to be labeled with the target hidden point to obtain the position information of the labeled line in the image includes: In response to the selection operation of any point on the outer edge main line of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the outer edge main line of the parking space to obtain the straight line equation of the outer edge main line of the parking space in the image; and, In response to the selection operation of any point on the inner edge of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the inner edge of the parking space to obtain the straight line equation of the inner edge of the parking space in the image.

13. The method according to any one of claims 1-3, wherein multiple parking spaces whose outer edge main line and inner edge main line are on the same straight line constitute a parking space group, wherein, When the line to be labeled is a secondary parking line of the parking space group, the step of constructing a line connecting any point on the line to be labeled with the target hidden point to obtain the position information of the labeled line in the image includes: For any parking space sub-line in the parking space group, in response to the selection operation of any point on the parking space sub-line in the image, a straight line is constructed between the target hidden point and the parking space vertex to obtain the straight line equation of the parking space sub-line in the image.

14. The method according to claim 13, wherein, Based on the positional information of the line to be labeled in the image, key points of the parking space are obtained, including: Two intersection points are obtained based on the linear equations of the main outer edge of the parking space and the two secondary parking space lines in the image; and... Two intersection points are obtained based on the equation of the straight line along the main line within the parking space in the image and the equations of the straight lines of the two secondary lines of the parking space in the image; The obtained intersection points are determined as the key points of the parking space.

15. An image-based parking space information processing device, comprising: The auxiliary line determination module is used to obtain N auxiliary lines for the parking space to be labeled in the image. The N auxiliary lines are parallel to the line to be labeled, and N is an integer greater than or equal to 3. The hidden point determination module includes: an intersection point determination unit, used to obtain the intersection point between every two auxiliary lines in the N auxiliary lines; an initial hidden point determination unit, used to perform data analysis on the intersection points between every two auxiliary lines to obtain an initial hidden point; and a hidden point optimization unit, used to adjust the position of the initial hidden point based on preset constraints to obtain the target hidden point of the N auxiliary lines; wherein, the preset constraints are used to ensure that the line connecting the adjusted initial hidden point and the target point on the auxiliary line is parallel to the N auxiliary lines; The construction module is used to construct a line connecting any point on the line to be labeled to the target hidden point, thereby obtaining the position information of the line to be labeled in the image; The key point determination module is used to obtain parking space key points based on the position information of the line to be labeled in the image.

16. The apparatus according to claim 15, wherein, The hidden point optimization unit is used for: Perform the following for each of the N auxiliary lines: Connect the initial hidden point and the midpoint of the auxiliary line to obtain a reference straight line; the midpoint is the target point. Obtain the distances from the endpoints of the auxiliary line to the reference line; The sum of the distances from the 2N endpoints of the N auxiliary lines to the corresponding reference lines is obtained. The minimum of the sum of the distances is used as the preset constraint condition. The position of the initial hidden point is adjusted to obtain the target hidden point of the N auxiliary lines.

17. The apparatus according to claim 15, wherein, The initial blanking point determination unit is used for: The initial hidden point is obtained by determining the mean value of the intersection points between every two auxiliary lines.

18. The apparatus according to any one of claims 15-17, wherein, The auxiliary line determination module is used for: In response to the annotation operation on the line segments in the image, N auxiliary lines parallel to the line to be annotated are obtained.

19. The apparatus according to claim 18, wherein, The auxiliary line determination module is used for: In response to the annotation operation on the parking space line to be annotated in the image, the line to be annotated is obtained; Candidate line segments in the image are obtained based on line detection. Among the candidate line segments, N line segments parallel to the line to be labeled are selected to obtain the N auxiliary lines.

20. The apparatus according to any one of claims 15-17, further comprising: The vehicle detection module is used to perform target detection on the acquired images to be processed, and to obtain the target vehicles; The parking space determination module is used to determine the parking space where the target vehicle is located as the target parking space based on the key points of the parking space and the positional relationship of the target vehicle in the image; The acquisition module is used to acquire the overlap between the target vehicle and the target parking space, as well as the relative angle between the orientation of the target vehicle and the orientation of the target parking space. A determination module is used to determine the parking type of the target vehicle based on the overlap and the relative angle.

21. The apparatus according to claim 20, wherein, The acquisition module is used for: Based on the key points of the target parking space, obtain the rectangular frame of the target parking space in the top plane of the world coordinate system. The vehicle bounding box of the target vehicle in the top plane of the world coordinate system is determined based on the vehicle key points of the target vehicle. The degree of overlap between the target vehicle and the target parking space is determined based on the overlapping area between the parking space rectangle and the vehicle rectangle.

22. The apparatus according to claim 20, wherein, The determining module is used for: If the overlap is greater than a first set threshold and less than a second set threshold, and the relative angle is greater than a first angle threshold, then the parking type is determined to include abnormal parking. If the parking type is not abnormal parking, the overlap is greater than the first set threshold and less than the third set threshold, and the relative angle is greater than the second angle threshold, then the parking type is determined to include angled parking; the third set threshold is greater than the second set threshold, and the second angle threshold is less than the first angle threshold. If the overlap is greater than the first set threshold and less than the fourth set threshold, the parking type is determined to include cross-space parking; the fourth set threshold is less than the second set threshold and greater than the first set threshold.

23. The apparatus according to claim 20, wherein the determining module is further configured to: Extract the key points at the front and rear of the target vehicle; The orientation of the target vehicle is determined based on the key points at the front and rear of the target vehicle. If the orientation of the target vehicle and the orientation of the target parking space are inconsistent, the parking type is determined to be reverse parking.

24. The apparatus according to any one of claims 15-17, further comprising a high-quantization module for: Calculate the hidden point in the vertical direction based on the auxiliary line in the vertical direction; Based on the hidden point and the height information of the marker in the vertical direction, the transformation relationship from the image coordinate system to the world coordinate system is determined.

25. The apparatus of claim 24, wherein the height quantization module is further configured to: Based on the transformation relationship from the image coordinate system to the world coordinate system, the height of the target vehicle in the image to be processed is determined.

26. The apparatus according to any one of claims 15-17, wherein multiple parking spaces whose outer edge main line and inner edge main line are on the same straight line constitute a parking space group, wherein, When the line to be labeled is the main line of the parking space group, the construction module, which connects any point on the line to be labeled with the target hidden point to obtain the position information of the labeled line in the image, is used to: In response to the selection operation of any point on the outer edge main line of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the outer edge main line of the parking space to obtain the straight line equation of the outer edge main line of the parking space in the image. as well as, In response to the selection operation of any point on the inner edge of the parking space group in the image, a line is constructed connecting the target hidden point and any point on the inner edge of the parking space to obtain the straight line equation of the inner edge of the parking space in the image.

27. The apparatus according to any one of claims 15-17, wherein multiple parking spaces whose outer edge main line and inner edge main line are on the same straight line constitute a parking space group, wherein, When the line to be labeled is a secondary parking line of the parking space group, the construction module, which connects any point on the line to be labeled with the target hidden point to obtain the position information of the labeled line in the image, is further configured to: For any parking space sub-line in the parking space group, in response to the selection operation of any point on the parking space sub-line in the image, a straight line is constructed between the target hidden point and the parking space vertex to obtain the straight line equation of the parking space sub-line in the image.

28. The apparatus according to claim 27, wherein, The key point determination module is also used for: Two intersection points are obtained based on the linear equations of the main outer edge of the parking space and the two secondary parking space lines in the image; and... Two intersection points are obtained based on the equation of the straight line along the main line within the parking space in the image and the equations of the straight lines of the two secondary lines of the parking space in the image; The obtained intersection points are determined as the key points of the parking space.

29. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-14.

30. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-14.

31. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-14.

Images