Parking space reconstruction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present disclosure disclose a parking space reconstruction method and device, electronic equipment and a storage medium, wherein the method comprises: determining, based on first parking space corner point detection results respectively corresponding to each frame in a current frame set, a first number of second parking space corner point detection results that meet a preset condition in terms of observation confidence of each detected parking space; determining a first state quantity respectively corresponding to each parking space in the current frame set; optimizing each first state quantity based on each second parking space corner point detection result respectively corresponding to each parking space in the current frame set and a preset optimization rule, to obtain an optimized second state quantity respectively corresponding to each parking space in the current frame set; and determining a target position in a first coordinate system respectively corresponding to each parking space based on the second state quantity respectively corresponding to each parking space. The embodiments of the present disclosure can effectively improve the robustness of parking space optimization, thereby improving the accuracy of the obtained parking space position.

Description

Technical Field

[0001] This disclosure relates to computer vision technology, and in particular to a parking space reconstruction method, apparatus, electronic device, and storage medium. Background Technology

[0002] Parking space reconstruction has become a crucial technology in parking scenarios such as HPA (Hyper-Learning Parking) and AVP (Valet Parking), and is an indispensable part of high-precision maps. Current parking space reconstruction typically relies on deep learning models to extract parking space corner points, and then uses a frame sliding window strategy to optimize these corner points based on the parking space observations within the sliding window. However, the frame sliding window strategy has poor robustness to false detections, false negatives, image calibration errors, and image stitching errors, resulting in poor accuracy of the optimized parking space locations. Summary of the Invention

[0003] To address the technical problems such as poor accuracy in parking space reconstruction caused by the aforementioned frame sliding window strategy, this disclosure is proposed. Embodiments of this disclosure provide a parking space reconstruction method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, a parking space reconstruction method is provided, comprising: determining, based on the first parking space corner detection results corresponding to each frame in a current frame set, a first number of second parking space corner detection results whose observation confidence satisfies a preset condition for each detected parking space, wherein the current frame set includes the current frame and a second number of historical frames, the first parking space corner detection results include parking space corner detection results corresponding to at least one parking space detected in its corresponding frame, and both the first parking space corner detection results and the second parking space corner detection results include the observation position of at least one corner point in a first coordinate system of the corresponding parking space; and determining a first state corresponding to each of the parking spaces in the current frame set. The quantities are as follows: for a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space; for a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the previously detected parking space is the optimized state quantity corresponding to the previously detected parking space in the previous frame set; based on the detection results of the corner points of each second parking space corresponding to each parking space in the current frame set and the preset optimization rules, each first state quantity is optimized to obtain the optimized second state quantity corresponding to each parking space in the current frame set; based on the second state quantity corresponding to each parking space, the target position in the first coordinate system corresponding to each parking space is determined.

[0005] According to another aspect of the present disclosure, a parking space reconstruction apparatus is provided, comprising: a first processing module, configured to determine, based on the first parking space corner detection results corresponding to each frame in the current frame set, a first number of second parking space corner detection results whose observation confidence satisfies a preset condition for each detected parking space, wherein the current frame set includes the current frame and a second number of historical frames, the first parking space corner detection results include parking space corner detection results corresponding to at least one parking space detected in its corresponding frame, and both the first parking space corner detection results and the second parking space corner detection results include the observation position of at least one corner point in a first coordinate system of the corresponding parking space; and a first determining module, configured to determine the first state corresponding to each of the parking spaces in the current frame set. The first state quantity is defined as follows: for a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space; for a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the detected parking space is the optimized state quantity corresponding to the detected parking space in the previous frame set; the second processing module is used to optimize each first state quantity based on the detection results of each second parking space corner point corresponding to each parking space in the current frame set and a preset optimization rule, to obtain the optimized second state quantity corresponding to each parking space in the current frame set; the third processing module is used to determine the target position in the first coordinate system corresponding to each parking space based on the second state quantity corresponding to each parking space.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the parking space reconstruction method described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the parking space reconstruction method according to any of the above embodiments of the present disclosure.

[0008] Based on the parking space reconstruction method, apparatus, electronic device, and storage medium provided in the above embodiments of this disclosure, the first number of second parking space corner point detection results corresponding to each parking space are determined by the observation confidence level. This realizes the observation sliding window based on the observation confidence level, which is used to optimize the state variables of the parking space and obtain the target position of the optimized parking space. This can effectively reduce the image caused by image calibration error, perception error, etc., effectively improve the robustness of parking space optimization, and thus improve the accuracy of the optimized parking space position.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the parking space reconstruction method provided in this disclosure;

[0012] Figure 2 This is a schematic flowchart of a parking space reconstruction method provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a flowchart illustrating a parking space reconstruction method provided in another exemplary embodiment of this disclosure;

[0014] Figure 4 This is a schematic diagram of distortion error provided by an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a flowchart illustrating step 2031 provided in an exemplary embodiment of this disclosure;

[0016] Figure 6 This is a schematic diagram of a parallelogram parking space provided in an exemplary embodiment of this disclosure;

[0017] Figure 7 This is a schematic diagram illustrating the principle of parking space orientation residuals provided in an exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram illustrating the principle of parking space length residual provided in an exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram illustrating the principle of the side projection residual of a parking space provided in an exemplary embodiment of this disclosure;

[0020] Figure 10 This is a flowchart illustrating a parking space reconstruction method provided in yet another exemplary embodiment of this disclosure;

[0021] Figure 11 This is a schematic diagram of the entry line alignment provided in an exemplary embodiment of this disclosure;

[0022] Figure 12 This is a schematic diagram of the structure of a parking space reconstruction device provided in an exemplary embodiment of this disclosure;

[0023] Figure 13This is a schematic diagram of the structure of a parking space reconstruction device provided in another exemplary embodiment of this disclosure;

[0024] Figure 14 This is a schematic diagram of the structure of one application embodiment of the electronic device disclosed herein. Detailed Implementation

[0025] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0026] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0027] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0028] It should also be understood that in the embodiments disclosed herein, "multiple" can refer to two or more, and "at least one" can refer to one, two or more.

[0029] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0030] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0032] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0033] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0034] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0035] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0036] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0037] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0038] This disclosure outlines

[0039] In the process of developing this disclosure, the inventors discovered that parking space reconstruction has become an important technology in parking scenarios such as HPA (Hyper-Learning Parking) and AVP (Valet Parking), and is an indispensable part of high-precision maps. Current parking space reconstruction methods typically extract parking space corner points based on deep learning models, and then optimize the parking space corner points based on the parking space observations in the window using a frame sliding window strategy. However, the frame sliding window strategy has poor robustness to false detections, missed detections, image calibration errors, and image stitching errors, resulting in poor accuracy of the optimized parking space locations.

[0040] Exemplary Overview

[0041] Figure 1 This is an exemplary application scenario of the parking space reconstruction method provided in this disclosure.

[0042] In parking scenarios, pre-configured corner detection models can be used to detect parking space corners, obtaining corner detection results (referred to as first parking space corner detection results) in a first coordinate system (e.g., world coordinate system) for each parking space. The corner detection model can be any implementable model, such as a deep learning-based corner detection model, and can be configured according to actual needs. For example, corner detection can be performed by capturing environmental images of the vehicle's surroundings using a camera, and then using these images to obtain the first parking space corner detection results for each space. Alternatively, the corner detection results can be obtained in an image coordinate system based on the environmental images, and then converted to the first coordinate system to obtain the first parking space corner detection results for each space. The specific implementation is not limited. Using the parking space reconstruction method disclosed herein, based on the first parking space corner detection results and observation confidence rules corresponding to each frame in the current frame set, a first number of second parking space corner detection results whose observation confidence meets preset conditions can be determined for each parking space. The current frame set can include the current frame and a second number of historical frames. The second number can be set according to actual needs; for example, it can include all historical frames or historical frames whose observation confidence meets preset conditions. The observation confidence can be determined in any feasible way, such as based on the error distribution description function or based on the parking space semantic segmentation results. The observation confidence characterizes the reliability of the first parking space corner detection result in other observation methods. The higher the observation confidence, the closer the first parking space corner detection result is to the real parking space corner. After determining the detection results of the corner points of each second parking space corresponding to each parking space, the first state quantity corresponding to each parking space in the current frame set can be determined. The first state quantity is the state quantity to be optimized for each parking space in the current frame set. For a new parking space detected for the first time in the current frame set, the initial state quantity of the new parking space is used as the first state quantity. For a previously detected parking space that has not been detected for the first time in the current frame set, the optimized state quantity corresponding to the previously detected parking space in the previous frame set is used as the first state quantity of the previously detected parking space in the current frame set. Then, based on the detection results of the corner points of each second parking space corresponding to each parking space in the current frame set, the first state quantity corresponding to each parking space is optimized to obtain the optimized second state quantity corresponding to each parking space. Based on the second state quantity corresponding to each parking space, the target position in the first coordinate system corresponding to each parking space can be determined. Since the first number of second parking space corner detection results are determined based on the observation confidence of the first parking space corner detection results for the optimization of state variables, an observation sliding window based on observation confidence is realized. Compared with the frame sliding window rules of related technologies, the parking space reconstruction method disclosed in this invention adopts better observation results, which can effectively improve the robustness to false detection, missed detection, image calibration error, image stitching error, etc., thereby improving the accuracy of the parking space position obtained by optimization.

[0043] The parking space reconstruction method disclosed herein is not limited to parking scenarios, but can also be applied to any other scenarios that require parking space reconstruction, such as the mapping scenario of parking lot maps, without any specific limitations.

[0044] Exemplary methods

[0045] Figure 2 This is a schematic flowchart of a parking space reconstruction method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, on an in-vehicle computing platform. Figure 2 As shown, it includes the following steps:

[0046] Step 201: Based on the first parking space corner detection results corresponding to each frame in the current frame set, determine a first number of second parking space corner detection results whose observation confidence meets the preset conditions for each detected parking space. The current frame set includes the current frame and a second number of historical frames. The first parking space corner detection result includes the parking space corner detection results corresponding to at least one parking space detected in its corresponding frame. Both the first parking space corner detection result and the second parking space corner detection result include the observation position of at least one corner point in the first coordinate system of its corresponding parking space.

[0047] The first and second quantities can be set according to actual needs. For example, the first quantity can be a preset value, such as 10, 20, etc. The second quantity can be the number of all historical frames before the current frame, or the number of historical frames whose observation confidence meets the preset conditions. The specific setting can be based on actual needs. The observation confidence can be determined in any feasible way, such as based on the error distribution description function or the parking space semantic segmentation result. The observation confidence characterizes the reliability of the first parking space corner detection result in other observation methods; the higher the observation confidence, the closer the first parking space corner detection result is to the actual parking space corner. The preset conditions can be set according to actual needs. The first coordinate system can be the world coordinate system. The first parking space corner detection result can be obtained using any feasible corner detection model, such as a deep learning-based corner detection model. The specific setting can be based on actual needs.

[0048] In one optional example, when performing corner detection in the current frame, the vehicle's surrounding environment image can be captured by the vehicle's camera. Parking space corner detection can then be performed based on this environmental image to obtain the first parking space corner detection result for each parking space in the current frame. This result can be used as the first parking space corner detection result for the current frame. Alternatively, the parking space corner detection result can be obtained in the image coordinate system based on the current frame's environmental image, and then transformed to the first coordinate system to obtain the first parking space corner detection result for each parking space in the current frame. The specific method is not limited. Each historical frame uses the historical time corresponding to that historical frame as the current frame, and the method for obtaining the corresponding first parking space corner detection result is the same as that for the current frame, and will not be repeated here. Each frame may include one or more parking spaces. The first parking space corner detection result corresponding to each frame includes the first parking space corner detection results corresponding to each parking space detected in that frame. Therefore, for a parking space, there may be a first parking space corner detection result in one or more frames. For example, for a certain parking space A, if parking space A is observed in 20 consecutive frames, then each of those 20 frames can correspond to the first parking space corner detection result for parking space A. Each first parking space corner detection result may include the observation position of at least one corner point in the first coordinate system of its corresponding parking space, such as including the observation positions corresponding to 4 corner points. For each parking space, the corresponding second parking space corner detection results are a portion determined from the corresponding first parking space corner detection results based on the observation confidence. Therefore, each second parking space corner detection result essentially corresponds to one first parking space corner detection result, and each second parking space corner detection result also includes the observation position of at least one corner point in the first coordinate system of its corresponding parking space.

[0049] In one optional embodiment, parking spaces in different frames can be tracked using a tracking algorithm to determine the correspondence between parking spaces in different frames. The specific principle will not be elaborated here.

[0050] In practical applications, the initial number for each parking space can be the same or different, depending on the actual needs. For example, when a parking space is first observed, there is only one observation frame, meaning there is only one detection result for the first parking space corner point, and the initial number is 1. As the number of observation frames increases, the initial number can increase. When the number of observation frames reaches the threshold, the initial number can be a fixed value, such as setting the initial number to 20 frames, which can be set according to the actual needs.

[0051] Step 202: Determine the first state quantity corresponding to each parking space in the current frame set. For a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space. For a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the previously detected parking space is the optimized state quantity corresponding to the previously detected parking space in the previous frame set.

[0052] In this context, the first state variable corresponding to each parking space is the state variable to be optimized for each parking space in the current frame set. For parking spaces that are not detected for the first time in the current frame set, they have already been optimized in the previous frame set. Based on the invariance of the actual position of the parking space, the optimized state variable corresponding to the detected parking space in the previous frame set can be used as the first state variable of the detected parking space in the current frame set. For new parking spaces detected for the first time in the current frame set, there is no corresponding state variable yet. Therefore, it is necessary to initialize their state variables to obtain an initialized state variable, which is then used as the first state variable corresponding to the new parking space. The number of new parking spaces is determined based on the actual detection results and may include one or more, or the current frame may not include any new parking spaces; there is no specific limitation. The initial state variable of a new parking space can be determined in any feasible way. For example, the state variable of the new parking space can be initialized based on the detection result of the first corner point of the new parking space to obtain the initialized state variable.

[0053] In an optional example, the first state variable X1 may include the parking space's heading angle θ1, interior angle α1, center point x-coordinate px1, center point y-coordinate py1, width W1, and length L1, and can be represented as:

[0054] X1 = [θ1, α1, px1, py1, W1, L1] T

[0055] Where T represents transpose.

[0056] Step 203: Based on the detection results of the corner points of each second parking space corresponding to each parking space in the current frame set and the preset optimization rules, optimize each first state variable to obtain the optimized second state variable corresponding to each parking space in the current frame set.

[0057] The preset optimization rules can be set according to actual needs. For example, any nonlinear least squares optimization algorithm can be used to optimize the state variables.

[0058] Step 204: Based on the second state variables corresponding to each parking space, determine the target position in the first coordinate system corresponding to each parking space.

[0059] In an optional embodiment, the target location may include the target coordinates of the four corner points of the parking space in the first world coordinate system. The target location corresponding to each parking space can be determined based on the transformation relationship between the second state variable and the corner points of the parking space.

[0060] In an optional example, the optimized second state variable for rectangular parking space A is represented as:

[0061] X2 = [θ2, α2, px2, py2, W2, l2] T

[0062] The coordinates of the four corner points of parking space A are obtained through the following transformation relationships:

[0063]

[0064]

[0065]

[0066]

[0067] Where P0, P1, P2 and P3 represent the coordinates of the four corner points of parking space A.

[0068] In an optional embodiment, each state variable in the second state variable can also be used as the target location of the parking space. For example, X2 mentioned above can be used as the target location of the parking space. The subsequent application can determine the coordinates of the four corner points of the parking space based on the target location. This embodiment does not limit this.

[0069] The parking space reconstruction method provided in this embodiment determines a first number of second parking space corner detection results that meet preset conditions from the first parking space corner detection results corresponding to each parking space in the current frame set based on observation confidence. These results are used to optimize the first state variables corresponding to each parking space to obtain the optimized second state variables corresponding to each parking space. Then, the target position corresponding to each parking space is determined based on the optimized second state variables. Since the state variables are optimized using the second parking space corner detection results with better observation, an observation sliding window based on observation confidence is realized. Compared with the frame sliding window rules of related technologies, this method can effectively improve the robustness to false detections, missed detections, image calibration errors, image stitching errors, etc., thereby improving the accuracy of the parking space positions obtained by optimization.

[0070] Figure 3 This is a flowchart illustrating a parking space reconstruction method provided in another exemplary embodiment of this disclosure.

[0071] In an optional embodiment, step 201 may specifically include the following steps:

[0072] Step 2011: Based on the detection results of each first parking space corner point and the pre-obtained error distribution description function, determine the first observation confidence of the detection results of each first parking space corner point corresponding to each parking space in each frame; and / or, based on the detection results of each first parking space corner point and the semantic segmentation results of each first parking space corresponding to each frame, determine the second observation confidence of the detection results of each first parking space corner point corresponding to each parking space in each frame.

[0073] Among them, the error distribution description function is used to describe the distortion error distribution of parking space observation. The larger the error, the lower the confidence level. Based on this principle, a mapping rule from error to confidence level can be set to map the error to the confidence level and determine the first observation confidence level of the first parking space corner detection result corresponding to each parking space.

[0074] In one optional embodiment, parking space corner detection is implemented based on an IPM (Inverse Perspective Mapping) bird's-eye view. For example, an bird's-eye view is obtained by transforming images acquired by a four-way surround-view fisheye camera using IPM. The coordinate system corresponding to the bird's-eye view can be the vehicle coordinate system or other coordinate systems that can represent the bird's-eye view. The bird's-eye view has certain distortion errors in both the horizontal and vertical directions. For example, Figure 4 This is a schematic diagram of distortion error provided by an exemplary embodiment of this disclosure. Brighter areas show smaller errors. The error distribution description function is used to simulate... Figure 4 The distortion error is shown. Determining the error requires the corner coordinates of other coordinate systems (such as image coordinate system, vehicle coordinate system). This can be obtained by performing coordinate transformation on the observed position of the corner point in the first coordinate system in the first parking space corner detection result. Alternatively, the corresponding corner coordinates in other coordinate systems can be pre-stored. Then, the corner coordinates in other coordinate systems can be obtained by using the corner observation position in the first parking space corner detection result and the pre-stored correspondence. The specific principle will not be elaborated here.

[0075] In one optional example, the error distribution description function can be a modified hyperbolic parabolic description function, which can be set according to actual needs.

[0076] In an optional embodiment, the first parking space semantic segmentation result of each frame may include the parking space contour pixels corresponding to each parking space observed in the frame. By transforming the parking space contour pixels to the first coordinate system, the parking space contour points in the first coordinate system can be obtained. The parking space line can be determined based on the parking space contour points. The second observation confidence of the first parking space corner detection result can be determined by the distance between the observation position of each corner point in the first parking space corner detection result and the parking space line (the larger the distance, the lower the confidence).

[0077] In an optional embodiment, the observation positions of each corner point in the first parking space corner detection result can be projected onto the image of the corresponding frame to obtain the projection point. The second observation confidence is determined by the distance between the projection point and the parking line in the image coordinate system determined by the parking space semantic segmentation result.

[0078] Step 2012: For any parking space, based on the first observation confidence and / or the second observation confidence of the corner detection results of each first parking space corresponding to that parking space, determine a first number of second parking space corner detection results whose observation confidence satisfies the preset conditions.

[0079] The preset conditions can be set according to actual needs. For example, the preset condition can be the detection results of the second parking space corner points with high observation confidence (TOPn, which are the top n points ranked from high to low observation confidence). n represents the first quantity.

[0080] In an alternative embodiment, the detection results of each second parking space corner point can be determined based solely on either the first observation confidence level or the second observation confidence level.

[0081] In one optional embodiment, the detection results of each second parking space corner point can be determined by combining the first observation confidence level and the second observation confidence level. For example, the detection results of the first parking space corner points with the highest first observation confidence level and the highest second observation confidence level can be selected, and after deduplication, a first number of second parking space corner point detection results can be formed. The specific combining rules can be set according to actual needs.

[0082] In this embodiment, the corner detection results of the second parking space corresponding to the better observation are determined by the first observation confidence level and / or the second observation confidence level, which are used to optimize the state variables and further improve the robustness and accuracy of corner detection.

[0083] In an optional embodiment, step 203, based on the detection results of the corner points of each second parking space corresponding to each parking space in the current frame set and the preset optimization rules, optimizes each first state variable to obtain the optimized second state variable corresponding to each parking space in the current frame set, including:

[0084] Step 2031: Based on the detection results of the corner points of each second parking space corresponding to each parking space and the preset objective function, the preset optimization algorithm is used to optimize each first state variable to obtain the optimized second state variables corresponding to each parking space in the current frame set. The preset objective function includes at least one of the following residual functions: parking space entrance corner point residual function, parking space orientation residual function, parking space side projection residual function, and parking space length residual function.

[0085] The preset objective function is constructed based on at least one of the following residual functions: parking space entrance corner residual function, parking space orientation residual function, parking space side projection residual function, and parking space length residual function. This residual function serves as the target for state variable optimization. The goal is to minimize the value of the preset objective function by continuously optimizing the state variables. The preset optimization algorithm can be a nonlinear least squares algorithm, which transforms the optimization of parking space corner points into a nonlinear least squares problem. This problem can be solved using certain algorithms (such as steepest gradient descent, Newton's method, Gauss-Newton method, etc.) to achieve iterative updates of the state variables, thereby optimizing the parking space position. The parking space entrance corner point residual function characterizes the reprojection residual between the state variable and the observed position of the corner point in the second parking space corner point detection result; the parking space orientation residual function characterizes the parking space orientation residual between the state variable and the second parking space corner point detection result; the parking space side projection residual function characterizes the projection residual between the parking space side determined by the state variable and the parking space side determined by the second parking space corner point detection result; and the parking space length residual function characterizes the residual between the parking space length determined by the state variable and the parking space length determined by the second parking space corner point detection result. By constraining the optimization of the state variable through at least one of the residual functions, the parking space position determined by the optimized state variable is closer to the actual parking space position.

[0086] This embodiment transforms the optimization of parking space corner points into a nonlinear optimization problem by using the detection results of the second parking space corner points corresponding to each parking space and a preset objective function. This facilitates the iterative updating of state variables through a certain solution algorithm, thereby achieving the optimization of parking space positions.

[0087] Figure 5 This is a flowchart illustrating step 2031 provided in an exemplary embodiment of this disclosure.

[0088] In an optional embodiment, step 2031, based on the detection results of the corner points of each second parking space corresponding to each parking space and a preset objective function, uses a preset optimization algorithm to optimize each first state variable to obtain the optimized second state variables corresponding to each parking space in the current frame set, including:

[0089] Step 20311: Based on the first state variables corresponding to each parking space and the detection results of the corner points of each second parking space, determine the parking space entrance corner residual, parking space orientation residual, parking space side projection residual, parking space length residual, and the current Jacobian matrix. The current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, parking space orientation residual, and parking space length residual with respect to the first state variables, or the current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, parking space side projection residual, and parking space length residual with respect to the first state variables. The parking space entrance corner residual can be determined based on the parking space entrance corner residual function, the parking space orientation residual can be determined based on the parking space orientation residual function, the parking space side projection residual can be determined based on the parking space side projection residual function, and the parking space length residual can be determined based on the parking space length residual function.

[0090] Since both the parking space orientation residual and the parking space side projection residual represent the parking space orientation residual, one of them can be used to reduce computational complexity.

[0091] In an optional embodiment, if parking space orientation residuals are used, the parking space entrance corner residual, parking space orientation residual, parking space length residual, and current Jacobian matrix can be determined based on the first state variable corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space. In this case, the current Jacobian matrix (which can be called the first current Jacobian matrix) includes the Jacobian of the parking space entrance corner residual, parking space orientation residual, and parking space length residual with respect to the first state variable.

[0092] In an optional embodiment, if the parking space side projection residual is used, the parking space entrance corner residual, parking space orientation residual, parking space side projection residual, parking space length residual and current Jacobian matrix can be determined based on the first state variable corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space. In this case, the current Jacobian matrix (which can be called the second current Jacobian matrix) includes the Jacobian of the parking space entrance corner residual, the parking space side projection residual and the parking space length residual with respect to the first state variable.

[0093] In an optional embodiment, both the parking space orientation residual and the parking space side projection residual can be used. In this case, the current Jacobian matrix (which can be called the third current Jacobian matrix) includes the Jacobian of the parking space entrance corner point residual, the parking space orientation residual, the parking space side projection residual, and the parking space length residual with respect to the first state variable. The specific settings can be configured according to actual needs.

[0094] In one optional embodiment, for any type of residual, it can be determined based on the first state variable corresponding to each parking space, the detection results of the corner points of each second parking space, and the determination rule corresponding to the residual. For example, for the parking space entrance corner point residual, for each parking space, the first state variable is mapped to the entrance corner point coordinates, and the difference between the entrance corner point coordinates and the entrance corner point observation position in the second parking space corner point detection results is used to determine the parking space entrance corner point residual. The determination rule corresponding to each residual can be set according to actual needs.

[0095] Step 20312: Determine the current iteration step size based on the parking space entrance corner residual, parking space orientation residual, parking space length residual, and the current Jacobian matrix; or, determine the current iteration step size based on the parking space entrance corner residual, parking space side projection residual, parking space length residual, and the current Jacobian matrix.

[0096] The current iteration step size can be determined using any feasible solution algorithm, such as the Gauss-Newton method, without any specific limitation. The current Jacobian matrix can be obtained by taking the first-order partial derivatives of each residual with respect to each of the six state variables of the first state variable (such as the heading angle, interior angle, x-coordinate of the center point, y-coordinate of the center point, width, and length). The specific principle for determining the first-order partial derivatives will not be elaborated here.

[0097] In an optional embodiment, referring to the foregoing, when choosing between the parking space orientation residual and the parking space side projection residual, if step 20311 uses the parking space orientation residual, the current iteration step size is determined based on the parking space entrance corner point residual, the parking space orientation residual, the parking space length residual, and the first current Jacobian matrix; if step 20311 uses the parking space side projection residual, the current iteration step size is determined based on the parking space entrance corner point residual, the parking space side projection residual, the parking space length residual, and the second current Jacobian matrix.

[0098] In an optional embodiment, if both the parking space orientation residual and the parking space side projection residual are used, the current iteration step size is determined based on the parking space entrance corner residual, the parking space orientation residual, the parking space side projection residual, the parking space length residual, and the third current Jacobian matrix.

[0099] Step 20313: Based on the current iteration step size, update each first state variable to obtain the second state variable corresponding to each parking space.

[0100] The current iteration step size represents the update step size of the first state variable. Therefore, by updating the first state variable using the current iteration step size, the optimized second state variable can be obtained.

[0101] This embodiment determines the current iteration step size by comprehensively considering the residuals of the parking space entrance corner point, the parking space orientation, or the parking space side projection, and the parking space length. The first state variable is then updated so that the parking space position determined by the optimized second state variable is closer to the actual parking space situation in terms of entrance corner point, parking space orientation, and parking space length, thereby further improving the accuracy of the parking space position.

[0102] In an optional embodiment, step 20311, which determines the parking space entrance corner residual based on the first state variable corresponding to each parking space and the corner detection results of each second parking space corresponding to each parking space, includes:

[0103] 1. For any parking space, based on the first state variable corresponding to the parking space, determine the coordinates of the first entrance corner point corresponding to each entrance corner point of the parking space.

[0104] 2. Based on the detection results of the corner points of each second parking space corresponding to the parking space, determine the observation position of the entrance corner point of the parking space in each frame.

[0105] 3. Based on the coordinates of each first entrance corner point corresponding to each parking space and the observation position of each parking space entrance corner point in each frame, determine the parking space entrance corner point residual.

[0106] The entrance corner points include the two corner points of the parking space entrance. The specific operation for determining the coordinates of each first entrance corner point based on the first state variable is similar to determining the corner point coordinates based on the second state variable. For example, for a rectangular parking space, refer to P0-P3 mentioned above. The four corner points of P0-P3 include the two corner points of the parking space entrance. In practical applications, entrance corner points can be marked for each parking space, ensuring that the same symbol is used for each entrance corner point throughout the observation process. For example, if P0 and P3 represent two entrance corner points, then the first state variable is converted into the coordinates of the first entrance corner points corresponding to each entrance corner point of the corresponding parking space based on the aforementioned conversion relationship between P0 and P3.

[0107] In one optional embodiment, the transformation relationship from state variables to corner coordinates of a parallelogram parking space differs from that for a rectangular parking space. For example, Figure 6 This is a schematic diagram of a parallelogram parking space provided in an exemplary embodiment of this disclosure. In this example, for parallelogram parking space B, based on the first state variable X1 = [θ1, α1, px1, py1, W1, L1] T The coordinates of the four corner points P0 are determined. ′ P1′, P2′, and P3′ can be represented as follows:

[0108]

[0109]

[0110]

[0111]

[0112] Based on the above transformation relationship, the coordinates of the first entrance corner points corresponding to the first state variables of parking space B can be determined.

[0113] The coordinates of each first entrance corner point determined by the first state variable of the i-th parking space (e.g., P0) ′ and P3 ′ The reprojection residuals of the observation positions of each entrance corner point (e.g., P0(1) and P3(1)) determined by the detection results of the j-th second parking space corner point of the parking space are expressed as:

[0114]

[0115] Among them, e ij (1) and e ij (2) Represent the residuals at the two entrance corners respectively.

[0116] The residual at the corner point of the parking space entrance is expressed as:

[0117]

[0118] Where N represents the total number of parking spaces observed in the current frame set, and M represents the number of second parking space corner detection results for each parking space, which is also the first number.

[0119] This embodiment uses the reprojection residual of the entry corner point as a residual term of a preset objective function for the optimization of state variables, thus ensuring the stability of the entry corner point.

[0120] In an optional embodiment, step 20311, which determines the parking space orientation residual based on the first state variable corresponding to each parking space and the corner detection results of each second parking space corresponding to each parking space, includes:

[0121] 1. For any parking space, based on the first state quantity corresponding to the parking space, determine the first straight line between the midpoint of the first entrance line and the midpoint of the first far end line corresponding to the first state quantity.

[0122] The midpoint of the first entrance line and the midpoint of the first far end line can be determined by the coordinates of the four corner points determined by the first state quantity. The first straight line is a straight line passing through the midpoint of the first entrance line and the midpoint of the first far end line, which is the centerline of the parking space determined by the first state quantity.

[0123] 2. Based on the detection results of the corner points of each second parking space corresponding to the parking space, determine the midpoint of the first observation far end line and the second straight line where the first observation far end line is located, respectively, corresponding to the detection results of the corner points of each second parking space.

[0124] For any second parking space corner detection result, the corresponding first observation far end line refers to the observation parking space line between the observation positions of the two far end corner points in the second parking space corner detection result, and the midpoint of the first observation far end line refers to the midpoint between the observation positions of the two far end corner points.

[0125] 3. Based on the first straight line and each of the second straight lines, determine the first intersection point corresponding to the detection results of each second parking space corner point.

[0126] For any second parking space corner detection result, its corresponding first intersection point is the intersection point of its corresponding second straight line and the first straight line.

[0127] 4. Based on each first intersection point and the midpoint of each first observation far end line, determine the orientation residual corresponding to the parking space.

[0128] Specifically, for any second parking space corner detection result, the residual of the second parking space corner detection result can be represented by the distance between its corresponding first intersection point and the midpoint of the first observation far end line. Based on the residuals of each second parking space corner detection result, the orientation residual corresponding to the parking space is determined. If the parking space and second parking space corner detection results of the first state variable are consistent, the first intersection point should coincide with the midpoint of the first observation far end line. Therefore, the orientation residual of one observation can be characterized by the distance between the first intersection point and the midpoint of the first observation far end line.

[0129] 5. Determine the parking space orientation residual based on the orientation residual corresponding to each parking space.

[0130] For example, Figure 7 This is a schematic diagram illustrating the principle of parking space orientation residuals provided in an exemplary embodiment of this disclosure. The solid rectangle represents a parking space determined based on a first state variable; p1 represents the midpoint of the first entrance line; p2 represents the midpoint of the first far end line; l1 represents the first straight line; the dashed box represents the parking space corresponding to the second parking space corner detection result; the black dots at the four corners of the dashed box represent the observation positions of the four corner points included in the second parking space corner detection result; p3 represents the midpoint of the first observation far end line; l2 represents the second straight line; p4 ​​represents the first intersection point; and the distance between p4 and p3 is used to determine the orientation residual of the parking space. For example, the orientation residual of the k-th corner detection result of the s-th parking space is represented as e. sk The parking space orientation residual is expressed as:

[0131]

[0132] This embodiment constrains the parking space orientation angle by using the distance between the midpoint of the far end line of the parking space determined by the first state variable and the midpoint of the far end line of the second parking space corner detection result. This can avoid the adverse effects of the longitudinal error of the far end point on the parking space position, thereby further improving the accuracy of the parking space position.

[0133] In an optional embodiment, step 20311, which determines the parking space length residual based on the first state variable corresponding to each parking space and the corner point detection results of each second parking space corresponding to each parking space, includes:

[0134] 1. For any parking space, based on the first state quantity corresponding to the parking space, determine the midpoint of the second far end line corresponding to the first state quantity and the first vector from the midpoint of the second entrance line corresponding to the first state quantity to the midpoint of the second far end line.

[0135] The determination method for the midpoint of the second distal line is the same as that for the midpoint of the first distal line, and the determination method for the midpoint of the second inlet line is the same as that for the midpoint of the first inlet line, which will not be repeated here. The first vector represents the vector pointing from the midpoint of the second inlet line to the midpoint of the second source line, and the first vector can be a unit vector.

[0136] 2. Based on the detection results of the corner points of each second parking space corresponding to the parking space, determine the midpoint of the second observation far end line corresponding to the detection results of the corner points of each second parking space.

[0137] The method for determining the midpoint of the second observation distal line is the same as that for the midpoint of the first observation distal line, and will not be repeated here.

[0138] 3. Based on the midpoint of the second far end line and the midpoints of each second observation far end line, determine the second vector corresponding to the detection results of each second parking space corner point, pointing from the midpoint of the second far end line to the midpoint of the second observation far end line.

[0139] 4. Based on the first vector and each of the second vectors, determine the projection length of each of the second vectors onto the first vector.

[0140] The second vector represents the relationship between the midpoint of the second far end line and the midpoint of the second observation far end line. If the parking space determined by the first state variable is consistent with the parking space determined by the second parking space corner detection result, the projection length of the second vector on the first vector should be 0. Therefore, by using the projection length of the second vector on the first vector as a residual term of the preset function, the length of the constrained parking space is minimized through the projection length, ensuring the stability of the parking space length and further improving the accuracy of the parking space position.

[0141] 5. Based on the length of each projection, determine the length residual of the parking space.

[0142] 6. Determine the parking space length residual based on the length residual corresponding to each parking space.

[0143] For example, Figure 8 This is a schematic diagram illustrating the principle of parking space length residuals provided in an exemplary embodiment of this disclosure. The solid rectangle represents a parking space determined based on a first state variable. p1 represents the midpoint of the second entrance line, p2 represents the midpoint of the second far end line, the dashed box represents the parking space corresponding to the second parking space corner detection result, the black dots at the four corners of the dashed box represent the observation positions of the four corner points included in the second parking space corner detection result, p3 represents the midpoint of the second observation far end line, the solid arrow represents the first vector, the dashed arrow from p2 to p3 represents the second vector, the second vector is projected onto the first vector to determine the projection length, the length residual of each parking space is determined based on the projection length corresponding to the corner detection result of each second parking space, and then the overall parking space length residual is determined based on the length residuals corresponding to each parking space. For example, the length residual of the nth corner detection result of the mth parking space is represented as e. mn The parking space length residual is expressed as:

[0144]

[0145] In an alternative embodiment, the first vector may also be a side line vector parallel to the centerline of the parking space determined by the first state variable, for example... Figure 8 The vector from corner point 3 to corner point 2 of the solid line frame, or the vector from corner point 0 to corner point 1, can be set according to actual needs.

[0146] In an optional embodiment, the parking space length residual can also be obtained directly using the difference between the observed length and the length determined by the first state quantity, which can be set according to actual needs.

[0147] This embodiment uses the vector projection length between the far midpoint of the state variable and the far midpoint of the observation variable as a residual term of a preset objective function to constrain the length of the parking space, ensure the stability of the parking space length, and further improve the accuracy of the parking space position.

[0148] In an optional embodiment, step 20311, which determines the side projection residual of the parking space based on the first state variable corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space, includes:

[0149] 1. For any parking space, based on the first state variable corresponding to the parking space, determine the first side vector and the second side vector corresponding to the parking space.

[0150] The first and second side vectors refer to the two side vectors adjacent to the entrance line, which can be determined by the coordinates of the two corner points of the side. For example, if the aforementioned corner points 0 and 3 are the entrance corner points, then the side between corner point 0 and corner point 1 is one side, and the side between corner point 3 and corner point 2 is the other side.

[0151] 2. Based on the detection results of the corner points of each second parking space corresponding to the parking space, determine the first observation side vector and the second observation side vector corresponding to the parking space.

[0152] The first observation side vector and the second observation side vector are similar to the first side vector and the second side vector, respectively, and are determined by the observation positions of the two corner points on the side in the second parking space corner point detection results. The details will not be elaborated further.

[0153] 3. Based on each first observation side vector, each second observation side vector, the first side vector, and the second side vector, determine the side projection residual corresponding to the parking space.

[0154] Specifically, the first side vector is rotated 90 degrees clockwise to determine the third vector. For any first observed side vector, the first observed side vector is projected onto the third vector to obtain the first projection length. The same process is applied to the other side to obtain the second projection length. The first projection length and the second projection length are used as the side projection residuals corresponding to the detection results of the second parking space corner point. Based on the side projection residuals corresponding to the detection results of each second parking space corner point, the side projection residuals corresponding to the parking space are determined.

[0155] In an alternative embodiment, the third vector can be determined by rotating the heading angle θ1 of the first state variable by 90 degrees (θ1+π / 2), and the first observation side vector can be projected onto the third vector. The specific settings can be made according to actual needs.

[0156] 4. Determine the side projection residuals of each parking space based on the side projection residuals corresponding to each parking space.

[0157] For example, Figure 9 This is a schematic diagram illustrating the principle of parking space side projection residuals provided in an exemplary embodiment of this disclosure. The solid rectangle represents the parking space determined based on the first state variable, the dashed box represents the parking space corresponding to the second parking space corner detection result, the black dots at the four corners of the dashed box represent the observation positions of the four corner points included in the second parking space corner detection result, p1p2 represents the first observed side vector, p1'p2' represents the first side vector, and the dashed arrow represents the third vector. The projection length of the first observed side vector onto the third vector characterizes the degree of consistency between the direction of the first observed side and the direction of the first side. If they are consistent, the projection length should be 0. Based on this, the parking space side projection residual can be used to constrain the orientation of the parking space, replacing the aforementioned parking space orientation residual.

[0158] For example, the side projection residual of the detection result of the corner point of the d-th second parking space of the h-th parking space is represented as e. hd The side projection residual of the parking space is expressed as:

[0159]

[0160] In an optional embodiment, the preset objective function can be expressed as:

[0161] L = L(1) + L(2) + L(3)

[0162] In an optional embodiment, the preset objective function can be expressed as:

[0163] L = L(1) + L(4) + L(3)

[0164] In an optional embodiment, the preset objective function can be expressed as:

[0165] L = L(1) + L(2) + L(3) + L(4)

[0166] The goal of optimization is to minimize the objective function.

[0167] In one optional embodiment, two side weights can be configured for the side projection residual of the parking space. For example, if a side is occluded, the contribution of that side to the parking space orientation can be reduced, further improving the accuracy of parking space reconstruction. Furthermore, even if the parking space position shifts due to odometer inaccuracy, the vector dot product is unaffected by position, ensuring that the parking space heading angle constraint remains effective under all circumstances.

[0168] Figure 10 This is a schematic flowchart of a parking space reconstruction method provided in another exemplary embodiment of this disclosure.

[0169] In an optional embodiment, step 2031, based on the detection results of the corner points of each second parking space corresponding to each parking space and a preset objective function, uses a preset optimization algorithm to optimize each first state variable to obtain the optimized second state variables corresponding to each parking space in the current frame set, including:

[0170] Step 2031a: Based on the detection results of the corner points of each second parking space corresponding to each parking space, the preset objective function, and the preset adjacent parking space constraints, the Gauss-Newton iteration method is used to optimize each first state variable to obtain the optimized second state variables corresponding to each parking space in the current frame set.

[0171] The preset constraints for adjacent parking spaces can be set according to actual needs, such as adjacent parking spaces having the same heading angle (heading angle constraint), the same interior angle (interior angle constraint), or collinear entrance lines (collinear entrance line constraint), etc. The Gauss-Newton iteration method solves for the current iteration step size using the Jacobian matrix, the iteration step size, and the normal equation satisfied by the residual (objective function).

[0172] For example, the current Jacobian matrix is ​​denoted as J(x), the current iteration step size is denoted as Δx, the current residual is denoted as f(x), and x represents a state variable (such as the first state variable of this disclosure), satisfying the following equation:

[0173] J(x) T J(x)Δx=-J(x) T f(x)

[0174] Here, J(x) and f(x) are determined quantities. By solving this system of equations, the current iteration step size Δx can be obtained. Then, the first state variable can be updated based on the current iteration step size to obtain the optimized state variable. The iteration can continue to be updated based on the optimized state variable according to the above process until the iteration termination condition is met to obtain the optimized second state variable. The specific principle of the Gauss-Newton iteration method will not be elaborated here.

[0175] This embodiment constrains the characteristics that adjacent parking spaces satisfy by pre-setting adjacent parking space constraints, so that the parking spaces determined by the second state variables obtained by optimization are more in line with the actual parking space situation, effectively improving the parking space reconstruction effect. In addition, the Gauss-Newton iteration method can avoid calculating the second partial derivative, effectively reducing the computational complexity.

[0176] In an optional embodiment, the preset adjacent parking space constraints include constraints on equal orientation angles of adjacent parking spaces, equal interior angles of adjacent parking spaces, and collinearity of entrance lines of adjacent parking spaces; step 2031a, based on the detection results of corner points of each second parking space corresponding to each parking space, the preset objective function, and the preset adjacent parking space constraints, uses the Gauss-Newton iteration method to optimize each first state variable, obtaining the optimized second state variables corresponding to each parking space in the current frame set, including:

[0177] Based on the first state variables corresponding to each parking space and the preset clustering rules, each parking space is clustered to obtain at least one set of parking spaces corresponding to each category. In the process of optimizing each first state variable using the Gauss-Newton iteration method based on the detection results of each second parking space corner and the preset objective function, each set of parking spaces is constrained based on the preset adjacent parking space constraint conditions so that the parking spaces belonging to the same set of parking spaces in the obtained second state variables satisfy the preset adjacent parking space constraint conditions.

[0178] Among them, the preset clustering rules can be set according to actual needs. For example, parking space clustering can be achieved based on the following conditions:

[0179] 1. Parking space shape requirements: Parking spaces of the same type should have the same shape, such as all being rectangular parking spaces or all being parallelogram parking spaces.

[0180] 2. Unit vector from the parking space entrance to the far end: The dot product of the unit vectors of any two parking spaces of the same type should be greater than 0 to avoid grouping parking spaces that are connected end to end into one type.

[0181] 3. Midpoint of the entrance line: The midpoint of the entrance lines of any two parking spaces of the same type should be on the same straight line.

[0182] 4. Observation frame count condition: For example, when the number of observation frames for a parking space is small (such as the first observation frame), the parking space is not included in the parking space cluster because there are too few observations and the parking space is not yet stable.

[0183] 5. Parking space orientation: Among the same type of parking spaces, the difference in the heading angle (Yaw, also known as orientation angle, denoted as θ) between any two parking spaces should be less than the heading angle threshold, that is, the orientation of any two parking spaces should be the same or similar.

[0184] 6. Interior angle of parking space: In the same type of parking space, the difference between the interior angles (denoted as α) of any two parking spaces should be less than the interior angle threshold, that is, the interior angles of any two parking spaces are the same or similar.

[0185] 7. The convex hull formed by the four corner points of the parking space: In the same type of parking space, there should be an overlap after the size expansion of each pair of parking spaces, that is, after the length and width of adjacent parking spaces are increased by a certain amount, there will be a certain overlap.

[0186] 8. For any two parking spaces in step 7 that have no overlap after size expansion, these two parking spaces will not be clustered into one class in the future. This is to avoid the situation where the conditions of the two parking spaces are just around the threshold, causing the clustering to jump between success and failure.

[0187] Based on the above conditions, each parking space is clustered to obtain at least one set of parking spaces corresponding to each category. Each set of parking spaces corresponding to each category is taken as a group of adjacent parking spaces. During the optimization process, each group of adjacent parking spaces is constrained based on the preset adjacent parking space constraint conditions so that the parking spaces corresponding to each of the obtained second state variables, which belong to the same set of parking spaces, satisfy the preset adjacent parking space constraint conditions.

[0188] This embodiment clusters parking spaces, grouping adjacent parking spaces into one class, which facilitates subsequent constraints on adjacent parking spaces and helps to optimize the overall parking space structure better, thus improving the parking space reconstruction effect. Furthermore, the more parking spaces involved in the joint optimization, the better the effects of image distortion, IPM stitching errors, and uneven ground can be eliminated.

[0189] In an optional embodiment, after applying adjacent parking space constraints, to avoid situations where adjacent parking spaces in a class of parking spaces do not actually meet the adjacent parking space constraints (e.g., the entrance lines of adjacent parking spaces are not actually collinear, or the heading angles of adjacent parking spaces are not the same), but the adjacent parking space constraints are imposed, causing the results to be inconsistent with reality, the adjacent parking space constraints can be removed for parking spaces with large deviation results after optimization, and single parking space optimization can be performed instead to ensure the accuracy of parking space reconstruction.

[0190] In one optional embodiment, the adjacent parking space constraint can be imposed after each state variable update, or the constraint can be added during the optimization solution process. The specific setting can be determined according to actual needs.

[0191] In an optional example, for heading angle constraints and interior angle constraints, the state variables of adjacent parking spaces can be implemented by sharing the same parameters during the optimization process. For example, the heading angle state variables of adjacent parking spaces can use the same heading angle.

[0192] In an optional example, for the collinearity constraint of the entrance lines, after each state variable update, for a group of adjacent parking spaces, an average entrance position can be determined based on the updated state variable. Each parking space in the group of adjacent parking spaces is then translated according to the entrance line to the average entrance position, so that the entrance lines of each parking space in the group of adjacent parking spaces are on the same straight line.

[0193] In an optional example, the collinearity constraint for entrance lines can be incorporated during the optimizer's solution process. This constraint can be implemented by establishing a mathematical model for entrance line alignment. For instance, the midpoint of the entrance to the first parking space in a set of parking spaces of the same type can be used as an anchor point. The entrance lines of other parking spaces can be moved to the line containing the anchor point. This operation is incorporated into the construction of the optimization problem, and the derivative of this process is calculated. Specifically, when determining the residuals for each term, the corresponding residuals are determined based on the moved parking space state variables, and the current Jacobian matrix is ​​determined, thereby determining the current iteration step size.

[0194] For example, Figure 11 This is a schematic diagram of entrance line alignment provided in an exemplary embodiment of this disclosure. The alignment line is a straight line perpendicular to the centerline of the first parking space, where the center point of the first parking space is located. Black dots represent the center points of each parking space before alignment, and gray dots represent the center points of each parking space after alignment. The entrance center point of the first parking space in this type of parking space set is used as the anchor point. Since the parking space position state quantity uses the center point coordinates, the line containing the anchor point is moved by half the length of the parking space to be moved, and the entrance lines of other parking spaces are moved to the line containing the anchor point. It is necessary to determine the new center point coordinates (gray dots) after the parking spaces are aligned. The anchor point is denoted as m, and the center point of the parking space to be moved is denoted as p. It is necessary to determine the derivative of the anchor point with respect to the anchored parking space state quantity, denoted as... Since the state variables of the anchored parking space have been differentiated, the anchor point is not a fixed point. The equation of the anchoring line is represented by l3, as follows:

[0195] l1: y = kx - k*mx + my + Δb

[0196] Where x and y represent the variables of the anchoring line equation, Δb is the change in intercept after the anchoring line is moved, mx represents the abscissa of the anchor point, my represents the ordinate of the anchor point, and k represents the slope of the anchoring line.

[0197] The equation of the centerline of the parking space to be moved is expressed as:

[0198] l4: y = tan(θ)(x - px) + py

[0199] Where px and py represent the x-coordinate and y-coordinate of the center point of the parking space to be moved, respectively.

[0200] The intersection of l3 and l4 is the new center point of the parking space (gray point).

[0201] Let the new parking space center point be represented as p_new = f(m, k, p, θ, Δb). The derivative of the new parking space center point with respect to the anchored parking space parameters is expressed as follows:

[0202]

[0203] f'(k)*dk / dθ

[0204] The derivative of the new parking space center with respect to its own parking space parameters is determined as follows:

[0205] f'(p)

[0206] f'(θ)

[0207]

[0208] Based on the derivatives mentioned above, the current Jacobian matrix is ​​determined and used to solve for the current iteration step size, thereby achieving the collinearity constraint of the entry lines during the solution process.

[0209] In an optional embodiment, step 202, determining the first state quantity corresponding to each parking space in the current frame set, includes:

[0210] Step 2021: For any new parking space, determine the initial state quantity of the new parking space based on the detection result of the first parking space corner point corresponding to the new parking space, and use the initial state quantity as the first state quantity of the new parking space.

[0211] In an optional embodiment, the heading angle, interior angle, center point x-coordinate, center point y-coordinate, width, and length can be determined based on the observed positions of each corner point included in the first parking space corner point detection results, forming initialization state variables.

[0212] Step 2022: For any inspected parking space, obtain the optimized state quantity corresponding to the inspected parking space in the previous frame set, and use the optimized state quantity corresponding to the inspected parking space in the previous frame set as the first state quantity of the inspected parking space.

[0213] For an inspected parking space, which has already been optimized in the previous frame set, based on the invariance of the actual position of the parking space, the optimized state variable corresponding to the inspected parking space in the previous frame set can be used as the first state variable of the inspected parking space in the current frame set.

[0214] This embodiment determines the initial state variables of a newly detected parking space based on the detection results of the first parking space corner point, thereby realizing observation-based initialization of state variables. This makes the initial state variables closer to the actual parking space situation and improves the convergence speed.

[0215] In an optional embodiment, the second state quantity includes the coordinates of the center point of the corresponding parking space in the first coordinate system, the length of the parking space, the width of the parking space, the orientation angle of the parking space, and the interior angle of the parking space; step 204, based on the second state quantity corresponding to each parking space, determines the target position in the first coordinate system corresponding to each parking space, including:

[0216] Step 2041: Based on the coordinates of the center point of each parking space, the length of the parking space, the width of the parking space, the orientation angle of the parking space, the interior angle of the parking space, and the mapping relationship between the state variables and the corner point coordinates, determine the corner point coordinates of each parking space.

[0217] Step 2042: Use the coordinates of the corner points of each parking space as the target positions for each parking space.

[0218] The mapping relationship between state variables and corner coordinates is described above and will not be repeated here.

[0219] This embodiment maps the second state variable obtained through optimization to the coordinates of the parking space corner point, and uses the coordinates of the parking space corner point as the target location of the parking space, which facilitates subsequent applications.

[0220] The embodiments or optional examples disclosed above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit it.

[0221] Any of the parking space reconstruction methods provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the parking space reconstruction methods provided in this disclosure can be executed by a processor, such as by a processor executing any of the parking space reconstruction methods mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0222] Exemplary device

[0223] Figure 12 This is a schematic diagram of a parking space reconstruction device provided in an exemplary embodiment of this disclosure. The device in this embodiment can be used to implement corresponding method embodiments of this disclosure, such as… Figure 12 The device shown includes: a first processing module 501, a first determining module 502, a second processing module 503, and a third processing module 504.

[0224] The first processing module 501 is used to determine, based on the first parking space corner detection results corresponding to each frame in the current frame set, a first number of second parking space corner detection results whose observation confidence meets a preset condition for each detected parking space. The current frame set includes the current frame and a second number of historical frames. The first parking space corner detection result includes the parking space corner detection results corresponding to at least one parking space detected in its corresponding frame. Both the first parking space corner detection result and the second parking space corner detection result include the observation position of at least one corner point in the first coordinate system of its corresponding parking space.

[0225] The first determining module 502 is used to determine the first state quantity corresponding to each parking space in the current frame set. For a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space. For a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the previously detected parking space is the optimized state quantity corresponding to the previously detected parking space in the previous frame set.

[0226] The second processing module 503 is used to optimize each first state variable based on the detection results of the corner points of each second parking space corresponding to each parking space in the current frame set and the preset optimization rules, so as to obtain the optimized second state variables corresponding to each parking space in the current frame set.

[0227] The third processing module 504 is used to determine the target position in the first coordinate system corresponding to each parking space based on the second state quantity corresponding to each parking space.

[0228] Figure 13 This is a schematic diagram of the structure of a parking space reconstruction device provided in another exemplary embodiment of this disclosure.

[0229] In an optional embodiment, the first processing module 501 includes a first determining unit 5011 and / or a second determining unit 5012. The first processing module 501 further includes a third determining unit 5013.

[0230] The first determining unit 5011 is used to determine the first observation confidence level of the first parking space corner detection results corresponding to each parking space in each frame based on the detection results of each first parking space corner and the pre-obtained error distribution description function.

[0231] The second determining unit 5012 is used to determine the second observation confidence of the first parking space corner detection results corresponding to each parking space in each frame based on the detection results of each first parking space corner and the semantic segmentation results of the first parking space corresponding to each frame.

[0232] The third determining unit 5013 is used to determine, for any parking space, a first number of second parking space corner detection results whose observation confidence satisfies a preset condition, based on the first observation confidence and / or the second observation confidence of the corner detection results of each first parking space corresponding to that parking space.

[0233] In an optional embodiment, the second processing module 503 includes: a first processing unit 5031, used to optimize each first state variable based on the detection results of each second parking space corner corresponding to each parking space and a preset objective function, using a preset optimization algorithm to obtain the optimized second state variables corresponding to each parking space in the current frame set. The preset objective function includes at least one of the following residual functions: parking space entrance corner residual function, parking space orientation residual function, parking space side projection residual function, and parking space length residual function.

[0234] In an optional embodiment, the first processing unit 5031 is specifically used for:

[0235] Based on the first state variables corresponding to each parking space and the detection results of the corner points of each second parking space, the following are determined: parking space entrance corner residual, parking space orientation residual, parking space side projection residual, parking space length residual, and current Jacobian matrix. The current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, parking space orientation residual, and parking space length residual with respect to the first state variables, or the current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, parking space side projection residual, and parking space length residual with respect to the first state variables. Based on the parking space entrance corner residual, parking space orientation residual, parking space length residual, and current Jacobian matrix, the current iteration step size is determined; or, based on the parking space entrance corner residual, parking space side projection residual, parking space length residual, and current Jacobian matrix, the current iteration step size is determined. Based on the current iteration step size, each first state variable is updated to obtain the second state variables corresponding to each parking space.

[0236] In an optional embodiment, the first processing unit 5031 is specifically used for:

[0237] For any parking space, based on the first state variable corresponding to the parking space, determine the coordinates of the first entrance corner point corresponding to each entrance corner point of the parking space; based on the detection results of the second parking space corner points corresponding to the parking space, determine the observation position of the entrance corner point corresponding to the parking space in each frame; based on the coordinates of the first entrance corner points corresponding to each parking space and the observation position of the entrance corner point corresponding to each parking space in each frame, determine the parking space entrance corner point residual.

[0238] In an optional embodiment, the first processing unit 5031 is specifically used for:

[0239] For any parking space, based on the first state variable corresponding to the parking space, determine the first straight line between the midpoint of the first entrance line and the midpoint of the first far end line corresponding to the first state variable; based on the detection results of each corner point of the second parking space corresponding to the parking space, determine the second straight line where the midpoint of the first observation far end line and the first observation far end line are located respectively, corresponding to the detection results of each corner point of the second parking space; based on the first straight line and each second straight line, determine the first intersection point corresponding to the detection results of each corner point of the second parking space; based on each first intersection point and the midpoint of each first observation far end line, determine the orientation residual corresponding to the parking space; based on the orientation residual corresponding to each parking space, determine the orientation residual of the parking space.

[0240] In an optional embodiment, the first processing unit 5031 is specifically used for:

[0241] For any parking space, based on the first state variable corresponding to the parking space, determine the midpoint of the second distal line corresponding to the first state variable and the first vector from the midpoint of the second entrance line corresponding to the first state variable to the midpoint of the second distal line; based on the detection results of each corner point of the second parking space corresponding to the parking space, determine the midpoint of the second observation distal line corresponding to each corner point detection result of the second parking space; based on the midpoint of the second distal line and each midpoint of the second observation distal line, determine the second vector from the midpoint of the second distal line to the midpoint of the second observation distal line corresponding to each corner point detection result of the second parking space; based on the first vector and each second vector, determine the projection length of each second vector on the first vector; based on each projection length, determine the length residual of the parking space; based on the length residual corresponding to each parking space, determine the length residual of the parking space.

[0242] In an optional embodiment, the first processing unit 5031 is specifically used for:

[0243] For any parking space, based on the first state variable corresponding to the parking space, determine the first side vector and the second side vector corresponding to the parking space; based on the detection results of each corner point of each second parking space corresponding to the parking space, determine each first observation side vector and the second observation side vector corresponding to the parking space; based on each first observation side vector, each second observation side vector, the first side vector and the second side vector, determine the side projection residual corresponding to the parking space; based on the side projection residual corresponding to each parking space, determine the side projection residual of the parking space.

[0244] In an optional embodiment, the first processing unit 5031 is specifically used to: optimize each first state variable by using the Gauss-Newton iteration method based on the detection results of each second parking space corner corresponding to each parking space, the preset objective function, and the preset adjacent parking space constraints, so as to obtain the optimized second state variables corresponding to each parking space in the current frame set.

[0245] In an optional embodiment, the preset adjacent parking space constraints include constraints on equal orientation angles of adjacent parking spaces, equal interior angles of adjacent parking spaces, and collinearity of entrance lines of adjacent parking spaces; the first processing unit 5031 is specifically used for:

[0246] Based on the first state variables corresponding to each parking space and the preset clustering rules, each parking space is clustered to obtain at least one set of parking spaces corresponding to each category. In the process of optimizing each first state variable using the Gauss-Newton iteration method based on the detection results of each second parking space corner and the preset objective function, each set of parking spaces is constrained based on the preset adjacent parking space constraint conditions so that the parking spaces belonging to the same set of parking spaces in the obtained second state variables satisfy the preset adjacent parking space constraint conditions.

[0247] In an optional embodiment, the first determining module 502 includes a fourth determining unit 5021 and a fifth determining unit 5022.

[0248] The fourth determining unit 5021 is used to determine the initial state quantity of any new parking space based on the detection result of the first parking space corner point corresponding to the new parking space, and use the initial state quantity as the first state quantity of the new parking space.

[0249] The fifth determining unit 5022 is used to obtain the optimized state quantity corresponding to the inspected parking space in the previous frame set for any inspected parking space, and use the optimized state quantity corresponding to the inspected parking space in the previous frame set as the first state quantity of the inspected parking space.

[0250] In an optional embodiment, the second state quantity includes the coordinates of the center point of the parking space, the length of the parking space, the width of the parking space, the orientation angle of the parking space, and the interior angle of the parking space in the first coordinate system; the third processing module 504 includes: a second processing unit 5041 and a third processing unit 5042.

[0251] The second processing unit 5041 is used to determine the corner coordinates of each parking space based on the center point coordinates, length, width, orientation angle, interior angle, and mapping relationship between state variables and corner coordinates of each parking space. The third processing unit 5042 is used to use the corner coordinates of each parking space as the target position of each parking space.

[0252] The modules or units in the above embodiments of this disclosure can be further divided into finer-grained units according to actual needs. The specific division can be set according to actual needs, and this disclosure does not limit it.

[0253] Exemplary electronic devices

[0254] This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, wherein when the computer program is executed, it implements the parking space reconstruction method described in any of the above embodiments of this disclosure.

[0255] Figure 14 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0256] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0257] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0258] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0259] For example, the input device 13 may be the microphone or microphone array described above, used to capture the input signal of the sound source.

[0260] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0261] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0262] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0263] Exemplary computer program products and computer-readable storage media

[0264] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0265] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0266] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0267] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0268] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0269] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0270] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0271] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0272] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0273] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0274] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A parking space reconstruction method, comprising: Based on the first parking space corner detection results corresponding to each frame in the current frame set, a first number of second parking space corner detection results are determined for each detected parking space whose observation confidence meets the preset conditions. The current frame set includes the current frame and a second number of historical frames. The first parking space corner detection result includes the parking space corner detection results corresponding to at least one parking space detected in its corresponding frame. Both the first parking space corner detection result and the second parking space corner detection result include the observation position of at least one corner point in the first coordinate system of its corresponding parking space. Determine the first state quantity corresponding to each parking space in the current frame set. For a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space. For a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the previously detected parking space is the optimized state quantity corresponding to the previously detected parking space in the previous frame set. Based on the detection results of the corner points of the second parking spaces corresponding to each parking space in the current frame set and the preset optimization rules, each of the first state variables is optimized to obtain the optimized second state variables corresponding to each parking space in the current frame set. Based on the second state quantity corresponding to each parking space, the target position in the first coordinate system corresponding to each parking space is determined.

2. The method according to claim 1, wherein, The step of determining a first number of second parking space corner detection results whose observation confidence levels satisfy preset conditions for each detected parking space, based on the first parking space corner detection results corresponding to each frame in the current frame set, includes: Based on the detection results of each first parking space corner point and the pre-obtained error distribution description function, determine the first observation confidence level of the detection results of the first parking space corner points corresponding to each parking space in each frame; and / or, Based on the detection results of each first parking space corner point and the semantic segmentation results of the first parking space corresponding to each frame, the second observation confidence of the detection results of the first parking space corner point corresponding to each parking space in each frame is determined. For any of the parking spaces, based on the first observation confidence and / or the second observation confidence of each of the first parking space corner detection results corresponding to that parking space, a first number of second parking space corner detection results whose observation confidence satisfies a preset condition are determined.

3. The method according to claim 1, wherein, The step of optimizing each first state variable based on the corner detection results of each second parking space corresponding to each parking space in the current frame set and the preset optimization rules to obtain the optimized second state variables corresponding to each parking space in the current frame set includes: Based on the detection results of the corner points of each of the second parking spaces corresponding to each of the parking spaces, and the preset objective function, a preset optimization algorithm is used to optimize each of the first state variables to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set. The preset objective function includes at least one of the following residual functions: parking space entrance corner point residual function, parking space orientation residual function, parking space side projection residual function, and parking space length residual function.

4. The method according to claim 3, wherein, Based on the detection results of the corner points of each of the second parking spaces corresponding to each of the parking spaces, and a preset objective function, a preset optimization algorithm is used to optimize each of the first state variables to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set, including: Based on the first state variables corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space, the parking space entrance corner residual, parking space orientation residual, parking space side projection residual, parking space length residual, and current Jacobian matrix are determined. The current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, the parking space orientation residual, and the parking space length residual with respect to the first state variables, or the current Jacobian matrix includes the Jacobian ratios of the parking space entrance corner residual, the parking space side projection residual, and the parking space length residual with respect to the first state variables. Based on the parking space entrance corner residual, the parking space orientation residual, the parking space length residual, and the current Jacobian matrix, determine the current iteration step size; or, The current iteration step size is determined based on the parking space entrance corner residual, the parking space side projection residual, the parking space length residual, and the current Jacobian matrix. Based on the current iteration step size, each of the first state variables is updated to obtain the second state variable corresponding to each parking space.

5. The method according to claim 4, wherein, The step of determining the parking space entrance corner residual based on the first state quantity corresponding to each of the parking spaces and the corner detection results of each of the second parking spaces corresponding to each of the parking spaces includes: For any of the parking spaces, based on the first state quantity corresponding to the parking space, determine the coordinates of the first entrance corner point corresponding to each entrance corner point of the parking space; Based on the detection results of the corner points of each second parking space corresponding to the parking space, the observation position of the entrance corner point of the parking space in each frame is determined; Based on the coordinates of each first entrance corner point corresponding to each parking space and the observation position of each parking space corresponding to the entrance corner point in each frame, the parking space entrance corner point residual is determined.

6. The method according to claim 4, wherein, Based on the first state variable corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space, the parking space orientation residual is determined, including: For any of the parking spaces, based on the first state quantity corresponding to the parking space, a first straight line is determined between the midpoint of the first entrance line and the midpoint of the first far end line corresponding to the first state quantity. Based on the detection results of each corner point of the second parking space corresponding to the parking space, the midpoint of the first observation far end line and the second straight line where the first observation far end line is located are determined respectively according to the detection results of each corner point of the second parking space. Based on the first straight line and each of the second straight lines, determine the first intersection point corresponding to the detection results of each of the second parking space corner points; Based on each of the first intersection points and the midpoint of each of the first observation far-end lines, determine the orientation residual corresponding to the parking space; The orientation residual of each parking space is determined based on the orientation residual corresponding to each parking space.

7. The method according to claim 4, wherein, Based on the first state variable corresponding to each parking space and the corner detection results of each second parking space corresponding to each parking space, the parking space length residual is determined, including: For any of the parking spaces, based on the first state quantity corresponding to the parking space, determine the midpoint of the second far end line corresponding to the first state quantity and the first vector from the midpoint of the second entrance line corresponding to the first state quantity to the midpoint of the second far end line. Based on the detection results of each corner point of the second parking space corresponding to the parking space, the midpoint of the second observation far end line corresponding to each corner point detection result of the second parking space is determined; Based on the midpoint of the second far end line and the midpoint of each of the second observation far end lines, determine the second vector corresponding to the detection result of each of the second parking space corner points, pointing from the midpoint of the second far end line to the midpoint of the second observation far end line; Based on the first vector and each of the second vectors, determine the projection length of each of the second vectors onto the first vector; Based on the projected lengths, determine the length residual of the parking space; The length residual of each parking space is determined based on the length residual corresponding to each parking space.

8. The method according to claim 4, wherein, Based on the first state variable corresponding to each parking space and the detection results of the corner points of each second parking space corresponding to each parking space, the side projection residual of the parking space is determined, including: For any of the parking spaces, based on the first state quantity corresponding to the parking space, determine the first side vector and the second side vector corresponding to the parking space; Based on the detection results of each corner point of the second parking space corresponding to the parking space, determine each first observation side vector and second observation side vector corresponding to the parking space; Based on each of the first observed side vectors, each of the second observed side vectors, the first side vector, and the second side vector, determine the side projection residual corresponding to the parking space; The side projection residuals of each parking space are determined based on the side projection residuals corresponding to each parking space.

9. The method according to claim 3, wherein, Based on the detection results of the corner points of each of the second parking spaces corresponding to each of the parking spaces, and a preset objective function, a preset optimization algorithm is used to optimize each of the first state variables to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set, including: Based on the detection results of the corner points of each of the second parking spaces corresponding to each of the parking spaces, the preset objective function, and the preset adjacent parking space constraints, the Gauss-Newton iteration method is used to optimize each of the first state variables to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set.

10. The method according to claim 9, wherein, The preset adjacent parking space constraints include the constraint that adjacent parking spaces have equal orientation angles, the constraint that adjacent parking spaces have equal interior angles, and the constraint that adjacent parking spaces have collinear entrance lines. Based on the detection results of the corner points of each of the second parking spaces corresponding to each of the parking spaces, the preset objective function, and the preset adjacent parking space constraints, the Gauss-Newton iteration method is used to optimize each of the first state variables to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set, including: Based on the first state quantity corresponding to each parking space and the preset clustering rules, each parking space is clustered to obtain a set of parking spaces corresponding to each category in at least one category. In the process of optimizing each first state variable using the Gauss-Newton iteration method based on the detection results of each second parking space corner corresponding to each parking space and the preset objective function, each parking space set is constrained based on the preset adjacent parking space constraint condition, so that among the parking spaces corresponding to each second state variable, each parking space belonging to the same parking space set satisfies the preset adjacent parking space constraint condition.

11. The method according to claim 1, wherein, Determining the first state quantity corresponding to each parking space in the current frame set includes: For any new parking space, the initial state quantity of the new parking space is determined based on the detection result of the first parking space corner point corresponding to the new parking space, and the initial state quantity is used as the first state quantity of the new parking space. For any of the inspected parking spaces, obtain the optimized state quantity corresponding to the inspected parking space in the previous frame set, and use the optimized state quantity corresponding to the inspected parking space in the previous frame set as the first state quantity of the inspected parking space.

12. The method according to any one of claims 1-11, wherein, The second state quantity includes the coordinates of the center point of the corresponding parking space in the first coordinate system, the length of the parking space, the width of the parking space, the orientation angle of the parking space, and the interior angle of the parking space; The step of determining the target position in the first coordinate system corresponding to each of the parking spaces based on the second state quantity corresponding to each of the parking spaces includes: Based on the coordinates of the center point of each parking space, the length of the parking space, the width of the parking space, the orientation angle of the parking space, the interior angle of the parking space, and the mapping relationship between the state variables and the corner point coordinates, the corner point coordinates of each parking space are determined. The coordinates of the corner points of each parking space are taken as the target positions of each parking space.

13. A parking space reconstruction device, comprising: The first processing module is used to determine, based on the first parking space corner detection results corresponding to each frame in the current frame set, a first number of second parking space corner detection results whose observation confidence satisfies a preset condition for each detected parking space. The current frame set includes the current frame and a second number of historical frames. The first parking space corner detection results include the parking space corner detection results corresponding to at least one parking space detected in its corresponding frame. Both the first parking space corner detection results and the second parking space corner detection results include the observation position of at least one corner point in the first coordinate system of its corresponding parking space. The first determining module is used to determine the first state quantity corresponding to each parking space in the current frame set. For a new parking space detected for the first time in the current frame set, the first state quantity of the new parking space is the initial state quantity of the new parking space. For a previously detected parking space that has not been detected for the first time in the current frame set, the first state quantity of the previously detected parking space is the optimized state quantity corresponding to the previously detected parking space in the previous frame set. The second processing module is used to optimize each of the first state variables based on the detection results of the corner points of each of the second parking spaces corresponding to each parking space in the current frame set and the preset optimization rules, so as to obtain the optimized second state variables corresponding to each of the parking spaces in the current frame set. The third processing module is used to determine the target position in the first coordinate system corresponding to each parking space based on the second state quantity corresponding to each parking space.

14. A computer-readable storage medium storing a computer program for performing the parking space reconstruction method according to any one of claims 1-12.

15. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the parking space reconstruction method according to any one of claims 1-12.

Images