A method and device for real-time correction of parking segmentation based on visual detection

Abstract

This invention provides a method and apparatus for real-time segmented correction of parking based on visual detection. The method includes: acquiring a parking area of ​​interest for the vehicle, wherein the parking area of ​​interest includes parking spaces; performing visual detection on the shape of the parking spaces; and, based on the results of the visual detection, dividing the automatic parking process into a trajectory segment before parking and a trajectory segment after parking; and sequentially correcting errors existing in the trajectory segments before and after parking in real time, thereby improving the correction accuracy of the automatic parking process and enabling the vehicle to accurately park in the parking space. This invention solves the technical problems of untimely error correction and persistent lateral and heading errors after parking, which exist in traditional methods that rely on a fixed trajectory segment to correct dynamic errors during the parking process, by dividing the automatic parking process into trajectory segments and adopting different real-time correction strategies for different trajectory segments, thus improving the parking accuracy of the vehicle.

Description

Technical Field

[0001] This invention relates to the field of automatic parking technology in intelligent driving, and in particular to a method and apparatus for real-time correction of parking segments based on vision detection. Background Technology

[0002] Automatic parking involves using onboard sensors to collect information about the vehicle's surroundings, transmitting the collected image and location data to a perception module for analysis to obtain the target parking point, a planning module to calculate the parking trajectory based on the target point analyzed by the perception module, and finally, a control module to control the vehicle to park in the parking space.

[0003] Although real-time correction based on vision detection can correct dynamic errors in positioning and control during parking, as well as accuracy errors in parking space search, the parking trajectory is not fixed (here, it is mainly divided into the trajectory before and after parking based on visual parking space detection). Most existing solutions correct the parking process based on the last fixed trajectory. Since the last trajectory is almost in the parking space, the calculated lateral error of the parking space cannot be corrected in time, and ultimately, lateral and heading errors still exist.

[0004] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0005] This invention provides a solution to the technical problems of untimely error correction and persistent lateral and heading errors after parking, which exist in existing technologies that correct dynamic errors during vehicle parking based on the last fixed trajectory.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a method for real-time segmented parking correction based on visual detection, comprising:

[0008] Obtain the parking areas that the vehicle is interested in, including parking spaces;

[0009] Visual detection is performed on the shape of the parking space. Based on the results of the visual detection, the automatic parking process of the vehicle is divided into the trajectory segment before entering the parking space and the trajectory segment after entering the parking space.

[0010] Errors in the trajectory segments before and after parking are corrected in real time to improve the accuracy of automatic parking and enable the vehicle to park precisely in the parking space.

[0011] Preferably, obtaining the parking area of ​​interest to the vehicle includes:

[0012] The vehicle is in a parked state as detected by the CAN bus.

[0013] Cameras positioned around the vehicle are used to visually inspect the area surrounding the vehicle; these cameras also have parking space search and error correction functions.

[0014] The surrounding area containing parking space information found in the search is identified as the parking area that the vehicle is interested in.

[0015] Preferably, determining the surrounding area including parking space information as the parking area of ​​interest to the vehicle includes:

[0016] Use the corresponding camera to locate the parking space information in the search state;

[0017] Based on the positioning information and fixed transformation formula, the parking space coordinates are transformed to the corresponding image coordinate system of the correction camera, resulting in the rectangular outline of the parking space.

[0018] Using the rectangular outline of the parking space as the reference center, a visual algorithm is used to crop the surrounding area image, which includes parking space information, found by the corresponding camera, to obtain the parking area of ​​interest to the vehicle.

[0019] Preferably, the step of visually detecting the parking space shape and dividing the automatic parking process into a trajectory segment before parking and a trajectory segment after parking based on the visual detection results includes:

[0020] Cameras placed around the vehicle capture the outline of the parking space in real time;

[0021] When the corresponding camera detects that the outline of the parking space is closed, it determines that the automatic parking process is in the trajectory segment before entering the parking space; when the corresponding camera detects that the outline of the parking space is open, it determines that the automatic parking process is in the trajectory segment after entering the parking space.

[0022] Preferably, the real-time correction of errors existing in the trajectory segment before storage includes:

[0023] Use the camera's search function to search for parking spaces in parking areas that interest the vehicle;

[0024] The coordinates of the searched parking spaces are transformed into a coordinate system based on the vehicle body;

[0025] Calculate the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates;

[0026] The lateral deviation and heading angle deviation values ​​are sent as compensation values ​​to the planning and control modules to realize real-time correction of errors existing in the trajectory segment before entering the database.

[0027] Preferably, the calculation of the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates includes:

[0028] Obtain the final parking point coordinates and vehicle body coordinates, and use the average of the final parking point coordinates and vehicle body coordinates as the visual observation value of the camera detecting the parking space;

[0029] The final parking point coordinates are converted into the target point in the vehicle coordinate system, and the lateral deviation and heading angle deviation between the visual observation value and the target point are calculated.

[0030] Preferably, the real-time correction of errors existing in the trajectory segments after data entry includes:

[0031] Use the camera's correction function to detect parking spaces in parking areas of interest to the vehicle;

[0032] The detected parking space coordinates are transformed into a coordinate system based on the vehicle body;

[0033] Calculate the longitudinal deviation between the final parking point coordinates and the vehicle body coordinates;

[0034] The longitudinal deviation value is sent as a compensation value to the planning module and the control module to realize real-time correction of errors in the trajectory segment after it is entered into the database.

[0035] Secondly, the present invention provides a device for real-time correction of parking segments based on visual detection, comprising:

[0036] The image acquisition module is used to collect parking space information and location data in the area surrounding the vehicle;

[0037] The information sensing module communicates with the image acquisition module and is used to analyze and acquire parking target points;

[0038] The planning module, which communicates with the perception module, is used to calculate the parking trajectory; and,

[0039] The control module, which communicates with the planning module, is used to control the vehicle to park in the parking space.

[0040] Thirdly, the present invention provides an electronic device, comprising:

[0041] At least one processor; and,

[0042] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the vision-based real-time parking segment correction method as described in the first aspect.

[0043] Fourthly, the present invention provides a non-transitory computer-readable storage medium storing a computer software program for implementing a vision-based real-time parking segment correction method as described in the first aspect.

[0044] In view of the shortcomings of the prior art, the beneficial effects that the present invention can achieve are as follows:

[0045] Based on the results of visual detection, this invention divides the automatic parking process of a vehicle into a trajectory segment before parking and a trajectory segment after parking. Different real-time correction strategies are adopted for different trajectory segments. This can solve the technical problems of untimely error correction and lateral and heading errors that still exist after parking, which are caused by the traditional method of correcting dynamic errors in the vehicle parking process based on the last fixed trajectory segment. This improves the accuracy of vehicle parking. Attached Figure Description

[0046] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0047] Figure 1 This is a flowchart illustrating a method for real-time segmented parking correction based on visual detection, provided in an embodiment of the present invention.

[0048] Figure 2 yes Figure 1 A flowchart illustrating one of the steps;

[0049] Figure 3 yes Figure 2 A flowchart illustrating one of the steps;

[0050] Figure 4 This is a schematic diagram of one of the states of a vehicle when it is parked;

[0051] Figure 5 yes Figure 1 A flowchart illustrating another step;

[0052] Figure 6 This is a schematic diagram showing the state of a vehicle in different trajectory segments during the automatic parking process;

[0053] Figure 7 yes Figure 1 A flowchart of another step;

[0054] Figure 8 yes Figure 7 A flowchart illustrating one of the steps;

[0055] Figure 9 This is a schematic diagram of the trajectory correction state of a vehicle during the automatic parking process before entering the parking space.

[0056] Figure 10 yes Figure 1 One of the steps is illustrated in the flowchart.

[0057] Figure 11 This is a schematic diagram of the trajectory correction state after the vehicle has entered the parking space during the automatic parking process.

[0058] Figure 12 This is a schematic diagram of a device for real-time correction of parking segments based on visual detection, provided in an embodiment of the present invention.

[0059] Figure 13 This is a schematic diagram of an electronic device structure provided by an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined with each other to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0061] To address the technical problems of untimely error correction and persistent lateral and heading errors after parking, which exist in existing technologies that rely on a fixed trajectory for the final segment of the parking process, this invention provides, in a first aspect, a real-time segmented parking correction method based on visual detection. Figure 1 As shown, it includes:

[0062] S100: Obtain the parking area of ​​interest to the vehicle, which includes parking spaces.

[0063] In specific implementation, such as Figure 2 As shown, obtaining the parking area of ​​interest to the vehicle includes:

[0064] S110: Obtain information from the CAN bus that the vehicle is in a parked state.

[0065] The CAN bus is a serial communication protocol bus used for real-time applications. In this step, the parking space detection function will only be triggered after a signal indicating that the vehicle is in a parking state is obtained from the CAN bus.

[0066] The S120 uses cameras positioned around the vehicle to visually inspect the area surrounding the vehicle; these cameras also have parking space search and error correction functions.

[0067] In practical applications, 360° panoramic cameras can also be used to replace the cameras set around the vehicle. The cameras have search and correction functions, which are triggered as needed.

[0068] S130 identifies the surrounding area containing parking space information as the parking area of ​​interest to the vehicle.

[0069] In specific implementation, the area surrounding the searched parking space information is determined as the parking area of ​​interest to the vehicle, such as... Figure 3 As shown, it includes:

[0070] S131, use the corresponding camera to locate the parking space information in the search state.

[0071] S132, perform parking space coordinate transformation based on positioning information and fixed transformation formula, transform the parking space information to the image coordinate system of the corresponding correction camera, and obtain the rectangular outline of the parking space.

[0072] The fixed conversion formulas involved in this step are existing technologies and will not be described in detail here.

[0073] S133 uses the rectangular outline of the parking space as the reference center and uses a visual algorithm to crop the surrounding area image, which includes parking space information, found by the corresponding camera, to obtain the parking area of ​​interest to the vehicle.

[0074] After trimming, the parking area that the vehicle is interested in can be evenly and completely covered across the entire parking space.

[0075] The visual algorithms involved in this step are existing technologies, and will not be described in detail here.

[0076] like Figure 4 The diagram shown is one of the states of a vehicle in a parking state. At this time, the corresponding coordinate system has been established, and the rectangular outline of the parking space and the parking area of ​​interest to the vehicle have been determined.

[0077] The S200 performs visual detection on the parking space shape and, based on the results of the visual detection, divides the automatic parking process into a trajectory segment before entering the parking space and a trajectory segment after entering the parking space.

[0078] In specific implementation, the parking space shape is visually detected, and based on the results of the visual detection, the automatic parking process is divided into a trajectory segment before entering the parking space and a trajectory segment after entering the parking space, such as... Figure 5 As shown, it includes:

[0079] S210 uses cameras placed around the vehicle to capture the outline of the parking space in real time.

[0080] S220: When the corresponding camera detects that the outline of the parking space is closed, it determines that the automatic parking process of the vehicle is in the trajectory segment before entering the parking space; when the corresponding camera detects that the outline of the parking space is open, it determines that the automatic parking process of the vehicle is in the trajectory segment after entering the parking space.

[0081] like Figure 6 The diagram shows the vehicle's state during the automatic parking process at different trajectory segments. The "U" or "n" shape represents the outline of the parking space. Figure 6 'a' represents the trajectory segment before the vehicle enters the parking space during the automatic parking process. Figure 6 b represents the trajectory segment after the vehicle enters the parking space during the automatic parking process.

[0082] The S300 performs real-time corrections on the trajectory segments before and after parking, improving the accuracy of the automatic parking process and enabling the vehicle to park precisely in the parking space.

[0083] The error in the trajectory segment before storage is corrected in real time, such as... Figure 7 As shown, it includes:

[0084] The S310 uses the camera's search function to search for parking spaces in parking areas of interest to the vehicle.

[0085] When the vehicle is in the trajectory segment before entering the parking space during the automatic parking process, the camera in search mode and the camera in correction mode may capture the target parking space. Therefore, in practical applications, parking space retrieval needs to be performed in the clipped area (i.e., the parking area that the vehicle is interested in).

[0086] S320 transforms the searched parking space coordinates into a coordinate system based on the vehicle body.

[0087] S330, calculate the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates.

[0088] In this step, the calculation of the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates is as follows: Figure 8 As shown, it includes:

[0089] S331, obtain the final parking point coordinates and vehicle body coordinates, and use the average of the final parking point coordinates and vehicle body coordinates as the visual observation value of the camera detecting the parking space.

[0090] S332 transforms the final parking point coordinates into the target point in the vehicle coordinate system, and calculates the lateral deviation and heading angle deviation between the visual observation value and the target point.

[0091] S340 sends the lateral deviation value and heading angle deviation value as compensation values ​​to the planning module and control module to realize real-time correction of errors existing in the trajectory segment before entering the storage.

[0092] For example, let's say the coordinates of the found parking space are... The final parking point coordinates are Due to the influence of camera calibration, there is a certain difference between these two coordinates. The average of the two is taken as the visual observation value of the parking space detected by the camera. It should be noted that if one camera fails to detect a parking space, the coordinates of the parking point where the space was detected are set as the actual observed value of the detected parking space; the target point coordinates are... ,calculate and as well as and deviation value , This deviation value , The compensation value is sent to the planning and control modules for correction.

[0093] like Figure 9 The diagram shown is a schematic of the trajectory correction state before the vehicle enters the parking space during the automatic parking process.

[0094] Similar to the implementation principle and calculation method of S310-S340, in this step, the error existing in the trajectory segment after data entry is corrected in real time, such as... Figure 10 As shown, it includes:

[0095] The S350 uses the camera's correction function to detect parking spaces in parking areas of interest to the vehicle.

[0096] S360 transforms the detected parking space coordinates into a coordinate system based on the vehicle body.

[0097] S370, calculate the longitudinal deviation between the final parking point coordinates and the vehicle body coordinates.

[0098] S380 sends the longitudinal deviation value as a compensation value to the planning module and the control module to realize real-time correction of errors in the trajectory segment after it is entered into the database.

[0099] Following the example above, in S370, the calculation is performed. as well as Deviation between , the deviation value The compensation value used for longitudinal correction is sent to the planning and control modules for correction.

[0100] like Figure 11 The diagram shown is a schematic of the trajectory correction state after the vehicle has entered the parking space during the automatic parking process.

[0101] In summary, this invention divides the automatic parking process into a trajectory segment before parking and a trajectory segment after parking, and adopts different real-time correction strategies for different trajectory segments. Before parking, lateral deviation and heading angle deviation are corrected in real time, and after parking, longitudinal deviation is corrected in real time. This can solve the technical problems of untimely error correction and the existence of lateral and heading errors after parking, which are present in traditional methods that correct dynamic errors in the vehicle parking process based on the last fixed trajectory segment.

[0102] In general, the solution proposed in this invention has the following advantages:

[0103] Advantage 1: Parking space recognition relies solely on cameras and traditional algorithms, resulting in lower recognition costs.

[0104] Advantage 2: The recognition strategy is applicable to different trajectory segments, which can improve the accuracy of correction.

[0105] Secondly, the present invention provides a device for real-time correction of parking segments based on visual detection, such as... Figure 12 As shown, it includes:

[0106] The image acquisition module is used to collect parking space information and location data in the area surrounding the vehicle;

[0107] The information sensing module communicates with the image acquisition module and is used to analyze and acquire parking target points;

[0108] The planning module, which communicates with the perception module, is used to calculate the parking trajectory; and,

[0109] The control module, which communicates with the planning module, is used to control the vehicle to park in the parking space.

[0110] Thirdly, the present invention provides an electronic device, such as... Figure 13 As shown, it includes:

[0111] At least one processor; and,

[0112] A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the vision-based real-time parking segment correction method as described in the first aspect.

[0113] Fourthly, the present invention provides a non-transitory computer-readable storage medium storing a computer software program for implementing a vision-based real-time parking segment correction method as described in the first aspect.

[0114] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0115] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, electronic devices, or computer software program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0116] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, systems, electronic devices, or computer software program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

Claims

1. A method for real-time correction of parking segmentation based on visual detection, characterized in that, include: Obtain the parking areas that the vehicle is interested in, including parking spaces; Visual detection is performed on the shape of the parking space. Based on the results of the visual detection, the automatic parking process of the vehicle is divided into the trajectory segment before entering the parking space and the trajectory segment after entering the parking space. The system corrects errors in the trajectory segments before and after parking in real time, improving the accuracy of the automatic parking process and enabling the vehicle to park precisely in the parking space. The process involves visually detecting the shape of the parking space and, based on the results, dividing the automatic parking process into a trajectory segment before and after parking, including: Cameras placed around the vehicle capture the outline of the parking space in real time; When the corresponding camera detects that the outline of the parking space is closed, it determines that the automatic parking process is in the trajectory segment before entering the parking space; when the corresponding camera detects that the outline of the parking space is open, it determines that the automatic parking process is in the trajectory segment after entering the parking space. Real-time correction of errors existing in the trajectory segment before data entry, including: Use the camera's search function to search for parking spaces in parking areas that interest the vehicle; The coordinates of the searched parking spaces are transformed into a coordinate system based on the vehicle body; Calculate the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates; The lateral deviation and heading angle deviation values ​​are sent as compensation values ​​to the planning and control modules to realize real-time correction of errors existing in the trajectory segment before entering the database. Real-time correction of errors in the trajectory segments after data entry, including: Use the camera's correction function to detect parking spaces in parking areas of interest to the vehicle; The detected parking space coordinates are transformed into a coordinate system based on the vehicle body; Calculate the longitudinal deviation between the final parking point coordinates and the vehicle body coordinates; The longitudinal deviation value is sent as a compensation value to the planning module and the control module to realize real-time correction of errors in the trajectory segment after it is entered into the database.

2. The method of real-time correction of parking segmentation based on visual detection according to claim 1, characterized in that, The acquisition of the parking area of ​​interest to the vehicle includes: The vehicle is in a parked state as detected by the CAN bus. Cameras positioned around the vehicle are used to visually inspect the area surrounding the vehicle; these cameras also have parking space search and error correction functions. The surrounding area containing parking space information found in the search is identified as the parking area that the vehicle is interested in.

3. The method for real-time segmented parking correction based on vision detection according to claim 2, characterized in that, The step of determining the surrounding area containing parking space information as the parking area of ​​interest to the vehicle includes: Use the corresponding camera to locate the parking space information in the search state; Based on the positioning information and fixed transformation formula, the parking space coordinates are transformed to the corresponding image coordinate system of the correction camera, resulting in the rectangular outline of the parking space. Using the rectangular outline of the parking space as the reference center, a visual algorithm is used to crop the surrounding area image, which includes parking space information, found by the corresponding camera, to obtain the parking area of ​​interest to the vehicle.

4. The method of real-time correction of parking segmentation based on visual detection according to claim 1, characterized in that, The calculation of the lateral deviation and heading angle deviation between the final parking point coordinates and the vehicle body coordinates includes: Obtain the final parking point coordinates and vehicle body coordinates, and use the average of the final parking point coordinates and vehicle body coordinates as the visual observation value of the camera detecting the parking space; The final parking point coordinates are converted into the target point in the vehicle coordinate system, and the lateral deviation and heading angle deviation between the visual observation value and the target point are calculated.

5. A device for real-time correction of parking segmentation based on visual detection, for implementing the method of real-time correction of parking segmentation based on visual detection according to any one of claims 1 to 4, characterized in that, include: The image acquisition module is used to collect parking space information and location data in the area surrounding the vehicle; The information sensing module communicates with the image acquisition module and is used to analyze and acquire parking target points; The planning module, which communicates with the perception module, is used to calculate the parking trajectory; as well as, The control module, which communicates with the planning module, is used to control the vehicle to park in the parking space.

6. An electronic device, comprising: include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the processor for performing the vision-based real-time parking segment correction method as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium, comprising: The storage medium stores a computer software program for implementing a vision-based real-time parking segment correction method as described in any one of claims 1-4.

Images