Parking method, device and equipment based on visual perception and medium
Through the parking method based on visual perception, the vehicle camera and dead calculation algorithm are used to adjust the parking target position in real time, solving the parking accuracy problem caused by error accumulation in the automatic parking system, and improving the parking accuracy and experience.
Patent Information
- Application Number
- CN202411401633.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2024-12-06
AI Technical Summary
The existing automatic parking system relies on dead calculating algorithms, resulting in poor parking accuracy and serious error accumulation. Especially in the absence of high-precision maps and GPS, it is difficult to achieve accurate parking target position perception and adjustment.
The parking method based on visual perception is adopted to obtain the initial and real image coordinates of the parking target position through the vehicle camera, and the calculated relative motion coordinates are obtained in combination with the dead estimation algorithm, and the real-time conversion matrix is corrected to achieve dynamic adjustment of the vehicle. Improve parking accuracy.
Accurate perception and real-time adjustment of the parking target position are achieved, and the accuracy and parking experience of the automatic parking system are improved. It is suitable for complex scenarios such as underground garages and slope environments.
Smart Images

Figure CN119078807A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle control technology, and in particular to a parking method, device, equipment and medium based on visual perception. Background Art
[0002] Currently, the existing APA (Auto Parking Assistant) / AVP (Automated Valet Parking) generally adopts a dead reckoning algorithm.
[0003] However, the odometer coordinate system (Odom coordinate system) of the dead reckoning algorithm inevitably brings errors and error accumulation, which ultimately leads to inaccurate parking. Summary of the invention
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a parking method, device, equipment and medium based on visual perception.
[0005] The present disclosure provides a parking method based on visual perception, including:
[0006] When the parking target position is a preset landmark, in response to the parking instruction, obtaining the initial image coordinates and initial relative motion coordinates of the parking target position based on the camera of the vehicle;
[0007] During parking, the real landmark image coordinates of the parking target position are acquired based on the camera, the estimated relative motion coordinates of the parking target position are acquired based on a preset dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into target image coordinates;
[0008] A real-time conversion matrix is determined based on the target image coordinates and the real mark image coordinates, and the target image coordinates are converted based on the real-time conversion matrix to obtain a correction value of the parking target position, and parking adjustments are made to the vehicle based on the correction value.
[0009] Optionally, the parking target position is not the preset landmark, and the method further includes:
[0010] Obtaining initial marker image coordinates and initial marker relative motion coordinates of the preset marker;
[0011] During parking, based on the dead reckoning algorithm and the initial marker relative motion coordinates, the dead reckoning marker relative motion coordinates are obtained, and the dead reckoning marker relative motion coordinates are converted into target marker image coordinates;
[0012] During parking, obtaining the real image coordinates of the preset marker based on the camera;
[0013] The real-time transformation matrix is determined based on the target marker image coordinates and the real marker image coordinates.
[0014] Optionally, determining a real-time conversion matrix based on the target image coordinates and the real marker image coordinates includes:
[0015] Calculate based on the target image coordinates and the real mark image coordinates to obtain first deviation data;
[0016] The real-time conversion matrix is obtained by performing calculation based on the first deviation data and the actual relative motion deviation data of the vehicle.
[0017] Optionally, determining the real-time conversion matrix based on the target marker image coordinates and the real marker image coordinates includes:
[0018] Calculating based on the target mark image coordinates and the real mark image coordinates to obtain second deviation data;
[0019] The real-time conversion matrix is obtained by performing calculation based on the second deviation data and the actual relative motion deviation data of the vehicle.
[0020] Optionally, in the parking method based on visual perception, a plurality of cameras are arranged on the vehicle, and the method further comprises:
[0021] A plurality of distances between each of the cameras and the parking target position are obtained, and the image coordinates of the parking target position and / or the preset marker are determined using the image data obtained by the camera with the smallest distance among the plurality of distances at the target time.
[0022] Optionally, the preset markers include but are not limited to physical parking spaces, the parking target position, a parking target position determined based on a human-computer interface, a custom ground shape area, a self-selected target parking space, a ground fixed projection pattern, a ground projection of a vehicle-mounted fixed projection device, and one of the target positions of the vehicle obtained through a vehicle-mounted camera.
[0023] Optionally, the method further includes:
[0024] Acquiring a target area including at least one parking position within a target range of the vehicle based on the camera;
[0025] The target area including at least one parking position is displayed based on a human-machine interface, and an input instruction is received, and the parking target position is determined from the target area.
[0026] The present disclosure also provides a parking device based on visual perception, including:
[0027] a response acquisition module, for acquiring, in response to a parking instruction, initial image coordinates and initial relative motion coordinates of the parking target position based on a camera of the vehicle when the parking target position is a preset marker;
[0028] an acquisition conversion module, used for acquiring the real sign image coordinates of the parking target position based on the camera during the parking process, acquiring the estimated relative motion coordinates of the parking target position based on a preset dead reckoning algorithm and the initial relative motion coordinates, and converting the estimated relative motion coordinates into target image coordinates;
[0029] A first determination module, used for determining a real-time transformation matrix based on the target image coordinates and the real marker image coordinates;
[0030] A processing module is used to transform the target image coordinates based on the real-time transformation matrix to obtain a correction value of the parking target position, and perform parking adjustment on the vehicle based on the correction value.
[0031] An embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement a parking method based on visual perception as provided in an embodiment of the present disclosure.
[0032] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the parking method based on visual perception as provided in the embodiment of the present disclosure.
[0033] The embodiment of the present disclosure further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the parking method based on visual perception described in the aforementioned aspect.
[0034] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: the parking solution based on visual perception provided by the embodiment of the present disclosure includes: when the parking target position is a preset marker, in response to the parking instruction, the initial image coordinates and initial relative motion coordinates of the parking target position are obtained based on the camera of the vehicle; during the parking process, the real marker image coordinates of the parking target position are obtained based on the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into the target image coordinates; the real-time conversion matrix is determined based on the target image coordinates and the real marker image coordinates, and the target image coordinates are converted based on the real-time conversion matrix to obtain the correction value of the parking target position, and the vehicle is adjusted for parking based on the correction value. The technical problem of the poor parking accuracy of the existing method is solved, and the position and posture of the parking target position relative to the vehicle can be accurately perceived continuously and provide a dynamic true value reference for the parking system to adjust the vehicle in time, thereby improving the APA / AVP parking accuracy and improving the parking experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.
[0036] Figure 1A An example diagram of an initial parking target parking space provided by an embodiment of the present disclosure;
[0037] Figure 1B An example diagram of a target parking space during a parking process provided by an embodiment of the present disclosure;
[0038] Figure 1C An example diagram of a parking method provided in an embodiment of the present disclosure;
[0039] Figure 2 A schematic diagram of a parking method based on visual perception provided in an embodiment of the present disclosure;
[0040] Figure 3 A schematic diagram of a flow chart of another parking method based on visual perception provided by an embodiment of the present disclosure;
[0041] Figure 4A An example diagram of a marker provided in an embodiment of the present disclosure;
[0042] Figure 4B An exemplary diagram of another marker provided in an embodiment of the present disclosure;
[0043] Figure 4CAn example diagram of dead reckoning EKF smoothing provided in an embodiment of the present disclosure;
[0044] Figure 5A An example diagram of a target parking space and an Odom coordinate system provided in an embodiment of the present disclosure;
[0045] Figure 5B An example diagram of a Marker used for parking provided in an embodiment of the present disclosure;
[0046] Figure 5C An example diagram of an adjusted parking target parking space provided by an embodiment of the present disclosure;
[0047] Figure 5D An example diagram of a target parking space that is gradually adjusted during parking provided by an embodiment of the present disclosure;
[0048] Figure 6 An example diagram of parking based on visual perception provided by an embodiment of the present disclosure;
[0049] Figure 7 An example diagram of a parking correction effect based on visual perception provided by an embodiment of the present disclosure;
[0050] Figure 8 A schematic diagram of the structure of a parking device based on visual perception provided by an embodiment of the present disclosure;
[0051] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0052] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0053] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0054] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.
[0055] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0056] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0057] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0058] Usually, the calibration process of the vehicle camera is to find the mapping process from the real world coordinates to the camera image coordinate system (3D-2D). This mapping relationship is often used when performing detection and measurement based on image content; the dead reckoning algorithm is also called the relative positioning algorithm. It mainly detects the mileage of the vehicle at each moment through the wheel pulse sensor, and calculates the position of the vehicle relative to the previous moment by combining the vehicle acceleration, angular velocity and other information, and then obtains the vehicle's position at each moment. Only by locating the position of the vehicle in real time can the vehicle be accurately controlled in real time, such as path planning, path tracking, etc.
[0059] Among them, dead reckoning is a navigation technology that uses the current position and speed of an object to estimate its future position and direction. It has been applied to many transportation technologies, but it is easily affected by the accumulation of errors. For example, the displacement is measured by the number of rotations of the wheel, but the wheel shrinks due to temperature changes and produces slight errors. Since the previous estimate contains errors, continuous subsequent calculations will cause error propagation.
[0060] At present, the existing APA / AVP generally adopts the dead reckoning algorithm. With the development of automobile intelligence, automobile intelligent products are developing from traditional advanced driving assistance systems to automatic driving systems, and the application scenarios are gradually upgraded from simple to complex, and they are increasingly dependent on the results of visual perception. The odometer coordinate system (Odom coordinate system) of the dead reckoning algorithm (hereinafter referred to as DR) adopted by the existing APA / AVP (automatic parking assistance system) inevitably brings errors and error accumulation. The sources of odometer errors include limited resolution during integration; changes in wheel outer diameter (such as tire pressure); changes in wheel contact points; non-uniform ground contact and friction coefficients that change at different positions causing slippage (such as passing speed bumps); non-uniform ground contact and friction coefficients that change at different positions causing slippage, etc. One or more.
[0061] Among them, the odometer, also known as the driving speedometer, is an instrument used to measure the mileage of a vehicle. The odometer is usually installed on the dashboard of the vehicle and calculates the mileage of the vehicle by recording the number of times the vehicle tires rotate or the distance the wheels rotate. Traditional mechanical odometers use mechanical gears or electronic sensors for measurement, while modern cars usually use speed sensors built into the vehicle transmission system to implement the odometer function. The Odom coordinate system refers to the relative motion coordinate system of the robot (such as a vehicle). Assuming that the robot starts now, the current position is set as the origin, and then the coordinate system is established. The coordinate point relative to the starting position is obtained based on the distance and direction of movement.
[0062] In the disclosed embodiment, the initial target parking space information provided by the image perception algorithm is based on the image data obtained by a certain vehicle-mounted camera. During the parking process, due to the change in the position of the target parking space (i.e., the parking target position) relative to different on-board cameras, the camera data used in the parking process is not the same as the camera used to initially detect the parking space (such as a 360-degree surround stitching image or an Around View Monitor (AVM) image). Due to the different calibration effects of the cameras themselves and the different image accuracies of the target parking space at different positions of the cameras, for example, in the application of surround view image parking, the parking space detection is performed by the right-side camera (which can be captured by the camera), and the parking for reverse parking is mainly performed by the rear-side camera (which can be captured by the camera). The target parking space in the same physical world is located in different areas of different cameras due to the movement during the parking process. The deviation of the surround view stitching may exceed the width of a parking space line (compared with the ground truth). At this time, the surround view image detection method is always used. The deviation for high-precision parking is uncontrollable. At this time, it is necessary to use the image data mainly from a certain single-channel camera for dynamic adjustment. Figure 1AThe initial parking target parking space shown in the figure has four cameras on the vehicle, located at the front, back, left and right of the vehicle. Figure 1B The target parking space for the parking process is shown.
[0063] Generally, the general perception method is to extract the bounding box (or feature points) of the parking target in the image through the visual perception recognition algorithm before parking and convert it to the vehicle body coordinate system (Odom coordinate system). In the subsequent parking process, the parking vehicle estimates the position coordinates of the parking target based on this bounding box (or feature points). However, the parking target position coordinates given by this method can only give an initial unchanged parking target (Odom coordinate system). Due to the accumulated error of the aforementioned odometer, the error between the posture and true value (physical true value, which can be measured by measuring tools such as rangefinders and total station instruments with the origin of the system coordinate system) obtained by the parking vehicle based on DR will gradually accumulate, and it is impossible to accurately perceive the position and posture of the parking target relative to the vehicle in real time and make dynamic adjustments, that is, the system is in an open-loop state. If the vehicle is controlled based on this information, it is easy for the parking posture to fail to reach the ideal / expected state: crossing the line, not being centered (front and back / left and right), and the front of the vehicle protruding, which is easy to cause the risk of collision with surrounding vehicles due to exceeding the expected target box or limit box.
[0064] In addition, for manually selected parking spaces or parking spaces at any designated location or imaginary parking spaces, there may be no marked parking spaces on the ground in the actual physical world (visual perception cannot detect them specifically), and the above-mentioned method of identifying the parking target boundary box is not feasible at all. Figure 1C The parking method shown (imaginary parking space / parking space at a designated location).
[0065] The existing automatic parking in any area and scene, especially in closed scenes and slopes such as underground garages, tunnels, indoor floors, etc., lacks high-precision maps, GPS (Global Positioning System), high-precision SLAM (Simultaneous Localization and Mapping), and other precise positioning, and its positioning accuracy is far from meeting the accuracy requirements required for parking (single pixel, such as within 3 degrees), which seriously affects the parking effect and parking experience.
[0066] The disclosed implementation proposes a parking method based on visual perception. When the parking target position is a preset marker, in response to the parking instruction, the initial image coordinates and initial relative motion coordinates of the parking target position are obtained based on the camera of the vehicle; during the parking process, the real marker image coordinates of the parking target position are obtained based on the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into the target image coordinates; the real-time conversion matrix is determined based on the target image coordinates and the real marker image coordinates, and the target image coordinates are converted based on the real-time conversion matrix to obtain the correction value of the parking target position, and the vehicle is adjusted for parking based on the correction value. The technical problem of the poor parking accuracy of the existing method is solved, and the position and posture of the parking target position relative to the vehicle can be accurately perceived continuously to provide a dynamic true value reference for the parking system to adjust the vehicle in time, thereby improving the APA / AVP parking accuracy and enhancing the parking experience.
[0067] Figure 2 The present invention provides a flow chart of a parking method based on visual perception, which can be performed by a parking device based on visual perception, wherein the device can be implemented by software and / or hardware, and can generally be integrated into an electronic device. Figure 2 As shown, the method includes:
[0068] Step 201 : When the parking target position is a preset landmark, in response to a parking instruction, the initial image coordinates and initial relative motion coordinates of the parking target position are acquired based on a camera of the vehicle.
[0069] In the disclosed embodiment, before initiating the automatic parking command, it is necessary to predetermine the markers, and the preset markers include but are not limited to a physical parking space, a parking target position, a parking target position determined based on a human-computer interface, a custom ground shape area, a self-selected target parking space, a fixed ground projection pattern, a ground projection of a vehicle-mounted fixed projection device, and one of the target positions of the vehicle obtained through a vehicle-mounted camera.
[0070] In the disclosed embodiment, a parking target position may be randomly selected, or a target area including at least one parking position within a target range of the vehicle may be acquired based on a camera, the target area including at least one parking position may be displayed on a human-machine interface, and an input instruction may be received to determine the parking target position from the target area.
[0071] In the embodiment of the present disclosure, the case where the parking target position is a preset marker means that the parking target position and the marker are the same. When multiple cameras are set on the vehicle, multiple distances between each camera and the parking target position are obtained, and the image data obtained by the camera with the smallest distance among the multiple distances at the target time is used to determine the image coordinates of the parking target position and / or the preset marker.
[0072] In the embodiment of the present disclosure, when the parking target position is a preset landmark, in response to a parking instruction (triggered by the user through a screen, a remote control device, or manually, etc.), the initial image coordinates and initial relative motion coordinates of the parking target position are acquired through the vehicle's camera; wherein the initial image coordinates refer to the coordinate information of the parking target position in the image coordinate system; the initial relative motion coordinates refer to the relative coordinate information of the vehicle relative to the target parking position in the Odom coordinate system.
[0073] Step 202: During the parking process, the real landmark image coordinates of the parking target position are obtained based on the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into target image coordinates.
[0074] During the parking process, the real landmark image coordinates of the parking target position are continuously obtained through the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates; among which, only during the first reckoning, the initial relative motion coordinates are the relative coordinate information of the vehicle relative to the target parking position in the Odom coordinate system when the parking instruction is initiated. During the second or more reckonings, the initial relative motion coordinates are constantly changing, and the initial relative motion coordinates used for each reckoning are the initial relative motion coordinates obtained in the previous reckoning.
[0075] In the disclosed embodiment, in order to improve calculation efficiency, the estimated relative motion coordinates are converted into target image coordinates, that is, the coordinate information of the vehicle distance to the parking target position in the Odom coordinate system is converted into the image coordinate information of the target parking position in the image.
[0076] Step 203: determine a real-time conversion matrix based on the target image coordinates and the real sign image coordinates, convert the target image coordinates based on the real-time conversion matrix, obtain a correction value of the parking target position, and make parking adjustments to the vehicle based on the correction value.
[0077] In the embodiments of the present disclosure, there are many ways to determine the real-time transformation matrix based on the target image coordinates and the real sign image coordinates. Specifically, calculations are performed based on the observed image coordinates (such as the target image coordinates) and the real sign image coordinates to obtain deviation data, and calculations are performed based on the deviation data and the actual relative motion deviation data of the vehicle to obtain the real-time transformation matrix.
[0078] Furthermore, the target image coordinates are transformed based on the real-time conversion matrix to obtain a correction value of the parking target position, and the vehicle is adjusted for parking based on the correction value. It can be understood that during the parking process, the real-time conversion matrix is continuously obtained to continuously transform the observed target image coordinates to obtain a correction value of the parking target position, and the vehicle is adjusted for parking based on the correction value to ensure the accuracy of parking.
[0079] The parking solution based on visual perception provided by the embodiment of the present disclosure includes: when the parking target position is a preset marker, in response to the parking instruction, the initial image coordinates and initial relative motion coordinates of the parking target position are obtained based on the camera of the vehicle; during the parking process, the real marker image coordinates of the parking target position are obtained based on the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into the target image coordinates; the real-time conversion matrix is determined based on the target image coordinates and the real marker image coordinates, and the target image coordinates are converted based on the real-time conversion matrix to obtain the correction value of the parking target position, and the vehicle is adjusted for parking based on the correction value. The technical problem of the relatively poor parking accuracy of the existing method is solved, and the position and posture of the parking target position relative to the vehicle can be accurately perceived continuously to provide a dynamic true value reference for the parking system to adjust the vehicle in time, thereby improving the APA / AVP parking accuracy and enhancing the parking experience.
[0080] Figure 3 This is a flow chart of another parking method based on visual perception provided by an embodiment of the present disclosure. This embodiment further optimizes the parking method based on visual perception based on the above embodiment. Figure 3 As shown, the method includes:
[0081] Step 301: When the parking target position is a preset landmark, in response to a parking instruction, the initial image coordinates and initial relative motion coordinates of the parking target position are acquired based on a camera of the vehicle.
[0082] Step 302: During the parking process, the real landmark image coordinates of the parking target position are obtained based on the camera, and the estimated relative motion coordinates of the parking target position are obtained based on the dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into target image coordinates.
[0083] It should be noted that steps 301-302 are the same as steps 201-202. For detailed description, please refer to the description of steps 201-202 in the aforementioned embodiment.
[0084] Step 303: Calculate based on the target image coordinates and the real marker image coordinates to obtain first deviation data, and calculate based on the first deviation data and the actual relative motion deviation data of the vehicle to obtain a real-time conversion matrix.
[0085] In the disclosed embodiment, the target image coordinates and the real sign image coordinates are subjected to difference processing, such as subtracting the coordinate values of each dimension of the three-dimensional coordinate point, so as to obtain first deviation data, and obtain actual relative motion deviation data between the vehicle and the parking target position based on the vehicle odometer or the like, and determine a real-time conversion matrix based on the first deviation data and the actual relative motion deviation data of the vehicle.
[0086] Step 304: If the parking target position is not a preset marker, the initial marker image coordinates and the initial marker relative motion coordinates of the preset marker are obtained.
[0087] In the disclosed embodiment, the parking target position is not a preset marker, that is, the parking target position and the marker are not the same. In this case, the marker can be one of a custom ground shape area, a self-selected target parking space, a fixed ground projection pattern, a ground projection of a vehicle-mounted fixed projection device, and the target position of the vehicle (handlebars, etc.) obtained through a vehicle-mounted camera.
[0088] In the embodiment of the present disclosure, the initial marker image coordinates refer to the coordinate information of the marker in the image coordinate system; the initial marker relative motion coordinates refer to the relative coordinate information of the vehicle relative to the marker in the Odom coordinate system.
[0089] Step 305: During the parking process, the dead reckoning algorithm and the initial marker relative motion coordinates are used to obtain the calculated marker relative motion coordinates of the preset marker, and the calculated marker relative motion coordinates are converted into target marker image coordinates. The real marker image coordinates of the preset marker are obtained based on the camera.
[0090] In the embodiment of the present disclosure, the calculated marker relative motion coordinates of a preset marker are obtained based on a preset dead reckoning algorithm and the initial marker relative motion coordinates. During the first calculation, the initial marker relative motion coordinates are the relative coordinate information of the vehicle relative to the marker in the Odom coordinate system when the parking command is initiated. During the second or more calculations, the initial marker relative motion coordinates are constantly changing, and the initial marker relative motion coordinates used for each calculation are the initial marker relative motion coordinates obtained in the previous calculation.
[0091] In the disclosed embodiment, in order to improve calculation efficiency, the estimated relative motion coordinates of the marker are converted into the target marker image coordinates, that is, the coordinate information of the vehicle distance marker in the Odom coordinate system is converted into the image coordinate information of the marker in the image.
[0092] Step 306: Calculate based on the target marker image coordinates and the real marker image coordinates to obtain second deviation data, and calculate based on the second deviation data and the actual relative motion deviation data of the vehicle to obtain a real-time conversion matrix.
[0093] In the disclosed embodiment, the target sign image coordinates and the real sign image coordinates are subjected to difference processing, for example, the coordinate values of each dimension of the three-dimensional coordinate point are subtracted, so as to obtain second deviation data, and the actual relative motion deviation data between the vehicle and the sign is obtained based on the vehicle odometer or the like, and the real-time conversion matrix is determined based on the second deviation data and the actual relative motion deviation data of the vehicle.
[0094] Step 307: transform the target image coordinates based on the real-time transformation matrix to obtain a correction value of the parking target position, and perform parking adjustments on the vehicle based on the correction value.
[0095] In an embodiment of the present disclosure, a plurality of cameras are arranged on the vehicle, and the method further comprises: obtaining a plurality of distances between each camera and a parking target position, and determining the image coordinates of the parking target position and / or a preset marker using the image data obtained by the camera with the smallest distance among the plurality of distances at the target time.
[0096] In the disclosed embodiment, preset markers include but are not limited to physical parking spaces, parking target positions, parking target positions determined based on a human-computer interface, custom ground shape areas, self-selected target parking spaces, ground fixed projection patterns, ground projections of vehicle-mounted fixed projection equipment, and one of the target positions of the vehicle obtained through a vehicle-mounted camera.
[0097] In the disclosed embodiment, a target area including at least one parking position within a target range of the vehicle is acquired based on a camera, the target area including at least one parking position is displayed based on a human-machine interface, and an input instruction is received to determine the parking target position from the target area.
[0098] Specifically, the disclosed embodiment is based on the vehicle camera and the vehicle body odometer information (dead reckoning algorithm), combined with the camera reference marker image data, the marker image data can be a real physical world line drawing target parking space, or artificial AR (Augmented Reality, augmented reality) / VR (Virtual Reality, virtual reality technology) image data, such as Figure 4AFrom left to right, the actual target parking space with lines drawn on it, the manually selected position, and the vehicle-mounted (equipment) projected onto the ground pattern are shown. Figure 4B Shown from left to right are the vehicle body component structure and the text information patterns unique to the target parking space surrounding.
[0099] Specifically, the image data collection diagram of the calibrated vehicle-mounted camera and its parking process; dynamic switching or area selection of camera data during parking: the vehicle-mounted camera data comes from single-channel, fisheye surround, perimeter view, etc., to maximize the parking accuracy and convenience of the parking system. According to the relative position of the vehicle body position and the parking target and reference marker, the camera position is selected. The data position and time that is closest and most centered is conducive to maximum accuracy, such as Figure 1A and 1B As shown in the figure, the data position and time with the highest accuracy are the position and time when the acquisition camera is closest to the target parking space; by relying only on the dead reckoning mileage and information and the reference marker image information, it is simple and reliable, with a wide range of applications, supporting closed indoor, underground, ramp and other scenes, without the need for additional high-precision maps, GPS, SLAM data; it can also effectively reduce the adjustment fluctuation of the target position and the influence of outliers based on the smoothing technology of filter optimization, and provide ideal data for the regulation and control module, such as Figure 4C As shown, EKF (Extended Kalman Filter) filtering is performed on the collected true values (i.e., the true image coordinates described above), the coordinates obtained by the track calculation algorithm, and the observed values (i.e., the coordinates obtained by GPS). EKF is a highly efficient recursive filter (autoregressive filter) that can estimate the state of a dynamic system from a series of measurements that do not completely contain noise.
[0100] More specifically, step s1: when or before the parking command is initiated, the parking system receives the parking target position information (such as the vehicle screen selection, which can be one or more, multiple can be selected, or manual screen selection) of the target parking position and the initial value of the relative position of the vehicle (Odom coordinates), such as Figure 1A and Figure 1B As shown; Step s2: When or after the parking instruction is started, the vehicle camera collects the original image data (image coordinates) containing the parking target position in real time, such as Figure 1A and Figure 1B As shown; Step s3: When or before the parking instruction is started, the Marker (initial value) in the vehicle image data is selected interactively (manually), or the parking target image content is selected by default, and the vehicle camera collects the Marker original image data containing the parking target in real time for error correction. In particular, if the parking target and the Marker are the same object, then Step s2 and Step s3 are exactly the same or the same operation, such as Figure 1A and Figure 1B shown.
[0101] Step s4: During the parking process, the dead reckoning method of the parking system is used to obtain the Marker position information (estimated value) according to step s3, and the Marker position information is converted to the image data of the currently used vehicle camera through coordinate conversion (image coordinates); Step s5: During the parking process, the Marker detected on the image content in step s3 is used to obtain its position information (observed value, true value detected by the image algorithm, image coordinates); Step s6: During the parking process, the parking target (estimated value) is obtained according to step s2 using the dead reckoning method of the parking system, and the Marker position information is converted to the image data of the currently used vehicle camera through coordinate conversion (image coordinates); Step s7: According to the image content of step s4 and step s5, the deviation between the dead reckoned Marker position and the image true value Marker detected by the image is calculated, and the position is converted to the odometer coordinate system (Odom coordinates), and the real-time transformation matrix tf (R|t, rotation and translation, Odom coordinates) is obtained;
[0102] Step s8: Perform EKF filtering and smoothing filtering on tf in step s7, and remove wild values and abnormal data in combination with the constraints of the physical world coordinate points to obtain a smooth, stable and effective tf; Step s9: According to the transformation matrix tf in step s7, transform the parking target estimated value in step s6, that is, obtain the correction value of the parking target (odom coordinates). In particular, if the parking target and the Marker are the same object, then step s4 and step s6 are exactly the same or the same operation; s10: According to the correction value obtained in step s9, send it to the parking regulation and control module for dynamic adjustment.
[0103] Furthermore, the marker image information mentioned in the embodiments of the present invention can be obtained through the camera image data, or can be converted into the image data content obtained by the camera, including the point set (3 points or more), line set (2 or more), shape set (closed or non-closed asymmetric point set or line set) synthesized by artificial AR / VR based on the image true value content through calculation and superposition. The image information of the marker is mapped to the true value information of the real physical world relative to the parking ground or the parking body, and has the following characteristics: the shape and size of the marker are known, or can be indirectly (calculated) obtained through camera image data; the marker can be uniquely identified, or indirectly uniquely identified through calculation, and the physical world can only be uniquely distinguished; the marker on the image data is real physical content, or based on the real physical world. The AR / VR artificial data Marker (lines, shapes, etc.) generated by the content can be mapped to the real physical world space and mapped to the image content, including but not limited to the following: the actual parking space in the real physical world, the parking target position (that is, the target parking space and the Marker are the same object), or the imaginary parking target position (available, obstacle-free, and unaffected) parking space selected by the HMI human-machine interface, the self-selected ground area of any shape (available, obstacle-free, and unaffected), the self-selected target parking space, the ground fixed projection pattern, the ground projection of the vehicle-mounted fixed projection device, the vehicle body, self-components, self-components, and wheels of the vehicle fixed to the vehicle body that can be obtained through the vehicle-mounted camera (which can be obtained through the vehicle-mounted camera); the image data content is the image obtained by the vehicle-mounted camera of the parked vehicle (including artificially superimposed Markers, which can be used for vision-based perception detection algorithms).
[0104] Furthermore, in the embodiments of the present disclosure, the vehicle-mounted camera and data used for deviation correction may be from monocular, binocular, periscopic, surround (fisheye), BEV (multi-camera), etc.
[0105] Furthermore, in the disclosed embodiment, the vehicle-mounted camera used for deviation correction is calibrated in advance (parameters and internal reference data are known, or the conversion relationship from image content to the real physical world is known.
[0106] Furthermore, in the embodiments of the present disclosure, all detection data acquired based on camera images (including artificial, virtual, and markers superimposed on camera image data) and subsequent data calculations used by the regulation and control module are converted to the odometry coordinate system.
[0107] Specifically, the present invention discloses the vehicle mileage calculation error correction based on the APA / AVP vehicle camera visual perception, which can improve the parking accuracy and parking effect, obtain a good parking experience, and can be applied to any complex closed, indoor, underground garage, and slope scene. As an example of a scene, the method includes: (1) online real-time acquisition of parking vehicle odometer information; (2) online real-time acquisition of vehicle camera image data (the vehicle camera has been calibrated, and the internal and external reference data are known); (3) the parking target position information required when the parking command is initiated is converted to the odometer coordinate system, such as Figure 5A The target parking space and Odom coordinate system shown in FIG. 4 are as follows; (4) when the parking instruction is initiated and during the subsequent parking process, the marker image information used for error correction is obtained, such as Figure 5B (5) After starting parking and during parking, the parking target position in the odometer coordinate system is adjusted online and continuously using the marker image information, such as Figure 5C (6) sending the adjusted parking target position information, the control module can continuously make corresponding adjustments according to the updated parking target position in the odometer coordinate system, such as Figure 5D The target parking space (Odom coordinate system) that is gradually adjusted during the parking process is shown in the figure, from left to right, respectively: step 1: before s1 adjustment / after s2 adjustment; step 2: before s2 adjustment / after s3 adjustment; step 3: before s3 adjustment / after s4 adjustment; (7) based on the relative position of the adopted marker and the on-board camera, the camera and camera image data that are conducive to the most accurate parking time and parking position (the situation with high image resolution and full Marker image data in the center) can be selected; (8) based on the relative position of the adopted marker and the on-board camera, the camera data at the parking position that is most conducive to the parking effect can be selected.
[0108] The disclosed embodiment utilizes the image data from the vehicle-mounted camera and adjusts the accumulated error of the odometer based on the stationary markers Markers relative to the vehicle body or the ground that can be obtained by the camera, so as to achieve an ideal parking effect at the target storage location and ultimately realize a good posture of the vehicle body relative to the target storage location (the parking is centered and the posture is correct relative to the parking target).
[0109] As an example, Figure 6The steps shown include: step 6.1 start parking; step 6.2 activate the vehicle camera (dynamic switching / optional); step 6.3 acquire image data (including parking target / Marker); step 6.4 DR acquires parking target position; step 6.5 DR / Marker pose conversion to current image; step 6.6 real-time detection of current image Marker (image true value); step 6.7 image coordinate conversion to Odom coordinate system; step 6.8 DR deviation adjustment matrix tf; step 6.9 tf filtering and data legitimacy verification; step 6.10 DR parking target position adjustment; step 6.11 parking is completed.
[0110] As another example, the existing automatic parking assistance systems such as APA / AVP have insufficient accuracy, limited application range, and the parking experience and effect need to be improved. Figure 7 As shown, the image content shows the effect diagram based on the embodiment of the present disclosure. The parking scene is a sloping ground, and the parking target is a parking space with ground lines. Frame A is the parking target adjustment effect based on the reference marker Marker (in this case, it is a parking space with ground lines, which is consistent with the parking target), and Frame B is the original and corrected parking target effect using the dead reckoning algorithm. It can be seen that the posture adjustment effect is obvious.
[0111] In view of the problems that the existing automatic parking assistance systems such as APA / AVP are insufficient in accuracy, have a limited scope of application, and need to improve the parking experience and effect, Figure 7 The error correction effect of the reference marker based on perception is displayed using image content (true value). The parking scene is a sloped ground, and the parking target is a parking space with lines drawn on the ground. The green frame is the parking target adjustment effect based on the reference marker (in this case, it is a parking space with lines drawn on the ground, which is consistent with the parking target). The yellow frame is the original and corrected parking target effect using the dead reckoning algorithm. It can be seen that the posture adjustment effect is obvious.
[0112] Therefore, a parking method based on visual perception is provided, which can continuously and accurately perceive the position and posture of the parking target relative to the vehicle (visual closed-loop system), provide a dynamic true value reference for regulation and control, improve APA / AVP parking accuracy and enhance the parking experience.
[0113] Figure 8 A schematic diagram of the structure of a parking device based on visual perception provided in an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device.
[0114] like Figure 8 As shown, the device comprises:
[0115] A response acquisition module 801 is used for acquiring the initial image coordinates and initial relative motion coordinates of the parking target position based on the camera of the vehicle in response to the parking instruction when the parking target position is a preset marker;
[0116] an acquisition conversion module 802, configured to acquire, during a parking process, the real landmark image coordinates of the parking target position based on the camera, acquire the estimated relative motion coordinates of the parking target position based on the dead reckoning algorithm and the initial relative motion coordinates, and convert the estimated relative motion coordinates into target image coordinates;
[0117] A first determination module 803, configured to determine a real-time transformation matrix based on the target image coordinates and the real marker image coordinates;
[0118] The processing module 804 is used to transform the target image coordinates based on the real-time transformation matrix to obtain a correction value of the parking target position, and perform parking adjustment on the vehicle based on the correction value.
[0119] Optionally, the parking target position is not the preset landmark, and the device further includes:
[0120] A first acquisition module is used to acquire the initial marker image coordinates and the initial marker relative motion coordinates of the preset marker;
[0121] A second acquisition module is used to acquire the dead reckoning relative motion coordinates of the preset marker based on a preset dead reckoning algorithm and the initial marker relative motion coordinates during parking, and convert the dead reckoning relative motion coordinates into target marker image coordinates;
[0122] A third acquisition module, used for acquiring the real marker image coordinates of the preset marker based on the camera during parking;
[0123] The second determination module is used to determine the real-time conversion matrix based on the target marker image coordinates and the real marker image coordinates.
[0124] Optionally, the first determining module is specifically configured to:
[0125] Calculate based on the target image coordinates and the real mark image coordinates to obtain first deviation data;
[0126] The real-time conversion matrix is obtained by performing calculation based on the first deviation data and the actual relative motion deviation data of the vehicle.
[0127] Optionally, the second determining module is specifically configured to:
[0128] Calculating based on the target mark image coordinates and the real mark image coordinates to obtain second deviation data;
[0129] The real-time conversion matrix is obtained by performing calculation based on the second deviation data and the actual relative motion deviation data of the vehicle.
[0130] Optionally, in the parking method based on visual perception, a plurality of cameras are arranged on the vehicle, and the device further comprises:
[0131] The fourth acquisition module is used to acquire multiple distances between each of the cameras and the parking target position, and determine the image coordinates of the parking target position and / or the preset marker using the image data acquired by the camera with the smallest distance among the multiple distances at the target time.
[0132] Optionally, the device further includes:
[0133] A fifth acquisition module, configured to acquire, based on the camera, a target area within the target range of the vehicle including at least one parking position;
[0134] The display determination module is used to display the target area including at least one parking position based on a human-machine interface, receive an input instruction, and determine the parking target position from the target area.
[0135] The parking device based on visual perception provided in the embodiments of the present disclosure can execute the parking method based on visual perception provided in any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution method.
[0136] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the parking method based on visual perception provided by any embodiment of the present disclosure.
[0137] Fig. 9 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Fig. 9 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure. The electronic device 900 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0138] like Fig. 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0139] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig. 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.
[0140] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the parking method based on visual perception of the embodiment of the present disclosure are executed.
[0141] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0142] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0143] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0144] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: when the parking target position is a preset marker, in response to a parking instruction, obtains the initial image coordinates and initial relative motion coordinates of the parking target position based on the vehicle's camera; during the parking process, obtains the real marker image coordinates of the parking target position based on the camera, obtains the estimated relative motion coordinates of the parking target position based on the dead reckoning algorithm and the initial relative motion coordinates, and converts the estimated relative motion coordinates into target image coordinates; determines a real-time conversion matrix based on the target image coordinates and the real marker image coordinates, converts the target image coordinates based on the real-time conversion matrix, obtains a correction value of the parking target position, and makes parking adjustments to the vehicle based on the correction value.
[0145] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
[0146] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0147] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.
[0148] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0149] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0150] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:
[0151] processor;
[0152] a memory for storing instructions executable by the processor;
[0153] The processor is used to read the executable instructions from the memory and execute the instructions to implement any visual perception-based parking method provided in the present disclosure.
[0154] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute any of the visual perception-based parking methods provided by the present disclosure.
[0155] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.
[0156] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.
[0157] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.
Claims
1. A parking method based on visual perception, characterized in that: include: When the parking target position is a preset landmark, in response to the parking instruction, obtaining the initial image coordinates and initial relative motion coordinates of the parking target position based on the camera of the vehicle; During parking, the real landmark image coordinates of the parking target position are acquired based on the camera, the estimated relative motion coordinates of the parking target position are acquired based on a preset dead reckoning algorithm and the initial relative motion coordinates, and the estimated relative motion coordinates are converted into target image coordinates; A real-time conversion matrix is determined based on the target image coordinates and the real mark image coordinates, and the target image coordinates are converted based on the real-time conversion matrix to obtain a correction value of the parking target position, and parking adjustments are made to the vehicle based on the correction value.
2. The parking method based on visual perception according to claim 1, characterized in that: The parking target position is not the preset landmark, and the method further includes: Obtaining initial marker image coordinates and initial marker relative motion coordinates of the preset marker; During parking, based on the dead reckoning algorithm and the initial marker relative motion coordinates, the dead reckoning marker relative motion coordinates are obtained, and the dead reckoning marker relative motion coordinates are converted into target marker image coordinates; During parking, obtaining the real image coordinates of the preset marker based on the camera; The real-time transformation matrix is determined based on the target marker image coordinates and the real marker image coordinates.
3. The parking method based on visual perception according to claim 1, characterized in that: The determining of the real-time conversion matrix based on the target image coordinates and the real marker image coordinates comprises: Calculate based on the target image coordinates and the real mark image coordinates to obtain first deviation data; The real-time conversion matrix is obtained by performing calculation based on the first deviation data and the actual relative motion deviation data of the vehicle.
4. The parking method based on visual perception according to claim 2, characterized in that: The determining the real-time conversion matrix based on the target marker image coordinates and the real marker image coordinates comprises: Calculating based on the target mark image coordinates and the real mark image coordinates to obtain second deviation data; The real-time conversion matrix is obtained by performing calculation based on the second deviation data and the actual relative motion deviation data of the vehicle.
5. The parking method based on visual perception according to claim 1, characterized in that: The vehicle is provided with a plurality of cameras, and the method further comprises: A plurality of distances between each of the cameras and the parking target position are obtained, and the image coordinates of the parking target position and / or the preset marker are determined using the image data obtained by the camera with the smallest distance among the plurality of distances at the target time.
6. The parking method based on visual perception according to claim 1, characterized in that: The preset markers include but are not limited to physical parking spaces, the parking target positions, parking target positions determined based on a human-computer interface, custom ground shape areas, self-selected target parking spaces, ground fixed projection patterns, ground projections of vehicle-mounted fixed projection equipment, and one of the target positions of the vehicle obtained through a vehicle-mounted camera.
7. The parking method based on visual perception according to claim 1, characterized in that: The method further comprises: Acquiring a target area including at least one parking position within a target range of the vehicle based on the camera; The target area including at least one parking position is displayed based on a human-machine interface, and an input instruction is received, and the parking target position is determined from the target area.
8. A parking device based on visual perception, characterized in that: include: a response acquisition module, for acquiring, in response to a parking instruction, initial image coordinates and initial relative motion coordinates of the parking target position based on a camera of the vehicle when the parking target position is a preset marker; an acquisition conversion module, used for acquiring the real sign image coordinates of the parking target position based on the camera during the parking process, acquiring the estimated relative motion coordinates of the parking target position based on a preset dead reckoning algorithm and the initial relative motion coordinates, and converting the estimated relative motion coordinates into target image coordinates; A first determination module, used for determining a real-time transformation matrix based on the target image coordinates and the real marker image coordinates; A processing module is used to transform the target image coordinates based on the real-time transformation matrix to obtain a correction value of the parking target position, and perform parking adjustment on the vehicle based on the correction value.
9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the parking method based on visual perception as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the parking method based on visual perception as described in claims 1-7 above.
