Scene reconstruction method with optical device for machine parking
By calculating optical flows in specific ROIs and dynamically adjusting them based on the moving body's movement, the method addresses the challenges of low parallax scenarios, achieving efficient and accurate 3D reconstruction for autonomous parking.
Patent Information
- Application Number
- PCT/EP2026/058358
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-16
- Filing Date
- 2026-03-24
- Publication Date
- 2026-11-19
AI Technical Summary
Existing 3D information detection methods for autonomous vehicles and mobile robots face challenges in low parallax scenarios, leading to high errors due to noise and increased processing time, which are not suitable for real-time applications like machine parking.
A method that calculates optical flows in specific regions of interest (ROIs) within the detected area, using previous knowledge of the 3D profile of machine parking slots to minimize noise and reduce processing time, and dynamically adjusts ROIs based on the moving body's movement to enhance obstacle detection and trajectory planning.
This approach ensures accurate and efficient 3D reconstruction, providing a reliable solution for safe autonomous parking by minimizing noise interference and reducing processing time, thus overcoming the limitations of existing methods.
Smart Images

Figure EP2026058358_19112026_PF_FP_ABST
Abstract
Description
[0001] 202501974
[0002] 1
[0003] Scene reconstruction method with optical device for machine parking
[0004] The present invention relates to a method and system for detecting 3D information in various areas, particularly in machine parking scenarios, using cameras and other sensors. This method aims to enhance the accuracy and efficiency of obstacle detection and trajectory planning for autonomous vehicles and mobile robots by leveraging optical flow analysis within defined regions of interest (ROIs).
[0005] State-of-the-art methods for 3D information detection, such as structure from motion (SfM), rely on detecting and tracking features across multiple frames as the ego-vehicle moves. These methods face significant challenges, particularly in low parallax scenarios where the apparent movement of tracked features is minimal, leading to high errors due to noise. For example, one existing method improves traditional SfM by using an iterative reweighted least square method, setting a threshold based on parallax angles to filter out objects with high uncertainty. While this reduces errors, it increases processing time. Another method involves visual simultaneous localization and mapping (SLAM), incorporating bundle adjustment and key frame techniques to refine 3D poses and increase parallax. However, this also relies on iterative filtering, resulting in high processing time. Additionally, these methods often struggle with the trade-off between accuracy and processing time, making them less effective in real-time applications such as machine parking, where rapid and reliable 3D information is crucial.
[0006] Another method in this field is 3DiffTection, which leverages features from a 3D-aware diffusion model for 3D object detection from single images. This method addresses the challenge of annotating large-scale image data by fine-tuning a diffusion model to perform novel view synthesis conditioned on a single image. While 3DiffTection shows promise in bridging the gap between 2D and 3D tasks and excels in identifying cross-view point correspondences, it still faces limitations related to the computational cost and complexity of the model. Additionally, the reliance on pretrained models and the need for extensive fine-tuning can be resource-intensive and time-consuming.202501974
[0007] 2
[0008] The present invention addresses these limitations by introducing a method that calculates optical flows in specific ROIs within the detected area, focusing on minimizing noise and reducing processing time. By using previous knowledge of the 3D profile of existing machine parking slots, the method optimally sets ROIs to detect obstacles and quickly filter out irrelevant flows. A reference ROI on the ground outside the detected slot is used to enhance the robustness of obstacle detection. This approach ensures accurate and efficient 3D reconstruction, providing a reliable solution for safe autonomous parking and overcoming the critical limitations of existing SfM methods.
[0009] The disclosed solution is provided by the subject-matter of the appended independent claims. The appended dependent claims address optional embodiments of the present disclosure.
[0010] According to a first aspect of the present disclosure, there is provided a method for detecting 3D information in an area, the method comprising: acquiring data from one or more sensors mounted on a moving body; detecting an area within the acquired data; defining regions of interest (ROIs) within the detected area where one or more obstacles exist; estimating a height of each of the obstacles within the ROIs, computing a trajectory of the moving body based on the estimated heights of the obstacles; and controlling the movement of the moving body based on the computed trajectory.
[0011] By acquiring data (e.g. images) from one or more sensors (e.g. camera), detecting the area (e.g. parking slot) within the acquired data, defining ROIs for instance dynamically based on the movement of the moving body, estimating the height of the obstacles within the ROIs, computing the trajectory based on said height information, and controlling the movement based on the computed trajectory, efficient and precise execution and planning of the trajectory is achieved by reducing the processing time to one area out of many and by defining the ROIs inside said area. This integrated approach addresses the limitations of existing methods, such as high processing time and low accuracy in low parallax scenarios, providing robust and efficient 3D information detection.202501974
[0012] 3
[0013] In embodiments, the one or more sensors may comprise at least one of monocular cameras, stereo cameras, fisheye cameras, Time-of-Flight cameras, radar, and lidar. This enhances the versatility and applicability of the method by allowing the use of various types of sensors. Monocular cameras for instance provide cost-effective visual data, while stereo cameras offer depth information. Fisheye cameras capture wide-angle views, and Time-of-Flight cameras provide precise distance measurements. Radar and lidar sensors add robustness to obstacle detection. The inclusion of multiple sensor types ensures comprehensive data acquisition, improving the accuracy and reliability of 3D information detection compared to methods relying on a single sensor type.
[0014] In embodiments, detecting the area within the acquired data may comprise using machine learning models (e.g., convolutional neural networks, CNN) or rule-based algorithms to detect said area. This leverages machine learning techniques to enhance the accuracy and efficiency of area detection. CNNs for instance can process complex visual data and identify patterns with high precision, while rule-based algorithms provide reliable detection based on predefined criteria. The use of machine learning models addresses the limitations of manual annotation and low detection accuracy, by automating the detection process and improving performance in diverse environments.
[0015] In embodiments, defining the ROIs within the detected area may comprise dynamically adjusting the ROIs based on the movement of the moving body. This introduces dynamic adjustment of ROIs, ensuring real-time responsiveness to changes in the moving body's position and orientation. By continuously updating the ROIs, the method maintains accurate obstacle detection and height estimation, even in complex and changing environments. This dynamic approach addresses the limitations of static ROI definitions in existing methods, providing improved adaptability and precision.
[0016] In embodiments, the method may further comprise normalizing the apparent motion of optical flows within the ROIs to compare them effectively. This enhances the202501974
[0017] 4
[0018] accuracy of optical flow comparison by normalizing the apparent motion to a common reference frame, such as a top-down view. Normalization minimizes the effects of noise and variations in the captured data, ensuring consistent and reliable height estimation of obstacles. This improvement addresses the challenges of noise interference in existing methods, providing more accurate 3D information detection.
[0019] In embodiments, estimating the height of obstacles within the ROIs may comprise comparing the amount of optical flow in each ROI to a reference optical flow in a first reference (e.g., ground) ROI. This introduces a reference ROI to serve as a baseline for height estimation. By comparing the optical flow in other ROIs to the reference flow, the method accurately determines the relative heights of obstacles. This approach enhances the precision of height estimation, addressing the limitations of methods that struggle with low parallax scenarios and noise interference.
[0020] In embodiments, a second reference ROI may be adopted or used if the first reference ROI becomes invisible, said second reference ROI may comprise a ROI with the lowest height or the current ROI the moving body is moving on. This ensures continuous and reliable height estimation by adopting a new reference ROI if the initial ground reference becomes invisible. The second reference ROI can be chosen based on its height or the current ROI the moving body is moving on, maintaining accurate obstacle detection even as the environment changes. This improvement addresses the limitations of methods that rely on a single reference ROI, providing enhanced robustness and adaptability.
[0021] In embodiments, estimating the height of the obstacles within the ROIs may further comprise extracting representative optical flows in each ROI using flow extraction methods, including the average length, median, mode, or weighted methods based on the resolution and noise levels of the optical flows. This introduces various flow extraction methods to determine the representative optical flow within each ROI. By using statistical and computational techniques such as average length, median, mode, or weighted methods, the method ensures accurate height estimation based on the resolution and noise levels of the optical flows. This improvement addresses202501974
[0022] 5
[0023] the limitations of single-method approaches, providing enhanced accuracy and reliability.
[0024] In embodiments, the method may further comprise using optimization methods based on previously known profiles of the area to filter out incoherent optical flows. This leverages optimization techniques that use prior knowledge of the area's typical profiles to enhance obstacle detection. By applying probabilistic filtering, machine learning algorithms, or other optimization methods, the method filters out incoherent flows, improving the accuracy of height estimation. This approach addresses the limitations of methods that struggle with noise and irrelevant data, providing more reliable 3D information detection.
[0025] In embodiments, the moving body may be selected from the group consisting of a vehicle, ego-vehicle, robot, or any other mobile entity. This defines the types of moving bodies that can be used with the method, including various autonomous and mobile platforms. By accommodating different types of moving bodies, such as vehicles, ego-vehicles, robots, and other mobile entities, the method ensures broad applicability and versatility. This improvement addresses the limitations of methods restricted to specific platforms, providing enhanced flexibility.
[0026] In embodiments, the area may be a parking slot, a defined path, a workspace, or any other specified region where obstacle detection is required. By accommodating various environments that require accurate obstacle detection, the method ensures broad applicability and versatility. This improvement addresses the limitations of methods restricted to specific areas, providing enhanced flexibility.
[0027] According to another aspect of the invention, there is provided a system for detecting 3D information in an area, comprising a camera mounted on a moving body, and a processing unit configured to: detect an area within captured data of the camera, define regions of interest (ROIs) within the detected area, estimate the height of obstacles within the ROIs, compute a trajectory of the moving body based on the estimated heights of obstacles, and control the movement of the moving body based on the computed trajectory.202501974
[0028] 6
[0029] This system integrates hardware and software components to implement the method, ensuring comprehensive data acquisition, processing, and execution.
[0030] In embodiments, the system may further comprise a self-position estimation method, such as odometry, lidar, or GPS, to provide motion data for the moving body. This enhances the system's accuracy by incorporating self-position estimation methods such as odometry, lidar, or GPS. These methods provide reliable motion data, ensuring precise navigation and obstacle avoidance.
[0031] In embodiments, the processing unit may be further configured to dynamically adjust the ROIs based on the movement of the moving body. This ensures real-time responsiveness by dynamically adjusting the ROIs based on the moving body's position and orientation. The continuous update of ROIs maintains accurate obstacle detection and height estimation, even in complex and changing environments. This improvement addresses the limitations of static ROI definitions, providing enhanced adaptability and precision.
[0032] In embodiments, the processing unit may be further configured to extract optical flows of the ROIs and to use optimization methods based on previously known profiles of the area to filter out incoherent optical flows. This enhances the system's accuracy by extracting optical flows and applying optimization methods based on known profiles of the area. By filtering out incoherent flows, the system improves the reliability of height estimation and obstacle detection. This improvement addresses the limitations of methods that struggle with noise and irrelevant data, providing more reliable 3D information detection.
[0033] According to another aspect of the invention, there is provided an apparatus comprising: one or more interfaces for communication; a memory; and a data processing circuit configured to carry out the method disclosed herein.
[0034] Examples will now be further described with reference to the figures in which:202501974
[0035] 7
[0036] Fig. 1 shows a flow diagram illustrating a computer-implemented method 100 for detecting 3D information in an area.
[0037] Fig. 2 shows a schematic illustrating a system comprising a plurality of sensors used according to method 100.
[0038] Fig. 3 shows a schematic illustrating an area and ROIs according to method 100. Fig. 4 illustrates the process of normalization and profile estimation by optical flow in the context of parking slot detection and obstacle height estimation.
[0039] Fig. 5 schematically illustrates a block diagram of an embodiment of an apparatus to execute the methods described herein.
[0040] The present invention discloses a method for detecting 3D information in an area. In the context of the method for detecting 3D information in an area, "3D information" may refer to the spatial characteristics and dimensions of objects within the area, including their height, width, depth, and / or relative positions. This information may be derived from the analysis of optical flows from camera sensor, and / or other sensor data (e.g. from radar or lidar), enabling the accurate reconstruction of the three-dimensional environment. The 3D information may encompass the identification and measurement of obstacles, such as steps, gaps, pillars, walls, and other features that may pose a hazard to the movement of the ego-vehicle or mobile entity.
[0041] The term area refers to any region within which the 3D information is to be detected. This can include, but is not limited to, parking slots, defined paths, workspaces, machine parking scenarios, or any other specified regions where obstacle detection is required. For example, in the case of a parking slot, the area would be the boundaries of the parking space where the vehicle intends to park. In a defined path scenario, the area could be the route that an autonomous mobile robot is programmed to follow. In a workspace, the area might be the operational zone where a robot performs tasks, requiring precise detection of obstacles to avoid collisions.
[0042] The method described herein is applicable to various environments and scenarios beyond those explicitly mentioned. For instance, it can be used in construction sites202501974
[0043] 8
[0044] to navigate autonomous vehicles around equipment and materials, in warehouses to guide robots through aisles and storage areas, or in agricultural fields to assist autonomous machinery in avoiding obstacles such as rocks and uneven terrain. By acquiring data from sensors and processing this information to detect and measure obstacles, the method ensures safe and efficient navigation within any defined area, providing a robust solution for 3D information detection in diverse applications.
[0045] Step 110 of method 100 comprises acquiring data from one or more sensors mounted on a moving body.
[0046] The term "sensors" may refer to devices that may include monocular cameras, stereo cameras, fisheye cameras, Time-of-Flight (ToF) cameras, lidar, radar, and / or any other type of sensor capable of acquiring relevant environmental data. These sensors may be designed to capture or acquired visual information, depth information, or other types of data necessary for 3D reconstruction and obstacle detection. For example, a monocular camera may capture or acquired 2D images, while a stereo camera may provide depth information by capturing images from two slightly different perspectives.
[0047] The "moving body" may refer to any mobile entity that may be equipped with sensors for data acquisition. This may include vehicles, ego-vehicles, autonomous mobile robots, drones, or any other type of mobile platform. The moving body may be designed to navigate various environments, such as parking lots, warehouses, construction sites, or agricultural fields. For instance, an autonomous vehicle may use sensors to navigate a parking slot, while a mobile robot may use sensors to move through a workspace.
[0048] The term "acquire" may refer to the process of acquiring data from the sensors mounted on the moving body. This may involve recording images, depth information, and / or other sensor data at regular intervals as the moving body moves through the environment. Data acquisition may be continuous or periodic, depending on the requirements of the application. For example, acquiring data may202501974
[0049] 9
[0050] involve recording video frames from a monocular camera or collecting point cloud data from a lidar sensor.
[0051] The term “data” may refer to the information acquired by the sensors, which may include visual images, depth measurements, point clouds, and / or other relevant environmental information. This data may be used to detect and measure obstacles, estimate their heights, and compute the trajectory of the moving body. For instance, data captured by a stereo camera may include depth information that helps in reconstructing the 3D environment, while data from a lidar sensor may provide detailed point cloud information for obstacle detection.
[0052] Examples of acquiring data from sensors mounted on a moving body may include an autonomous vehicle using monocular and / or stereo cameras to navigate a parking slot, a mobile robot equipped with lidar and / or radar sensors to move through a warehouse, or a drone using fisheye cameras to survey an agricultural field.
[0053] Step 120 of method 100 comprises detecting an area within the acquired data. This may involve using algorithms to analyze the data and determine the boundaries of the area where the moving body is intended to operate or park or be inserted. The detection may be performed using techniques such as machine learning models (e.g. Convolutional Neural Networks (CNN)) and / or rule-based algorithms, which can process visual images, depth information, or other sensor data to identify the area. For example, detecting an area may involve identifying a parking slot within the captured images or determining the boundaries of a workspace in a warehouse.
[0054] Step 130 of method 100 comprises defining regions of interest (ROIs) within the detected area where one or more obstacles exist. This may involve segmenting, using segmentation techniques, the identified area into smaller regions where potential obstacles may be located. This step may include dynamically adjusting the ROIs based on the movement of the moving body to ensure accurate obstacle detection. The ROIs may be defined based on known profiles of the area, such as the typical locations of obstacles in a parking slot or the layout of a workspace. For202501974
[0055] 10
[0056] instance, ROIs may be defined within a parking slot to identify areas where steps, gaps, or pillars are likely to exist, or within a warehouse to detect shelves and other obstacles.
[0057] Step 140 of method 100 comprises estimating a height of each of the obstacles within the ROIs. This may involve analyzing the optical flows and / or other sensor data within each ROI to determine the vertical and / or horizontal dimensions of each of the obstacles. This step may include comparing the amount of flow in each ROI to a reference flow in a ground ROI, which may serve as a baseline for height estimation. Various flow extraction methods may be used, such as calculating the average length, median, mode, or applying weighted methods based on resolution and noise levels. For example, estimating the height of obstacles may involve determining the height of steps or pillars within a parking slot or measuring the height of shelves in a warehouse.
[0058] Step 150 of method 100 comprises computing a trajectory of the moving body based on the estimated heights of the obstacles involves planning the path that the moving body will follow to navigate through the detected area safely. This step may include using the estimated heights to avoid high obstacles and navigate through lower areas. The trajectory computation may take into account the dynamic adjustments of the ROIs and the movement patterns of the moving body. For instance, computing a trajectory may involve planning the path of an autonomous vehicle to park in a slot without hitting obstacles or determining the route for a mobile robot to move through a warehouse while avoiding shelves.
[0059] Step 160 of method 100 comprises controlling the movement of the moving body based on the computed trajectory. This may comprise executing the planned path by adjusting the movement of the moving body. This may involve sending control signals to the actuators of the moving body to follow the computed trajectory accurately. The control may be continuous or periodic, ensuring that the moving body navigates through the detected area while avoiding obstacles. For example, controlling the movement may involve steering an autonomous vehicle into a202501974
[0060] 11
[0061] parking slot or guiding a mobile robot through a warehouse, adjusting its speed and direction based on the computed trajectory.
[0062] In embodiments, defining the ROIs within the detected area comprises dynamically adjusting the ROIs based on the movement of the moving body. This means that the regions of interest (ROIs) are not static but are continuously updated as the moving body navigates through the area. The dynamic adjustment may involve recalculating the positions, sizes, and shapes of the ROIs to ensure accurate obstacle detection and height estimation. For example, as an autonomous vehicle moves into a parking slot, the ROIs may be adjusted to account for changes in the vehicle's position and orientation, ensuring that obstacles are detected in real-time.
[0063] In embodiments, the method 100 may further comprise normalizing an apparent motion of optical flows within the ROIs to compare them effectively. Normalizing the apparent motion may involve adjusting the optical flow data to a common reference frame, such as a top-down view, to facilitate comparison between different ROIs. This step helps to minimize the effects of noise and variations in the captured data, ensuring that the optical flows within each ROI can be accurately compared. For instance, normalizing the optical flows may involve transforming the data to account for the movement of the moving body, allowing for consistent and reliable height estimation of obstacles.
[0064] In embodiments, estimating the height of obstacles within the ROIs may comprise comparing the amount of optical flow in each ROI to a reference optical flow in a first reference (e.g., ground) ROI. This involves using a baseline ROI, typically located on the ground, to serve as a reference for height estimation. The optical flow in the reference ROI represents the actual motion of the moving body, and by comparing the flow in other ROIs to this reference, the relative heights of obstacles can be determined. For example, if the optical flow in an ROI is greater than the reference flow, it indicates a higher obstacle, while a smaller flow suggests a lower obstacle.
[0065] In embodiments, a second reference ROI may be adopted and / or used if the first reference ROI becomes invisible, said second reference ROI may comprise a ROI202501974
[0066] 12
[0067] with the lowest height or the current ROI the moving body is moving on. This means that if the initial ground reference ROI is no longer visible, a new reference ROI is selected to continue accurate height estimation. The second reference ROI may be chosen based on its height, with the lowest height ROI or the current ROI the moving body is moving on serving as the new reference. For instance, as the moving body progresses into a parking slot, the ground reference may become obscured, necessitating the adoption of a new reference ROI to maintain reliable obstacle detection.
[0068] In embodiments, estimating the height of the obstacles within the ROIs may further comprise extracting representative optical flows in each ROI using flow extraction methods, including the average length, median, mode, or weighted methods based on the resolution and noise levels of the optical flows. This involves using various statistical and computational techniques to determine the representative optical flow within each ROI. The average length method calculates the mean flow length, while the median and mode methods use the middle and most frequent values, respectively. Weighted methods give higher influence to flows with better resolution and lower noise. For example, in a parking slot, the representative flow may be extracted using the average length of optical flows to estimate the height of a step or pillar.
[0069] In embodiments, the method 100 may further comprise using optimization methods based on previously known profiles of the area to filter out incoherent optical flows. This means applying optimization techniques that leverage prior knowledge of the area's typical profiles to enhance the accuracy of obstacle detection. These methods may include probabilistic filtering, machine learning algorithms, or other techniques that use historical data to identify and exclude incoherent flows. For instance, in a warehouse, optimization methods may filter out optical flows that do not match the expected layout of shelves and aisles, improving the reliability of height estimation.
[0070] In embodiments, the moving body may be selected from the group consisting of a vehicle, ego-vehicle, robot, or any other mobile entity. This defines the types of202501974
[0071] 13
[0072] moving bodies that can be used with the method, including various autonomous and mobile platforms. A vehicle may refer to any car or truck, an ego-vehicle specifically denotes an autonomous vehicle, and a robot includes any mobile robotic system. Other mobile entities may encompass drones, automated guided vehicles, or any platform capable of navigating through an area. For example, the method may be applied to an autonomous car parking in a slot or a mobile robot navigating a warehouse.
[0073] In embodiments, the area may be a parking slot, a defined path, a workspace, or any other specified region where obstacle detection is required. This specifies the types of areas where the method can be applied, including various environments that require accurate obstacle detection. A parking slot may refer to a designated space for vehicle parking, a defined path may be a route for autonomous navigation, and a workspace includes areas where robots perform tasks. Other specified regions may include construction sites, agricultural fields, or any environment where obstacle detection is critical. For instance, the method may be used to navigate a vehicle into a parking slot or guide a robot through a workspace.
[0074] According to another aspect of the invention, there is provided a system for detecting 3D information in an area, comprising a camera mounted on a moving body and a processing unit configured to detect an area within captured data of the camera, define regions of interest (ROIs) within the detected area, estimate the height of obstacles within the ROIs, compute a trajectory of the moving body based on the estimated heights of obstacles, and control a movement of the moving body based on the computed trajectory. This system includes the hardware and software components necessary for implementing the method. The camera captures data, and the processing unit performs the detection, definition, estimation, computation, and control steps. For example, the system may be installed on an autonomous vehicle to navigate a parking slot or on a mobile robot to move through a warehouse.
[0075] In embodiments, the system may further comprise a self-position estimation method, such as odometry, lidar, or GPS, to provide motion data for the moving body. This means that the system includes additional sensors or methods to202501974
[0076] 14
[0077] accurately determine the position and movement of the moving body. Odometry uses wheel encoders, lidar provides detailed distance measurements, and GPS offers global positioning data. These methods ensure reliable motion data for accurate navigation. For instance, an autonomous vehicle may use GPS for positioning, while a mobile robot may use lidar for precise distance measurements.
[0078] In embodiments, the processing unit may be configured to dynamically adjust the ROIs based on the movement of the moving body. This means that the processing unit continuously updates the ROIs as the moving body navigates through the area. The dynamic adjustment ensures that obstacle detection remains accurate and relevant to the current position and orientation of the moving body. For example, as an autonomous vehicle moves into a parking slot, the processing unit adjusts the ROIs to account for changes in the vehicle's position.
[0079] In embodiments, the processing unit may be configured to extract optical flows of the ROIs and to use optimization methods based on previously known profiles of the area to filter out incoherent optical flows. This means that the processing unit performs optical flow extraction and applies optimization techniques to enhance obstacle detection. The optimization methods use historical data and known profiles to filter out irrelevant flows, improving the accuracy of height estimation. For instance, in a warehouse, the processing unit may use optimization methods to exclude optical flows that do not match the expected layout of shelves and aisles.
[0080] Fig. 2 shows a schematic illustrating a system comprising a plurality of sensors used according to method 100.
[0081] The figure illustrates two scenarios involving a moving body 200 equipped with an optical device 201 observing an object 202 on a road 203. These scenarios highlight the challenges of structure from motion (SfM) in different speed conditions.
[0082] In the high-speed scenario (Fig. 2a), the moving body 200 is traveling at a high speed (Va) on the road 203. The object 202 appears to move significantly relative to202501974
[0083] 15
[0084] the optical device 201, represented by flow 204a. This distinct flow is easily distinguishable from noise caused by wrong tracking 205 or far objects 206. The clear separation between the actual movement and noise allows for accurate estimation of the 3D information of the object 202.
[0085] In contrast, the low-speed scenario (Fig. 2b) shows the moving body 200 traveling at a much lower speed (Vb), where Vb is significantly less than Va. The apparent movement of the object 202 is much smaller, represented by flow 204b. At this lower speed, the magnitude of flow 204b becomes close to the noise level 205, making it difficult to discriminate the actual movement of the object 202 from noise. This results in challenges in accurately reconstructing the 3D information of the object 202 using SfM.
[0086] Overall, the figure demonstrates that at high speeds, the apparent movement of objects is more pronounced, making it easier to distinguish from noise and accurately estimate 3D information. Conversely, at low speeds, the minimal apparent movement makes it difficult to differentiate from noise, impacting the accuracy of SfM. This highlights the need for robust methods to handle low parallax scenarios in SfM.
[0087] The invention described herein (method 100) addresses the low parallax scenario by dynamically defining and adjusting regions of interest (ROIs) within the detected area, focusing on minimizing noise and enhancing the accuracy of 3D information detection. By normalizing the apparent motion of optical flows within these ROIs and comparing them to a reference flow in a ground RO I, the method effectively distinguishes between actual object movement and noise, even at low speeds. Additionally, the use of optimization methods based on previously known profiles of the area further filters out incoherent flows, ensuring robust height estimation of obstacles. This approach allows the system to accurately compute the trajectory of the moving body and control its movement, thereby overcoming the challenges posed by low parallax scenarios and ensuring safe navigation.202501974
[0088] 16
[0089] Fig. 3a shows a schematic illustrating a detected area according to method 100, and Fig. 3b shows the corresponding ROIs.
[0090] The figures 3a and 3b illustrates the process of defining Regions of Interest (ROIs) before the ego-vehicle starts to park in the slot. This step is useful for safety and accuracy, ensuring that potential obstacles are identified before the vehicle moves into the identified slot area 310. The ego-vehicle (not shown) is approaching a detected parking slot. The ROIs are defined as follows:
[0091] ROI1 (Ground Reference) 320: This region is set between the detected slot and the ego-vehicle. It serves as a reference for comparing optical flows and estimating the height of obstacles. The ground reference ROI is crucial for establishing a baseline flow that represents the actual motion of the ego-vehicle.
[0092] ROI2 to ROI6 330: These regions are inside the detected slot where potential obstacles, such as steps or walls, may exist. The number of ROIs can be adjusted as necessary, but for simplicity, six ROIs are shown in the figure. These ROIs are strategically placed to cover areas where obstacles are likely to be present, ensuring comprehensive detection.
[0093] The method described herein (method 100) involves several key steps that are illustrated by this figure:
[0094] Acquiring Data: The ego-vehicle captures data from its sensors as it approaches the parking slot 310. This data includes visual information and other sensor readings necessary for detecting the slot and defining ROIs 320, 330.
[0095] Detecting the Area: The parking slot 310 is detected within the acquired data using algorithms such as Convolutional Neural Networks (CNN) or rule-based methods. Once the slot is detected and chosen by the user, the process of defining ROIs begins.202501974
[0096] 17
[0097] Defining ROIs: Before the ego-vehicle starts moving into the slot, ROIs 320, 330 are defined within the detected area. ROI1 320 is set as the ground reference, while ROI2 to ROI6 330 are placed inside the slot where obstacles may exist. This step ensures that the ROIs are optimally positioned to detect potential obstacles.
[0098] Estimating Heights: The height of obstacles within each ROI is estimated by comparing the optical flows in these regions to the reference flow in ROI1. This comparison helps to identify areas with higher or lower obstacles, ensuring accurate 3D reconstruction. For instance, the height of elevated metal boundaries in ROI2 and ROI6 may be identified.
[0099] Computing Trajectory: Based on the estimated heights of obstacles, the trajectory of the ego-vehicle is computed. This trajectory is planned to navigate the vehicle safely into the parking slot, avoiding any detected obstacles, e.g. the metal boundaries in ROI2 and ROI6.
[0100] Controlling Movement: The movement of the ego-vehicle is controlled based on the computed trajectory. This ensures that the vehicle follows the planned path and parks safely without colliding with obstacles.
[0101] The trajectory of the ego-vehicle can also include rotations, which may be accounted for when setting the ROIs. This dynamic adjustment of ROIs ensures that obstacle detection remains accurate even as the vehicle changes its orientation.
[0102] Overall, the figures 3a and 3b show how the method described in the claims handles the process of defining ROIs and estimating obstacle heights to ensure safe and efficient parking of the ego-vehicle.
[0103] Fig. 4 illustrates the process of normalization and profile estimation by optical flow in the context of parking slot detection and obstacle height estimation.
[0104] The top part of Fig. 4 (Normalization (Top View)) shows a parking area 410 with an ego-vehicle approaching it. A rectangular region 410 indicates the application of a202501974
[0105] 18
[0106] top view normalization process. This normalization step ensures that the apparent motion of optical flows can be compared effectively. By applying a top view, the optical flows in the Regions of Interest (ROIs) will only differ from each other if there is a height difference, indicating the presence of obstacles such as steps, gaps, or pillars.
[0107] To the right, there is a diagram showing six top view ROIs 420 labeled ROI1 to ROI6, arranged in parallel strips. ROI1 is set as the ground reference region between the detected slot and the ego-vehicle. ROIs 2 to 6 are defined within the detected slot where potential obstacles may exist. This setup allows for the comparison of optical flows in these regions to detect height differences.
[0108] The bottom part (Profile Estimation by Optical Flow (Top View)) of Fig. 4 focuses on the profile estimation of the detected slot using optical flow analysis. The process involves extracting and tracking features within the ROIs to minimize the influence of noise. When the moving body (e.g. ego vehicle) is approaching the defined area (e.g. parking slot), for each time instance t1 , 12, and t3, the optical flows in each ROI (ROI2 to ROI6) are compared to the reference flow in ROH (e.g. ground flow), which represents the actual motion of the ego-vehicle (e.g. estimated by odometry or any self-localization technique).
[0109] The comparison helps determine the relative heights of obstacles within the ROIs, at each time step. If the optical flow in an ROI is higher than the reference flow in ROH , it indicates a higher area (represented by a longer arrow 430). Conversely, a smaller flow suggests a lower area relative to the ground. The bottom right figure corresponds to time instance t3 and shows the height of the ROIs represented by the length of the arrows of the obtained flows: same-length arrows at the bottom denote the reference height of the reference ground flow, the short arrows with discs indicate small-height obstacles, the longer arrows with squares indicate longer obstacles, and the longest arrows with stars indicate the obstacles with the most height. This estimates a relative height profile for the flow in each ROI.202501974
[0110] 19
[0111] Different methods can be applied to extract the representative amount of flow in each ROI, such as using the average length, median, mode, or weighted methods based on resolution and noise levels. Optimization methods based on previously known profiles of the existing slot can also be used to filter out incoherent flows. For example, parking slots are often symmetrical and continuous in the longitudinal direction, allowing the method to filter out probabilistically incoherent flows.
[0112] The estimated relative height profile provides sufficient information for safe parking, as the ego-vehicle can navigate within the lowest areas and avoid higher ones. The invention can further estimate the real height of each flow using the actual displacement of the ego-vehicle measured by odometry and converting pixels to meters by geometry. If the ground reference ROI becomes invisible as the vehicle moves into the slot, a new reference ROI can be adopted and used, such as the ROI with the lowest height or the current ROI the ego-vehicle is moving on. This detailed process ensures accurate and efficient 3D reconstruction of the parking slot, enabling safe navigation and obstacle avoidance for the ego-vehicle.
[0113] In the present disclosure, any feature described in an embodiment, example, and / or method (e.g. method 100) or method step, may be combined with any feature of another embodiment, example, and / or method step, unless otherwise specified.
[0114] Fig. 5 schematically illustrates a block diagram of an embodiment of an apparatus to execute the methods described herein.
[0115] The apparatus 500 comprising one or more interfaces 510 for communication and one or more processors or data processing circuits 520, configured to execute any one of the methods described herein. The apparatus further comprises a memory or a storage medium 530.
[0116] The storage medium 530 (or a data carrier, or a computer-readable data carrier, or a computer-readable medium) may comprise, stored thereon, a computer program or computer-executable instructions for performing one of the methods described herein when it is performed by one or more processors 520. The processors 520,202501974
[0117] 20
[0118] may communicate with the storage medium 530 directly and / or via the one or more interfaces 510.
[0119] In embodiments, the one or more interfaces 510 may comprise wired and / or wireless interfaces for transmitting and / or receiving communication signals in connection with the execution of the proposed concept. In practice, the interfaces may comprise at least one of: pins, wires, antennas, a plurality of cameras 515 and / or other sensors (e.g. lidar, radar) 515. As well, the interfaces may comprise means for (analog and / or digital) signal or data processing in connection with the communication, e.g., filters, samples, analog-to-digital converters, signal acquisition and / or reconstruction means as well as signal amplifiers, compressors and / or any encryption / decryption means.
[0120] The data processing circuit 520, may correspond to or comprise any type of programable hardware. So, examples of the data processing circuit 520, e.g., comprise a memory, microcontroller, field programmable gate arrays, one or more central and / or graphical processing units. To execute the proposed method, the data processing circuit 520 may be configured to access or retrieve an appropriate computer program for the execution of the proposed method from a memory of the data processing circuit 520 or a separate memory which is communicatively coupled to the data processing circuit 520.
[0121] Some or all of the method steps described above with regard to Figures 1 to 4 may be implemented by a computer in that they are executed by (or using) a processor, a microprocessor, an electronic circuit, or the data processing circuit 520. For example, the implementation can be performed using a non-transitory storage medium such as a computer-readable storage medium or data carrier. Such computer-readable media include any media that can be accessed by either a general-purpose or a specialized computer system.
[0122] Generally, the methods described herein can be implemented as a computer program product with a program code or computer-executable instructions, the program code or computer-executable instructions being operative for performing202501974
[0123] 21
[0124] one of the methods when the computer program products run on a computer. The program code or the computer-executable instructions may, for example, be stored on the computer-readable storage medium 530.
[0125] The apparatus 500 may also include a cloud implementation, which enhances its capabilities by leveraging cloud-based resources. This cloud implementation comprises several key components. The cloud implementation may comprise cloud interfaces 510’ designed for communication between the local apparatus and the cloud. These interfaces enable data transfer, remote access, and integration with cloud services, ensuring seamless connectivity.
[0126] Additionally, the cloud implementation may include cloud processors or data processing circuits 520’. These may be processing units located in the cloud, configured to execute any of the methods described for the apparatus. By utilizing cloud processors, the system can handle more complex computations and larger datasets without being limited by the local hardware's capabilities. This allows for greater flexibility and efficiency in processing tasks.
[0127] Furthermore, the cloud implementation may include a cloud memory or storage medium 530’ that may be responsible for storing data, datasets, and program / method code in the cloud. Cloud storage offers scalability, redundancy, and accessibility, ensuring that data is securely stored and can be accessed from anywhere. This enhances the overall reliability and availability of the system.
[0128] The cloud implementation allows for flexibility in how the methods are executed. Specifically, any of the methods described herein can be partially executed in the cloud, where some parts of the method may be processed locally while others are offloaded to the cloud. This hybrid approach optimizes performance and resource utilization. Alternatively, the entire method can be fully executed in the cloud, leveraging the cloud's extensive computational power and storage capabilities. This dual implementation (local and cloud) ensures that the apparatus can efficiently handle a wide range of tasks, from simple local processing to complex cloud-based computations, providing a robust and scalable solution.
Claims
20250197422Patent Claims:
1. A method for detecting 3D information in an area, comprising:acquiring (110) data from one or more sensors mounted on a moving body; detecting (120) an area within the acquired data;defining (130) regions of interest, ROIs, within the detected area where one or more obstacles exist;estimating (140) a height of each of the obstacles within the ROIs; computing (150) a trajectory of the moving body based on the estimated heights of the obstacles; andcontrolling (160) a movement of the moving body based on the computed trajectory.
2. The method of claim 1, wherein the one or more sensors comprise at least one of monocular cameras, stereo cameras, fisheye cameras, Time-of-Flight cameras, radar, and lidar.
3. The method of claim 1 or 2, wherein detecting the area within the captured data comprises using machine learning models or rule-based algorithms to detect said area.
4. The method of any previous claim, wherein defining the ROIs within the detected area comprises dynamically adjusting the ROIs based on the movement of the moving body.
5. The method of any previous claim, further comprising normalizing an apparent motion of optical flows within the ROIs to compare them effectively.
6. The method of any previous claim, wherein estimating the height of each of the obstacles within the ROIs comprises comparing the amount of optical flow in each RO I to a reference optical flow in a first reference RO I.
7. The method of claim 6, wherein a second reference ROI is used if the first reference ROI becomes invisible, said second reference ROI comprises a ROI with the lowest height or the current ROI the moving body is moving on.202501974238. The method of any claim 6 or 7, wherein estimating the height of each of the obstacles within the ROIs further comprises extracting representative optical flows in each ROI using flow extraction methods, including the average length, median, mode, or weighted methods based on the resolution and noise levels of the optical flows.
9. The method of any one of claims 6 to 8, further comprising using optimization methods based on previously known profiles of the area to filter out incoherent optical flows.
10. The method of any previous claim, wherein the moving body is selected from the group consisting of a vehicle, ego-vehicle, robot, or any other mobile entity.
11. The method of any previous claim, wherein the area is a parking slot, a defined path, a workspace, or any other specified region where obstacle detection is required.
12. A system for detecting 3D information in an area, comprising:a camera mounted on a moving body;a processing unit configured to:detect an area within data acquired by the camera;define regions of interest, ROIs, within the detected area;estimate the height of obstacles within the ROIs;compute a trajectory of the moving body based on the estimated heights of obstacles; andcontrol a movement of the moving body based on the computed trajectory.
13. The system of claim 12, further comprising a self-position estimation method, such as odometry, lidar, or GPS, to provide motion data for the moving body.
14. The system of claim 12 or 13, wherein the processing unit is configured to dynamically adjust the ROIs based on the movement of the moving body.2025019742415. The system of any one of claims 12 to 14, wherein the processing unit is configured to extract optical flows of the RO Is and to use optimization methods based on previously known profiles of the area to filter out incoherent optical flows.
