Apparatus for controlling vehicle and method thereof

Abstract

The invention relates to an apparatus for controlling a vehicle and a method thereof. The apparatus includes a camera, a processor, and a memory, the apparatus configured to identify whether a vehicle enters a point in an indoor environment from a point in an outdoor environment, obtain a point cloud of at least one object identified using the camera based on identifying that the vehicle enters the point in the indoor environment, generate an indoor environment map using the point cloud, and transmit the generated indoor environment map to a server. The indoor environment map represents at least a portion of the indoor environment along a movement path of the vehicle in the indoor environment, and a point in the indoor environment is determined as a starting point of the indoor environment map by mapping the point in the outdoor environment and the point in the indoor environment using an outdoor environment map representing the outdoor environment.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of priority to Korean Patent Application No. 10-2024-0060625, filed with the Korean Intellectual Property Office on May 8, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to devices and methods for controlling vehicles, and more specifically, to techniques for generating maps and controlling vehicles based on those maps. Background Technology

[0004] A scheme for identifying the location of moving objects such as vehicles and creating a map of their surroundings is called Simultaneous Localization and Mapping (SLAM). The location of a vehicle can be estimated by creating a map of the surrounding environment using sensors via SLAM. The estimated vehicle location (such as a vehicle and / or equipment on it) can be corrected based on loop closure (e.g., loop closure detection). In cases where moving objects such as vehicles are captured by a camera in two-dimensional images, there is a need to investigate schemes for correcting the estimated vehicle location based on loop closure detection. Summary of the Invention

[0005] The following overview presents a simplified overview of some features. This overview is not exhaustive and is not intended to identify key or critical elements.

[0006] Systems, devices, and methods for controlling a vehicle are described. One device for a vehicle may include: a camera; one or more processors; and a memory storing instructions configured, when executed by the one or more processors, to cause the device to: identify whether the vehicle has moved from a first outdoor point in an outdoor environment to a first indoor point in an indoor environment; based on identifying that the vehicle has moved to the first indoor point in the indoor environment, obtain a point cloud of at least one indoor object identified in an image acquired using the camera; based on the point cloud, generate an indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path; based on the outdoor environment map representing the outdoor environment, map the first outdoor point in the outdoor environment to the first indoor point in the indoor environment; based on the mapping from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment, determine the first indoor point in the indoor environment as the starting point of the indoor environment map; and control the operation of the vehicle based on the generated indoor environment map.

[0007] Alternatively, a method (e.g., for controlling a vehicle and / or performed by the vehicle and / or devices of the vehicle) may include: identifying whether the vehicle has moved from a first outdoor point in an outdoor environment to a first indoor point in an indoor environment; obtaining a point cloud of at least one indoor object identified in an image acquired using a camera, based on identifying that the vehicle has moved to the first indoor point in the indoor environment; generating an indoor environment map based on the point cloud, the indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path; mapping the first outdoor point in the outdoor environment to the first indoor point in the indoor environment based on an outdoor environment map representing the outdoor environment; determining the first indoor point in the indoor environment as the starting point of the indoor environment map based on the mapping from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment; and controlling the operation of the vehicle based on the generated indoor environment map.

[0008] These and other features and advantages are described in more detail below. Attached Figure Description

[0009] The above and other objects, features, and advantages of this disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0010] Figure 1 This is a block diagram illustrating an embodiment of a vehicle control device according to an embodiment of the present disclosure;

[0011] Figure 2A and Figure 2B This is a diagram illustrating an embodiment of the operation of a vehicle control device identifying the location of a vehicle according to an embodiment of the present disclosure;

[0012] Figure 3 This is a diagram illustrating an embodiment of the operation of a vehicle control device calibrating an indoor environment map according to an embodiment of the present disclosure;

[0013] Figure 4 This is a flowchart illustrating an embodiment of the operation of a vehicle control device according to an embodiment of the present disclosure;

[0014] Figure 5 This is a diagram illustrating an embodiment of the operation of a vehicle control device that selects at least one indoor environment map from a plurality of indoor environment maps according to an embodiment of the present disclosure;

[0015] Figure 6 This is a flowchart illustrating a method for controlling a vehicle according to embodiments of the present disclosure; and

[0016] Figure 7 This is a block diagram illustrating a computing system or method for controlling a vehicle in relation to a vehicle control device according to embodiments of the present disclosure. Detailed Implementation

[0017] In the following, some embodiments of the present disclosure will be described in detail with reference to the exemplary accompanying drawings. Furthermore, when adding reference numerals to the components of the various drawings, note that identical or equivalent components are designated by the same numerals, even if these identical or equivalent components are shown in other drawings. Additionally, in describing embodiments of the present disclosure, detailed descriptions of related known configurations or functions will be omitted if it is determined that such descriptions would interfere with the understanding of the embodiments of the present disclosure.

[0018] Furthermore, or alternatively, terms such as first, second, A, B, (a), (b), etc., may be used herein when describing components of this disclosure. These terms are used only to distinguish elements from other elements, and the nature, order, sequence, and number of elements are not limited by these terms. Additionally, or alternatively, unless otherwise defined, all terms used herein (including technical or scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in commonly used dictionaries shall be interpreted as having a meaning consistent with the context of the relevant art and shall not be interpreted as having an ideal or overly formal meaning unless clearly defined in the description of this disclosure.

[0019] As used herein, the term "module" can include units implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "component," or "circuit." A module can be a single integrated component, or a minimum unit or portion thereof, adapted to perform one or more functions. According to embodiments, modules may be implemented in the form of application-specific integrated circuits (ASICs). According to various embodiments, operations performed by a module, program, or other component may be performed sequentially, in parallel, or repeatedly, or one or more operations may be performed in a different order, omitted, or one or more additional operations may be added.

[0020] The various embodiments described herein can be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., internal or external memory) readable by a machine (e.g., a computing device, such as device 100 for controlling a vehicle). For example, one or more processors (e.g., processor 110) of the machine (e.g., device 100 for controlling a vehicle) can invoke at least one of the one or more instructions stored in the storage medium and execute it with or without one or more other components under the control of the processor. This allows the machine to be operated to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. Machine-readable storage media may be provided in the form of non-transitory storage media. The term "non-transitory" simply means that the storage medium is a tangible device and does not include the signal (e.g., electromagnetic wave) itself. "Non-transitory" does not distinguish between data that is semi-permanently stored in the storage medium and data that is temporarily stored in the storage medium.

[0021] In the following text, reference will be made to Figures 1 to 7 The embodiments of this disclosure are described in detail.

[0022] Figure 1 This is a block diagram illustrating an embodiment of a vehicle control device according to an embodiment of the present disclosure.

[0023] See Figure 1 The vehicle control device 100 according to embodiments of the present disclosure can be implemented inside and / or outside a vehicle. Some components included in the vehicle control device 100 can be implemented inside and / or outside the vehicle. The vehicle control device 100 may be integrally formed with an in-vehicle control unit, and / or may be implemented as a separate device connected to / communicating with the vehicle's control unit (e.g., via a separate connection device). For example, the vehicle control device 100 may also include... Figure 1 Components not shown in the diagram.

[0024] The vehicle control device 100 according to an embodiment may include at least one of a processor 110, a memory 120, a sensor 130, a camera 150, and / or a communication circuit 160. The processor 110, memory 120, sensor 130, camera 150, and communication circuit 160 may be electrically connected and / or operatively coupled to each other via electronic components including a communication bus. In an embodiment, the camera 150 may be part of and / or included in the sensor 130 (e.g., a camera, blind spot monitoring sensor, lane departure warning sensor, parking sensor, light sensor, rain sensor, traction control sensor, anti-lock braking system sensor, tire pressure monitoring sensor, seatbelt sensor, airbag sensor, fuel sensor, emission sensor, throttle position sensor, etc.). Hereinafter, operatively coupled hardware may mean a direct or indirect connection between the hardware components established wired or wirelessly, such that a second piece of hardware is controlled by a first piece of hardware. Although illustrated based on different blocks, this disclosure is not limited thereto, and Figure 1 Some of the hardware (e.g., at least a portion of processor 110, memory 120, and communication circuitry 160) may be included in a single integrated circuit (such as a system-on-a-chip (SoC)).

[0025] The processor 110 of the vehicle control device 100 according to an embodiment may include hardware components for processing data based on one or more instructions. For example, the hardware components for processing data may include an arithmetic logic unit (ALU), a floating-point unit (FPU), a field-programmable gate array (FPGA), a central processing unit (CPU), a microcontroller unit (MCU), and / or an application processor (AP). The number of processors 110 may be one or more. For example, the processor 110 may have a multi-core processor architecture, including dual-core, quad-core, hexa-core, or octa-core processors.

[0026] The memory 120 of the vehicle control device 100 according to an embodiment may include hardware components for storing data and / or instructions input to and / or output to the processor 110. For example, the memory 120 may include volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM)). For example, the non-volatile memory may include at least one of programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), flash memory, hard disk, compact disk, and embedded multimedia card (eMMC). The processor 110 and / or memory 120 may be associated with a fuel cell system for controlling the fuel cell and / or managing the temperature.

[0027] According to an embodiment, the sensor 130 of the vehicle control device 100 can generate electrical information that will be processed by the processor 110 and / or memory 120 of the vehicle control device 100 based on non-electronic information related to the vehicle control device 100.

[0028] In an embodiment, sensor 130 may include one or more sensors. For example, sensor 130 may be attached to different locations on the vehicle. Sensor 130 may face one or more different directions (e.g., configured to sense or receive signals / information from one or more different directions). For example, sensor 130 may be attached to the front, side, rear, and / or roof of the vehicle to face various directions, such as forward, rearward, and sideways. However, this disclosure is not limited thereto.

[0029] In an embodiment, sensor 130 may include an image sensor, such as a camera (e.g., a high dynamic range camera). Sensor 130 may include one or more non-visual sensors. For example, sensor 130 may include radar, light detection and ranging (LiDAR), and / or ultrasonic sensors as a complement to or alternative to the image sensor.

[0030] In embodiments, sensor 130 may include attitude sensors (e.g., yaw, roll, and / or pitch sensors), collision sensors, wheel sensors, speed sensors, tilt sensors, weight sensors, heading sensors, gyroscope sensors, accelerometer sensors, inertial measurement units (IMUs), position modules, vehicle forward / reverse sensors, battery sensors, fuel sensors, tire sensors, steering sensors rotated via a steering wheel, vehicle interior temperature sensors, vehicle interior humidity sensors, ultrasonic sensors, illuminance sensors, accelerator pedal position sensors, and / or brake pedal position sensors. For example, the IMU sensor may include means that can use a combination of accelerometers, gyroscopes, and / or magnetometers to measure and / or report specific forces, angular velocities, and / or magnetic fields of a subject. IMU sensors can track the movement and orientation of an object in 3D space, providing data on acceleration, rotation, and sometimes orientation. IMUs can be used in applications requiring precise motion tracking and stability, such as smartphones, drones, virtual reality systems, and / or autonomous vehicles. By integrating this motion data, IMUs enable devices to navigate, stabilize, and interact with their environment more effectively.

[0031] For example, vehicle control device 100 can obtain vehicle attitude information, vehicle collision information, vehicle direction information, vehicle position information (e.g., Global Positioning System (GPS) information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, headlight information, vehicle interior temperature information, vehicle interior humidity information, and sensing data about steering wheel rotation angle, vehicle exterior illuminance, pressure applied to accelerator pedal and / or pressure applied to brake pedal via sensor 130.

[0032] The vehicle control device 100 according to an embodiment can control notification systems, including warning systems for notifying the driver of driving events such as approaching a destination or a potential collision. For example, the vehicle control device 100 can control sensors 130 of the vehicle. For example, the vehicle control device 100 can modify the orientation of sensor 130. The vehicle control device 100 can change the output resolution and / or format type of sensor 130. The vehicle control device 100 can change (e.g., increase or decrease) the capture rate. The vehicle control device 100 can adjust the dynamic range of sensor 130. The vehicle control device 100 can control (e.g., turn on or off) the operation of sensor 130 individually or collectively.

[0033] According to an embodiment, the vehicle control device 100 can perform deep learning analysis on sensor data received from sensor 130. The vehicle control device 100 can be coupled to a memory 120 via an input / output interface, the memory 120 being configured to provide instructions to processes that cause the determination of deep learning results for at least partially autonomous operation of the vehicle. For example, the vehicle control device 100 can process commands for vehicle control output from processor 110. To control various modules of the vehicle, the vehicle control device 100 can translate the output of processor 110 into commands for controlling the modules of the vehicle. One or more features and / or operations described herein can be used to control the autonomous driving of the vehicle. For example, an indoor environment map and / or the starting point of the indoor environment map can be used for autonomous driving control of the vehicle.

[0034] According to the Society of Automotive Engineers (SAE), the automation levels of autonomous vehicles can be classified as follows: Level 0, corresponding to "no automation," involves the autonomous driving system temporarily intervening and / or only providing warnings (e.g., blind spot warning, lane departure warning, etc.) in emergency situations (e.g., automatic emergency braking), and expects the driver to operate the vehicle. Level 1, corresponding to "driver assistance," involves the system performing some driving functions (e.g., steering, acceleration, braking, lane centering, adaptive cruise control, etc.), while the driver operates the vehicle within normal operating ranges, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. Level 2, corresponding to "partial automation," involves the system performing steering, acceleration, and / or braking under driver supervision, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. At Level 3 of autonomous driving, the SAE classification standard can correspond to "conditional automation," where the system drives the vehicle under limited conditions (e.g., performing driving functions such as steering, acceleration, and / or braking), but transfers driving control to the driver when the required conditions are not met. The driver is expected to determine the system's operating state and / or timing, and take over control in emergency situations, but is not required to operate the vehicle in other situations (e.g., steering, acceleration, and / or braking). At Level 4 of autonomous driving, the SAE classification standard can correspond to "high automation," where the system performs all driving functions and the driver is expected to control the vehicle only in emergency situations. At Level 5 of autonomous driving, the SAE classification standard can correspond to "full automation," where the system performs full driving functions in all situations, including emergency situations, without any assistance from the driver, and the driver is not expected to perform any driving functions other than determining the system's operating state. While this disclosure can apply the SAE classification standard to autonomous driving classification, other classification methods and / or algorithms can be used in one or more configurations described herein. One or more features associated with autonomous driving control can be activated based on the configured autonomous driving control settings (e.g., based on at least one of the following: autonomous driving classification, selection of the vehicle's autonomous driving level, etc.).

[0035] According to an embodiment, the vehicle control device 100 may use the sensor 130 to obtain sensor information about the vehicle's position, the vehicle's movement path, the vehicle's attitude, and / or at least one object located around the vehicle.

[0036] For example, after a vehicle enters a point in an indoor environment (e.g., based on the vehicle's entry into that point in the indoor environment), the vehicle control device 100 can use the sensor 130 to identify sensor information about the vehicle's position in the indoor environment, the vehicle's path of movement, and / or the vehicle's attitude.

[0037] For example, vehicle control device 100 can identify at least one object via camera 150 based on sensor information obtained by using sensor 130, according to the vehicle's position, the vehicle's movement path, and / or the vehicle's attitude.

[0038] For example, vehicle control device 100 may acquire a point cloud of at least one object. For example, vehicle control device 100 may acquire the point cloud based on the execution of a point cloud information generator. Vehicle control device 100 may perform feature detection or simultaneous localization and mapping (SLAM) on images acquired via a camera based on the execution of the point cloud information generator. Vehicle control device 100 may use the point cloud to generate an indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path. For example, the indoor environment may include an indoor parking lot for parking.

[0039] According to an embodiment, the camera 150 of the vehicle control device 100 may include one or more optical sensors (e.g., charge-coupled device (CCD) sensors or complementary metal-oxide-semiconductor (CMOS) sensors) that generate electrical signals representing light input (e.g., the color and / or brightness of light). Multiple optical sensors included in the camera 150 may be arranged in a two-dimensional array. The camera 150 may acquire electrical signals from each of the multiple optical sensors substantially simultaneously and generate two-dimensional frame data corresponding to the light arriving at the optical sensors of the two-dimensional grid. For example, photographic data captured using the camera 150 may mean one or more two-dimensional frame data obtained from the camera 150. For example, video data captured using the camera 150 may mean a sequence of multiple two-dimensional frame data obtained from the camera 150 according to the frame rate. The camera 150 may be positioned facing forward of the vehicle. By being arranged to face forward of the vehicle, the camera 150 can acquire an image representing the outdoor environment corresponding to the front of the vehicle. The camera 150 may include a black box camera (driving recorder) based on acquiring an image representing the outdoor environment. Camera 150 may include a depth camera for identifying distances (e.g., a depth map) between the vehicle and at least one object located around the vehicle. The depth map may be an image (or image channels) containing information related to distances from a viewpoint to the surfaces of one or more objects. A depth map can be presented by acquiring multiple images from one or more viewpoints and determining distances from one or more pixels to one or more image sensors (e.g., cameras). For a depth camera (e.g., an RGB-depth camera), it may include means for simultaneously capturing both color images (RGB) and depth information. A depth camera combines the capabilities of a conventional color camera with the capabilities of a depth sensor that uses red, green, and blue channels to capture the visual appearance of a scene, and a depth sensor that measures the distances between the camera and objects in the scene. This dual data capture enables the camera to create a 3D representation of the environment, making it useful for applications such as 3D modeling, robotics, augmented reality, and / or gesture recognition. An RGB-depth camera can be useful when both color and spatial structure of a scene need to be understood.

[0040] According to an embodiment, the vehicle control device 100 can use a camera 150 (e.g., an image obtained / acquired via the camera 150) to identify when a vehicle enters an indoor environment from a point in the outdoor environment (e.g., by moving from a first point (the point in the outdoor environment) to a second point (the point in the indoor environment)). For example, the vehicle control device 100 can use the camera 150 to identify the point in the indoor environment from a point in the outdoor environment based on (e.g., by) identifying entrance and / or exit information corresponding to that point in the indoor environment.

[0041] For example, entrance and / or exit information may include information about outdoor objects (e.g., height restriction signs indicating height restrictions for vehicles that may enter the indoor environment, barriers to temporarily prevent vehicles from entering the indoor environment, and / or speed bumps to reduce vehicle speed).

[0042] For example, vehicle control device 100 can use outdoor environment map 125 to identify a point in the indoor environment from which a vehicle deviates from an outdoor road. However, this disclosure is not limited thereto.

[0043] According to an embodiment, the vehicle control device 100 can identify at least one indoor object in an indoor environment based on a point where the vehicle enters the indoor environment by using a camera 150 (e.g., based on an image obtained / acquired by the camera 150).

[0044] For example, vehicle control device 100 can identify at least one object by using image recognition and / or feature detection on an image representing at least a portion of the indoor environment obtained using camera 150.

[0045] For example, vehicle control device 100 can acquire / determine / receive a point cloud of at least one object. A point cloud can represent a set of points located in three-dimensional space. However, this disclosure is not limited thereto. For example, vehicle control device 100 can use sensor 130 to acquire a point cloud of at least one object located around the vehicle. For example, the point cloud can include information about at least one object (e.g., location or type). For example, a point cloud can include a set of data points in a three-dimensional coordinate system representing the outer surface of an object or environment. Each point in the cloud may have its own set of X, Y, and Z coordinates, and / or additional information (e.g., color or intensity). Point clouds are typically generated by 3D scanners, LiDAR, or photogrammetry techniques and can be used in a variety of applications such as 3D modeling, computer vision, and / or robotics. They can provide highly detailed and / or accurate representations of complex surfaces and / or structures, making them ideal for tasks such as object recognition, environment mapping, and / or digital reconstruction.

[0046] For example, a point cloud can include feature points. The vehicle control device 100 can obtain feature points of at least one object by object recognition in an image obtained using the camera 150.

[0047] According to an embodiment, the vehicle control device 100 can generate an indoor environment map based on / using point clouds, the indoor environment map representing at least a portion of the vehicle's movement path within the indoor environment.

[0048] For example, vehicle control device 100 may generate an indoor environment map based on / via Simultaneous Localization and Mapping (SLAM). The indoor environment map may be generated based on / via SLAM according to the vehicle's movement path. For example, vehicle control device 100 may generate an indoor environment map based on / via the execution of a map data generator. However, this disclosure is not limited thereto. Vehicle control device 100 may obtain an indoor environment map by sending data obtained from the execution of a parking lot entrance determination and time processor and / or point cloud information generator to an external server including a map data generator.

[0049] For example, vehicle control device 100 can obtain virtual objects representing at least one object from a point cloud based on a grid. Vehicle control device 100 can generate a 3D-based map 127 of the indoor environment that includes the virtual objects.

[0050] For example, vehicle control device 100 may obtain virtual objects based on a mesh to visually represent a point cloud. The virtual objects may include color information. Vehicle control device 100 may represent the virtual objects in three-dimensional space based on a mesh. For example, virtual objects obtained based on a mesh may include virtual objects with shapes such as vertices, edges, and / or polygons. However, this disclosure is not limited thereto.

[0051] According to an embodiment, after generating an indoor environment map / based on the generated indoor environment map, the vehicle control device 100 may temporarily stop acquiring point clouds based on the identification that the vehicle has moved to another point (e.g., a third point) in the outdoor environment.

[0052] For example, vehicle control device 100 may identify that the vehicle has moved to another point in the outdoor environment based on received GPS signals. For example, vehicle control device 100 may identify that the vehicle has moved to another point in the outdoor environment by / based on the use of a camera to identify objects (e.g., obstacles or roads) that represent another point in the outdoor environment. However, this disclosure is not limited thereto.

[0053] For example, vehicle control device 100 may identify the start and / or end point of indoor environment map 127 based on a generated indoor environment map (e.g., based on the generated indoor environment map). Vehicle control device 100 may use outdoor environment map 125 to identify the start and end point of indoor environment map 127.

[0054] According to an embodiment, the vehicle control device 100 can use an outdoor environment map 125 representing the outdoor environment to map points in the outdoor environment and / or points in the indoor environment, thereby determining a point in the indoor environment as the starting point of the indoor environment map 127.

[0055] For example, vehicle control device 100 can use outdoor environment map 125 to identify the coordinates representing a point in the indoor environment. Vehicle control device 100 can identify the starting point of indoor environment map 127 by using / based on the coordinates indicating a point in the indoor environment. For example, a point in the indoor environment may include a parking lot entrance.

[0056] For example, vehicle control device 100 can determine a point in the indoor environment as the starting point of indoor environment map 127 by matching the coordinates of a point in the indoor environment with the coordinates of a point in the outdoor environment. For example, vehicle control device 100 can map points in the outdoor environment and / or the indoor environment based on optimization (e.g., by applying an optimization algorithm to map points in the outdoor environment to points in the indoor environment). For example, based on optimization, vehicle control device 100 can combine data representing indoor environment map 127 with data representing outdoor environment map 125 to correct / modify positions in indoor environment map 127 based on outdoor environment map 125. However, this disclosure is not limited thereto.

[0057] According to an embodiment, the vehicle control device 100 can generate a virtual route that includes points in the outdoor environment by using an outdoor environment map 125. For example, the vehicle control device 100 can generate a virtual route based on the shortest route among multiple routes that include points (e.g., at least two points) in the outdoor environment. However, this disclosure is not limited thereto.

[0058] According to an embodiment, the vehicle control device 100 can obtain a closed curve that includes the virtual route and the vehicle's movement path based on loop closure (e.g., loop closure detection / loop closure algorithm).

[0059] According to an embodiment, the vehicle control device 100 can determine the point in the indoor environment as the starting point of the indoor environment map 127 by correcting a point in the indoor environment using a point in the outdoor environment.

[0060] For example, the vehicle control device 100 can determine the point in the indoor environment as the starting point of the indoor environment map 127 by matching the coordinates of the point in the indoor environment with the coordinates of the point in the indoor environment map 127.

[0061] According to an embodiment, the vehicle control device 100 can identify that a vehicle has moved from one point in an indoor environment to another point in an outdoor environment. For example, the vehicle control device 100 can temporarily stop acquiring point clouds of at least one object based on the identification that the vehicle has moved to another point in the outdoor environment. However, this disclosure is not limited thereto.

[0062] According to an embodiment, the vehicle control device 100 can map another point in the outdoor environment to another point in the indoor environment based on a closed loop using an outdoor environment map 125.

[0063] For example, if the first point in the indoor environment (the first indoor point) is the entrance to the indoor environment, and the second point in the indoor environment (the second indoor point) is the exit from the indoor environment, then the coordinates of the first point in the indoor environment and the coordinates of the second point in the indoor environment can be different.

[0064] For example, if the first point in the indoor environment is an entrance / exit, then the coordinates of the first point in the indoor environment and the coordinates of the second point in the indoor environment can be the same. In this case, the vehicle control device 100 can map other points in the indoor environment to points in the indoor environment, points in the outdoor environment, and / or other points in the outdoor environment. However, this disclosure is not limited thereto.

[0065] According to an embodiment, the vehicle control device 100 can determine the endpoint of the indoor environment map by mapping another point in the outdoor environment to another point in the indoor environment.

[0066] For example, vehicle control device 100 can use outdoor environment map 125 to identify coordinates representing other points in the indoor environment. Vehicle control device 100 can identify the endpoint of indoor environment map 127 by using coordinates representing other points in the indoor environment.

[0067] For example, vehicle control device 100 can match the coordinates of another point in the indoor environment with the coordinates of another point in the outdoor environment, thereby determining the other point in the indoor environment as the endpoint of indoor environment map 127. For example, vehicle control device 100 can map other points in the outdoor environment to other points in the indoor environment based on optimization. Optimization may include compensating for the indoor environment based on the outdoor environment.

[0068] According to the embodiment, the vehicle control device 100 can map another point in the outdoor environment and another point in the indoor environment by using a closed curve, and determine the other point in the indoor environment as the endpoint of the indoor environment map 127.

[0069] For example, vehicle control device 100 can obtain a closed curve including vehicle movement path and virtual route by mapping a point in the indoor environment to a point in the outdoor environment and mapping another point in the indoor environment to another point in the outdoor environment.

[0070] According to an embodiment, the communication circuit 160 of the vehicle control device 100 may include hardware components for supporting the transmission and / or reception of electrical signals between the vehicle control device 100 and external electronic devices. For example, the communication circuit 160 may include at least one of a modem, an antenna, and an optoelectronic (O / E) converter. The communication circuit 160 may support the transmission and / or reception of electrical signals based on various types of protocols such as Ethernet, Local Area Network (LAN), Wide Area Network (WAN), Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), ZigBee, Long Term Evolution (LTE), or 5G New Radio (NR).

[0071] According to an embodiment, the vehicle control device 100 can receive GPS signals indicating the vehicle's location from an external server using a communication circuit 160. For example, if a vehicle traveling in an outdoor environment enters an indoor environment, the reception of GPS signals via the communication circuit 160 may be temporarily interrupted. For example, if the reception of GPS signals indicating the vehicle's location from an external server is temporarily interrupted and / or if the vehicle's speed decreases, the vehicle control device 100 can identify that the vehicle has entered (or moved) from a point in the outdoor environment to a point in the indoor environment. However, this disclosure is not limited thereto.

[0072] As described above, the vehicle control device 100 according to the embodiment can use a camera 150 to identify whether a vehicle has moved from an outdoor environment to an indoor environment. The vehicle control device 100 can use images obtained via the camera 150 to identify that the vehicle has moved into the indoor environment based on object recognition or feature detection. The vehicle control device 100 can obtain a point cloud of at least one object identified using the camera 150 based on the vehicle's movement path within the indoor environment. The vehicle control device 100 can use the point cloud to generate an indoor environment map representing at least a portion of the indoor environment. For example, the vehicle control device 100 can use an outdoor environment map to correct the position of the indoor environment map. The vehicle control device 100 can improve the accuracy of the indoor environment map by correcting its position based on loop closure. The vehicle control device 100 can reduce the cost of generating the indoor environment map by using a camera to generate the map.

[0073] Figure 2A and Figure 2B This is a diagram illustrating an embodiment of the operation of a vehicle control device identifying the location of a vehicle according to an embodiment of the present disclosure. Figure 2A and Figure 2B The vehicle control device 100 can be called Figure 1 Vehicle control equipment 100.

[0074] See Figure 2AWhen the vehicle is in motion, the vehicle control device 100 according to the embodiment can use a camera (e.g., Figure 1 The vehicle control device 100 uses a camera 150 to acquire an image representing at least a portion of the outdoor environment corresponding to the front of the vehicle. The vehicle control device 100 can identify a point in the outdoor environment from which the vehicle enters the indoor environment based on image recognition (or object recognition) of the image.

[0075] In one embodiment 210, the vehicle control device 100 according to the embodiment can use a camera to acquire image 211. The vehicle control device 100 can analyze image 211.

[0076] For example, vehicle control device 100 can identify an outdoor object 212 within image 211. Outdoor object 212 may include a barrier for temporarily preventing a vehicle from entering. Outdoor object 212 may include a speed bump for reducing vehicle speed. However, this disclosure is not limited thereto.

[0077] For example, vehicle control device 100 can use parking information to identify an outdoor object 212 located in front of the parking lot entrance. Vehicle control device 100 can predict a vehicle's entry from the outdoor environment into the indoor environment by identifying the outdoor object 212. However, this disclosure is not limited thereto.

[0078] In one embodiment 220, the vehicle control device 100 according to the embodiment can use a camera to acquire an image 222. The vehicle control device 100 can identify an outdoor object 223 within the image 222, which indicates a height at which entry into the indoor environment is possible. By identifying the outdoor object 223, the vehicle control device 100 can determine that the vehicle has moved from the outdoor environment to the indoor environment.

[0079] Reference Figure 2B In one embodiment 230, an image 231 is shown, which includes a visual object 232 indicating the location of a vehicle. Image 231 may represent an outdoor environment map (e.g., Figure 1 Outdoor environment map 125) and / or indoor environment map (e.g., Figure 1 Map information for the indoor environment map (127).

[0080] According to an embodiment, the vehicle control device 100 can identify the vehicle's changing position over time. The vehicle control device 100 can identify the vehicle's position based on GPS signals received from an external server. The vehicle control device 100 can also identify the vehicle's position on an outdoor environment map (e.g., [missing information]) based on received GPS signals. Figure 1 The location is shown on the outdoor environment map (125).

[0081] For example, the vehicle control device 100 can use a visual object 232 on an outdoor environment map to represent the vehicle's position. The visual object 232 can represent the position of the identified vehicle as it changes over time.

[0082] For example, the length between one visual object and another can vary depending on the vehicle's speed. Because the vehicle control device 100 identifies the vehicle's position based on the same amount of time, the length between the visual objects may become longer as the vehicle's speed increases. For example, since the lengths between visual objects within region 235 are relatively short, the vehicle control device 100 can detect a decrease in vehicle speed.

[0083] For example, vehicle control device 100 can identify the position of a moving vehicle based on a specified orientation 233 in an outdoor environment. For example, vehicle control device 100 can identify the position of a moving vehicle based on the use of sensors (e.g., Figure 1 The sensor 130 in the middle identifies the vehicle's direction of movement and / or speed to identify the vehicle's position.

[0084] When the vehicle is traveling in an outdoor environment, the vehicle control device 100 according to the embodiment can use the communication circuit 160 to receive GPS signals indicating the vehicle's location from an external server. For example, if the vehicle traveling in an outdoor environment enters an indoor environment, the reception of GPS signals via the communication circuit 160 may be temporarily interrupted.

[0085] For example, if the reception of GPS signals indicating the vehicle's location from an external server is temporarily interrupted, and / or if the vehicle's speed decreases, the vehicle control device 100 can identify that the vehicle has moved from a point in the outdoor environment to a point in the indoor environment.

[0086] For example, vehicle control device 100 can use an outdoor environment map to identify the location of the entrance to the indoor environment. After the vehicle is within a specified range of the entrance to the indoor environment, if the reception of the GPS signal is temporarily interrupted, vehicle control device 100 can recognize that the vehicle has entered the indoor environment.

[0087] For example, if a vehicle is traveling on a connecting road leading to a parking lot, the vehicle control device 100 can use parking lot information to identify that the vehicle has entered an indoor environment. However, this disclosure is not limited thereto. For example, the vehicle control device 100 can use sensors, based on parking lot entry determination and the execution of a time processor, to identify that the vehicle has entered an indoor environment using the identified vehicle's location information, an outdoor environment map, and / or parking lot information.

[0088] As described above, the vehicle control device 100 according to an embodiment can use images obtained from a camera to identify whether a vehicle has moved from an outdoor environment to an indoor environment. According to an embodiment, the vehicle control device 100 can identify a vehicle's movement from an outdoor environment to an indoor environment based on a temporary interruption of GPS signal reception and / or a decrease in vehicle speed. After the vehicle moves to the indoor environment (e.g., after identifying that the vehicle has moved from an outdoor environment to an indoor environment), the vehicle control device 100 can use at least one object identified in the image of the indoor environment obtained by the camera 150 to obtain a point cloud of at least one object in the indoor environment to generate a map of the indoor environment. The vehicle control device 100 can use the point cloud to generate an indoor environment map representing at least a portion of the indoor environment. Compared to generating an indoor environment map using, for example, LiDAR, the vehicle control device can generate an indoor environment map based on images from the camera 150 at a lower cost.

[0089] Figure 3 This is an illustration of an embodiment of the operation of correcting an indoor environment map using a vehicle control device according to an embodiment of the present disclosure. Figure 3 The vehicle control equipment 100 can be seen Figure 1 Vehicle control equipment 100.

[0090] In embodiment 310, the vehicle control device 100 can use an outdoor environment map 311 to map an indoor environment map 312. The indoor environment map 312 may be included in... Figure 1 The indoor environment map 127. The outdoor environment map 311 can be included in. Figure 1 The outdoor environment map is shown in map 125.

[0091] According to an embodiment, the vehicle control device 100 can generate an indoor environment map 312 based on the vehicle's movement path within the indoor environment. The indoor environment map 312 can represent at least a portion of the indoor environment based on the vehicle's movement path. The indoor environment map 312 may include information about the path the vehicle can travel along within the indoor environment, information about at least one object included in the indoor environment, and / or information about available parking areas within the indoor environment (e.g., the number of parking spaces available only for disabled persons). However, this disclosure is not limited thereto.

[0092] For example, outdoor environment map 311 may include information about the location of at least one object included in the outdoor environment.

[0093] According to an embodiment, the vehicle control device 100 can use an outdoor environment map 311 to identify the location of an entrance 315 for entering an indoor environment and / or an exit 316 for moving from an indoor environment to an outdoor environment.

[0094] According to an embodiment, the vehicle control device 100 can use a camera to obtain a point cloud representing at least a portion of the indoor environment to generate an indoor environment map 312 based on the vehicle moving from a point in the outdoor environment (e.g., a parking lot entrance) to a point in the indoor environment.

[0095] According to an embodiment, the vehicle control device 100 can temporarily stop acquiring point clouds based on the vehicle moving from one point in the indoor environment to another point in the outdoor environment. In other words, the vehicle control device 100 can temporarily stop generating the indoor environment map 312 based on the vehicle moving to another point in the outdoor environment.

[0096] In this embodiment, because GPS signals may be temporarily interrupted (e.g., based on the vehicle's movement to a point in the indoor environment), the starting point 313 of the indoor environment map 312 generated by the vehicle control device 100 may differ from a point in the outdoor environment (e.g., entrance 315). That is, even if the vehicle moves continuously, drift or jumps may occur between the starting point 313 of the indoor environment map 312 and a point in the outdoor environment (e.g., entrance 315) due to the loss of GPS signals. Similarly, even if the vehicle moves continuously from another point in the indoor environment (e.g., endpoint 314) to another point in the outdoor environment (e.g., exit 316), drift or jumps may occur between the second point in the indoor environment (e.g., endpoint 314) and a second point in the outdoor environment (a second outdoor point, e.g., exit 316).

[0097] In embodiment 320, the vehicle control device 100 according to the embodiment can use (e.g., based on) an outdoor environment map 311 to correct the start point 313 and / or end point 314 of an indoor environment map 312. For example, the operation of correcting the start point 313 and / or end point 314 of the indoor environment map 312 may include the operation of correcting the start point and / or end point of the point cloud corresponding to the indoor environment map 312.

[0098] For example, before using the point cloud to generate the indoor environment map 312, the vehicle control device 100 can correct the start and end points of the point cloud and then generate the indoor environment map. However, this disclosure is not limited thereto.

[0099] According to an embodiment, the vehicle control device 100 can use an outdoor environment map 311 to generate a virtual route 325 that includes points in the outdoor environment (e.g., entrance 315 or exit 316).

[0100] For example, a virtual route 325 can be generated based on information about roads that vehicles can travel on in an outdoor environment.

[0101] According to an embodiment, the vehicle control device 100 may generate a virtual route 325 based on the shortest route among multiple routes including points in the outdoor environment (e.g., a first point and a second point). However, this disclosure is not limited thereto.

[0102] According to an embodiment, the vehicle control device 100 can obtain a closed curve 330 of the vehicle's movement path (e.g., from 312 to 314) in an indoor environment and the virtual route 325 based on loop closure.

[0103] For example, vehicle control device 100 can map one end 324 of virtual route 325 (e.g., a point in the outdoor environment) to a point in the indoor environment based on the location of entrance 315, so as to determine / identify / generate / select that point in the indoor environment as the starting point 313 of indoor environment map 312.

[0104] For example, vehicle control device 100 may map the opposite end 326 of virtual route 325 (e.g., a second point in the outdoor environment) to a second point in the indoor environment based on the location of exit 316, so as to determine the second point in the indoor environment as the endpoint 314 of indoor environment map 312.

[0105] According to an embodiment, the vehicle control device 100 can use a closed curve 330 obtained based on loop closure to map the indoor environment map 312 to roads in the outdoor environment. The vehicle control device 100 can use the outdoor environment map 311 to set the coordinates of the indoor environment map 312 in the outdoor environment.

[0106] As described above, according to the embodiment, the vehicle control device 100 can generate an indoor environment map 312 and then use an outdoor environment map 311 to correct the positions of the start point 313 and / or end point 314 of the indoor environment map 312. The vehicle control device 100 can obtain / generate more accurate map information (e.g., an indoor environment map) by correcting the positions of the start point 313 and / or end point 314 of the indoor environment map 312 using the outdoor environment map 311. By mapping the indoor environment map 312 to roads, the vehicle control device 100 can continuously provide map information to the user independently of the vehicle's position (position within the indoor environment or position within the outdoor environment).

[0107] Figure 4 This is a flowchart illustrating an embodiment of the operation of a vehicle control device according to an embodiment of the present disclosure. In the following, it is assumed that... Figure 1 The vehicle control equipment 100 executes Figure 4 The process. Furthermore, or alternatively, in Figure 4 As can be understood from the description, the operations described as being performed by the device are controlled by the processor 110 of the vehicle control device 100. Figure 4 Each operation in the process can be executed sequentially, but not necessarily in that order. For example, the order of each operation can be changed, and at least two operations can be executed in parallel.

[0108] Reference Figure 4 In operation S410, the vehicle control device according to the embodiment can use a camera to identify the parking lot entrance.

[0109] For example, a vehicle control device may acquire an image representing at least a portion of an outdoor environment via a camera, the portion of which corresponds to the area in front of a vehicle traveling on a road in the outdoor environment.

[0110] For example, vehicle control equipment can identify at least one object in an image acquired by camera 150 based on object recognition.

[0111] For example, a vehicle control device can determine whether at least one object identified in an image is a parking lot entrance road. The vehicle control device can determine whether the at least one object is a parking lot entrance road using parking lot information. However, this disclosure is not limited thereto.

[0112] In one embodiment, the vehicle control device can identify a vehicle moving from an outdoor environment to an indoor environment based on a parking lot entry road in an image.

[0113] For example, vehicle control equipment can identify when a vehicle moves from an outdoor environment to an indoor environment based on a temporary interruption in the reception of GPS signals.

[0114] For example, vehicle control equipment can use outdoor environment maps (e.g., Figure 1 The outdoor environment map 125 identifies the vehicle's position within the outdoor environment while simultaneously identifying the position where the vehicle leaves the road. The vehicle control device can then determine whether the vehicle has moved from the outdoor environment to the indoor environment based on the identified position where the vehicle leaves the road. However, this disclosure is not limited thereto.

[0115] See Figure 4 In operation S420, the vehicle control device according to the embodiment can obtain a point cloud. For example, the operation of the vehicle control device obtaining the point cloud may include performing a point cloud generation operation.

[0116] For example, the vehicle control device may acquire an image representing at least a portion of an indoor environment via a camera (e.g., camera 150) based on the vehicle moving into an indoor environment (e.g., based on a vehicle movement path instructing the vehicle to move into the indoor environment). The vehicle control device may acquire a point cloud of at least one object in the image (e.g., identified in the image).

[0117] For example, based on the obtained point cloud, the vehicle control device can generate an indoor environment map representing at least a portion of the indoor environment according to the vehicle's movement path.

[0118] In an embodiment, if the indoor environment includes an indoor parking lot, the vehicle control equipment can recognize an ignition shutdown for parking. This ignition shutdown may include operation of the vehicle's ignition system.

[0119] For example, vehicle control equipment can recognize that the vehicle's ignition system is turned on after the vehicle's ignition system has been turned off. This ignition shutdown can include the engagement or disengagement of the vehicle's ignition system.

[0120] For example, the vehicle control equipment can maintain point cloud acquisition operations (e.g., operation S420) in an indoor environment, independently of whether the vehicle ignition is off or on. However, this disclosure is not limited thereto.

[0121] See Figure 4 In operation S430, the vehicle control device 100 according to the embodiment can identify whether the vehicle has entered an outdoor environment (e.g., from an indoor environment and / or approaching an indoor environment).

[0122] For example, the vehicle control device may use a camera (e.g., camera 150) to identify at least a portion of the outdoor environment (e.g., the road), so that it can be identified that the vehicle has moved into the outdoor environment.

[0123] For example, vehicle control equipment can identify when a vehicle has moved into an outdoor environment based on the recognition of GPS signals.

[0124] See Figure 4 In operation S440, the vehicle control device according to the embodiment can end (or stop) the acquisition of point clouds based on the recognition that the vehicle has entered the outdoor environment.

[0125] For example, vehicle control equipment can use point cloud acquisition to correct indoor environment maps generated from point clouds.

[0126] For example, vehicle control equipment can use an outdoor environment map to determine where an indoor environment map will be placed in the outdoor environment.

[0127] As described above, the vehicle control device according to the embodiment can provide the user with a more accurate indoor environment map by determining the location of the indoor environment map.

[0128] Figure 5 This is an illustration of an embodiment of the operation of a vehicle control device according to an embodiment of the present disclosure, in selecting at least one indoor environment map from a plurality of indoor environment maps. See also Figure 5 , Figure 5 The vehicle control device 100 can be called Figure 1 Vehicle control equipment 100. Figure 5 The vehicle control device 100 can be implemented outside the vehicle. If it is implemented outside the vehicle, then Figure 5 The vehicle control device 100 may be referred to as a server and / or a remote controller.

[0129] According to an embodiment, the vehicle control device 100 can receive (e.g., from one or more vehicles) information about point clouds corresponding to each of the plurality of vehicles.

[0130] For example, vehicle control device 100 may obtain a first point cloud 511 based on the movement path of the first vehicle in an indoor environment (e.g., from the first vehicle and / or another computing device associated with the first vehicle).

[0131] For example, vehicle control device 100 may obtain (e.g., from the second vehicle and / or another computing device associated with the second vehicle) a second point cloud 521 based on the movement path of the second vehicle in an indoor environment.

[0132] The first point cloud 511 and the second point cloud 521 may be different from each other (e.g., due to the vehicle's movement path, vehicle sensor information (e.g., IMU sensor information, cloud acquisition time point and point cloud acquisition time point)).

[0133] Based on the first point cloud 511 and the second point cloud 521, the vehicle control device 100 according to the embodiment can use at least one of the first point cloud 511 and / or the second point cloud 521 to generate an indoor environment map using the outdoor environment map 510.

[0134] For example, vehicle control device 100 can use outdoor environment map 510 to identify the locations of entrances 515 and exits 516 in the indoor environment.

[0135] For example, the vehicle control device 100 can identify the position information of the first point cloud 511 and the position information of the second point cloud 521.

[0136] For example, vehicle control device 100 can use the location information of first point cloud 511 and / or second point cloud 521 to identify at least one point cloud that is relatively adjacent (best fit and / or closest to) the entrance 515 and / or exit 516 of the indoor environment.

[0137] For example, if the starting point (or ending point) corresponding to the second point cloud 521 is closer to the entrance 515 than the starting point corresponding to the first point cloud 511, the vehicle control device 100 can generate an indoor environment map by using the second point cloud 521 to generate a closed loop.

[0138] In this embodiment, since point clouds representing different parts of the indoor environment are obtained based on the vehicle’s movement path, the vehicle control device 100 can use the first point cloud 511 and the second point cloud 521 to generate an indoor environment map (e.g., using a portion of the first point cloud 511 and another portion of the second point cloud 521).

[0139] For example, if the first point cloud 511 represents a portion of the indoor environment and the second point cloud 521 represents another portion of the indoor environment (or the remaining portion excluding that portion), the vehicle control device 100 can use the first point cloud 511 and the second point cloud 521 to generate an indoor environment map representing the entire indoor environment. However, this disclosure is not limited thereto.

[0140] According to an embodiment, the vehicle control device 100 can use a point cloud to determine the order of information for identifying at least one object included in the point cloud.

[0141] For example, if at least one object included in the first point cloud 511 is identified using the second point cloud 521, the vehicle control device 100 may prioritize operations for identifying information about the at least one object. However, this disclosure is not limited thereto.

[0142] As described above, the vehicle control device 100 according to the embodiment can obtain point clouds representing at least a portion of the indoor environment from multiple vehicles, and can select at least one point cloud from the obtained point clouds that is adjacent to the location of an exit or entrance in the outdoor environment, and can generate an indoor environment map by using the selected at least one point cloud. The vehicle control device 100 can improve the accuracy of the indoor environment map by receiving point clouds containing information about different indoor environments from multiple vehicles.

[0143] Figure 6 This is a flowchart illustrating a method for controlling a vehicle according to an embodiment of the present disclosure.

[0144] In the following text, reference will be made to Figure 6 A method for controlling a vehicle according to another embodiment of the present disclosure is described in detail. Figure 6 This is a flowchart illustrating a method for controlling a vehicle according to another embodiment of the present disclosure.

[0145] In the following text, it is assumed that... Figure 1 The vehicle control equipment 100 executes Figure 6 The process. Furthermore, or alternatively, in Figure 6 As can be understood from the description, the operations described as being performed by the device are controlled by the processor 110 of the vehicle control device 100. Figure 6Each operation in the process can be executed sequentially, but not necessarily in that order. For example, the order of each operation can be changed, and at least two operations can be executed in parallel.

[0146] See Figure 6 In operation S610, the method for controlling a vehicle according to the embodiment may include an operation to identify whether the vehicle enters a point in an indoor environment from a point in an outdoor environment (e.g., by moving from a first point (first outdoor point) in the outdoor environment to a first point in the indoor environment).

[0147] For example, methods for controlling a vehicle may include identifying a point in the indoor environment from a point in the outdoor environment by using a camera to identify entrance and / or exit information corresponding to a point in the indoor environment.

[0148] For example, a method for controlling a vehicle may include: if the reception of GPS signals via a communication circuit is temporarily interrupted, identifying the operation of the vehicle moving from a point in the outdoor environment to a point in the indoor environment.

[0149] For example, points in an outdoor environment and / or points in an indoor environment can be included in an area containing an entrance for access to the indoor environment.

[0150] refer to Figure 6 In operation S620, the method for controlling the vehicle according to the embodiment may include obtaining a point cloud of at least one object identified by using a camera (e.g., in an image obtained / acquired by the camera) based on identifying points where the vehicle enters an indoor environment.

[0151] For example, a method of controlling a vehicle may include, after the vehicle enters an indoor environment, using a camera to obtain an image representing at least a portion of the indoor environment based on the vehicle's movement path.

[0152] For example, methods for controlling a vehicle may include operations that identify at least one object based on object recognition of an image.

[0153] For example, methods for controlling a vehicle may include operations that obtain a point cloud containing information about at least one object.

[0154] See Figure 6 In operation S630, the method for controlling a vehicle according to an embodiment may include the operation of generating an indoor environment map using a point cloud, the indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path in the indoor environment.

[0155] For example, a method for controlling a vehicle may include operations based on a grid applied to a point cloud to obtain (e.g., generating / receiving) data indicating a virtual object representing at least one object from the point cloud. The method may also include operations to generate a 3D-based map of an indoor environment including the virtual object.

[0156] See Figure 6 In operation S640, the method for controlling the vehicle according to the embodiment may include determining a point in the indoor environment as the starting point of an indoor environment map. A point in the indoor environment can be determined as the starting point by mapping a point in the outdoor environment to a point in the indoor environment using an outdoor environment map representing the outdoor environment.

[0157] For example, methods for controlling a vehicle may include using an outdoor environment map to generate a virtual route based on the shortest path among multiple paths that include points in the outdoor environment.

[0158] For example, a method for controlling a vehicle can map points in the outdoor environment to points in the indoor environment by generating a closed loop using the trajectory of a point cloud (or the vehicle's movement path) and the identified virtual route, based on the vehicle's movement path in the indoor environment.

[0159] For example, points in the mapped outdoor environment and points in the indoor environment can be included in an area containing the entrance to the indoor environment.

[0160] In one embodiment, the method for controlling the vehicle can improve the accuracy of the indoor environment map by using an outdoor environment map (e.g., using corresponding points in the outdoor environment map) to correct at least a portion of the indoor environment map.

[0161] Figure 7 This is a block diagram illustrating a computing system or method for controlling a vehicle associated with a vehicle control device 100 according to an embodiment of the present disclosure.

[0162] See Figure 7 The computing system 1000 may include at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage device 1600, and a network interface 1700 connected via a system bus 1200.

[0163] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory 1300 and / or storage device 1600.

[0164] The memory 1300 and storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include ROM (Read-Only Memory) 1310 and RAM (Random Access Memory) 1320.

[0165] Therefore, the operation of the methods or algorithms described in conjunction with the embodiments disclosed herein can be directly embodied in hardware or software modules, or combinations thereof, executed by processor 1100. Software modules may reside on storage media such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, and CD-ROMs (i.e., memory 1300 and / or storage device 1600).

[0166] An exemplary storage medium may be coupled to processor 1100, and processor 1100 may read information from and record information in the storage medium. Alternatively, the storage medium may be integrated with processor 1100. Processor 1100 and storage medium may reside in an application-specific integrated circuit (ASIC). The ASIC may reside within a user terminal. In another scenario, processor 1100 and storage medium may reside as separate components in the user terminal.

[0167] This disclosure has been made to address the aforementioned problems in the prior art while maintaining the advantages achieved by the prior art.

[0168] One aspect of this disclosure provides an apparatus and method for controlling a vehicle, the apparatus being able to identify whether the vehicle has entered an indoor environment.

[0169] Another aspect of this disclosure provides an apparatus and method for controlling a vehicle, wherein if the vehicle moves into an indoor environment, the apparatus is capable of using a camera to perform point cloud generation.

[0170] Another aspect of this disclosure provides an apparatus and method for controlling a vehicle that can use the location of the entrance to correct an indoor environment map.

[0171] The technical problems to be solved by this disclosure are not limited to those described above, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art from the following description.

[0172] According to one aspect of this disclosure, an apparatus for controlling a vehicle includes a camera, a processor, and a memory. The processor can identify whether the vehicle enters an indoor environment from a point in an outdoor environment, obtain a point cloud of at least one object identified by the camera based on the identified point cloud, generate an indoor environment map using the point cloud, the indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path in the indoor environment, and determine the point in the indoor environment as the starting point of the indoor environment map by mapping the points in the outdoor environment to the points in the indoor environment using an outdoor environment map representing the outdoor environment.

[0173] According to an embodiment, the processor can use an outdoor environment map to generate a virtual route including points in the outdoor environment, obtain a closed curve including the virtual route and the movement path based on loop closure, and map the points in the outdoor environment to the points in the indoor environment by using the obtained closed curve, thereby determining the points in the indoor environment as the starting point of the indoor environment map.

[0174] According to an embodiment, the processor can generate a virtual route based on the shortest route among multiple routes including points in an outdoor environment.

[0175] According to an embodiment, the processor can identify that the vehicle moves from one point in the indoor environment to another point in the outdoor environment, and determine the other point in the indoor environment as the endpoint of the indoor environment map by mapping the other point in the indoor environment to the other point in the outdoor environment based on loop closure using an outdoor environment map.

[0176] According to an embodiment, the processor may temporarily stop acquiring point clouds based on the recognition that the vehicle has moved to another point in the outdoor environment.

[0177] According to an embodiment, the processor can identify the point in the indoor environment from which a vehicle enters the indoor environment by using a camera to identify entrance and exit information corresponding to points in the indoor environment.

[0178] According to an embodiment, the entrance and exit information may include information about at least one of the following: an outdoor object indicating the height of a vehicle capable of entering the indoor environment, an outdoor object used to temporarily prevent entry into the indoor environment, or a combination thereof.

[0179] According to an embodiment, if the reception of a Global Positioning System (GPS) signal indicating the vehicle's location from an external server is temporarily interrupted, or if the vehicle's speed decreases, the processor can identify a point in the indoor environment where the vehicle has entered.

[0180] According to an embodiment, the device may also include sensors. After the vehicle enters the indoor environment, the processor can use the sensors to identify the vehicle's movement path within the indoor environment and generate an indoor environment map based on the vehicle's movement path using Simultaneous Localization and Mapping (SLAM).

[0181] According to an embodiment, the processor can obtain virtual objects representing at least one object from a point cloud and generate an indoor environment map that includes the virtual objects and is based on three dimensions.

[0182] According to another aspect of this disclosure, a method for controlling a vehicle includes: identifying whether the vehicle enters an indoor environment from a point in an outdoor environment; obtaining a point cloud of at least one object identified using a camera based on the identification of the point where the vehicle enters the indoor environment; generating an indoor environment map using the point cloud, the indoor environment map representing at least a portion of the indoor environment along the movement path of the vehicle in the indoor environment; and determining the point in the indoor environment as the starting point of the indoor environment map by mapping the points in the outdoor environment to the points in the indoor environment using an outdoor environment map representing the outdoor environment.

[0183] According to an embodiment, determining the point in the indoor environment as the starting point of the indoor environment map may include: generating a virtual route that includes points in the outdoor environment using an outdoor environment map; obtaining a closed curve that includes the virtual route and the movement path based on loop closure; and determining the point in the indoor environment as the starting point of the indoor environment map by mapping the points in the outdoor environment and the points in the indoor environment using the obtained closed curve.

[0184] According to an embodiment, generating a virtual route may include generating a virtual route based on the shortest route among multiple routes including points in an outdoor environment.

[0185] According to an embodiment, the method may further include identifying whether the vehicle has moved from another point in the indoor environment to another point in the outdoor environment, and based on identifying that the vehicle has moved to another point in the outdoor environment, determining the other point in the indoor environment as the endpoint of the indoor environment map by mapping the other point in the outdoor environment to the other point in the indoor environment based on loop closure using an outdoor environment map.

[0186] According to an embodiment, identifying vehicle movement may include temporarily halting point cloud acquisition based on the identification that the vehicle has moved to another point in the outdoor environment.

[0187] According to an embodiment, identifying the point where a vehicle enters an indoor environment may include: identifying the point where the vehicle enters the indoor environment from a point in the outdoor environment based on entrance and exit information corresponding to the point in the indoor environment using a camera.

[0188] According to an embodiment, the entrance and exit information may include information about at least one of an outdoor object indicating the height of a vehicle capable of entering the indoor environment, an outdoor object used to temporarily prevent entry into the indoor environment, or a combination thereof.

[0189] According to an embodiment, identifying the point where a vehicle enters an indoor environment may include: identifying the point where the vehicle enters an indoor environment if the reception of a Global Positioning System (GPS) signal indicating the vehicle's location from an external server is temporarily interrupted, or if the vehicle's speed decreases.

[0190] According to an embodiment, the method may further include: after the vehicle enters a point in the indoor environment, using sensors to identify the vehicle's movement path in the indoor environment; and generating an indoor environment map based on simultaneous localization and mapping (SLAM) according to the vehicle's movement path.

[0191] According to an embodiment, generating an indoor environment map may include obtaining a virtual object representing the at least one object from a point cloud, and generating an indoor environment map that includes the virtual object and is based on three dimensions.

[0192] This technology can determine whether a vehicle has entered an indoor environment.

[0193] If the vehicle moves into an indoor environment, this technique can use a camera to perform point cloud generation.

[0194] Alternatively, this technology can be used to correct the indoor environment map by using the location of the entrance.

[0195] In addition, or alternatively, various effects may be provided that are understood directly or indirectly from this disclosure.

[0196] Although embodiments of the present disclosure have been described for illustrative purposes, those skilled in the art will recognize that various modifications, additions, and substitutions are possible without departing from the scope and spirit of the present disclosure.

[0197] Therefore, the embodiments disclosed herein are provided for illustrative purposes and are not intended to limit the technical concept of this disclosure. It should be understood that such embodiments are not intended to limit the scope of the technical concept of this disclosure. The scope of protection of this invention should be determined by the scope of the claims, and all equivalent technical concepts should be determined by the scope of the claims.

Claims

1. A device for controlling a vehicle, the device comprising: camera; One or more processors; as well as A memory storing instructions that, when executed by the one or more processors, are configured to cause the device to: Identify whether the vehicle has moved from a first outdoor point in an outdoor environment to a first indoor point in an indoor environment; Based on identifying the first indoor point where the vehicle moves into the indoor environment, a point cloud of at least one indoor object identified in the image acquired using the camera is obtained; Based on the point cloud, an indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path is generated. Based on the outdoor environment map representing the outdoor environment, the first outdoor point in the outdoor environment is mapped to the first indoor point in the indoor environment; Based on the mapping from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment, the first indoor point in the indoor environment is determined as the starting point of the indoor environment map; and The vehicle operation is controlled based on the generated indoor environment map.

2. The device according to claim 1, wherein, The instructions, when executed by the one or more processors, are also configured to cause the device to: Based on the outdoor environment map, a virtual route including the first outdoor point in the outdoor environment is generated; as well as Based on loop closure, a closed curve including the virtual route and the movement path is obtained; and Using the obtained closed curve, the first outdoor point in the outdoor environment is mapped to the first indoor point in the indoor environment.

3. The device according to claim 2, wherein, When executed by the one or more processors, the instructions are configured to cause the device to generate the virtual route based on the shortest route among multiple routes that include the first outdoor point and at least one other outdoor point in the outdoor environment.

4. The device according to claim 1, wherein, The instructions, when executed by the one or more processors, are also configured to cause the device to: Identify whether the vehicle has moved from a second indoor point in the indoor environment to a second outdoor point in the outdoor environment; Based on the identification of the vehicle moving to the second outdoor point in the outdoor environment, the outdoor environment map is used to map the second outdoor point in the outdoor environment to the second indoor point in the indoor environment based on loop closure. as well as Based on the mapping from the second outdoor point in the outdoor environment to the second indoor point in the indoor environment, the second indoor point in the indoor environment is determined as the endpoint of the indoor environment map.

5. The device according to claim 4, wherein, When executed by the one or more processors, the instructions are configured to cause the device to temporarily stop acquiring the point cloud based on the identification that the vehicle has moved to a second outdoor point in the outdoor environment.

6. The device according to claim 1, wherein, When executed by the one or more processors, the instructions are configured to cause the device to: identify, based on the use of the camera to identify, in the image acquired by the camera, entrance or exit information corresponding to the first indoor point in the indoor environment, and thus identify that the vehicle has moved from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment.

7. The device according to claim 6, wherein, The inbound or outbound information includes information about one or more of the following: Outdoor objects indicating height restrictions for vehicles entering the indoor environment; and Used to temporarily prevent outdoor objects from entering the indoor environment.

8. The device according to claim 1, wherein, When executed by the one or more processors, the instructions are configured to cause the device to identify that the vehicle has moved from a first outdoor point in the outdoor environment to a first indoor point in the indoor environment based on one or more of the following: Interruption of reception of the GPS signal indicating the vehicle's location, and The vehicle's speed decreased.

9. The device according to claim 1, further comprising: sensor, The instructions, when executed by the one or more processors, are further configured to cause the device to: After the vehicle moves to the first indoor point in the indoor environment, the movement path of the vehicle in the indoor environment is identified based on data acquired by the sensors; and The indoor environment map is generated based on the vehicle's movement path and simultaneous localization and mapping (SMR).

10. The device according to claim 1, wherein, The instructions, when executed by the one or more processors, are also configured to cause the device to: Obtain a virtual object representing the at least one indoor object from the point cloud; and Generate a three-dimensional map of the indoor environment that includes the virtual object.

11. A method for controlling a vehicle, comprising: Identify whether the vehicle has moved from a first outdoor point in an outdoor environment to a first indoor point in an indoor environment; Based on identifying the first indoor point where the vehicle moves into the indoor environment, a point cloud of at least one indoor object identified in the image acquired using the camera is obtained; Based on the point cloud, an indoor environment map representing at least a portion of the indoor environment along the vehicle's movement path is generated. Based on the outdoor environment map representing the outdoor environment, the first outdoor point in the outdoor environment is mapped to the first indoor point in the indoor environment; Based on the mapping from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment, the first indoor point in the indoor environment is determined as the starting point of the indoor environment map; and The vehicle operation is controlled based on the generated indoor environment map.

12. The method of claim 11, further comprising: Based on the outdoor environment map, a virtual route including the first outdoor point in the outdoor environment is generated; as well as Based on loop closure, a closed curve including the virtual route and the movement path is obtained; and Specifically, the first outdoor point in the outdoor environment is mapped to the first indoor point in the indoor environment based on the obtained closed curve.

13. The method according to claim 12, wherein, The virtual route is generated based on the shortest route among multiple routes that include the first outdoor point and at least one other outdoor point in the outdoor environment.

14. The method of claim 11, further comprising: Identify whether the vehicle has moved from a second indoor point in the indoor environment to a second outdoor point in the outdoor environment; Based on the identification of the vehicle moving to the second outdoor point in the outdoor environment, the outdoor environment map is used to map the second outdoor point in the outdoor environment to the second indoor point in the indoor environment based on loop closure. as well as Based on the mapping, the second indoor point in the indoor environment is determined as the endpoint of the indoor environment map.

15. The method of claim 14, further comprising: Based on the identification that the vehicle has moved to the second outdoor point in the outdoor environment, the acquisition of the point cloud is temporarily stopped.

16. The method according to claim 11, wherein, Based on the entrance or exit information identified in the image acquired using the camera that corresponds to the first indoor point in the indoor environment, it is determined that the vehicle moved from the first outdoor point in the outdoor environment to the first indoor point in the indoor environment.

17. The method according to claim 16, wherein, The inbound or outbound information includes information about one or more of the following: Outdoor objects indicating height restrictions for vehicles entering the indoor environment, and Used to temporarily prevent outdoor objects from entering the indoor environment.

18. The method according to claim 11, wherein, The vehicle is identified as moving from a first outdoor point in the outdoor environment to a first indoor point in the indoor environment based on one or more of the following: Interruption of reception of the GPS signal indicating the vehicle's location; and The vehicle's speed decreased.

19. The method of claim 11, further comprising: After the vehicle moves to the first indoor point in the indoor environment, the movement path of the vehicle in the indoor environment is identified based on data acquired by the sensors; as well as The indoor environment map is generated based on the vehicle's movement path and simultaneous localization and mapping (SMR).

20. The method according to claim 11, wherein, Generating the indoor environment map includes: Obtain a virtual object representing the at least one indoor object from the point cloud; and Generate a three-dimensional map of the indoor environment that includes the virtual object.

Images