Apparatus for recognizing object and method thereof

Abstract

The invention relates to an apparatus for recognizing an object and a method thereof. The object recognition device may include: a sensor configured to obtain a point cloud associated with an object; and a processor. The processor may be configured to divide a specified space into a plurality of subspaces based on a size of the point cloud. The point cloud may be projected into a specified space. The specified space may include a plurality of voxels included in the plurality of subspaces. The processor may also be configured to: identify, among the plurality of voxels and based on sequentially exploring the plurality of voxels, one or more active voxels including at least a portion of the point cloud; assigning at least one index to the one or more active voxels; identifying an object based on the one or more valid voxels and the at least one index; and controlling an operation of the vehicle based on the identified object.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims the benefit of priority to Korean Patent Application No. 10-2024-0103380, filed on August 2, 2024, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. TECHNICAL FIELD

[0003] The disclosure relates to an apparatus for recognizing an object and a method thereof, and more particularly, to processing data based on voxels. BACKGROUND

[0004] Various researches have been conducted to recognize external objects using various sensors (e.g., Light Detection and Ranging (LiDAR), radar, camera, etc.) for assisting or automatically driving a vehicle.

[0005] Specifically, various research efforts have been made to voxelize point clouds obtained by sensors and classify valid voxels. SUMMARY

[0006] The disclosure has been made to solve the above problems occurring in at least some implementations, while maintaining advantages achieved by those implementations unaffected.

[0007] An aspect of the disclosure provides an object recognition apparatus and an object recognition method capable of effectively recognizing valid voxels.

[0008] An aspect of the disclosure provides an object recognition apparatus and an object recognition method capable of dynamically generating a valid voxel set.

[0009] An aspect of the disclosure provides an object recognition apparatus and an object recognition method capable of obtaining a relationship between valid voxels.

[0010] The technical problems to be solved by the disclosure are not limited to the above problems, and any other technical problems not mentioned herein will be clearly understood by a person skilled in the art from the following description.

[0011] According to one or more example embodiments of the present disclosure, an object recognition device can include a sensor configured to obtain a point cloud associated with an object, and a processor. The processor can be configured to divide a designated space into a plurality of sub-spaces based on a size of the point cloud. The point cloud can be projected into the designated space. The designated space can include a plurality of voxels included in the plurality of sub-spaces. The processor can be further configured to identify one or more valid voxels including at least a portion of the point cloud in the plurality of voxels and based on sequentially exploring the plurality of voxels, assign at least one index associated with at least one of the plurality of sub-spaces, the plurality of voxels, and the one or more valid voxels to the one or more valid voxels, identify the object based on the one or more valid voxels and the at least one index, and control an operation of a vehicle based on the identified object.

[0012] The processor can be configured to assign the at least one index by assigning at least one of a first index associated with the plurality of sub-spaces, a second index associated with the plurality of voxels, a third index associated with an axis used to obtain the plurality of voxels, and a fourth index associated with a number of sub-voxels included in the plurality of sub-spaces and associated with a number of axes to the one or more valid voxels.

[0013] The processor can be configured to divide the designated space by identifying a virtual frame including the point cloud, identifying a diagonal line length of the virtual frame based on a designated direction of viewing the point cloud, and dividing the designated space into the plurality of sub-spaces based on the diagonal line length.

[0014] The processor can be configured to identify the one or more valid voxels by identifying that the one or more valid voxels are further based on sequentially exploring a lateral axis extending from a right side of the vehicle to a left side of the vehicle, a longitudinal axis extending from a rear side of the vehicle to a front side of the vehicle, and a vertical axis extending from a bottom of the vehicle to a top of the vehicle.

[0015] The processor can be further configured to identify the number of sub-voxels based on the designated space being divided into a plurality of regions according to the number of axes, and obtain the fourth index based on the number of axes and the number of sub-voxels.

[0016] The processor can be further configured to obtain the first index based on an order of exploring the plurality of sub-spaces.

[0017] The processor can be further configured to identify coordinates of the one or more valid voxels, and obtain the second index based on the coordinates of the one or more valid voxels.

[0018] The coordinates of the one or more effective voxels can include coordinate values identified in a vehicle coordinate system of the vehicle. The vehicle coordinate system can include a longitudinal axis extending from a rear side of the vehicle to a front side of the vehicle, a lateral axis extending from a right side of the vehicle to a left side of the vehicle, and a vertical axis extending from a bottom of the vehicle to a top of the vehicle.

[0019] The processor can be further configured to obtain a third index based on a number of axes. The axes can include at least one of a spatial axis and a temporal axis.

[0020] At least one of a size of each voxel of the plurality of voxels and a size of each subspace of the plurality of subspaces can be set by at least one of a user and a provider.

[0021] The processor can be further configured to: generate a mapping table including at least one of the first index, the second index, the third index, and the fourth index; and output the mapping table.

[0022] According to one or more example embodiments of the disclosure, a method performed by a device of a vehicle can include dividing a designated space into a plurality of subspaces based on a size of a point cloud associated with an object. The point cloud can be projected into the designated space. The designated space can include a plurality of voxels included in the plurality of subspaces. The method can further include: identifying one or more effective voxels including at least a portion of the point cloud in the plurality of voxels and based on sequentially exploring the plurality of voxels; and assigning at least one index associated with at least one of the plurality of subspaces, the plurality of voxels, and the one or more effective voxels to the one or more effective voxels; identifying the object based on the one or more effective voxels and the at least one index; and controlling an operation of the vehicle based on the identified object.

[0023] The assigning of the at least one index can include assigning at least one of: a first index associated with the plurality of subspaces, a second index associated with the plurality of voxels, a third index associated with axes used to acquire the plurality of voxels, and a fourth index associated with a number of sub-voxels included in the plurality of subspaces and associated with a number of axes.

[0024] The dividing of the designated space can include: identifying a virtual box including the point cloud; identifying a diagonal length of the virtual box based on a designated direction of viewing the point cloud; and dividing the designated space into the plurality of subspaces based on the diagonal length.

[0025] The identifying of the one or more effective voxels can include identifying the one or more effective voxels based on sequentially exploring: a lateral axis extending from a right side of the vehicle to a left side of the vehicle, a longitudinal axis extending from a rear side of the vehicle to a front side of the vehicle, and a vertical axis extending from a bottom of the vehicle to a top of the vehicle.

[0026] The method may further include: identifying the number of sub-voxels by dividing a specified space into multiple regions based on the number of axes; and obtaining a fourth index based on the number of axes and the number of sub-voxels.

[0027] The method may also include obtaining a first index based on the order in which multiple subspaces are explored.

[0028] The method may further include: identifying the coordinates of one or more effective voxels; and obtaining a second index based on the coordinates of one or more effective voxels.

[0029] The coordinates of one or more valid voxels may include coordinate values ​​identified in the vehicle's vehicle coordinate system. The vehicle coordinate system may include a longitudinal axis extending from the rear to the front of the vehicle, a lateral axis extending from the right to the left of the vehicle, and a vertical axis extending from the bottom to the top of the vehicle.

[0030] The method may further include obtaining a third index based on the number of axes. The axes may include at least one of a spatial axis and a time axis. Attached Figure Description

[0031] The above and other objects, features, and advantages of this disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings:

[0032] Figure 1 An example of a block diagram related to an object recognition device is shown;

[0033] Figure 2 An example of dividing a specified space into multiple subspaces is shown;

[0034] Figure 3 An example of exploring effective voxels is shown;

[0035] Figure 4 An example of a mapping table is shown;

[0036] Figure 5 An example of a flowchart associated with an object recognition method is shown; and

[0037] Figure 6 A computing system related to an object recognition device or object recognition method is shown. Detailed Implementation

[0038] In the following, one or more exemplary embodiments of the present disclosure will be described in detail with reference to the exemplary accompanying drawings. When adding reference numerals to components in each drawing, it should be noted that identical or equivalent components are indicated by the same numerals even when shown in other drawings. Furthermore, in describing exemplary embodiments of the present disclosure, detailed descriptions of well-known features or functions will be excluded so as not to unnecessarily obscure the spirit of the disclosure.

[0039] In describing components according to exemplary embodiments of this disclosure, terms such as first, second, "A", "B", (a), (b), etc., may be used. These terms are intended only to distinguish one component from another, and they do not limit the nature, order, or sequence of the constituent components. Unless otherwise defined, all terms used herein (including technical or scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. Such terms, as defined in commonly used dictionaries, should be interpreted as having a meaning equivalent to that in the context of the relevant field and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined as having an ideal or overly formal meaning in this application.

[0040] For the purposes of this application and claims, the exemplary phrases “at least one of A; B; or C” or “at least one of A, B, or C” are used, which means “at least one A, or at least one B, or at least one C, or at least one A, at least one B, and at least one C.” Furthermore, exemplary phrases such as “A, B, and C,” “A, B, or C,” “at least one of A, B, and C,” “at least one of A, B, or C,” etc., as used herein, may refer to each listed item or all possible combinations of listed items. For example, “at least one of A or B” may mean (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.

[0041] According to the Society of Automotive Engineers (SAE), the automation levels of autonomous vehicles can be categorized as follows: At Level 0, the SAE classification corresponds to "No Automation," where the autonomous driving system temporarily handles emergency situations (e.g., automatic emergency braking) and / or only provides warnings (e.g., blind spot warning, lane departure warning, etc.), and expects the driver to operate the vehicle. At Level 1, the SAE classification corresponds to "Driver Assistance," where the system performs some driving functions (e.g., steering, acceleration, braking, lane centering, adaptive cruise control, etc.) while the driver operates the vehicle within normal operating ranges, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. At Level 2, the SAE classification corresponds to "Partial Automation," where the system performs steering, acceleration, and / or braking under driver supervision, and expects the driver to determine the system's operating status and / or timing, perform other driving functions, and handle (e.g., resolve) emergency situations. At Level 3 of autonomous driving, the SAE classification standard can correspond to "conditional automation," where the system drives the vehicle under limited conditions (e.g., performing driving functions such as steering, acceleration, and / or braking), but transfers driving control to the driver when the desired conditions are not met. The driver is expected to determine the system's operating state and / or timing and take over control in emergency situations, but not otherwise operate the vehicle (e.g., steering, acceleration, and / or braking). At Level 4 of autonomous driving, the SAE classification standard can correspond to "high automation," where the system performs all driving functions and the driver is expected to control the vehicle only in emergency situations. At Level 5 of autonomous driving, the SAE classification standard can correspond to "full automation," where the system performs all driving functions without any assistance from the driver, including in emergency situations, and the driver is not expected to perform any driving functions other than determining the system's operating state. While this disclosure can apply the SAE classification standard to autonomous driving classification, other classification methods and / or algorithms can be used in one or more configurations described herein. One or more features associated with autonomous driving control can be activated based on the configured autonomous driving control settings (e.g., based on at least one of the following: autonomous driving classification, selection of the vehicle's autonomous driving level, etc.).

[0042] Based on one or more features described herein (e.g., assigning an index to effective voxels in a point cloud), vehicle operation can be controlled. Vehicle control can include various operational controls associated with the vehicle (e.g., autonomous driving control, sensor control, braking control, braking time control, acceleration control, acceleration rate of change control, warning timing control, forward collision warning timing control, etc.).

[0043] One or more auxiliary devices (e.g., engine braking, exhaust braking, hydraulic decelerators, electric decelerators, regenerative braking, etc.) can also be controlled, for example, based on one or more features described herein (e.g., assigning an index to effective voxels in a point cloud). One or more communication devices (e.g., modems, network adapters, radio transceivers, antennas, etc., capable of communicating via one or more wired or wireless communication protocols such as Ethernet, Wi-Fi, Near Field Communication (NFC), Bluetooth, Long Term Evolution (LTE), 5G New Radio (NR), Vehicle-to-Everything (V2X), etc.) can also be controlled, for example, based on one or more features described herein (e.g., assigning an index to effective voxels in a point cloud).

[0044] Minimum Risk Maneuvering (MRM) operations can also be controlled, for example, based on one or more features described herein (e.g., assigning an index to effective voxels in a point cloud). Minimum Risk Maneuvering operations (e.g., minimum risk maneuvering, lowest risk maneuvering) can be maneuvers performed to minimize (e.g., reduce) the risk of collisions with surrounding vehicles to achieve a reduced (e.g., lowest) risk state. Minimum Risk Maneuvering can be an operation activated during autonomous driving when the driver is unable to respond to an intervention request. During Minimum Risk Maneuvering, one or more processors in the vehicle can control the vehicle's driving operations for a set time period.

[0045] Bias drive operation can also be controlled, for example, based on one or more features described herein (e.g., the index for assigning effective voxels in a point cloud). The drive control device can perform bias drive control. To perform bias driving, the drive control device can control the vehicle to drive within the lane by maintaining the lateral distance between the vehicle's center position and the lane center. For example, the drive control device can control the vehicle to remain within the lane but not at the center of the lane.

[0046] Drive control devices can identify a target lateral distance for bias drive control. For example, the target lateral distance may include an intentionally adjusted lateral distance that the vehicle may aim to maintain from a reference point (such as the center of the lane or another vehicle) during maneuvers (such as lane changes). This adjustment can be made to improve the vehicle's stability, safety, and / or performance under changing driving conditions. For instance, during lane changes, the drive control system may bias the lateral distance to maintain a safer clearance from adjacent vehicles, taking into account factors such as vehicle speed, road conditions, and / or the presence of obstacles.

[0047] One or more sensors (e.g., IMU sensors, cameras, LiDAR, RADAR, blind spot monitoring sensors, lane departure warning sensors, parking sensors, light sensors, rain sensors, traction control sensors, anti-lock braking system sensors, tire pressure monitoring sensors, seat belt sensors, airbag sensors, fuel sensors, emission sensors, throttle position sensors, inverters, converters, motor controllers, power distribution units, high-voltage wiring and connectors, auxiliary power modules, charging interfaces, etc.) can also be controlled, for example, based on one or more features described herein (e.g., assigning an index to effective voxels in a point cloud).

[0048] Operational control for autonomous driving of vehicles can include various driving controls of the vehicle by the vehicle control unit (e.g., acceleration, deceleration, steering control, gear shifting control, braking system control, traction control, stability control, cruise control, lane keeping assist control, collision avoidance system control, emergency braking assist control, traffic sign recognition control, adaptive headlight control, etc.).

[0049] In the following text, reference will be made to Figures 1 to 6 One or more exemplary embodiments of this disclosure are described in detail.

[0050] Figure 1 An example block diagram related to an object recognition device is shown.

[0051] refer to Figure 1 The object recognition device 100 can be implemented inside or outside the vehicle, and some components of the object recognition device 100 can be implemented inside or outside the vehicle. In this case, the object recognition device 100 can be integrally formed with the vehicle's internal control unit, or it can be implemented as a separate device and connected to the vehicle's control unit via a separate connection means. For example, the object recognition device 100 may also include Figure 1 Components not shown in the diagram.

[0052] The object recognition device 100 may include a processor 110 and a sensor 120. The object recognition device 100 may also include a memory 130. The processor 110, sensor 120, or memory 130 may be electronically and / or operatively coupled to each other via electronic components including a communication bus.

[0053] In the following text, operatively combinable hardware may include direct and / or indirect connections between hardware established in a wired and / or wireless manner, such that the second hardware is controlled by the first hardware in the hardware.

[0054] Although hardware blocks are shown in different boxes, this disclosure is not limited thereto. Figure 1A portion of the hardware may be included in a single integrated circuit, including a system-on-a-chip (SoC). The type and / or quantity of hardware included within the object recognition device 100 are not limited to... Figure 1 The type and / or quantity shown. For example, object recognition device 100 may include only... Figure 1 A portion of the hardware is shown in the image.

[0055] The object recognition device 100 may include hardware for processing data based on one or more instructions. The hardware for processing the data may include a processor 110. For example, the hardware for processing the data may include an arithmetic and logic unit (ALU), a floating-point unit (FPU), a field-programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The processor 110 may have a single-core processor architecture, or a multi-core processor architecture including dual-core, quad-core, hexa-core, or octa-core cores.

[0056] The object recognition device 100 may include a sensor 120 for acquiring a point cloud. For example, the point cloud may include a set of points corresponding to external objects.

[0057] For example, sensor 120 may include at least one or any combination of optical detection and ranging (LiDAR), time-of-flight (ToF) sensor, structured light sensor, ultrasonic sensor, infrared sensor, optical distance sensor, and radio detection and ranging (RADAR).

[0058] The memory 130 of the object recognition device 100 may include hardware components for storing data and / or instructions input to and / or output from the processor 110 of the object recognition device 100. For example, the memory 130 may include volatile memory or non-volatile memory, the volatile memory including random access memory (RAM) and the non-volatile memory including read-only memory (ROM).

[0059] For example, volatile memory may include at least one of dynamic RAM (DRAM), static RAM (SRAM), cache RAM, and pseudo SRAM (PSRAM) or any combination thereof.

[0060] For example, non-volatile memory may include at least one or any combination of a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), flash memory, a hard disk, an optical disk, a solid-state drive (SSD), and an embedded multimedia card (eMMC).

[0061] The memory 130 may include a neural network model. For example, the neural network model may project a point cloud into a specified space based on input from a point cloud acquired via sensor 120. For example, the neural network model may output information related to sub-voxels comprising at least a portion of the point cloud based on the input from the point cloud.

[0062] The processor 110 of the object recognition device 100 can acquire point clouds via the sensor 120. For example, the processor 110 can project the point cloud onto a specified space that includes multiple voxels. For example, the specified space may be referred to as the voxel space.

[0063] For example, processor 110 can divide the specified space into multiple subspaces by using the size of the point cloud, based on projecting the point cloud onto a specified space comprising multiple voxels. For example, each of the multiple subspaces can be referred to as an exploration space.

[0064] For example, processor 110 can recognize virtual bounding boxes that include point clouds. For example, virtual bounding boxes that include point clouds can include virtual bounding boxes corresponding to external objects. For example, virtual bounding boxes can be represented in a three-dimensional virtual coordinate system.

[0065] For example, processor 110 can identify the diagonal length of a virtual bounding box based on recognizing a virtual bounding box including a point cloud. For example, processor 110 can identify the diagonal length of the virtual bounding box when viewed along a specified direction based on recognizing a virtual bounding box including a point cloud. The diagonal length of the point cloud can be identified based on the specified direction of viewing the point cloud. This specified direction can determine the orientation of the virtual bounding box. For example, the specified direction can include the z-axis direction in a three-dimensional virtual coordinate system. For example, viewing the virtual bounding box along the specified direction can include viewing the virtual bounding box from a bird's-eye view.

[0066] For example, when viewing a virtual frame along a specified direction, the processor 110 can divide the specified space into multiple subspaces based on the identified diagonal length.

[0067] Processor 110 can sequentially explore multiple voxels included in multiple subspaces based on specified conditions. For example, processor 110 can sequentially explore multiple voxels based on specified conditions associated with multiple axes forming multiple subspaces.

[0068] For example, the processor 110 can sequentially explore multiple voxels based on a y-axis (e.g., a lateral axis) extending from the right side of the vehicle to the left side, an x-axis (e.g., a longitudinal axis) extending from the rear side of the vehicle to the front side, and a z-axis (e.g., a vertical axis) extending from the bottom of the vehicle to the top of the vehicle.

[0069] For example, processor 110 can identify at least a portion of the point cloud among multiple voxels by sequentially exploring multiple voxels included in multiple subspaces according to specified conditions.

[0070] For example, an effective voxel may include a voxel that has at least one point within the voxel.

[0071] For example, processor 110 can sequentially explore the y-axis, which increases towards the left side of the vehicle; the x-axis, which increases towards the front of the vehicle; and the z-axis, which increases towards the top of the vehicle. For example, processor 110 can identify at least a portion of the point cloud in at least some of a plurality of voxels based on the sequential exploration of the y-axis, x-axis, and z-axis. In other words, the y-axis can extend from the right side of the vehicle to the left side. The x-axis can extend from the rear side of the vehicle to the front side. The z-axis can extend from the bottom of the vehicle to the top of the vehicle.

[0072] For example, processor 110 can identify effective voxels, including at least a portion of the point cloud, among a plurality of voxels by sequentially exploring the y-axis increasing toward the left side of the vehicle, the x-axis increasing toward the front of the vehicle, and the z-axis increasing toward the top of the vehicle.

[0073] For example, processor 110 may assign an index to an effective voxel. For example, processor 110 may assign at least one index to an effective voxel that is associated with at least one of a plurality of subspaces, a plurality of voxels, and an effective voxel, or any combination thereof.

[0074] For example, processor 110 may assign a first index associated with multiple subspaces to effective voxels. For example, processor 110 may assign a second index associated with multiple voxels to effective voxels. For example, processor 110 may assign a third index associated with axes used to acquire multiple voxels to effective voxels. For example, processor 110 may assign a fourth index associated with the number of sub-voxels included in the multiple subspaces and the number of axes used to acquire multiple voxels to effective voxels.

[0075] For example, processor 110 may assign to effective voxels at least one or any combination thereof a first index associated with a plurality of subspaces, a second index associated with a plurality of voxels, a third index associated with axes used to acquire the plurality of voxels, and a fourth index associated with the number of sub-voxels and axes included in the plurality of subspaces.

[0076] Processor 110 can output information about effective voxels, including at least one index, based on assigning at least one index associated with at least one of a plurality of subspaces, a plurality of voxels, and effective voxels, or any combination thereof, to effective voxels.

[0077] For example, processor 110 may output information about effective voxels based on information that generates effective voxels including at least one index. For example, the information about effective voxels may include a mapping table that includes at least one index assigned to effective voxels.

[0078] Processor 110 can obtain the first index based on the order in which multiple subspaces are explored.

[0079] Processor 110 can identify the coordinates of at least a portion of the voxels in the identified point cloud. For example, processor 110 can identify the coordinates of valid voxels. For example, processor 110 can obtain a second index based on the voxel coordinates. For example, processor 110 can obtain a second index corresponding to the voxel coordinates.

[0080] For example, the coordinates of a voxel can include coordinate values ​​identified in a vehicle coordinate system formed around the vehicle. For example, the vehicle coordinate system can include an x-axis that increases toward the front of the vehicle, a y-axis that increases toward the left side of the vehicle, and a z-axis that increases toward the top of the vehicle.

[0081] The processor 110 may obtain a third index based on the number of axes. For example, these axes may include at least one or any combination of spatial axes associated with space and time axes associated with time (e.g., time axes).

[0082] For example, spatial axes associated with space may include at least one or any combination of the x-axis, y-axis, and z-axis. For example, time-associated axes may include a time axis associated with the time point at which the point cloud was acquired.

[0083] The processor 110 can identify the specified number of axes and the number of sub-voxels based on the specified number of times the specified space is divided into axes. For example, the specified number of axes may include at least one or any combination of the x-axis, y-axis, z-axis, and time axis.

[0084] For example, processor 110 can obtain a fourth index based on a specified number and the number of sub-voxels.

[0085] Processor 110 can generate a mapping table that includes at least one of a first exponent, a second exponent, a third exponent, and a fourth exponent, or any combination thereof. For example, processor 110 can output the generated mapping table.

[0086] For example, processor 110 can output information including the effective voxels of the mapping table.

[0087] As described above, the object recognition device 100 can assign at least one index to effective voxels for classification. The object recognition device 100 can effectively classify effective voxels by assigning at least one index to effective voxels according to specified conditions.

[0088] Figure 2 An example of dividing a specified space into multiple subspaces is shown.

[0089] refer to Figure 2 Object recognition devices (e.g., Figure 1 The processor of the object recognition device 100 (e.g., Figure 1 The processor 110 in the system can identify a virtual frame 203 corresponding to an external object in the designated space 200. For example, the virtual frame 203 may include a sensor (e.g., Figure 1 The point cloud obtained by sensor 120.

[0090] The processor can identify a virtual frame 203 located in front of the vehicle 201 within the designated space 200. For example, the front of the vehicle 201 may include the orientation of the first axis 211 relative to the vehicle in the first axis 211 and the second axis 213.

[0091] The processor can select a first virtual frame 221 from the virtual frames 203 based on the identification of the virtual frame 203. For example, the processor can identify the diagonal length 223 of the first virtual frame 221.

[0092] The processor can divide the specified space 200 into multiple subspaces based on the identification of the diagonal length 223. For example, subspace 230 may include one of the multiple subspaces.

[0093] The processor can set the size of the subspace 230 based on the diagonal length 223.

[0094] For example, the length 231 of subspace 230 in the y-axis direction can be equal to the diagonal length 223. For example, the length 233 of subspace 230 in the x-axis direction can be equal to the diagonal length 223. For example, the length 235 of subspace 230 in the z-axis direction can be equal to the height of the specified space 200.

[0095] For example, subspace 230 may be referred to as a voxel exploration space. For example, subspace 230 may include spaces with the following shapes... Tensor data. For example, the total number of voxels constituting subspace 230 could be... The above. This can include a specified space with a height of 200. (The above...) This can include a diagonal length of 223. (The above...) This can include the height of the voxel. (The above) This can include the length of the voxel in the y-direction. (The above...) It can include the length of the voxel in the x-direction.

[0096] The processor can identify at least one valid voxel included in the voxels of the subspace 230 based on the obtained subspace 230.

[0097] Figure 3 An example of exploring effective voxels is shown.

[0098] refer to Figure 3 Object recognition devices (e.g., Figure 1 The processor of the object recognition device 100 (e.g., Figure 1 The processor 110 can divide the specified space 300 into multiple subspaces 305.

[0099] The processor can identify voxels 310 and 320 included in subspace 305. For example, the processor can identify valid voxels 310 and 320 that include at least a portion of the point cloud. For example, the processor can identify empty voxels 320 and 310 that do not include at least a portion of the point cloud.

[0100] For example, the processor can explore voxels 310 and 320 sequentially. For example, the processor can explore voxels 310 and 320 along the y-axis. For example, the processor can explore voxels 310 and 320 along the x-axis based on exploring voxels 310 and 320 along the x-axis. For example, the processor can explore voxels 310 and 320 along the z-axis based on exploring voxels 310 and 320 along the x-axis.

[0101] For example, the processor can move by a shift of 1 in the x-axis direction, and based on exploring voxels 310 and 320 in the y-axis direction, explore voxels 310 and 320 again in the y-axis direction. If the above process is repeatedly performed to explore all voxels 310 and 320 located on the first plane, the processor can move by a shift of 1 in the z-axis direction to explore voxels 310 and 320 in the y-axis direction.

[0102] As described above, the processor can first explore voxels 310 and 320 located in each layer along the y-axis, and then perform the exploration while moving 1 unit along the x-axis. By repeating the above process, the processor can identify valid voxels 310.

[0103] The processor can identify the coordinate values ​​of the effective voxel 310. For example, the processor can identify the coordinate values ​​corresponding to the position of the effective voxel 310. The coordinate values ​​corresponding to the position of the effective voxel 310 can be referred to as system voxel space coordinates.

[0104] Figure 4 An example of a mapping table is shown.

[0105] refer to Figure 4 Object recognition devices (e.g., Figure 1 The processor of the object recognition device 100 (e.g., Figure 1 The processor 110 can generate a mapping table. For example, the mapping table may include at least one or any combination of system voxel space coordinates, exploration space indices, voxel set indices, and set interior indices.

[0106] For example, the exploration space index may include a first index. For example, the system voxel space coordinates may include a second index. For example, the set interior index may include a third index. For example, the voxel set index may include a fourth index.

[0107] The process of obtaining the system voxel space coordinates, exploration space index, voxel set index, and set interior index is described below.

[0108] The processor can identify the coordinates of a valid voxel based on identifying a valid voxel comprising at least a portion of a point cloud. For example, the coordinates of a valid voxel can be represented based on a vehicle coordinate system formed around the vehicle. For example, the coordinates of a valid voxel can include the x, y, and z coordinates of the voxel, which comprises at least a portion of the point cloud in the vehicle coordinate system formed around the vehicle.

[0109] For example, the processor can obtain system voxel space coordinates, including the coordinates of effective voxels. Based on the obtained system voxel space coordinates, the processor can store the system voxel space coordinates in at least a portion of a mapping table.

[0110] The processor can obtain exploration space indices. For example, exploration space indices can include indices assigned to subspaces included in a specified space. For example, the processor can assign an index to each subspace to indicate the location of the subspace, or assign an index based on the order in which the specified space is divided into subspaces (e.g., regions).

[0111] For example, the processor can explore subspaces sequentially according to exploration space indices. For instance, the processor can explore a subspace assigned a first exploration space index, and subsequently explore a subspace assigned a second exploration space index. As described above, the processor can explore voxels included in a subspace according to the order in which exploration space indices are assigned to the subspaces.

[0112] The processor can obtain a voxel set index. For example, the processor can obtain the voxel set index based on the number of voxels included in the exploration space and the size of the voxel set. For example, the voxel set index can be related to the number of dynamic voxel sets.

[0113] For example, the number of voxels included in the exploration space can include the total number of voxels included in the subspace. For example, the size of the voxel set can include a value obtained by dividing the number of voxels included in the exploration space by the exploration space partition size. For example, the exploration space partition size can be represented as... .here, This can include the number of axes that form the voxel space. Therefore, if the voxel space is formed by the x-axis, y-axis, and z-axis, then... It could be 3. For example, if the voxel space is formed by the x-axis, y-axis, z-axis, and time axis, then... It can be 4. For example, if the value obtained by dividing the number of voxels included in the exploration space by the size of the voxel set is not a natural number, the processor can obtain a natural number by rounding the value up to the nearest decimal place and setting the obtained natural number as the voxel set index.

[0114] The processor can obtain the set's internal index. For example, the set's internal index can be related to the size of the voxel set. For example, as mentioned above, the size of the voxel set can include a value obtained by dividing the number of voxels included in the exploration space by the size of the exploration space partition.

[0115] As described above, the processor of the object recognition device can generate a mapping table that includes system voxel space coordinates, exploration space indices, voxel set indices, and set interior indices. The processor can output the generated mapping table or use the generated mapping table to control the vehicle.

[0116] Figure 5 An example flowchart associated with an object recognition method is shown.

[0117] In the following text, it is assumed that... Figure 1 Object recognition device 100 performs Figure 5 The processing. Furthermore, in Figure 5 In the description, the operation described as being performed by the device can be understood as being controlled by the processor 110 of the object recognition device 100.

[0118] Figure 5 At least one operation can be performed by Figure 1 The object recognition device 100 performs the operation. Figure 5 At least one operation can be performed by Figure 1 The processor 110 executes. Figure 5 Operations in the program can be executed sequentially, but they do not necessarily have to be. For example, the order of operations can be changed, and at least two operations can be executed in parallel.

[0119] refer to Figure 5 In operation S501, the object recognition method may include: dividing the specified space into multiple subspaces by projecting a point cloud onto a specified space that includes multiple voxels.

[0120] For example, an object recognition method may include recognizing a virtual bounding box comprising a point cloud. For example, an object recognition method may include recognizing the diagonal length if the virtual bounding box is viewed along a specified direction. For example, an object recognition method may include recognizing the diagonal length if the virtual bounding box comprising a point cloud is viewed along a specified direction. For example, an object recognition method may include dividing a specified space into multiple subspaces using the diagonal length.

[0121] For example, the size of each voxel among multiple voxels can be set by at least one of the users and suppliers, or any combination thereof. Similarly, the size of each subspace among multiple subspaces can be set by at least one of the users and suppliers, or any combination thereof. For example, the size of each voxel among multiple voxels and the size of each subspace among multiple subspaces, or any combination thereof, can be set by at least one of the users and suppliers, or any combination thereof.

[0122] In operation S503, the object recognition method may include: based on specified conditions, sequentially exploring multiple voxels included in each of multiple subspaces, and identifying valid voxels among the multiple voxels that include at least a portion of the point cloud.

[0123] For example, object recognition methods may include identifying effective voxels comprising at least a portion of a point cloud by sequentially exploring a y-axis increasing toward the left side of the vehicle, an x-axis increasing toward the front of the vehicle, and a z-axis increasing toward the top of the vehicle.

[0124] In operation S505, the object recognition method may include: outputting information about effective voxels including at least one index based on assigning at least one index to effective voxels, the at least one index being associated with at least one or any combination of a plurality of subspaces, a plurality of voxels, and effective voxels. A vehicle may identify objects associated with a point cloud based on effective voxels including at least one index (e.g., determining the object's location, orientation, size, etc.). Vehicle operations (e.g., autonomous driving) may be performed based on the identified objects.

[0125] For example, an object recognition method may include at least one or any combination thereof of: assigning an effective voxel a first index associated with a plurality of subspaces, a second index associated with a plurality of voxels, a third index associated with an axis used to acquire the plurality of voxels, and a fourth index associated with the number of sub-voxels included in the plurality of subspaces and the number of axes.

[0126] For example, object recognition methods may include obtaining a first index based on the order of exploring multiple subspaces.

[0127] For example, an object recognition method may include identifying the coordinates of valid voxels comprising at least a portion of a point cloud. For example, an object recognition method may include obtaining a second index based on the coordinates of the valid voxels.

[0128] For example, the coordinates of an effective voxel may include coordinate values ​​identified in a vehicle coordinate system formed around the vehicle.

[0129] For example, a vehicle coordinate system may include an x-axis that increases toward the front of the vehicle, a y-axis that increases toward the left side of the vehicle, and a z-axis that increases toward the top of the vehicle.

[0130] For example, object recognition methods may include obtaining a third index based on the number of axes. For example, the axis may include at least one or any combination of a spatial axis associated with space and a temporal axis associated with time.

[0131] For example, an object recognition method may include identifying a specified number of axes and a number of sub-voxels by dividing a specified space into one or more regions based on a specified number of axes. For example, an object recognition method may include obtaining a fourth index based on a specified number of axes and a number of sub-voxels.

[0132] Object recognition methods may include generating a mapping table that includes at least one or any combination of a first index, a second index, a third index, and a fourth index. For example, an object recognition method may include generating a mapping table that includes a first index, a second index, a third index, and a fourth index. For example, an object recognition method may include outputting the generated mapping table.

[0133] Figure 6 A computational system related to an object recognition device or object classification method is shown.

[0134] refer to Figure 6 The computing system 1000 may include at least one processor 1100, a memory 1300, a user interface input device 1400, a user interface output device 1500, a storage device 1600, and a network interface 1700 connected to each other via a bus 1200.

[0135] Processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in memory 1300 and / or storage device 1600. Memory 1300 and storage device 1600 may include various types of volatile or non-volatile storage media. For example, memory 1300 may include read-only memory (ROM) and random access memory (RAM).

[0136] Therefore, the operation of the methods or algorithms described herein can be directly embodied in hardware or software modules or combinations thereof executed by processor 1100. Software modules can reside on storage media such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disks, removable disks, and CD-ROMs (i.e., memory 1300 and / or storage device 1600).

[0137] An exemplary storage medium may be coupled to processor 1100, and processor 1100 may read information from the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with processor 1100. The processor and storage medium may reside in an application-specific integrated circuit (ASIC). The ASIC may reside within the user terminal. In another case, the processor and storage medium may reside as separate components in the user terminal.

[0138] According to one aspect of this disclosure, an object recognition device includes a sensor and a processor for acquiring a point cloud. The processor can identify valid voxels comprising at least a portion of the point cloud based on projecting the point cloud into a specified space comprising a plurality of voxels, dividing the specified space into a plurality of subspaces using the size of the point cloud, sequentially exploring the plurality of voxels included in the plurality of subspaces according to specified conditions, identifying valid voxels comprising at least a portion of the point cloud among the plurality of voxels, and outputting information about the valid voxels including at least one index based on assigning at least one index associated with at least one of the plurality of subspaces, the plurality of voxels, and the valid voxels, or any combination thereof.

[0139] The processor can assign at least one or any combination thereof to the effective voxels: a first index associated with multiple subspaces, a second index associated with multiple voxels, a third index associated with axes used to acquire multiple voxels, and a fourth index associated with the number of multiple sub-voxels and axes included in multiple subspaces.

[0140] If a virtual box containing a point cloud is viewed along a specified direction, the processor can identify the diagonal length based on the identified virtual box and divide the specified space into multiple subspaces by using the diagonal length.

[0141] The processor can identify effective voxels, including at least a portion of the point cloud, by sequentially exploring the y-axis increasing toward the left side of the vehicle, the x-axis increasing toward the front of the vehicle, and the z-axis increasing toward the top of the vehicle.

[0142] The processor can identify a specified number of axes and a specified number of voxels based on a specified number of axes that divide a specified space into axes, and obtain a fourth index based on the specified number of axes and the number of sub-voxels.

[0143] The processor can obtain the first index based on the order in which multiple subspaces are explored.

[0144] The processor can identify the coordinates of effective voxels, including at least a portion of the point cloud, and obtain a second index based on the coordinates of the effective voxels.

[0145] The coordinates of an effective voxel can include coordinate values ​​identified in a vehicle coordinate system relative to the vehicle. The vehicle coordinate system can include an x-axis that increases toward the front of the vehicle, a y-axis that increases toward the left side of the vehicle, and a z-axis that increases toward the top of the vehicle.

[0146] The processor can obtain a third index based on the number of axes. The axes may include at least one or any combination of spatial axes associated with space and time axes associated with time.

[0147] The size of each voxel in the plurality of voxels, and at least one or any combination thereof, of the sizes of each subspace in the plurality of subspaces, may be set by at least one or any combination thereof, of the user and the supplier.

[0148] The processor can generate a mapping table that includes at least one of the first, second, third, and fourth indices or any combination thereof, and output the mapping table.

[0149] According to one aspect of this disclosure, an object recognition method includes: dividing the specified space into multiple subspaces by a processor using the size of the point cloud based on projecting a point cloud onto a specified space comprising multiple voxels; identifying valid voxels comprising at least a portion of the point cloud based on sequentially exploring multiple voxels comprising the multiple subspaces according to specified conditions; and outputting information about the valid voxels comprising at least one index based on assigning at least one index associated with at least one of the multiple subspaces, the multiple voxels, and the valid voxels or any combination thereof to the valid voxels.

[0150] The object recognition method may further include: assigning at least one of the following to an effective voxel: a first index associated with a plurality of subspaces, a second index associated with a plurality of voxels, a third index associated with an axis used to acquire the plurality of voxels, and a fourth index associated with the number of the plurality of sub-voxels included in the plurality of subspaces and the number of axes, or any combination thereof.

[0151] The object recognition method may further include: if a virtual box including a point cloud is viewed along a specified direction based on the recognition virtual box, then the diagonal length is identified, and the specified space is divided into multiple subspaces by using the diagonal length.

[0152] The object recognition method may further include identifying effective voxels comprising at least a portion of the point cloud by sequentially exploring the y-axis increasing toward the left side of the vehicle, the x-axis increasing toward the front of the vehicle, and the z-axis increasing toward the top of the vehicle.

[0153] The object recognition method may further include: identifying a specified number of axes and a specified number of sub-voxels based on a specified number of axes that divide a specified space into axes, and obtaining a fourth index based on the specified number of axes and the specified number of sub-voxels.

[0154] Object recognition methods may also include obtaining a first index based on the order of exploring multiple subspaces.

[0155] The object recognition method may further include: identifying the coordinates of effective voxels comprising at least a portion of the point cloud, and obtaining a second index based on the coordinates of the effective voxels.

[0156] The coordinates of an effective voxel may include coordinate values ​​identified in a vehicle coordinate system relative to the vehicle. The vehicle coordinate system may include an x-axis extending towards the front of the vehicle, a y-axis extending towards the left side of the vehicle, and a z-axis extending towards the top of the vehicle.

[0157] The object recognition method may further include obtaining a third index based on the number of axes. The axes may include at least one of a spatial axis associated with space and a temporal axis associated with time, or any combination thereof.

[0158] The above description is merely an illustration of the technical concept of this disclosure, and those skilled in the art to which this disclosure pertains can make various modifications and changes without departing from the essential characteristics of this disclosure.

[0159] Therefore, the one or more exemplary embodiments disclosed herein are not intended to limit the technical concept of this disclosure, but rather to describe it, and the scope of the technical concept of this disclosure is not limited by the exemplary embodiments. The scope of protection of this disclosure should be interpreted by the appended claims, and all technical concepts within the scope of equivalents thereto should be interpreted as included within the scope of this disclosure.

[0160] This technology can effectively identify effective voxels.

[0161] Furthermore, this technology can dynamically generate effective voxel sets.

[0162] Furthermore, this technique can obtain the relationship between effective voxels.

[0163] In addition, various effects can be provided directly or indirectly through this disclosure.

[0164] While this disclosure has been described above with reference to exemplary embodiments and accompanying drawings, it is not limited thereto. Various modifications and alterations may be made by those skilled in the art without departing from the spirit and scope of this disclosure as claimed in the appended claims.

Claims

1. An object recognition device, comprising: The sensor is configured to acquire a point cloud associated with an object; as well as processor, The processor is configured as follows: Based on the size of the point cloud, a specified space is divided into multiple subspaces, wherein the point cloud is projected into the specified space, and wherein the specified space includes multiple voxels included in the multiple subspaces; Among the plurality of voxels and based on sequential exploration of the plurality of voxels, one or more valid voxels comprising at least a portion of the point cloud are identified; Assign at least one index associated with at least one of the plurality of subspaces, the plurality of voxels, and the one or more effective voxels to the one or more effective voxels; The object is identified based on the one or more effective voxels and the at least one index; and The vehicle's operation is controlled based on the identified objects.

2. The object recognition device according to claim 1, wherein, The processor is configured to allocate the at least one index by the following steps: Assign at least one of the following to the one or more effective voxels: The first index is associated with the plurality of subspaces. The second index is associated with the plurality of voxels. The third index is associated with the axis used to obtain the plurality of voxels, and The fourth index is associated with the number of sub-voxels included in the plurality of subspaces and with the number of axes.

3. The object recognition device according to claim 1, wherein, The processor is configured to partition the designated space by the following steps: Identify virtual bounding boxes including the point cloud; The diagonal length of the virtual bounding box is identified based on a specified direction of viewing the point cloud; and The specified space is divided into the plurality of subspaces based on the diagonal length.

4. The object recognition device according to claim 1, wherein, The processor is configured to identify the one or more valid voxels by the following steps: Identifying the one or more valid voxels is also based on sequentially exploring: a lateral axis extending from the right side of the vehicle to the left side of the vehicle, a longitudinal axis extending from the rear side of the vehicle to the front side of the vehicle, and a vertical axis extending from the bottom of the vehicle to the top of the vehicle.

5. The object recognition device according to claim 2, wherein, The processor is also configured to: The number of sub-voxels is identified by dividing the designated space into multiple regions according to the number of axes; and The fourth index is obtained based on the number of axes and the number of sub-voxels.

6. The object recognition device according to claim 2, wherein, The processor is also configured to: The first index is obtained based on the order in which the multiple subspaces are explored.

7. The object recognition device according to claim 2, wherein, The processor is also configured to: Identify the coordinates of the one or more effective voxels; and The second index is obtained based on the coordinates of the one or more effective voxels.

8. The object recognition device according to claim 7, wherein, The coordinates of the one or more effective voxels include coordinate values ​​identified in the vehicle's vehicle coordinate system, and The vehicle coordinate system includes a longitudinal axis extending from the rear of the vehicle to the front of the vehicle, a transverse axis extending from the right side of the vehicle to the left side of the vehicle, and a vertical axis extending from the bottom of the vehicle to the top of the vehicle.

9. The object recognition device according to claim 2, wherein, The processor is also configured to obtain the third index based on the number of axes; and The axis includes at least one of a spatial axis and a time axis.

10. The object recognition device according to claim 1, wherein, At least one of the dimensions of each of the plurality of voxels and the dimensions of each of the plurality of subspaces is set by at least one of the user and the supplier.

11. The object recognition device according to claim 2, wherein, The processor is also configured to: Generate a mapping table including at least one of the first index, the second index, the third index, and the fourth index; and Output the mapping table.

12. A method performed by a device in a vehicle, the method comprising the following steps: Based on the size of the point cloud associated with the object, a specified space is divided into multiple subspaces, wherein the point cloud is projected into the specified space, and wherein the specified space includes multiple voxels included in the multiple subspaces; Among the plurality of voxels and based on sequential exploration of the plurality of voxels, one or more valid voxels comprising at least a portion of the point cloud are identified; and Assign at least one index associated with at least one of the plurality of subspaces, the plurality of voxels, and the one or more effective voxels to the one or more effective voxels; The object is identified based on the one or more effective voxels and the at least one index; and The operation of the vehicle is controlled based on the identified object.

13. The method according to claim 12, wherein, Allocating the at least one index includes the following steps: Assign at least one of the following to the one or more effective voxels: The first index is associated with the plurality of subspaces. The second index is associated with the plurality of voxels. The third index is associated with the axis used to obtain the plurality of voxels, and The fourth index is associated with the number of sub-voxels included in the plurality of subspaces and with the number of axes.

14. The method according to claim 12, wherein, The partitioning of the specified space includes the following steps: Identify virtual bounding boxes including the point cloud; The diagonal length of the virtual bounding box is identified based on a specified direction of viewing the point cloud; and The specified space is divided into multiple subspaces based on the diagonal length.

15. The method according to claim 12, wherein, Identifying the one or more valid voxels includes the following steps: Identifying the one or more valid voxels is also based on sequentially exploring: a lateral axis extending from the right side of the vehicle to the left side of the vehicle, a longitudinal axis extending from the rear side of the vehicle to the front side of the vehicle, and a vertical axis extending from the bottom of the vehicle to the top of the vehicle.

16. The method of claim 13, further comprising the step of: The number of sub-voxels is identified by dividing the specified space into multiple regions according to the number of axes. as well as The fourth index is obtained based on the number of axes and the number of sub-voxels.

17. The method of claim 13, further comprising the step of: The first index is obtained based on the order in which the multiple subspaces are explored.

18. The method of claim 13, further comprising the step of: Identify the coordinates of the one or more effective voxels; and The second index is obtained based on the coordinates of the one or more effective voxels.

19. The method according to claim 18, wherein, The coordinates of the one or more effective voxels include coordinate values ​​identified in the vehicle's vehicle coordinate system, and The vehicle coordinate system includes a longitudinal axis extending from the rear of the vehicle to the front of the vehicle, a transverse axis extending from the right side of the vehicle to the left side of the vehicle, and a vertical axis extending from the bottom of the vehicle to the top of the vehicle.

20. The method of claim 13, further comprising the step of: The third index is obtained based on the number of axes, and The axis includes at least one of a spatial axis and a time axis.

Images