Method and apparatus for pre-processing and post-processing for priority detection of high-speed moving object

Abstract

Provided are a method for performing wireless communication and sensing, and an apparatus supporting same. A first device may detect an object, obtain information related to mobility of the object, and transmit a message including the information about the object. The information about the object may be preferentially included in the message on the basis of the information related to mobility.

Description

Preprocessing and postprocessing methods and devices for high-speed mobile object priority detection

[0001] The present disclosure relates to a wireless communication system.

[0002] 5G NR, the successor to LTE (long-term evolution), is a new clean-slate mobile communications system characterized by high performance, low latency, and high availability. 5G NR can utilize all available spectrum resources, from low-frequency bands below 1 GHz, mid-frequency bands between 1 GHz and 10 GHz, and high-frequency (millimeter wave) bands above 24 GHz.

[0003] The 6G (wireless communication) system aims to achieve (i) very high data rates per device, (ii) a very large number of connected devices, (iii) global connectivity, (iv) very low latency, (v) low energy consumption for battery-free Internet of Things (IoT) devices, (vi) ultra-reliable connectivity, and (vii) connected intelligence with machine learning capabilities. The vision of the 6G system can be divided into four aspects: intelligent connectivity, deep connectivity, holographic connectivity, and ubiquitous connectivity, and the 6G system can satisfy the requirements as shown in Table 1 below. For example, Table 1 can represent an example of the requirements of a 6G system.

[0004] Per device peak data rate 1 Tbps E2E latency 1 ms Maximum spectral efficiency 100 bps / Hz Mobility support Up to 1000 km / hr Satellite integration Fully AI Fully autonomous vehicle Fully XR Fully haptic communication Fully

[0005] In one embodiment, a method of performing sensing by a first device is provided. The method may include: detecting an object by the first device; transmitting a message including information about the object by the first device; and transmitting a message including information about the object by the first device. Information about the object may be preferentially included in the message based on information related to mobility.

[0006] In one embodiment, a first device configured to perform sensing is provided. The first device comprises at least one transceiver; at least one processor; and at least one memory coupled to the at least one processor and storing instructions, wherein the instructions, when executed by the at least one processor, cause the device to: detect an object; obtain information related to mobility of the object; and transmit a message including information about the object. The information about the object may be preferentially included in the message based on the information related to mobility.

[0007] In one embodiment, a processing device configured to control a first device is provided. The processing device includes at least one processor; and at least one memory coupled to the at least one processor and storing instructions, wherein the instructions, when executed by the at least one processor, cause the first device to: detect an object; obtain information related to mobility of the object; and transmit a message including information about the object. The information about the object may be preferentially included in the message based on the information related to mobility.

[0008] In one embodiment, a non-transitory computer-readable storage medium having recorded thereon instructions is provided. The instructions, when executed, may cause a first device to: detect an object; obtain mobility-related information about the object; and transmit a message including information about the object. Information about the object may be preferentially included in the message based on the mobility-related information.

[0009] FIG. 1 illustrates a communication structure that can be provided in a 6G system according to one embodiment of the present disclosure.

[0010] FIG. 2 illustrates an electromagnetic spectrum according to one embodiment of the present disclosure.

[0011] FIG. 3 illustrates a degradation in object detection performance and occurrence of false negatives due to motion blur according to an embodiment of the present disclosure.

[0012] FIG. 4 illustrates object detection candidates and blur metric values ​​according to one embodiment of the present disclosure.

[0013] FIG. 5 illustrates a method for high-speed moving object priority detection operation when using an AI / ML model of R-CNN according to one embodiment of the present disclosure.

[0014] FIG. 6 illustrates a method for skipping a high-speed moving object priority detection or classification operation according to one embodiment of the present disclosure.

[0015] FIG. 7 illustrates a method for a first device to perform wireless communication according to an embodiment of the present disclosure.

[0016] FIG. 8 illustrates a method for a second device to perform wireless communication according to one embodiment of the present disclosure.

[0017] FIG. 9 illustrates a communication system (1) according to one embodiment of the present disclosure.

[0018] FIG. 10 illustrates a wireless device according to an embodiment of the present disclosure.

[0019] FIG. 11 illustrates a signal processing circuit for a transmission signal according to one embodiment of the present disclosure.

[0020] FIG. 12 illustrates a wireless device according to one embodiment of the present disclosure.

[0021] FIG. 13 illustrates a mobile device according to one embodiment of the present disclosure.

[0022] FIG. 14 illustrates a vehicle or autonomous vehicle according to one embodiment of the present disclosure.

[0023] In this disclosure, "A or B" can mean "only A," "only B," or "both A and B." In other words, "A or B" in this disclosure can be interpreted as "A and / or B." For example, "A, B or C" in this disclosure can mean "only A," "only B," "only C," or "any combination of A, B and C."

[0024] As used herein, a slash ( / ) or a comma may mean "and / or." For example, "A / B" may mean "A and / or B." Accordingly, "A / B" may mean "only A," "only B," or "both A and B." For example, "A, B, C" may mean "A, B, or C."

[0025] In the present disclosure, “at least one of A and B” may mean “only A,” “only B,” or “both A and B.” Additionally, in the present disclosure, the expressions “at least one of A or B” or “at least one of A and / or B” may be interpreted identically to “at least one of A and B.”

[0026] Additionally, in the present disclosure, “at least one of A, B and C” can mean “only A,” “only B,” “only C,” or “any combination of A, B and C.” Additionally, “at least one of A, B or C” or “at least one of A, B and / or C” can mean “at least one of A, B and C.”

[0027] Additionally, parentheses used in the present disclosure may mean "for example." Specifically, when indicated as "control information (PDCCH)", "PDCCH" may be proposed as an example of "control information." In other words, "control information" in the present disclosure is not limited to "PDCCH," and "PDCCH" may be proposed as an example of "control information." Furthermore, even when indicated as "control information (i.e., PDCCH)", "PDCCH" may be proposed as an example of "control information."

[0028] In the following explanation, ‘when, if, in case of’ can be replaced with ‘based on’.

[0029] Technical features individually described in one drawing in this disclosure may be implemented individually or simultaneously.

[0030] In the present disclosure, higher layer parameters may be parameters set for the terminal, preset, or predefined. For example, a base station or network may transmit higher layer parameters to the terminal. For example, the higher layer parameters may be transmitted via radio resource control (RRC) signaling or medium access control (MAC) signaling.

[0031] In this disclosure, “setting or defining” may be interpreted as being preset to a device.

[0032] Wireless communication systems are multiple access systems that support communication with multiple users by sharing available system resources (e.g., bandwidth, transmission power, etc.). Examples of multiple access systems include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA), and multi-carrier frequency division multiple access (MC-FDMA).

[0033] Sidelink (SL) refers to a communication method that establishes a direct link between user equipment (UE), allowing voice or data to be exchanged directly between terminals without going through a base station (BS). SL is being considered as a solution to address the burden on base stations due to rapidly increasing data traffic.

[0034] V2X (vehicle-to-everything) refers to a communication technology that exchanges information with other vehicles, pedestrians, and infrastructure-based objects through wired / wireless communication. V2X can be divided into four types: V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2N (vehicle-to-network), and V2P (vehicle-to-pedestrian). V2X communication can be provided through the PC5 interface and / or Uu interface.

[0035] Meanwhile, as more and more communication devices demand greater communication capacity, the need for improved mobile broadband communication compared to existing radio access technology (RAT) is emerging. Accordingly, communication systems that consider services or terminals sensitive to reliability and latency are being discussed. Next-generation wireless access technologies that consider improved mobile broadband communication, massive machine type communication (MTC), and ultra-reliable and low latency communication (URLLC) can be called new radio access technology (RAT) or new radio (NR). NR can also support vehicle-to-everything (V2X) communication.

[0036] The technology proposed in the present disclosure can be used in various wireless communication systems such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), SC-FDMA (single carrier frequency division multiple access), etc. CDMA can be implemented with wireless technologies such as UTRA (universal terrestrial radio access) or CDMA2000. TDMA can be implemented with wireless technologies such as GSM (global system for mobile communications) / GPRS (general packet radio service) / EDGE (enhanced data rates for GSM evolution). OFDMA can be implemented with wireless technologies such as IEEE (Institute of Electrical and Electronics Engineers) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802-20, E-UTRA (evolved UTRA), etc. IEEE 802.16m is an evolution of IEEE 802.16e, providing backward compatibility with systems based on IEEE 802.16e. UTRA is part of the universal mobile telecommunications system (UMTS).3GPP (3rd generation partnership project) LTE (long term evolution) is a part of E-UMTS (evolved UMTS) that uses E-UTRA (evolved-UMTS terrestrial radio access). It employs OFDMA in the downlink and SC-FDMA in the uplink. LTE-A (advanced) is an evolution of 3GPP LTE.

[0037] The technology proposed in this disclosure can be implemented with 6G wireless technology and applied to various 6G systems. For example, 6G systems can have key factors such as enhanced mobile broadband (eMBB), ultra-reliable low latency communications (URLLC), massive machine-type communication (mMTC), artificial intelligence (AI) integrated communication, tactile internet, high throughput, high network capacity, high energy efficiency, low backhaul and access network congestion, and enhanced data security.

[0038] Figure 1 illustrates a communication structure that can be provided in a 6G system according to one embodiment of the present disclosure. The embodiment of Figure 1 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0039] 6G systems are expected to have 50 times the simultaneous wireless connectivity of 5G systems. URLLC, a key feature of 5G, will become even more crucial in 6G communications by providing end-to-end latency of less than 1 ms. 6G systems will have significantly higher volumetric spectral efficiency, compared to the commonly used area spectral efficiency. 6G systems can offer extremely long battery life and advanced battery technologies for energy harvesting, eliminating the need for separate charging for mobile devices in 6G systems. New network characteristics in 6G may include:

[0040] - Satellite integrated network: 6G is expected to integrate with satellites to provide a global mobile network. The integration of terrestrial, satellite, and airborne networks into a single wireless communications system is crucial for 6G.

[0041] - Connected Intelligence: Unlike previous generations of wireless communication systems, 6G is revolutionary, upgrading the wireless evolution from "connected objects" to "connected intelligence." AI can be applied at every stage of the communication process (or at every signal processing step, as described below).

[0042] - Seamless integration of wireless information and energy transfer: 6G wireless networks will transfer power to charge the batteries of devices such as smartphones and sensors. Therefore, wireless information and energy transfer (WIET) will be integrated.

[0043] - Ubiquitous super 3D connectivity: Access to networks and core network functions of drones and very low Earth orbit satellites will create super 3D connectivity in 6G ubiquitous.

[0044] Some general requirements for the new network characteristics of 6G, such as the above, may be as follows:

[0045] - Small cell networks: The concept of small cell networks was introduced to improve received signal quality in cellular systems by increasing throughput, energy efficiency, and spectral efficiency. Consequently, small cell networks are essential for 5G and beyond-5G (5GB) communication systems. Accordingly, 6G communication systems also adopt the characteristics of small cell networks.

[0046] Ultra-dense heterogeneous networks: Ultra-dense heterogeneous networks will be another key feature of 6G communication systems. Multi-tier networks comprised of heterogeneous networks improve overall QoS and reduce costs.

[0047] High-capacity backhaul: Backhaul connections are characterized by high-capacity backhaul networks to support high-volume traffic. High-speed fiber optics and free-space optics (FSO) systems may be potential solutions to this problem.

[0048] - Radar technology integrated with mobile technology: High-precision localization (or location-based services) through communications is a key feature of 6G wireless communication systems. Therefore, radar systems will be integrated with 6G networks.

[0049] - Softwarization and virtualization: Softwarization and virtualization are two critical features that form the foundation of the design process for 5GB networks to ensure flexibility, reconfigurability, and programmability. Furthermore, billions of devices can be shared on a shared physical infrastructure.

[0050] Below, the core implementation technologies of the 6G system are described.

[0051] - Artificial Intelligence (AI): The most important and newly introduced technology for 6G systems is AI. 4G systems did not involve AI. 5G systems will support partial or very limited AI. However, 6G systems will be fully AI-enabled for automation. Advances in machine learning will create more intelligent networks for real-time communications in 6G. Incorporating AI into communications can streamline and improve real-time data transmission. AI can use numerous analyses to determine how complex target tasks should be performed. This means AI can increase efficiency and reduce processing delays. Time-consuming tasks such as handovers, network selection, and resource scheduling can be performed instantly using AI. AI can also play a crucial role in machine-to-machine (M2M), machine-to-human, and human-to-machine communications. AI can also enable rapid communication in brain-computer interfaces (BCIs). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radios, self-sustaining wireless networks, and machine learning.

[0052] - THz communication (terahertz communication): Data rates can be increased by increasing the bandwidth. This can be achieved by utilizing sub-THz communication with a wide bandwidth and applying advanced massive MIMO technology. THz waves, also known as sub-millimeter waves, typically refer to the frequency range between 0.1 THz and 10 THz with corresponding wavelengths ranging from 0.03 mm to 3 mm. The 100 GHz to 300 GHz band (Sub-THz band) is considered a major portion of the THz band for cellular communications. Adding the Sub-THz band to the mmWave band will increase the capacity of 6G cellular communications. Among the defined THz bands, 300 GHz to 3 THz lies in the far infrared (IR) frequency band. While part of the optical band, the 300 GHz to 3 THz band lies at the boundary of the optical band, immediately following the RF band. Therefore, this 300 GHz to 3 THz band exhibits similarities to RF. Figure 2 illustrates the electromagnetic spectrum according to one embodiment of the present disclosure. The embodiment of FIG. 2 can be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted. Key characteristics of THz communications include (i) widely available bandwidth to support very high data rates, and (ii) high path loss at high frequencies (highly directional antennas are essential). The narrow beam width generated by the highly directional antenna reduces interference. The small wavelength of THz signals allows for a much larger number of antenna elements to be integrated into devices and BSs operating in this band. This enables the use of advanced adaptive array techniques to overcome range limitations.

[0053] - Large-scale MIMO technology

[0054] - Hologram beamforming (HBF)

[0055] - Optical wireless technology

[0056] - Free-space optical transmission backhaul network (FSO backhaul network)

[0057] - Non-Terrestrial Networks (NTN)

[0058] - Quantum communication

[0059] - Cell-free communication

[0060] - Integration of wireless information and power transmission

[0061] - Integration of wireless communication and sensing

[0062] - Integrated access and backhaul network

[0063] - Big data analysis

[0064] - Reconfigurable intelligent surface

[0065] - metaverse

[0066] - Blockchain

[0067] Unmanned aerial vehicles (UAVs): UAVs, or drones, will be a key element in 6G wireless communications. In most cases, high-speed data wireless connectivity can be provided using UAV technology. Base stations (BSs) can be installed on UAVs to provide cellular connectivity. UAVs may offer specific capabilities not found in fixed BS infrastructure, such as easy deployment, robust line-of-sight links, and controlled mobility. During emergencies such as natural disasters, deploying terrestrial communications infrastructure is not economically feasible and sometimes cannot provide services in volatile environments. UAVs can easily handle these situations. UAVs will become a new paradigm in wireless communications. This technology facilitates three fundamental requirements for wireless networks: enhanced mobile broadband (eMBB), URLLC, and mMTC. UAVs can also support various purposes, such as enhancing network connectivity, fire detection, disaster emergency services, security and surveillance, pollution monitoring, parking monitoring, and accident monitoring. Therefore, UAV technology is recognized as one of the most important technologies for 6G communications.

[0068] - Advanced air mobility (AAM): AAM is a higher concept than urban air mobility (UAM), which is an air transportation method available in urban areas, and can refer to a means of transportation that includes movement between regional hubs as well as urban areas.

[0069] - Autonomous driving (self-driving): For fully autonomous driving, vehicles must communicate with each other to alert each other of dangerous situations, and vehicles must communicate with infrastructure such as parking lots and traffic lights to confirm parking location information, signal change times, and other information. V2X (vehicle to everything), a key element of autonomous driving infrastructure, can be a technology that allows cars to communicate and share with various elements on the road for autonomous driving, such as vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) wireless communication. Fast transmission speeds and low-latency technologies are essential to maximize autonomous driving performance and ensure high safety. Furthermore, in the future, autonomous driving will go beyond simply providing warnings or guidance messages to drivers and may require active intervention and direct control of the vehicle in dangerous situations. To achieve this, the amount of information that must be transmitted and received can be enormous, so 6G is expected to maximize autonomous driving with faster transmission speeds and lower latency than 5G.

[0070] For clarity, the description focuses on 5G NR, but the technical concepts of one embodiment of the present disclosure are not limited thereto. Various embodiments of the present disclosure can also be applied to 6G communication systems.

[0071] Meanwhile, for example, in an Intelligent Transport System (ITS), road users can share status information via wireless communication, thereby improving traffic safety and efficiency. Furthermore, there may be a service (hereinafter referred to as a sensor data sharing service) that detects road users using sensors and shares the acquired information in the form of messages (e.g., ETSI TS 103 324 CPM, SAE J3224 SDSM). For example, this service may be provided by utilizing sensors mounted on infrastructure (e.g., Roadside Units (RSUs)) or mobile devices (e.g., vehicles, robots, and / or pedestrians). For example, a road user who receives a message from a sensor data sharing service can perceive the surrounding environment in more detail based on the information contained in the message, thereby enabling collision avoidance and efficient route planning. This, for example, can improve traffic safety and efficiency. For example, in providing an ITS sensor data sharing service, object detection using sensors may be the most important technological element. Additionally, for example, the performance of sensors for object detection, as well as artificial intelligence (AI) or machine learning (ML) (hereinafter, AI / ML), can play a significant role. For example, various sensors (e.g., RADAR, LiDAR, and / or cameras) can be used for object detection in ITS. For example, object detection using cameras in particular may be being developed and applied primarily. For example, various AI / ML methods are used not only in the ITS field but also for object detection in general, but among them, R-CNN (Regions with CNN features) and Yolo (You only look once) can be mainly used.For example, objects on the road (e.g., road users and / or objects) can be detected through a localization step that performs location search (e.g., search bounding box) for objects to be detected from sensor information acquired on the road, and a classification step that calculates the type of object.

[0072] Meanwhile, despite active development for real-time object detection, there may still be shortcomings in processing for high-performance detection and messaging. Typical examples include the detection rate and detection time for objects on the road. For example, data measured by sensors (e.g., camera images and / or LiDAR point clouds) may contain numerous detection candidates (e.g., bounding boxes). For example, items with a high detection probability may be processed first. Alternatively, for example, detection candidates with a probability below a certain value may be filtered out. For example, in object detection, 2,000 detection candidates (e.g., bounding boxes) that may contain objects may be selected. Furthermore, for example, a probability value (e.g., a confidence score) for the presence of an actual object within the space of the detection candidates (e.g., search boxes) may be calculated. Furthermore, for example, a conditional class probability (CLP) of the detection candidates may be calculated. For example, the detection object candidates for classification can be narrowed down based on the product of these two values ​​and the Jaccard coefficient or Intersection over Union (IoU), and a specified threshold (e.g., 0.5). For example, among multiple detection candidates, only those with high classification accuracy (e.g., bounding boxes) can be included in the sensor data sharing message. For example, if only object detection candidates with high classification accuracy are processed and provided to road users in the sensor data sharing service of ITS, information about fast-moving objects that are inaccurately represented in the detected sensor information may not be delivered to the service users.For example, image pixelation can occur in images captured by a camera, causing fast-moving objects to appear blurred (e.g., mosaic and / or blurry). For example, due to this phenomenon, fast-moving objects and / or road users may be excluded from detection candidates in object detection using sensors. For example, referring to Fig. 3, when detecting objects using a camera of an RSU installed at an intersection, pedestrians running on a crosswalk are not detected, and only stopped vehicles and pedestrians crossing at low speeds are detected.

[0073] FIG. 3 illustrates a degradation in object detection performance and the occurrence of false negatives due to motion blur according to an embodiment of the present disclosure. The embodiment of FIG. 3 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0074] Referring to the embodiment of Fig. 3, for example, this phenomenon of not being able to detect high-speed moving objects can be a big problem in ITS that must detect moving objects and prevent collisions between service users and moving objects. Referring to the embodiment of Fig. 3, for example, when a sensor installed at an intersection detects a road user on a crosswalk and provides a sensor data sharing service, from the perspective of a service user (e.g., a vehicle entering the intersection from below), more information may be needed on the fast-moving object among the high-speed crossing pedestrian (the running pedestrian on the right side of Fig. 3) and the slow-moving pedestrian (the walking pedestrian on the left side of Fig. 3, Person 7). However, for example, when performing an object detection AI / ML algorithm on fast-moving objects and slow-moving objects, among the many detection candidates present in the image, the fast-moving object may be excluded from the detection candidates due to a low object detection rate caused by the motion blur phenomenon. And, for example, the slow-moving object may be selected as the object detection target due to the clear image. For example, in such cases, users of sensor data sharing services (e.g., road users receiving messages) may only receive information about low-speed vehicles with a relatively low risk of collision. Furthermore, users of sharing services (e.g., road users receiving messages) may not receive information about high-speed vehicles with a high risk of collision. Consequently, collisions may occur, for example, at intersections.

[0075] Additionally, processing times may be extended for the type classification step for difficult-to-detect detection candidates. In such cases, information about objects whose detection time exceeds the reference validity time may not be included in the message. For example, important object information in traffic situations may not be conveyed to ITS service users, potentially reducing traffic safety.

[0076] In this disclosure, for object detection used in services that provide information acquired from sensor data, we propose an operation that prioritizes detection or classification operations by considering the sensor data characteristics of highly mobile objects. For example, in ITS services, it may be important to detect road users with a high risk of collision and provide them with information. For example, we propose an operation that calculates mobility representative values ​​for detection candidates detected by sensors under specific triggering conditions and, based on this, delivers the results of the prioritized detection or classification step with a label in a message. Furthermore, for example, by utilizing the method proposed in this disclosure, we propose that, in the process of selecting objects to be included in a message among multiple detected objects, the mobility representative value be used as a threshold value (e.g., a filtering condition) or comparison value that serves as a criterion for determining whether information about a specific detected object is included in the message. According to various embodiments of the present disclosure, for example, in situations where the ability to detect all object detection candidates is limited or it is not necessary to detect and process all objects, the detection operation or selective classification operation can be performed based on priority, thereby improving the overall quality of sensor data sharing services. Additionally, for example, for collision prevention, it is important that information about detected objects be delivered to service users within a short period of time, regardless of the type of high-speed object (e.g., the result of a classification process), and therefore, the operation proposed in this disclosure may be helpful.

[0077] In this disclosure, for example, in object detection utilized in sensor data sharing services of ITS, we propose an operation to skip the priority detection or classification step for objects with high mobility. For example, when detecting objects (e.g., obstacles and / or road users, etc.) using sensor data (e.g., camera images and / or LiDAR point clouds) measured by sensors (e.g., cameras, radars, and / or LiDAR) installed on infrastructure or mobile devices, high-speed mobile devices may need to be detected preferentially to enhance traffic safety in ITS. For example, for this purpose, the eigenvalues ​​of high-speed mobile devices can be measured / calculated through preprocessing or input parameters of an AI / ML model. For example, the present invention proposes that high-speed mobile devices be preferentially detected based on their eigenvalues ​​and included in service messages. Alternatively, for example, in order to include objects in a message that are not detected as high-speed moving objects or have a long detection processing time due to limitations of sensors and / or AI / ML that occur when detecting objects, the present disclosure proposes to omit the classification process under certain conditions and include only information about the object (e.g., location and / or velocity) in the message. The operation of prioritizing detection of high-speed moving objects and omitting the classification process under certain detection conditions proposed in the present disclosure can be performed in two steps.

[0078] For example, the first step may be to measure / calculate values ​​that represent mobility (e.g., speed and / or change of direction) to prevent high-speed moving objects from being excluded from detection candidates in measured sensor data (e.g., images and / or point clouds). For example, an object detection AI / ML algorithm may generate a large number of windows (e.g., possible bounding boxes) that are likely to contain objects from the input sensor data. For example, a mobility representative value for the object in each of these detection candidates can be calculated. For example, in images captured by a camera, highly mobile objects may exhibit a phenomenon in which pixels are mixed due to movement, blurring the edges of the image (e.g., pixelation), or the same motion blur phenomenon that occurs in point clouds measured by LiDAR. For example, this phenomenon may be stronger as mobility increases. For example, motion blur may be stronger in a fast-moving vehicle than in a walking pedestrian. For example, a value of the degree of motion blur can be calculated. For example, the calculated value of the degree of motion blur can be regarded as a representative value of mobility. For example, there can be various methods for measuring motion blur. For example, a blur kernel or blur metric can be calculated for each detection candidate. For example, a blur metric, which is an objective image quality evaluation that measures the amount of blur, can be a method of estimating the degree of blur by measuring the width between the start and end of an edge. For example, mathematical expression 1 can be utilized as an example according to the above calculation method.

[0079] [Mathematical Formula 1]

[0080]

[0081] Mathematical expression 1 shows a method for calculating a blur metric according to an embodiment of the present disclosure. For example, BM may represent a blur metric. For example, w k can represent the edge width. For example, K i can represent the number of edge pixels. For example, the edge width can be calculated according to mathematical expression 2.

[0082] [Equation 2]

[0083]

[0084] Mathematical expression 2 represents the edge width according to one embodiment of the present disclosure. For example, p s can represent the location of the pixel with the local maximum at each edge pixel. For example, p f can represent the location of a pixel having a local minimum.

[0085] For example, a blur kernel value, similar to a blur metric, can utilize metrics that express the degree of object shake and its trajectory, such as how much it has moved. For example, there are various methods for calculating or estimating blur metrics and blur kernels.

[0086] Alternatively, for example, motion vectors or optical flow values ​​can be utilized as a method of measuring the representative value of the mobility of a candidate detection object. For example, in the case of a stationary object, there may be no change in the image over time. On the other hand, for example, in the case of a moving object, sensor data may change over time. For example, not only the presence or absence of motion (e.g., separating an object from the background) but also the difference in sensor data (e.g., the differential image of two sequential images) can be used as a numerical value to measure the amount of object movement. For example, in the case of a high-speed moving object, the representative value of the mobility of each detection candidate can be obtained by using the difference in sensor data (e.g., numerical representation of motion vectors and / or optical flow, etc.).

[0087] For example, there may be various methods for measuring the degree of motion (e.g., blur kernel, blur metric, motion vector, and / or optical flow) of the object detection candidates described above. And, for example, the second step of the operation proposed in the present disclosure may be to use the mobility representative value of each detection candidate calculated from the sensor data as an input value of preprocessing for omitting the priority processing or classification task among the object detection candidates. For example, when the number of objects that can be included in one sensor data sharing service message is limited, at least one of the following operations may be performed on detection candidates that are higher or lower than the number of listed mobility representative values ​​(e.g., blur metric and / or motion vector) among a plurality of detection candidates. Or, for example, at least one of the following operations may be performed only on detection candidates that have a mobility representative value (e.g., blur metric and / or motion vector) equal to or greater than a threshold value (e.g., 0.75) among a plurality of detection candidates.

[0088] For example, the classification process can be performed first as follows.

[0089] - For example, classification can be performed starting with object detection candidates with high mobility representative values. For example, classification can be performed on an object detection candidate with an average motion vector of 20 before an object detection candidate with an average motion vector of 10.

[0090] - For example, to prioritize classification tasks, the confidence threshold of object detection candidates with high mobility representations can be differentiated from the confidence thresholds of other objects. For example, the confidence threshold of object detection candidates with high mobility representations can be lowered so that the bounding boxes of high-speed moving objects with low confidence scores can also be predicted.

[0091] - For example, a combination (e.g., sum and / or product) of the confidence scores of detection candidates and mobility representative values ​​(e.g., blur metrics and / or motion vectors) can be compared with a confidence threshold. For example, in a conventional classification method, a detection candidate may be excluded from detection if its confidence score is low (e.g., 0.4) but its blur metric value is high (e.g., 0.7), but its confidence score is lower than the confidence threshold (e.g., 0.5). However, for example, according to an embodiment proposed in the present disclosure, this detection candidate can be detected if the combination of its confidence score and its blur metric (e.g., 0.4+0.7=1.1) exceeds the confidence threshold.

[0092] For example, the classification process can be omitted as follows.

[0093] - For example, if the representative value of mobility is higher than a certain threshold, the classification task after localization or region proposal may be omitted. In this case, for example, the classification of the object (e.g., object class (ObjectClass), object type (ObjectType)) may be replaced with values ​​such as 'Unknown' or 'Unavailable' and / or may be preferentially included in the sensor data sharing service message including key information (e.g., location, speed, and / or direction).

[0094] Additionally, for example, detection candidates whose mobility representative values ​​are greater than a threshold may be preferentially included in messages. For example, this behavior allows information on high-speed vehicles important to traffic safety to be included in sensor data sharing service messages without being omitted or omitted from sensor information provision messages. Furthermore, for example, important high-speed vehicle information can be delivered to users, thereby enhancing traffic safety. For example, in ITS, where collision avoidance is more important than information on the type of high-speed vehicle (e.g., vehicle, pedestrian, and / or bicycle), information indicating an obstacle on the path may be prioritized.

[0095] Additionally, we propose trigger conditions that activate operations according to various embodiments proposed in this disclosure. For example, if at least one of the conditions specified below is satisfied, the eigenvalues ​​of object detection candidates may be measured / calculated, and classification tasks may be performed or skipped based on these values.

[0096] For example, a trigger condition may be a request for a preprocessing operation from an external source. For example, the proposed operation may be performed when a service requester sends a request message to perform a sensor data sharing service. For example, the service requester may be a service provider, a device manager, a message recipient, and / or a third-party service provider.

[0097] For example, a trigger condition may be a case where the number of detection candidates exceeds a certain value. For example, if the number of objects that can be included in a message is limited and the number of detection candidates is higher than the limited number of detections, operations according to various embodiments proposed in the present disclosure may be performed.

[0098] - For example, a trigger condition may be a case where the detection processing time of AI / ML exceeds a certain value. For example, if the processing time of AI / ML is higher than a set threshold, operations according to various embodiments proposed in the present disclosure may be performed. For example, if the detection processing time of AI / ML is long, the detected information may become invalid, and the invalidated information may not be included in the message. Or, for example, message generation may be delayed due to the long detection processing time of AI / ML. For example, this may cause disadvantages to the service user.

[0099] Additionally, for example, a preprocessing operation request may be transmitted by activating or deactivating a marker (e.g., a flag and / or an indicator) requesting an operation according to various embodiments proposed in the present disclosure when exchanging sensor information between servers.

[0100] In addition, for example, high-speed vehicles can be processed as detection targets of AI / ML through preprocessing for skipping the priority detection or classification work of high-speed vehicles proposed in the present disclosure. And, for example, when transmitting this information by including it in a message, it is proposed to include an indication (e.g., a flag and / or an indicator) or a processing mode in the message. For example, it is proposed that an information provider transmits information about the result of priority detection of high-speed objects to a V2X service message (e.g., CPM, SDSM, DENM, and / or RSM) to a service user. For example, the data elements indicated in the message in the CPS of ETSI ITS can be as shown in Table 2.

[0101] ObjectInformationList ::= SEQUENCE SIZE (1..16) OF ObjectInformationObjectInformation ::= SEQUENCE {objectID Identifier,position Position,speed Speed,heading Heading,classification ObjectClassPriorityDetection…}

[0102] Referring to Table 2, for example, information related to objects can be included in a message. For example, the size of a sequence can be included in a message. For example, an object identifier can be included in a message. For example, an object's location can be included in a message. For example, an object's velocity can be included in a message. For example, an object's direction can be included in a message. For example, an object's classification can be included in a message. For example, an object's detection priority can be included in a message. For example, an object's priority can be included in a message. For example, an object's classification can be omitted in a message. Furthermore, for example, a high-speed mobile object can be processed as a detection target for AI / ML through the proposed preprocessing for priority detection or classification omission of high-speed mobile objects. Furthermore, when including this information in a message along with other information and transmitting it, if the limited message size does not allow for all of the information to be included, it is proposed that this information be included in the message with priority. For example, the priority for inclusion in a message can be determined based on a mobility representative value. For example, mobility quotients can be used as a proxy for safety-related factors in traffic situations. Furthermore, it is proposed that mobility quotients be used as thresholds or comparison points for determining whether a message is included, or as conditions for filtering.

[0103] For example, the mobility representative values ​​of detection candidates detected by sensors under specific triggering conditions proposed in this disclosure can be calculated. For example, the operation of forwarding the results of the priority detection or classification step, skipping the priority detection or classification step based on the mobility representative value, along with the message, can proceed with the detection priority or selective classification task in situations where the ability to detect all object detection candidates is limited or there is no need to detect and process all objects. For example, this can have the effect of improving the overall quality of sensor data sharing services. For example, ITS services requiring real-time performance can face difficulties due to the significant data processing volume of images measured by cameras. However, for example, the method proposed in this disclosure can improve the real-time performance of the service. Furthermore, for example, since it is important to quickly deliver information to service users regardless of the type of high-speed object (e.g., the result of the classification process) for collision prevention, the operation proposed in this disclosure can be helpful.

[0104] FIG. 4 illustrates object detection candidates and blur metric values ​​according to an embodiment of the present disclosure. The embodiment of FIG. 4 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0105] Referring to the embodiment of FIG. 4, a service provider can send a request message to an RSU equipped with a sensor (camera) at an intersection to activate a priority detection mode. For example, by calculating the blur metric values ​​of the detection candidates in FIG. 3, the priority of priority detection can be determined when the situation is as shown in FIG. 4. For example, object candidates that were previously omitted in the classification task due to low detectability (e.g., confidence score), a problem pointed out in the prior art, can be detected through the classification task.

[0106] FIG. 5 illustrates a method for high-speed mobile object priority detection using an AI / ML model of R-CNN, according to an embodiment of the present disclosure. The embodiment of FIG. 5 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0107] Referring to the embodiment of FIG. 5, for example, the AI / ML model may first extract detection candidates based on a detection mode request. For example, the AI / ML model may calculate mobility representative values ​​for the detection candidates. For example, the AI / ML model may perform priority detection based on the calculated values, or may skip the classification task for detection candidates with high mobility values ​​and directly include relevant information in the message.

[0108] FIG. 6 illustrates a method for skipping high-speed mobile object priority detection or classification operations according to an embodiment of the present disclosure. The embodiment of FIG. 6 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0109] Referring to the embodiment of FIG. 6, for example, the device may receive a priority detection mode request. For example, the device may satisfy a triggering condition for skipping priority detection or classification tasks. For example, the device may search for candidate detection targets. For example, the device may calculate mobility representative values ​​of the detection candidates. For example, the device may prioritize detecting objects based on the calculated values. For example, the device may skip classification tasks for objects based on the calculated values. For example, the device may message information about detected objects. For example, the device may add a mark related to skipping priority detection or classification tasks to the message. For example, the device may transmit a sensor data sharing service message.

[0110] The method proposed in this disclosure may have various improved effects compared to prior art, although these effects are not limited to those presented in this disclosure. For example, traffic safety can be ensured by allowing road users to obtain information about high-speed moving objects in advance. For example, high-speed moving objects can be efficiently detected. For example, this can help road users avoid collisions with high-speed moving objects. For example, objects with a significant impact on traffic safety can be efficiently detected. For example, the efficiency of object detection can be increased. For example, traffic flow can be improved. For example, efficient traffic management can be enabled.

[0111] FIG. 7 illustrates a method for a first device to perform wireless communication according to an embodiment of the present disclosure. The embodiment of FIG. 7 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0112] Referring to FIG. 7, in step S710, the first device may detect an object. In step S720, the first device may obtain information related to the mobility of the object. In step S730, the first device may transmit a message including information about the object. For example, information about the object may be preferentially included in the message based on information related to the mobility.

[0113] For example, the information related to the mobility may include at least one of a value related to motion blur, a motion vector value, or an optical flow value.

[0114] For example, information about the object may be preferentially included in the message based on a mobility representative value greater than a threshold value.

[0115] For example, information about the object may be preferentially included in the message based on the information about the entire detected object being larger than the size limit of the message.

[0116] Additionally, for example, it may be determined whether to skip or prioritize classification tasks related to the object based on the mobility-related information. For example, classification tasks related to the object may be preferentially performed based on a mobility representative value exceeding a threshold value. For example, classification tasks related to the object may be preferentially performed based on a combination of mobility-related information and a confidence score. For example, classification tasks related to the object may be preferentially performed based on a sum of the mobility representative value and the confidence score exceeding a threshold value. For example, the priority of priority detection related to the classification task may be obtained based on a blur metric value. For example, classification tasks related to the object may be skipped based on a mobility representative value exceeding a threshold value.

[0117] For example, whether to skip or prioritize the above classification task may be determined based on receiving a request message from another device, the number of detection candidates being greater than a limit detection number, or the processing time associated with the detection being greater than a threshold.

[0118] For example, the message may include information regarding whether to skip classification work on the object or information regarding priority detection.

[0119] For example, information about the object may be preferentially included in a message based on AI (Artificial Intelligence) or ML (Machine Learning).

[0120] The proposed method can be applied to devices according to various embodiments of the present disclosure. First, the processor (102) of the first device (100) can control the first device (100) to detect an object. Then, the processor (102) of the first device (100) can control the first device (100) to obtain information related to the mobility of the object. Then, the processor (102) of the first device (100) can control the transceiver (106) to transmit a message including information about the object. For example, information about the object can be preferentially included in the message based on information related to the mobility.

[0121] According to one embodiment of the present disclosure, a first device configured to perform wireless communication may be provided. For example, the first device may include at least one transceiver; at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, the instructions, based on execution by the at least one processor, may cause the first device to: detect an object; obtain information related to mobility of the object; and transmit a message including information about the object. Information about the object may be preferentially included in the message based on information related to mobility.

[0122] According to one embodiment of the present disclosure, a processing device configured to control a first device may be provided. For example, the processing device may include at least one processor; and at least one memory coupled to the at least one processor and storing instructions. For example, the instructions, based on execution by the at least one processor, may cause the first device to: detect an object; obtain information related to mobility of the object; and transmit a message including information about the object. Information about the object may be preferentially included in the message based on information related to mobility.

[0123] According to one embodiment of the present disclosure, a non-transitory computer-readable storage medium storing instructions may be provided. For example, the instructions, when executed, may cause a first device to: detect an object; obtain information related to the mobility of the object; and transmit a message including information about the object. Information about the object may be preferentially included in the message based on the information related to the mobility.

[0124] FIG. 8 illustrates a method for a second device to perform wireless communication according to an embodiment of the present disclosure. The embodiment of FIG. 8 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0125] Referring to FIG. 8, in step S810, the second device may request detection of an object. In step S820, the second device may receive a message containing information about the object. For example, information about the object may be preferentially included in the message based on information related to the object's mobility.

[0126] The proposed method can be applied to devices according to various embodiments of the present disclosure. First, the processor (202) of the second device (200) can control the second device (200) to receive a request for detection of an object. Then, the processor (202) of the second device (200) can control the transceiver (206) to receive a message containing information about the object. For example, information about the object may be preferentially included in the message based on information related to the mobility of the object.

[0127] According to one embodiment of the present disclosure, a second device configured to perform wireless communication may be provided. For example, the second device may include at least one transceiver; at least one processor; and at least one memory connected to the at least one processor and storing instructions. For example, the instructions, based on execution by the at least one processor, may cause the second device to: request detection of an object; and receive a message including information about the object. For example, the information about the object may be preferentially included in the message based on information related to mobility of the object.

[0128] According to one embodiment of the present disclosure, a processing device configured to control a second device may be provided. For example, the processing device may include at least one processor; and at least one memory coupled to the at least one processor and storing instructions. For example, the instructions, based on execution by the at least one processor, may cause the second device to: request detection of an object; and receive a message including information about the object. For example, the information about the object may be preferentially included in the message based on information related to mobility of the object.

[0129] According to one embodiment of the present disclosure, a non-transitory computer-readable storage medium storing commands may be provided. For example, the commands, when executed, may cause a second device to: request detection of an object; and receive a message including information about the object. For example, information about the object may be preferentially included in the message based on information related to the mobility of the object.

[0130] The various embodiments of the present disclosure may be combined with each other, and some descriptions, functions, procedures, proposals, methods and / or operations of the various embodiments may be omitted.

[0131] Below, a description is given of devices to which various embodiments of the present disclosure can be applied.

[0132] Although not limited thereto, the various descriptions, functions, procedures, proposals, methods and / or operational flowcharts disclosed in this document may be applied to various fields requiring wireless communication / connectivity (e.g., 5G) between devices.

[0133] Hereinafter, more specific examples will be provided with reference to the drawings. In the drawings / descriptions below, the same drawing reference numerals may represent identical or corresponding hardware blocks, software blocks, or functional blocks, unless otherwise described.

[0134] FIG. 9 illustrates a communication system (1) according to one embodiment of the present disclosure. The embodiment of FIG. 9 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0135] Referring to FIG. 9, a communication system (1) to which various embodiments of the present disclosure are applied includes a wireless device, a base station, and a network. Here, the wireless device refers to a device that performs communication using a wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, the wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (eXtended Reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Things) device (100f), and an AI device / server (400). For example, the vehicle may include a vehicle equipped with a wireless communication function, an autonomous vehicle, a vehicle capable of performing vehicle-to-vehicle communication, etc. Here, the vehicle may include an Unmanned Aerial Vehicle (UAV) (e.g., a drone) and / or an Aerial Vehicle (AV) (e.g., an Advanced Air Mobility (AAM)). The XR device may include an Augmented Reality (AR) / Virtual Reality (VR) / Mixed Reality (MR) device, and may be implemented in the form of a Head-Mounted Device (HMD), a Head-Up Display (HUD) equipped in a vehicle, a television, a smartphone, a computer, a wearable device, a home appliance, a digital signage, a vehicle, a robot, etc. The portable device may include a smartphone, a smart pad, a wearable device (e.g., a smart watch, smart glasses), a computer (e.g., a laptop, etc.), etc. The home appliance may include a TV, a refrigerator, a washing machine, etc. The IoT device may include a sensor, a smart meter, etc. For example, a base station and a network may also be implemented as a wireless device, and a specific wireless device (200a) may operate as a base station / network node to other wireless devices.

[0136] Here, the wireless communication technology implemented in the wireless devices (100a to 100f) of the present specification may include not only LTE, NR, and 6G, but also Narrowband Internet of Things for low-power communication. At this time, for example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology, and may be implemented with standards such as LTE Cat NB1 and / or LTE Cat NB2, and is not limited to the above-described names. Additionally or alternatively, the wireless communication technology implemented in the wireless devices (100a to 100f) of the present specification may perform communication based on LTE-M technology. At this time, for example, LTE-M technology may be an example of LPWAN technology, and may be called by various names such as eMTC (enhanced Machine Type Communication). For example, LTE-M technology can be implemented by at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the above-described names. Additionally or alternatively, the wireless communication technology implemented in the wireless devices (100a to 100f) of the present specification can include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) considering low-power communication, and is not limited to the above-described names. For example, ZigBee technology can create personal area networks (PAN) related to small / low-power digital communication based on various standards such as IEEE 802.15.4, and can be called by various names.

[0137] Wireless devices (100a to 100f) can be connected to a network (300) via a base station (200). Artificial Intelligence (AI) technology can be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (400) via the network (300). The network (300) can be configured using a 3G network, a 4G (e.g., LTE) network, a 5G (e.g., NR) network, etc. The wireless devices (100a to 100f) can communicate with each other via the base station (200) / network (300), but can also communicate directly (e.g., sidelink communication) without going through the base station / network. For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (Vehicle to Vehicle) / V2X (Vehicle to Everything) communication). In addition, IoT devices (e.g., sensors) can communicate directly with other IoT devices (e.g., sensors) or other wireless devices (100a to 100f).

[0138] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a~100f) / base stations (200), and base stations (200) / base stations (200). Here, wireless communication / connection can be achieved through various wireless access technologies (e.g., 5G NR) such as uplink / downlink communication (150a), sidelink communication (150b) (or, D2D communication), and communication between base stations (150c) (e.g., relay, IAB (Integrated Access Backhaul). Through wireless communication / connection (150a, 150b, 150c), wireless devices and base stations / wireless devices, and base stations and base stations can transmit / receive wireless signals to each other. For example, wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, at least some of various configuration information setting processes for transmitting / receiving wireless signals, various signal processing processes (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), and resource allocation processes can be performed based on various proposals of the present disclosure.

[0139] FIG. 10 illustrates a wireless device according to an embodiment of the present disclosure. The embodiment of FIG. 10 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0140] Referring to FIG. 10, the first wireless device (100) and the second wireless device (200) can transmit and receive wireless signals via various wireless access technologies (e.g., LTE, NR). Here, {the first wireless device (100), the second wireless device (200)} can correspond to {the wireless device (100x), the base station (200)} and / or {the wireless device (100x), the wireless device (100x)} of FIG. 9.

[0141] A first wireless device (100) includes one or more processors (102) and one or more memories (104), and may further include one or more transceivers (106) and / or one or more antennas (108). The processor (102) controls the memories (104) and / or the transceivers (106), and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document. For example, the processor (102) may process information in the memory (104) to generate first information / signal, and then transmit a wireless signal including the first information / signal via the transceiver (106). Furthermore, the processor (102) may receive a wireless signal including second information / signal via the transceiver (106), and then store information obtained from signal processing of the second information / signal in the memory (104). The memory (104) may be connected to the processor (102) and may store various information related to the operation of the processor (102). For example, the memory (104) may perform some or all of the processes controlled by the processor (102), or may store software code including commands for performing the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. Here, the processor (102) and the memory (104) may be part of a communication modem / circuit / chip designed to implement a wireless communication technology (e.g., LTE, NR). The transceiver (106) may be connected to the processor (102) and may transmit and / or receive wireless signals via one or more antennas (108). The transceiver (106) may include a transmitter and / or a receiver. The transceiver (106) may be used interchangeably with an RF (Radio Frequency) unit. In the present disclosure, a wireless device may also mean a communication modem / circuit / chip.

[0142] A second wireless device (200) includes one or more processors (202), one or more memories (204), and may further include one or more transceivers (206) and / or one or more antennas (208). The processor (202) controls the memories (204) and / or the transceivers (206), and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document. For example, the processor (202) may process information in the memory (204) to generate third information / signals, and then transmit a wireless signal including the third information / signals via the transceivers (206). In addition, the processor (202) may receive a wireless signal including fourth information / signals via the transceivers (206), and then store information obtained from signal processing of the fourth information / signals in the memory (204). The memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, the memory (204) may perform some or all of the processes controlled by the processor (202), or may store software code including commands for performing the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. Here, the processor (202) and the memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). The transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals via one or more antennas (208). The transceiver (206) may include a transmitter and / or a receiver. The transceiver (206) may be used interchangeably with an RF unit. In the present disclosure, a wireless device may also mean a communication modem / circuit / chip.

[0143] Hereinafter, the hardware elements of the wireless device (100, 200) will be described in more detail. Although not limited thereto, one or more protocol layers may be implemented by one or more processors (102, 202). For example, one or more processors (102, 202) may implement one or more layers (e.g., functional layers such as PHY, MAC, RLC, PDCP, RRC, SDAP). One or more processors (102, 202) may generate one or more Protocol Data Units (PDUs) and / or one or more Service Data Units (SDUs) according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. One or more processors (102, 202) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. One or more processors (102, 202) can generate signals (e.g., baseband signals) including PDUs, SDUs, messages, control information, data or information according to the functions, procedures, proposals and / or methods disclosed herein, and provide the signals to one or more transceivers (106, 206). One or more processors (102, 202) can receive signals (e.g., baseband signals) from one or more transceivers (106, 206) and obtain PDUs, SDUs, messages, control information, data or information according to the descriptions, functions, procedures, proposals, methods and / or operational flowcharts disclosed herein.

[0144] One or more processors (102, 202) may be referred to as a controller, a microcontroller, a microprocessor, or a microcomputer. One or more processors (102, 202) may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), one or more Digital Signal Processing Devices (DSPDs), one or more Programmable Logic Devices (PLDs), or one or more Field Programmable Gate Arrays (FPGAs) may be included in one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. The descriptions, functions, procedures, suggestions, methods and / or operation flowcharts disclosed in this document may be implemented using firmware or software configured to perform one or more processors (102, 202) or stored in one or more memories (104, 204) and executed by one or more processors (102, 202). The descriptions, functions, procedures, suggestions, methods and / or operation flowcharts disclosed in this document may be implemented using firmware or software in the form of codes, instructions and / or sets of instructions.

[0145] One or more memories (104, 204) may be coupled to one or more processors (102, 202) and may store various forms of data, signals, messages, information, programs, codes, instructions, and / or commands. The one or more memories (104, 204) may be configured as ROM, RAM, EPROM, flash memory, hard drives, registers, cache memory, computer-readable storage media, and / or combinations thereof. The one or more memories (104, 204) may be located internally and / or externally to the one or more processors (102, 202). Additionally, the one or more memories (104, 204) may be coupled to the one or more processors (102, 202) via various technologies, such as wired or wireless connections.

[0146] One or more transceivers (106, 206) can transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or flowcharts of this document, to one or more other devices. One or more transceivers (106, 206) can receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or flowcharts of this document, from one or more other devices. For example, one or more transceivers (106, 206) can be connected to one or more processors (102, 202) and can transmit and receive wireless signals. For example, one or more processors (102, 202) can control one or more transceivers (106, 206) to transmit user data, control information, or wireless signals to one or more other devices. Additionally, one or more processors (102, 202) may control one or more transceivers (106, 206) to receive user data, control information, or wireless signals from one or more other devices. Additionally, one or more transceivers (106, 206) may be coupled to one or more antennas (108, 208), and one or more transceivers (106, 206) may be configured to transmit and receive user data, control information, wireless signals / channels, or the like, as referred to in the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed herein, via one or more antennas (108, 208). In this document, one or more antennas may be multiple physical antennas or multiple logical antennas (e.g., antenna ports). One or more transceivers (106, 206) can convert received user data, control information, wireless signals / channels, etc. from RF band signals to baseband signals in order to process the received user data, control information, wireless signals / channels, etc. using one or more processors (102, 202).One or more transceivers (106, 206) may convert user data, control information, wireless signals / channels, etc. processed by one or more processors (102, 202) from baseband signals to RF band signals. For this purpose, one or more transceivers (106, 206) may include an (analog) oscillator and / or filter.

[0147] Fig. 11 illustrates a signal processing circuit for a transmission signal according to an embodiment of the present disclosure. The embodiment of Fig. 11 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0148] Referring to FIG. 11, the signal processing circuit (1000) may include a scrambler (1010), a modulator (1020), a layer mapper (1030), a precoder (1040), a resource mapper (1050), and a signal generator (1060). Although not limited thereto, the operations / functions of FIG. 11 may be performed in the processor (102, 202) and / or the transceiver (106, 206) of FIG. 10. The hardware elements of FIG. 11 may be implemented in the processor (102, 202) and / or the transceiver (106, 206) of FIG. 10. For example, blocks 1010 to 1060 may be implemented in the processor (102, 202) of FIG. 10. Additionally, blocks 1010 to 1050 may be implemented in the processor (102, 202) of FIG. 10, and block 1060 may be implemented in the transceiver (106, 206) of FIG. 10.

[0149] The codeword can be converted into a wireless signal through the signal processing circuit (1000) of FIG. 11. Here, the codeword is an encoded bit sequence of an information block. The information block may include a transport block (e.g., an UL-SCH transport block, a DL-SCH transport block). The wireless signal may be transmitted through various physical channels (e.g., a PUSCH or a PDSCH).

[0150] Specifically, the codeword can be converted into a bit sequence scrambled by a scrambler (1010). The scramble sequence used for scrambling is generated based on an initialization value, and the initialization value may include ID information of the wireless device, etc. The scrambled bit sequence can be modulated into a modulation symbol sequence by a modulator (1020). The modulation method may include pi / 2-BPSK (pi / 2-Binary Phase Shift Keying), m-PSK (m-Phase Shift Keying), m-QAM (m-Quadrature Amplitude Modulation), etc. The complex modulation symbol sequence can be mapped to one or more transmission layers by a layer mapper (1030). The modulation symbols of each transmission layer can be mapped to the corresponding antenna port(s) by a precoder (1040) (precoding). The output z of the precoder (1040) can be obtained by multiplying the output y of the layer mapper (1030) by a precoding matrix W of N*M. Here, N is the number of antenna ports, and M is the number of transmission layers. Here, the precoder (1040) can perform precoding after performing transform precoding (e.g., DFT transform) on complex modulation symbols. In addition, the precoder (1040) can perform precoding without performing transform precoding.

[0151] The resource mapper (1050) can map modulation symbols of each antenna port to time-frequency resources. The time-frequency resources can include multiple symbols (e.g., CP-OFDMA symbols, DFT-s-OFDMA symbols) in the time domain and multiple subcarriers in the frequency domain. The signal generator (1060) generates a wireless signal from the mapped modulation symbols, and the generated wireless signal can be transmitted to another device through each antenna. To this end, the signal generator (1060) can include an Inverse Fast Fourier Transform (IFFT) module, a Cyclic Prefix (CP) inserter, a Digital-to-Analog Converter (DAC), a frequency uplink converter, etc.

[0152] The signal processing process for receiving signals in a wireless device can be configured in reverse order of the signal processing process (1010 to 1060) of FIG. 11. For example, a wireless device (e.g., 100, 200 of FIG. 10) can receive wireless signals from the outside through an antenna port / transceiver. The received wireless signals can be converted into baseband signals through a signal restorer. For this purpose, the signal restorer can include a frequency downlink converter, an analog-to-digital converter (ADC), a CP remover, and a fast Fourier transform (FFT) module. Thereafter, the baseband signal can be restored to a codeword through a resource demapper process, a postcoding process, a demodulation process, and a descrambling process. The codewords can be restored to the original information blocks through decoding. Accordingly, a signal processing circuit (not shown) for a received signal may include a signal restorer, a resource de-mapper, a postcoder, a demodulator, a de-scrambler, and a decoder.

[0153] Figure 12 illustrates a wireless device according to an embodiment of the present disclosure. The wireless device may be implemented in various forms depending on the use case / service (see Figure 9). The embodiment of Figure 12 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0154] Referring to FIG. 12, the wireless device (100, 200) corresponds to the wireless device (100, 200) of FIG. 10 and may be composed of various elements, components, units / units, and / or modules. For example, the wireless device (100, 200) may include a communication unit (110), a control unit (120), a memory unit (130), and additional elements (140). The communication unit may include a communication circuit (112) and a transceiver(s) (114). For example, the communication circuit (112) may include one or more processors (102, 202) and / or one or more memories (104, 204) of FIG. 10. For example, the transceiver(s) (114) may include one or more transceivers (106, 206) and / or one or more antennas (108, 208) of FIG. 10. The control unit (120) is electrically connected to the communication unit (110), the memory unit (130), and the additional elements (140) and controls the overall operation of the wireless device. For example, the control unit (120) may control the electrical / mechanical operation of the wireless device based on the program / code / command / information stored in the memory unit (130). In addition, the control unit (120) may transmit information stored in the memory unit (130) to an external device (e.g., another communication device) via a wireless / wired interface through the communication unit (110), or store information received from an external device (e.g., another communication device) via a wireless / wired interface in the memory unit (130).

[0155] The additional element (140) may be configured in various ways depending on the type of the wireless device. For example, the additional element (140) may include at least one of a power unit / battery, an input / output unit (I / O unit), a driving unit, and a computing unit. Although not limited thereto, the wireless device may be implemented in the form of a robot (Fig. 9, 100a), a vehicle (Fig. 9, 100b-1, 100b-2), an XR device (Fig. 9, 100c), a portable device (Fig. 9, 100d), a home appliance (Fig. 9, 100e), an IoT device (Fig. 9, 100f), a digital broadcasting terminal, a hologram device, a public safety device, an MTC device, a medical device, a fintech device (or a financial device), a security device, a climate / environmental device, an AI server / device (Fig. 9, 400), a base station (Fig. 9, 200), a network node, etc. Wireless devices may be mobile or stationary depending on the use / service.

[0156] In FIG. 12, various elements, components, units / parts, and / or modules within the wireless device (100, 200) may be interconnected entirely via a wired interface, or at least some may be wirelessly connected via a communication unit (110). For example, within the wireless device (100, 200), the control unit (120) and the communication unit (110) may be wired, and the control unit (120) and the first unit (e.g., 130, 140) may be wirelessly connected via the communication unit (110). In addition, each element, component, unit / part, and / or module within the wireless device (100, 200) may further include one or more elements. For example, the control unit (120) may be composed of one or more processor sets. For example, the control unit (120) may be composed of a set of a communication control processor, an application processor, an electronic control unit (ECU), a graphics processing processor, a memory control processor, etc. As another example, the memory unit (130) may be composed of a random access memory (RAM), a dynamic RAM (DRAM), a read only memory (ROM), a flash memory, a volatile memory, a non-volatile memory, and / or a combination thereof.

[0157] Below, the implementation example of Fig. 12 is described in more detail with reference to the drawings.

[0158] FIG. 13 illustrates a mobile device according to an embodiment of the present disclosure. The mobile device may include a smartphone, a smart pad, a wearable device (e.g., a smartwatch, smartglasses), or a portable computer (e.g., a laptop, etc.). The mobile device may be referred to as a Mobile Station (MS), a User Terminal (UT), a Mobile Subscriber Station (MSS), a Subscriber Station (SS), an Advanced Mobile Station (AMS), or a Wireless Terminal (WT). The embodiment of FIG. 13 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0159] Referring to FIG. 13, the portable device (100) may include an antenna unit (108), a communication unit (110), a control unit (120), a memory unit (130), a power supply unit (140a), an interface unit (140b), and an input / output unit (140c). The antenna unit (108) may be configured as a part of the communication unit (110). Blocks 110 to 130 / 140a to 140c correspond to blocks 110 to 130 / 140 of FIG. 12, respectively.

[0160] The communication unit (110) can transmit and receive signals (e.g., data, control signals, etc.) with other wireless devices and base stations. The control unit (120) can control components of the mobile device (100) to perform various operations. The control unit (120) can include an AP (Application Processor). The memory unit (130) can store data / parameters / programs / codes / commands required for operating the mobile device (100). In addition, the memory unit (130) can store input / output data / information, etc. The power supply unit (140a) supplies power to the mobile device (100) and can include a wired / wireless charging circuit, a battery, etc. The interface unit (140b) can support connection between the mobile device (100) and other external devices. The interface unit (140b) can include various ports (e.g., audio input / output ports, video input / output ports) for connection with external devices. The input / output unit (140c) can input or output video information / signals, audio information / signals, data, and / or information input from a user. The input / output unit (140c) may include a camera, a microphone, a user input unit, a display unit (140d), a speaker, and / or a haptic module.

[0161] For example, in the case of data communication, the input / output unit (140c) obtains information / signals (e.g., touch, text, voice, image, video) input by the user, and the obtained information / signals can be stored in the memory unit (130). The communication unit (110) converts the information / signals stored in the memory into wireless signals, and can directly transmit the converted wireless signals to other wireless devices or to a base station. In addition, the communication unit (110) can receive wireless signals from other wireless devices or base stations, and then restore the received wireless signals to the original information / signals. The restored information / signals can be stored in the memory unit (130) and then output in various forms (e.g., text, voice, image, video, haptic) through the input / output unit (140c).

[0162] FIG. 14 illustrates a vehicle or autonomous vehicle according to one embodiment of the present disclosure. The vehicle or autonomous vehicle may be implemented as a mobile robot, a car, a train, a manned / unmanned aerial vehicle (AV), a ship, etc. The embodiment of FIG. 14 may be combined with various embodiments of the present disclosure, and some descriptions, functions, procedures, proposals, methods, and / or operations of the embodiments may be omitted.

[0163] Referring to FIG. 14, a vehicle or autonomous vehicle (100) may include an antenna unit (108), a communication unit (110), a control unit (120), a driving unit (140a), a power supply unit (140b), a sensor unit (140c), and an autonomous driving unit (140d). The antenna unit (108) may be configured as a part of the communication unit (110). Blocks 110 / 130 / 140a to 140d correspond to blocks 110 / 130 / 140 of FIG. 12, respectively.

[0164] The communication unit (110) can transmit and receive signals (e.g., data, control signals, etc.) with external devices such as other vehicles, base stations (e.g., base stations, road side units, etc.), and servers. The control unit (120) can control elements of the vehicle or autonomous vehicle (100) to perform various operations. The control unit (120) can include an ECU (Electronic Control Unit). The drive unit (140a) can drive the vehicle or autonomous vehicle (100) on the ground. The drive unit (140a) can include an engine, a motor, a power train, wheels, brakes, a steering device, etc. The power supply unit (140b) supplies power to the vehicle or autonomous vehicle (100) and can include a wired / wireless charging circuit, a battery, etc. The sensor unit (140c) can obtain vehicle status, surrounding environment information, user information, etc. The sensor unit (140c) may include an IMU (inertial measurement unit) sensor, a collision sensor, a wheel sensor, a speed sensor, an incline sensor, a weight detection sensor, a heading sensor, a position module, a vehicle forward / backward sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, a pedal position sensor, etc. The autonomous driving unit (140d) may implement a technology for maintaining a driving lane, a technology for automatically controlling speed such as adaptive cruise control, a technology for automatically driving along a set path, a technology for automatically setting a path and driving when a destination is set, etc.

[0165] For example, the communication unit (110) can receive map data, traffic information data, etc. from an external server. The autonomous driving unit (140d) can generate an autonomous driving route and driving plan based on the acquired data. The control unit (120) can control the drive unit (140a) so that the vehicle or autonomous vehicle (100) moves along the autonomous driving route according to the driving plan (e.g., speed / direction control). During autonomous driving, the communication unit (110) can irregularly / periodically acquire the latest traffic information data from an external server and can acquire surrounding traffic information data from surrounding vehicles. In addition, during autonomous driving, the sensor unit (140c) can acquire vehicle status and surrounding environment information. The autonomous driving unit (140d) can update the autonomous driving route and driving plan based on newly acquired data / information. The communication unit (110) can transmit information regarding the vehicle location, autonomous driving route, driving plan, etc. to the external server. External servers can predict traffic information data in advance using AI technology or other technologies based on information collected from vehicles or autonomous vehicles, and provide the predicted traffic information data to the vehicles or autonomous vehicles.

[0166] The claims set forth in this specification may be combined in various ways. For example, the technical features of the method claims of this specification may be combined and implemented as a device, and the technical features of the device claims of this specification may be combined and implemented as a method. Furthermore, the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a device, and the technical features of the method claims and the technical features of the device claims of this specification may be combined and implemented as a method.

Claims

1. In the method, A first device comprises a step of detecting an object; The first device obtains information related to mobility of the object; and The first device comprises a step of transmitting a message including information about the object; A method in which information about the object is preferentially included in the message based on information related to the mobility.

2. In paragraph 1, A method wherein the information related to the mobility includes at least one of a value related to motion blur, a motion vector value, or an optical flow value.

3. In paragraph 1, A method in which information about the above object is preferentially included in the message based on a mobility representative value being greater than a threshold value.

4. In paragraph 1, A method in which information about the above object is preferentially included in the message based on the fact that information about the entire detected object is larger than the limit size of the message.

5. In paragraph 1, A method further comprising: a step of determining whether to skip or give priority to a classification task related to the object based on information related to the mobility; 6. In paragraph 5, A method in which classification tasks related to the above objects are preferentially performed based on a mobility representative value being greater than a threshold value.

7. In paragraph 5, A method in which the classification task related to the above object is preferentially performed based on a combination of information related to the above mobility and a confidence score.

8. In paragraph 7, A method in which a classification task related to the above object is preferentially performed based on the sum of the mobility representative value and the confidence score exceeding a threshold.

9. In paragraph 8, A method in which the priority of the priority detection related to the above classification task is obtained based on the blur metric value.

10. In paragraph 5, A method in which classification tasks related to the above objects are omitted based on a mobility representative value being higher than a threshold value.

11. In paragraph 5, A method in which whether to skip or prioritize the above classification task is determined based on whether a request message is received from another device, whether the number of detection candidates is greater than the limited number of detections, or whether the processing time associated with the detection is greater than a threshold.

12. In paragraph 5, A method wherein the above message includes information related to whether to skip classification work for the object or information related to priority detection.

13. In paragraph 1, A method in which information about the above object is preferentially included in a message based on AI (Artificial Intelligence) or ML (Machine Learning).

14. In the first device, At least one transmitter / receiver; at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor, wherein said first device causes: Detect objects; Obtaining information related to the mobility of the above object; and To transmit a message containing information about the above object; A first device, wherein information about the object is preferentially included in the message based on information related to the mobility.

15. In a processing device set to control the first device, at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor, wherein said first device causes: Detect objects; Obtaining information related to the mobility of the above object; and To transmit a message containing information about the above object; A processing device wherein information about the object is preferentially included in the message based on information related to the mobility.

16. A non-transitory computer-readable storage medium that records commands, The above commands, when executed, cause the first device to: Detect objects; Obtaining information related to the mobility of the above object; and To transmit a message containing information about the above object; A non-transitory computer-readable storage medium in which information about the object is preferentially included in the message based on information related to the mobility.

17. In the method, A second device requests detection of an object; and The second device comprises a step of receiving a message including information about the object; A method in which information about the object is preferentially included in the message based on information related to mobility of the object.

18. In the second device, At least one transmitter / receiver; at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor, wherein said second device causes: Request detection of an object; and To receive a message containing information about the above object; A second device wherein information about the object is preferentially included in the message based on information related to mobility of the object.

19. In a processing device set to control a second device, at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor, wherein said second device causes: Request detection of an object; and To receive a message containing information about the above object; A processing device wherein information about the object is preferentially included in the message based on information related to mobility of the object.

20. A non-transitory computer-readable storage medium that records commands, The above commands, when executed, cause the second device to: Request detection of an object; and To receive a message containing information about the object; A non-transitory computer-readable storage medium, wherein information about the object is preferentially included in the message based on information related to mobility of the object.

Images