Method for device to transmit message and device therefor in wireless communication system

By using an AI-based object recognition model to adjust image acquisition device settings in real-time, the method improves object recognition accuracy in V2X services by optimizing image characteristics for environmental conditions, addressing challenges in lighting, size, and movement speed.

WO2024248564A1PCT designated stage expired Publication Date: 2024-12-05LG ELECTRONICS INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2024/007559
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-06-01
Filing Date
2024-06-03
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Current object recognition technologies in V2X services face challenges such as reduced recognition accuracy due to lighting conditions, object size, movement speed, and environmental factors, leading to inefficient object detection and classification.

Method used

A method that involves an AI-based object recognition model to assess image acquisition device settings and adjust parameters like contrast, resolution, and frame rate based on real-time object recognition rates and environmental conditions, optimizing image characteristics for improved recognition accuracy.

Benefits of technology

Enhances object recognition accuracy by directly controlling image acquisition devices to adapt to road conditions, reducing computing power consumption and processing load, while minimizing additional correction or filtering operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2024007559_05122024_PF_FP_ABST
    Figure KR2024007559_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Disclosed are a method for a device to transmit a control message and a device therefor in a wireless communication system according to various embodiments. The method comprises the steps of: acquiring object recognition information from an artificial neural network-based object recognition model for recognizing and classifying objects in images of an image acquisition device; transmitting an object message to a peripheral device on the basis of object information recognized from the images included in the object recognition information; and determining whether to transmit the control message for controlling the image acquisition device, the determination being made on the basis of an object recognition ratio for the images included in the object recognition information.
Need to check novelty before this filing date? Find Prior Art

Description

Method for transmitting a message by a device in a wireless communication system and device therefor

[0001] A method for transmitting a control message for controlling an image acquisition device in a wireless communication system and a device therefor are provided.

[0002] Wireless communication systems are multiple access systems that support communication with multiple users by sharing available system resources (e.g., bandwidth, transmission power, etc.). Examples of multiple access systems include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA), and multi-carrier frequency division multiple access (MC-FDMA).

[0003] Sidelink (SL) refers to a communication method that establishes a direct link between user equipment (UE), allowing voice or data to be exchanged directly between terminals without going through a base station (BS). SL is being considered as a solution to address the burden on base stations due to rapidly increasing data traffic.

[0004] V2X (vehicle-to-everything) refers to a communication technology that exchanges information with other vehicles, pedestrians, and infrastructure-based objects through wired / wireless communication. V2X can be divided into four types: V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2N (vehicle-to-network), and V2P (vehicle-to-pedestrian). V2X communication can be provided through the PC5 interface and / or Uu interface.

[0005] Meanwhile, as more and more communication devices demand greater communication capacity, the need for improved mobile broadband communication compared to existing radio access technology (RAT) is emerging. Accordingly, communication systems that consider services or terminals sensitive to reliability and latency are being discussed. Next-generation wireless access technologies that consider improved mobile broadband communication, massive machine type communication (MTC), and ultra-reliable and low latency communication (URLLC) can be called new radio access technology (RAT) or new radio (NR). NR can also support vehicle-to-everything (V2X) communication.

[0006] Figure 1 is a diagram for comparing and explaining V2X communication based on RAT before NR and V2X communication based on NR.

[0007] In relation to V2X communication, in RATs prior to NR, methods for providing safety services based on V2X messages such as Basic Safety Message (BSM), Cooperative Awareness Message (CAM), and Decentralized Environmental Notification Message (DENM) were mainly discussed. V2X messages may include location information, dynamic information, attribute information, etc. For example, a terminal may transmit a CAM of a periodic message type and / or a DENM of an event triggered message type to another terminal.

[0008] For example, a CAM may include basic vehicle information such as dynamic vehicle status information, such as direction and speed, static vehicle data, such as dimensions, external lighting conditions, and route history. For example, a terminal may broadcast a CAM, and the latency of the CAM may be less than 100 ms. For example, in the event of an emergency, such as a vehicle breakdown or accident, a terminal may generate a DENM and transmit it to other terminals. For example, all vehicles within the transmission range of the terminal may receive the CAM and / or DENM. In this case, the DENM may have a higher priority than the CAM.

[0009] Since then, various V2X scenarios have been proposed in NR in relation to V2X communications. For example, various V2X scenarios may include vehicle platooning, advanced driving, extended sensors, and remote driving.

[0010] For example, based on vehicle platooning, vehicles can dynamically form groups and move together. For example, to perform platoon operations based on vehicle platooning, vehicles in the group can receive periodic data from the lead vehicle. For example, vehicles in the group can use this periodic data to narrow or widen the gap between vehicles.

[0011] For example, based on improved driving, vehicles can become semi-autonomous or fully automated. For example, each vehicle can adjust its trajectories or maneuvers based on data acquired from local sensors of nearby vehicles and / or nearby logical entities. Furthermore, for example, each vehicle can share driving intentions with nearby vehicles.

[0012] For example, based on extended sensors, raw data, processed data, or live video data acquired through local sensors can be exchanged between vehicles, logical entities, pedestrian terminals, and / or V2X application servers. Thus, for example, a vehicle can perceive its environment better than it can perceive using its own sensors.

[0013] For example, based on remote driving, a remote driver or V2X application can operate or control the remote vehicle for people who cannot drive or for remote vehicles located in hazardous environments. For example, in cases where the route is predictable, such as public transportation, cloud computing-based driving can be utilized to operate or control the remote vehicle. Additionally, access to a cloud-based back-end service platform, for example, can be considered for remote driving.

[0014] Meanwhile, a method to specify service requirements for various V2X scenarios, such as vehicle platooning, enhanced driving, expanded sensors, and remote driving, is being discussed in NR-based V2X communication.

[0015] The technical problem to be achieved by the present invention is to provide a method for transmitting and receiving messages for more accurate and efficient object recognition.

[0016] The technical challenges are not limited to the technical challenges mentioned above, and other technical challenges not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below.

[0017] A method for a device to transmit a control message in a wireless communication system according to one aspect may include: obtaining object recognition information from an artificial neural network-based object recognition model that recognizes and classifies an object from an image of an image acquisition device; transmitting an object message to a peripheral device based on object information recognized from the image included in the object recognition information; and determining whether to transmit the control message for controlling the image acquisition device based on an object recognition rate for the image included in the object recognition information.

[0018] Alternatively, the control message is characterized in that it is transmitted based on the object recognition rate being below a preset threshold.

[0019] Alternatively, the object recognition information further includes factor information on a cause of the object recognition model failing to classify at least one object from the image, and the control message is characterized in that it includes at least one control parameter determined based on the factor information.

[0020] Alternatively, the factor information is characterized in that it includes at least one of information on object classification failure according to lighting conditions, object classification failure according to object density, object classification failure according to object size, and object classification failure according to object movement speed.

[0021] Alternatively, based on the fact that the object recognition information further includes unclassified object information for an object having a mobility below a certain threshold, the control message is characterized in that it is transmitted even if the object recognition rate is above the preset threshold.

[0022] Alternatively, the control message is characterized in that it includes a control parameter that specifies a portion of an area to which an object corresponding to the unclassified object information belongs on the image.

[0023] Alternatively, the image acquisition device is characterized in that it does not transmit at least one image among the images to the object recognition model based on a difference in pixel values ​​between the acquired images.

[0024] Alternatively, the object recognition rate is characterized in that it is calculated based on a value obtained by dividing the number of objects classified by object type among the objects recognized by the object recognition module from the image by the number of objects.

[0025] According to another aspect, a device for performing the method of transmitting the control message described above may be provided.

[0026] According to another aspect, a processing device may be provided for controlling a device that performs the method of transmitting the control message described above.

[0027] According to another aspect, a method for an image acquisition device to receive a control message in a wireless communication system comprises the steps of: acquiring an image for a specific geographical area; transmitting the image to an artificial neural network-based object recognition model trained to recognize an object from the image; and receiving a control message from a first device, the control message including at least one control parameter instructing a change of a parameter related to acquisition of the image, wherein the control message can be received from the first device based on an object recognition rate included in object recognition information of the object recognition model being less than a preset threshold.

[0028] According to another aspect, a non-transitory computer-readable storage medium having recorded thereon commands for performing the method for receiving the above-described control message may be provided.

[0029] According to another aspect, an image acquisition device performing the method of receiving the control message described above may be provided.

[0030] According to another aspect, a processing device may be provided for controlling an image acquisition device that performs the method for receiving a control message described above.

[0031] According to one embodiment of the present invention, object recognition can be performed more accurately and efficiently through message transmission and reception in a wireless communication system.

[0032] The effects that can be obtained in various embodiments are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below.

[0033] The drawings attached to this specification are intended to provide an understanding of the present invention, illustrate various embodiments of the present invention, and together with the description of the specification serve to explain the principles of the present invention.

[0034] Figure 1 is a diagram for comparing and explaining V2X communication based on RAT before NR and V2X communication based on NR.

[0035] Figure 2 shows the structure of the LTE system.

[0036] Figure 3 shows the structure of the NR system.

[0037] Figure 4 shows the structure of a radio frame of NR.

[0038] Figure 5 shows the slot structure of an NR frame.

[0039] FIG. 6 illustrates a communication structure that can be provided in a 6G system according to one embodiment of the present disclosure.

[0040] FIG. 7 illustrates an electromagnetic spectrum according to one embodiment of the present disclosure.

[0041] FIG. 8 illustrates an example of a typical scenario of an NTN based on a transparent payload, according to one embodiment of the present disclosure.

[0042] FIG. 9 illustrates an example of a typical scenario of an NTN based on a regenerative payload, according to one embodiment of the present disclosure.

[0043] FIG. 10 illustrates an example of a sensing operation according to one embodiment of the present disclosure.

[0044] Figure 11 shows a radio protocol architecture for SL communication.

[0045] Figure 12 shows a terminal performing V2X or SL communication.

[0046] Figure 13 shows resource units for V2X or SL communication.

[0047] FIG. 14 illustrates an example of a BWP according to one embodiment of the present disclosure.

[0048] FIG. 15 illustrates a procedure for a terminal to perform V2X or SL communication according to a resource allocation mode, according to one embodiment of the present disclosure.

[0049] Figure 16 is a diagram illustrating an architecture capable of processing and acquiring adaptive sensing images.

[0050] FIGS. 17 to 19 are diagrams illustrating how a settings management module controls an image capture module or an image post-processing module.

[0051] FIG. 20 is a diagram illustrating how a device transmits a control message to control an image acquisition device.

[0052] Figure 21 is a drawing illustrating how an image acquisition device receives a control message from a device.

[0053] Figure 22 illustrates a communication system applied to the present invention.

[0054] Figure 23 illustrates a wireless device applicable to the present invention.

[0055] Figure 24 illustrates another example of a wireless device applicable to the present invention. The wireless device may be implemented in various forms depending on the use case / service.

[0056] Figure 25 illustrates a vehicle or autonomous vehicle to which the present invention is applied.

[0057] Figure 26 illustrates an AI device applied to the present invention.

[0058] A wireless communication system is a multiple access system that supports communication with multiple users by sharing available system resources (e.g., bandwidth, transmission power, etc.). Examples of multiple access systems include code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA), orthogonal frequency division multiple access (OFDMA), single carrier frequency division multiple access (SC-FDMA), and multi-carrier frequency division multiple access (MC-FDMA).

[0059] Sidelink refers to a communication method that establishes a direct link between user equipment (UE), allowing voice or data to be exchanged directly between terminals without going through a base station (BS). Sidelink is being considered as a solution to address the burden on base stations due to rapidly increasing data traffic.

[0060] V2X (vehicle-to-everything) refers to a communication technology that exchanges information with other vehicles, pedestrians, and infrastructure-based objects through wired / wireless communication. V2X can be divided into four types: V2V (vehicle-to-vehicle), V2I (vehicle-to-infrastructure), V2N (vehicle-to-network), and V2P (vehicle-to-pedestrian). V2X communication can be provided through the PC5 interface and / or Uu interface.

[0061] Meanwhile, as more and more communication devices demand greater communication capacity, the need for improved mobile broadband communication compared to existing radio access technology (RAT) is emerging. Accordingly, communication systems that consider services or terminals sensitive to reliability and latency are being discussed. Next-generation wireless access technologies that consider improved mobile broadband communication, massive MTC, and URLLC (Ultra-Reliable and Low Latency Communication) can be called new radio access technology (RAT) or new radio (NR). NR can also support V2X (vehicle-to-everything) communication.

[0062] The following technologies can be used in various wireless communication systems, such as CDMA (code division multiple access), FDMA (frequency division multiple access), TDMA (time division multiple access), OFDMA (orthogonal frequency division multiple access), and SC-FDMA (single carrier frequency division multiple access). CDMA can be implemented with wireless technologies such as UTRA (universal terrestrial radio access) or CDMA2000. TDMA can be implemented with wireless technologies such as GSM (global system for mobile communications) / GPRS (general packet radio service) / EDGE (enhanced data rates for GSM evolution). OFDMA can be implemented with wireless technologies such as IEEE (Institute of Electrical and Electronics Engineers) 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802-20, and E-UTRA (evolved UTRA). IEEE 802.16m is an evolution of IEEE 802.16e, providing backward compatibility with systems based on IEEE 802.16e. UTRA is part of UMTS (universal mobile telecommunications system). 3GPP (3rd generation partnership project) LTE (long term evolution) is a part of E-UMTS (evolved UMTS) that uses E-UTRA (evolved-UMTS terrestrial radio access), employing OFDMA in the downlink and SC-FDMA in the uplink.LTE-A (advanced) is an evolution of 3GPP LTE.

[0063] 5G NR, the successor to LTE-A, is a new clean-slate mobile communications system featuring high performance, low latency, and high availability. 5G NR can utilize all available spectrum resources, from low-frequency bands below 1 GHz, mid-frequency bands between 1 GHz and 10 GHz, and high-frequency (millimeter wave) bands above 24 GHz.

[0064] For clarity, the description will focus on LTE-A or 5G NR, but the technical ideas of the embodiment(s) are not limited thereto.

[0065] Figure 2 illustrates the architecture of an applicable LTE system. This may be referred to as an Evolved-UMTS Terrestrial Radio Access Network (E-UTRAN) or a Long Term Evolution (LTE) / LTE-A system.

[0066] Referring to FIG. 2, the E-UTRAN includes a base station (20; BS) that provides a control plane and a user plane to a terminal (10). The terminal (10) may be fixed or mobile, and may be referred to by other terms such as a mobile station (MS), a user terminal (UT), a subscriber station (SS), a mobile terminal (MT), a wireless device, etc. The base station (20) refers to a fixed station that communicates with the terminal (10), and may be referred to by other terms such as an evolved-NodeB (eNB), a base transceiver system (BTS), an access point, etc.

[0067] Base stations (20) can be connected to each other via the X2 interface. The base station (20) is connected to an EPC (Evolved Packet Core, 30) via the S1 interface, more specifically, to an MME (Mobility Management Entity) via the S1-MME, and to an S-GW (Serving Gateway) via the S1-U.

[0068] The EPC (30) consists of an MME, an S-GW, and a P-GW (Packet Data Network-Gateway). The MME holds information about terminal access and capabilities, and this information is primarily used for terminal mobility management. The S-GW is a gateway with the E-UTRAN as its endpoint, and the P-GW is a gateway with the PDN as its endpoint.

[0069] The layers of the radio interface protocol between the terminal and the network can be divided into L1 (Layer 1), L2 (Layer 2), and L3 (Layer 3) based on the three lower layers of the Open System Interconnection (OSI) standard model, which is widely known in communication systems. Among these, the physical layer belonging to Layer 1 provides an information transfer service using a physical channel, and the RRC (Radio Resource Control) layer located in Layer 3 controls radio resources between the terminal and the network. To this end, the RRC layer exchanges RRC messages between the terminal and the base station.

[0070] Figure 3 shows the structure of the NR system.

[0071] Referring to FIG. 3, the NG-RAN may include a gNB and / or an eNB that provides user plane and control plane protocol termination to the UE. FIG. 7 illustrates a case where only a gNB is included. The gNB and eNB are connected to each other via an Xn interface. The gNB and eNB are connected to the 5th generation core network (5G Core Network: 5GC) via the NG interface. More specifically, the gNB is connected to the access and mobility management function (AMF) via the NG-C interface, and the gNB is connected to the user plane function (UPF) via the NG-U interface.

[0072] Figure 4 shows the structure of a radio frame of NR.

[0073] Referring to FIG. 4, radio frames can be used for uplink and downlink transmission in NR. A radio frame has a length of 10 ms and can be defined as two 5 ms half-frames (Half-Frames, HF). A half-frame can include five 1 ms sub-frames (Subframes, SF). A sub-frame can be divided into one or more slots, and the number of slots within a sub-frame can be determined by the Subcarrier Spacing (SCS). Each slot can include 12 or 14 OFDM (A) symbols depending on the cyclic prefix (CP).

[0074] When normal CP is used, each slot can contain 14 symbols. When extended CP is used, each slot can contain 12 symbols. Here, the symbols can include OFDM symbols (or CP-OFDM symbols), SC-FDMA (Single Carrier - FDMA) symbols (or DFT-s-OFDM (Discrete Fourier Transform-spread-OFDM) symbols).

[0075] Table 1 below shows the number of symbols per slot ((N)) depending on the SCS setting (u) when normal CP is used. slot symb ), number of slots per frame ((N frame,u slot ) and the number of slots per subframe ((N subframe,u slot ) is an example.

[0076] SCS (15*2 u )N slot symb N frame,u slot N subframe,u slot 15KHz (u=0)1410130KHz (u=1)1420260KHz (u=2)14404120KHz (u=3)14808240KHz (u=4)1416016

[0077] Table 2 illustrates the number of symbols per slot, the number of slots per frame, and the number of slots per subframe according to SCS when extended CP is used.

[0078] SCS (15*2 u )N slot symb N frame,u slot N subframe,u slot 60KHz (u=2)12404

[0079] In an NR system, OFDM(A) numerology (e.g., SCS, CP length, etc.) may be set differently between multiple cells that are merged into a single terminal. Accordingly, the (absolute time) interval of a time resource (e.g., subframe, slot, or TTI) (conveniently referred to as TU (Time Unit)) consisting of the same number of symbols may be set differently between the merged cells.

[0080] In NR, multiple numerologies, or SCSs, can be supported to support various 5G services. For example, a 15 kHz SCS can support wide areas in traditional cellular bands, while a 30 kHz / 60 kHz SCS can support dense urban areas, lower latency, and wider carrier bandwidth. A 60 kHz or higher SCS can support bandwidths greater than 24.25 GHz to overcome phase noise.

[0081] The NR frequency band can be defined by two types of frequency ranges. The two types of frequency ranges can be FR1 and FR2. The numerical values ​​of the frequency ranges can be changed, and for example, the two types of frequency ranges can be as shown in Table 3 below. Among the frequency ranges used in the NR system, FR1 can mean the "sub 6 GHz range", and FR2 can mean the "above 6 GHz range" and can be called millimeter wave (mmW).

[0082] Frequency Range designationCorresponding frequency rangeSubcarrier Spacing (SCS)FR1450MHz - 6000MHz15, 30, 60kHzFR224250MHz - 52600MHz60, 120, 240kHz

[0083] As described above, the numerical value of the frequency range of the NR system can be changed. For example, FR1 may include a band from 410 MHz to 7125 MHz, as shown in Table 4 below. That is, FR1 may include a frequency band above 6 GHz (or 5850, 5900, 5925 MHz, etc.). For example, the frequency band above 6 GHz (or 5850, 5900, 5925 MHz, etc.) included within FR1 may include an unlicensed band. The unlicensed band may be used for various purposes, such as for vehicular communications (e.g., autonomous driving).

[0084] Frequency Range designationCorresponding frequency rangeSubcarrier Spacing (SCS)FR1410MHz - 7125MHz15, 30, 60kHzFR224250MHz - 52600MHz60, 120, 240kHz

[0085] Figure 5 shows the slot structure of an NR frame.

[0086] Referring to Figure 5, a slot includes multiple symbols in the time domain. For example, in the case of a normal CP, one slot may include 14 symbols, but in the case of an extended CP, one slot may include 12 symbols. Alternatively, in the case of a normal CP, one slot may include 7 symbols, but in the case of an extended CP, one slot may include 6 symbols.

[0087] A carrier includes multiple subcarriers in the frequency domain. An RB (Resource Block) can be defined as multiple (e.g., 12) consecutive subcarriers in the frequency domain. A BWP (Bandwidth Part) can be defined as multiple consecutive (P)RBs ((Physical) Resource Blocks) in the frequency domain, and can correspond to one numerology (e.g., SCS, CP length, etc.). A carrier can include up to N (e.g., 5) BWPs. Data communication can be performed through activated BWPs. Each element can be referred to as a Resource Element (RE) in the resource grid, and one complex symbol can be mapped to it.

[0088] Meanwhile, the wireless interface between terminals or between terminals and a network may be composed of an L1 layer, an L2 layer, and an L3 layer. In various embodiments of the present disclosure, the L1 layer may refer to a physical layer. Furthermore, for example, the L2 layer may refer to at least one of a MAC layer, an RLC layer, a PDCP layer, and an SDAP layer. Furthermore, for example, the L3 layer may refer to an RRC layer.

[0089] FIG. 6 illustrates a communication structure that can be provided in a 6G system according to an embodiment of the present disclosure. The embodiment of FIG. 6 can be combined with various embodiments of the present disclosure.

[0090] New network characteristics in 6G may include:

[0091] - Satellite integrated network

[0092] - Connected Intelligence: Unlike previous generations of wireless communication systems, 6G is revolutionary, upgrading the wireless evolution from "connected objects" to "connected intelligence." AI can be applied at every stage of the communication process (or at every signal processing step, as described below).

[0093] - Seamless integration of wireless information and energy transfer

[0094] - Ubiquitous super 3D connectivity: Access to networks and core network functions of drones and very low Earth orbit satellites will create super 3D connectivity in 6G ubiquitous.

[0095] Some general requirements for the new network characteristics of 6G, such as the above, may be as follows:

[0096] - small cell networks

[0097] - Ultra-dense heterogeneous network

[0098] - High-capacity backhaul

[0099] - Radar technology integrated with mobile technology: High-precision localization (or location-based services) through communications is a key feature of 6G wireless communication systems. Therefore, radar systems will be integrated with 6G networks.

[0100] - Softwarization and virtualization

[0101] Below, the core implementation technologies of the 6G system are described.

[0102] - Artificial Intelligence: Incorporating AI into communications can streamline and improve real-time data transmission. AI can use numerous analytics to determine how complex target tasks should be performed. This means AI can increase efficiency and reduce processing delays. Time-consuming tasks such as handovers, network selection, and resource scheduling can be performed instantly using AI. AI can also play a key role in machine-to-machine (M2M), machine-to-human, and human-to-machine communications. Furthermore, AI can facilitate rapid communication in brain-computer interfaces (BCIs). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radios, self-sustaining wireless networks, and machine learning.

[0103] - THz communication (terahertz communication): Data rates can be increased by increasing the bandwidth. This can be achieved by using sub-THz communication with wide bandwidths and applying advanced massive MIMO technology. THz waves, also known as sub-millimeter waves, typically refer to the frequency range between 0.1 THz and 10 THz, with corresponding wavelengths ranging from 0.03 mm to 3 mm. The 100 GHz to 300 GHz band (sub-THz band) is considered a key part of the THz spectrum for cellular communications. Adding the sub-THz band to the mmWave band will increase the capacity of 6G cellular communications. Among the defined THz bands, 300 GHz to 3 THz lies in the far infrared (IR) frequency band. While part of the optical band, the 300 GHz to 3 THz band lies at the boundary of the optical band, immediately following the RF band. Therefore, this 300 GHz to 3 THz band exhibits similarities to RF.

[0104] Figure 7 illustrates the electromagnetic spectrum according to one embodiment of the present disclosure. The embodiment of Figure 7 can be combined with various embodiments of the present disclosure. Key characteristics of THz communications include (i) a widely available bandwidth to support very high data rates, and (ii) high path loss at high frequencies (highly directional antennas are essential). The narrow beamwidth generated by the highly directional antenna reduces interference. The small wavelength of THz signals allows for a much larger number of antenna elements to be integrated into devices and base stations operating in this band. This enables the use of advanced adaptive array techniques to overcome range limitations.

[0105] - Large-scale MIMO technology

[0106] - Hologram beamforming (HBF)

[0107] - Optical wireless technology

[0108] - Free-space optical transmission backhaul network (FSO backhaul network)

[0109] - Quantum communication

[0110] - Cell-free communication

[0111] - Integration of wireless information and power transmission

[0112] - Integration of wireless communication and sensing

[0113] - Integrated access and backhaul network

[0114] - Big data analysis

[0115] - Reconfigurable intelligent surface

[0116] - metaverse

[0117] - Blockchain

[0118] Unmanned aerial vehicles (UAVs): UAVs, or drones, will be a key element in 6G wireless communications. In most cases, high-speed data wireless connectivity can be provided using UAV technology. Base stations (BSs) can be installed on UAVs to provide cellular connectivity. UAVs may offer specific capabilities not found in fixed BS infrastructure, such as easy deployment, robust line-of-sight links, and controlled mobility. During emergencies such as natural disasters, deploying terrestrial communications infrastructure is not economically feasible and sometimes cannot provide services in volatile environments. UAVs can easily handle these situations. UAVs will become a new paradigm in wireless communications. This technology facilitates three fundamental requirements for wireless networks: enhanced mobile broadband (eMBB), URLLC, and mMTC. UAVs can also support various purposes, such as enhancing network connectivity, fire detection, disaster emergency services, security and surveillance, pollution monitoring, parking monitoring, and accident monitoring. Therefore, UAV technology is recognized as one of the most important technologies for 6G communications.

[0119] - Autonomous driving (self-driving): V2X (vehicle to everything), a key element in building autonomous driving infrastructure, can be a technology that allows cars to communicate and share with various elements on the road for autonomous driving, such as vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) wireless communication. Fast transmission speeds and low-latency technologies are essential to maximize autonomous driving performance and ensure high safety. Furthermore, in the future, autonomous driving will go beyond simply providing warnings or guidance messages to drivers and may require active intervention in vehicle operation and direct control of the vehicle in dangerous situations. To this end, the amount of information that needs to be transmitted and received may become enormous, so 6G is expected to maximize autonomous driving with faster transmission speeds and lower latency than 5G.

[0120] - Non-terrestrial networks (NTN): NTN may refer to a network or network segment that uses radio frequency (RF) resources mounted on a satellite (or unmanned aerial system (UAS) platform). FIG. 8 illustrates an example of a typical NTN scenario based on a transparent payload according to an embodiment of the present disclosure. FIG. 9 illustrates an example of a typical NTN scenario based on a regenerative payload according to an embodiment of the present disclosure. The embodiments of FIG. 8 or FIG. 9 may be combined with various embodiments of the present disclosure. Referring to FIG. 8, a satellite (or UAS platform) may create a service link with a UE. The satellite (or UAS platform) may be connected to a gateway via a feeder link. The satellite may be connected to a data network via the gateway. A beam footprint may refer to an area where a signal transmitted by a satellite can be received. Referring to Figure 9, a satellite (or UAS platform) can establish a service link with a UE. A satellite (or UAS platform) connected to a UE can be connected to another satellite (or UAS platform) via an inter-satellite link (ISL). The other satellite (or UAS platform) can be connected to a gateway via a feeder link. Based on the replay payload, a satellite can be connected to a data network through another satellite and the gateway. If an ISL does not exist between a satellite and another satellite, a feeder link between the satellite and the gateway may be required. Figures 8 and 9 are merely examples of NTN scenarios, and NTN can be implemented based on various scenarios.For example, a satellite (or UAS platform) may implement a transparent or regenerative (with onboard processing) payload. For example, a satellite (or UAS platform) may generate multiple beams over a designated service area depending on the field of view of the satellite (or UAS platform). For example, the field of view of the satellite (or UAS platform) may vary depending on the onboard antenna diagram and minimum elevation angle. For example, a transparent payload may include radio frequency filtering, frequency conversion, and amplification. Therefore, the waveform signal repeated by the payload may not be altered. For example, a regenerative payload may include radio frequency filtering, frequency conversion and amplification, demodulation / decoding, switching and / or routing, and coding / modulation. For example, a regenerative payload may be substantially equivalent to equipping the satellite (or UAS platform) with all or part of the base station functionality.

[0121] - Integrated Sensing and Communication (ISAC): Wireless sensing is a technology that uses radio frequencies to determine the instantaneous linear velocity, angle, distance (range), etc. of an object, thereby obtaining information about the characteristics of the environment and / or objects within the environment. Because radio frequency sensing does not require a device to connect to the object through a network, it can provide a service for object positioning without a device. The ability to obtain range, velocity, and angle information from radio frequency signals can enable a wide range of new capabilities, such as various object detection, object recognition (e.g., vehicles, humans, animals, UAVs), and high-precision localization, tracking, and activity recognition. Wireless sensing services can provide information to a variety of industries (e.g., drones, smart homes, V2X, factories, railways, public safety, etc.), enabling applications such as intruder detection, assisted vehicle steering and navigation, trajectory tracking, collision avoidance, traffic management, and health and traffic management. In some cases, wireless sensing can utilize non-3GPP type sensors (e.g., radar, cameras) to further support 3GPP-based sensing. For example, the operation of a wireless sensing service, i.e., a sensing operation, may depend on the transmission, reflection, and scattering of wireless sensing signals. Therefore, wireless sensing may provide an opportunity to enhance existing communication systems from a communication network to a wireless communication and sensing network. FIG. 10 illustrates an example of a sensing operation according to an embodiment of the present disclosure. The embodiment of FIG. 10 may be combined with various embodiments of the present disclosure. Specifically, FIG. 10 (a) illustrates an example of sensing using a sensing receiver and a sensing transmitter located at the same location (e.g., monostatic sensing), and FIG. 10 (b) illustrates an example of sensing using a separated sensing receiver and a sensing transmitter (e.g., bistatic sensing).

[0122] Figure 11 illustrates a radio protocol architecture for SL communication. Specifically, Figure 11 (a) illustrates the user plane protocol stack of NR, and Figure 11 (b) illustrates the control plane protocol stack of NR.

[0123] Below, the SL synchronization signal (Sidelink Synchronization Signal, SLSS) and synchronization information are described.

[0124] SLSS is an SL-specific sequence and may include a Primary Sidelink Synchronization Signal (PSSS) and a Secondary Sidelink Synchronization Signal (SSSS). The PSSS may be referred to as a Sidelink Primary Synchronization Signal (S-PSS), and the SSSS may be referred to as a Sidelink Secondary Synchronization Signal (S-SSS). For example, length-127 M-sequences may be used for the S-PSS, and length-127 Gold sequences may be used for the S-SSS. For example, a terminal may detect an initial signal and acquire synchronization using the S-PSS. For example, a terminal may acquire detailed synchronization and detect a synchronization signal ID using the S-PSS and the S-SSS.

[0125] PSBCH (Physical Sidelink Broadcast Channel) may be a (broadcast) channel that transmits basic (system) information that a terminal must know first before transmitting or receiving an SL signal. For example, the basic information may be information related to SLSS, duplex mode (DM), TDD UL / DL (Time Division Duplex Uplink / Downlink) configuration, resource pool-related information, type of application related to SLSS, subframe offset, broadcast information, etc. For example, in NR V2X, for evaluating PSBCH performance, the payload size of PSBCH may be 56 bits, including a 24-bit CRC.

[0126] S-PSS, S-SSS and PSBCH may be included in a block format supporting periodic transmission (e.g., SL SS (Synchronization Signal) / PSBCH block, hereinafter referred to as S-SSB (Sidelink-Synchronization Signal Block)). The S-SSB may have the same numerology (i.e., SCS and CP length) as the PSCCH (Physical Sidelink Control Channel) / PSSCH (Physical Sidelink Shared Channel) in the carrier, and the transmission bandwidth may be within a (pre-)configured SL BWP (Sidelink BWP). For example, the bandwidth of the S-SSB may be 11 RBs (Resource Blocks). For example, the PSBCH may span 11 RBs. And, the frequency location of the S-SSB may be (pre-)configured. Therefore, the terminal does not need to perform hypothesis detection in the frequency to discover the S-SSB in the carrier.

[0127] Meanwhile, in the NR SL system, multiple numerologies having different SCS and / or CP lengths may be supported. In this case, as the SCS increases, the length of the time resource for a transmitting terminal to transmit an S-SSB may become shorter. Accordingly, the coverage of the S-SSB may decrease. Therefore, in order to ensure the coverage of the S-SSB, the transmitting terminal may transmit one or more S-SSBs to a receiving terminal within one S-SSB transmission period according to the SCS. For example, the number of S-SSBs that the transmitting terminal transmits to the receiving terminal within one S-SSB transmission period may be pre-configured or configured for the transmitting terminal. For example, the S-SSB transmission period may be 160 ms. For example, an S-SSB transmission period of 160 ms may be supported for all SCSs.

[0128] For example, when the SCS is 15 kHz at FR1, the transmitting terminal can transmit one or two S-SSBs to the receiving terminal within one S-SSB transmission period. For example, when the SCS is 30 kHz at FR1, the transmitting terminal can transmit one or two S-SSBs to the receiving terminal within one S-SSB transmission period. For example, when the SCS is 60 kHz at FR1, the transmitting terminal can transmit one, two, or four S-SSBs to the receiving terminal within one S-SSB transmission period.

[0129] For example, when the SCS is 60 kHz at FR2, the transmitting terminal can transmit 1, 2, 4, 8, 16, or 32 S-SSBs to the receiving terminal within one S-SSB transmission period. For example, when the SCS is 120 kHz at FR2, the transmitting terminal can transmit 1, 2, 4, 8, 16, 32, or 64 S-SSBs to the receiving terminal within one S-SSB transmission period.

[0130] Meanwhile, when the SCS is 60 kHz, two types of CP may be supported. In addition, the structure of the S-SSB transmitted by the transmitting terminal to the receiving terminal may be different depending on the CP type. For example, the CP type may be Normal CP (NCP) or Extended CP (ECP). Specifically, for example, when the CP type is NCP, the number of symbols to which the PSBCH is mapped within the S-SSB transmitted by the transmitting terminal may be 9 or 8. On the other hand, for example, when the CP type is ECP, the number of symbols to which the PSBCH is mapped within the S-SSB transmitted by the transmitting terminal may be 7 or 6. For example, the PSBCH may be mapped to the first symbol within the S-SSB transmitted by the transmitting terminal. For example, the receiving terminal receiving the S-SSB may perform an Automatic Gain Control (AGC) operation in the first symbol section of the S-SSB.

[0131] Figure 12 shows a terminal performing V2X or SL communication.

[0132] Referring to FIG. 12, the term "terminal" in V2X or SL communication may primarily refer to a user's terminal. However, if a network device such as a base station transmits and receives signals according to a communication method between terminals, the base station may also be considered a type of terminal. For example, terminal 1 may be a first device (100), and terminal 2 may be a second device (200).

[0133] For example, terminal 1 can select a resource unit corresponding to a specific resource within a resource pool, which represents a set of resources. Then, terminal 1 can transmit an SL signal using the resource unit. For example, terminal 2, which is a receiving terminal, can be configured with a resource pool in which terminal 1 can transmit a signal, and can detect a signal from terminal 1 within the resource pool.

[0134] Here, if terminal 1 is within the connection range of the base station, the base station can inform terminal 1 of the resource pool. On the other hand, if terminal 1 is outside the connection range of the base station, another terminal can inform terminal 1 of the resource pool, or terminal 1 can use a pre-configured resource pool.

[0135] In general, a resource pool can be composed of multiple resource units, and each terminal can select one or multiple resource units to use for its SL signal transmission.

[0136] Figure 13 shows resource units for V2X or SL communication.

[0137] Referring to Figure 13, the entire frequency resources of the resource pool can be divided into NF units, and the entire time resources of the resource pool can be divided into NT units. Therefore, a total of NF * NT resource units can be defined within the resource pool. Figure 13 illustrates an example where the resource pool repeats with a cycle of NT subframes.

[0138] As illustrated in Figure 13, a single resource unit (e.g., Unit #0) may appear periodically and repeatedly. Alternatively, to achieve diversity effects in the time or frequency dimensions, the index of the physical resource unit to which a single logical resource unit is mapped may change over time in a predetermined pattern. In this resource unit structure, a resource pool may refer to a set of resource units that a terminal wishing to transmit an SL signal can use for transmission.

[0139] Resource pools can be subdivided into several categories. For example, based on the content of the SL signal transmitted from each resource pool, resource pools can be categorized as follows:

[0140] (1) Scheduling Assignment (SA) may be a signal that includes information such as the location of resources used by a transmitting terminal for transmission of an SL data channel, MCS (Modulation and Coding Scheme) or MIMO (Multiple Input Multiple Output) transmission method required for demodulation of other data channels, and TA (Timing Advance). SA may also be transmitted multiplexed with SL data on the same resource unit, in which case the SA resource pool may mean a resource pool in which SA is multiplexed with SL data and transmitted. SA may also be called an SL control channel.

[0141] (2) The SL data channel (Physical Sidelink Shared Channel, PSSCH) may be a resource pool used by a transmitting terminal to transmit user data. If SA is multiplexed and transmitted together with SL data on the same resource unit, only the SL data channel excluding SA information may be transmitted from the resource pool for the SL data channel. In other words, the REs (Resource Elements) that were used to transmit SA information on individual resource units within the SA resource pool may still be used to transmit SL data in the resource pool of the SL data channel. For example, the transmitting terminal may transmit the PSSCH by mapping it to consecutive PRBs.

[0142] (3) A discovery channel may be a resource pool for transmitting terminals to transmit information such as their IDs. Through this, transmitting terminals can enable neighboring terminals to discover them.

[0143] Even if the content of the SL signal described above is the same, different resource pools may be used depending on the transmission and reception properties of the SL signal. For example, even if it is the same SL data channel or discovery message, it may be again divided into different resource pools depending on the transmission timing determination method of the SL signal (for example, whether it is transmitted at the time of reception of a synchronization reference signal or whether it is transmitted by applying a certain timing advance at the time of reception), the resource allocation method (for example, whether the base station designates transmission resources for individual signals to individual transmitting terminals or whether individual transmitting terminals independently select individual signal transmission resources within the resource pool), the signal format (for example, the number of symbols each SL signal occupies in one subframe or the number of subframes used for transmission of one SL signal), the signal strength from the base station, the transmission power strength of the SL terminal, etc.

[0144] FIG. 14 illustrates an example of a BWP according to an embodiment of the present disclosure. The embodiment of FIG. 14 can be combined with various embodiments of the present disclosure. In the embodiment of FIG. 14, it is assumed that there are three BWPs.

[0145] Referring to Figure 14, a common resource block (CRB) may be a carrier resource block numbered from one end of a carrier band to the other. Furthermore, a PRB may be a numbered resource block within each BWP. Point A may indicate a common reference point for the resource block grid.

[0146] The BWP can be set by Point A, an offset from Point A (NstartBWP), and a bandwidth (NsizeBWP). For example, Point A can be an outer reference point of a PRB of a carrier where subcarrier 0 of all numerologies (e.g., all numerologies supported by the network on that carrier) are aligned. For example, the offset can be the PRB spacing between the lowest subcarrier in a given numerology and Point A. For example, the bandwidth can be the number of PRBs in a given numerology.

[0147] SLSS (Sidelink Synchronization Signal) is a SL (sidelink) specific sequence and may include PSSS (Primary Sidelink Synchronization Signal) and SSSS (Secondary Sidelink Synchronization Signal). The PSSS may be referred to as S-PSS (Sidelink Primary Synchronization Signal) and the SSSS may be referred to as S-SSS (Sidelink Secondary Synchronization Signal). For example, length-127 M-sequences may be used for S-PSS and length-127 Gold sequences may be used for S-SSS. For example, a terminal may detect an initial signal (signal detection) and obtain synchronization using S-PSS. For example, the terminal can obtain detailed synchronization using S-PSS and S-SSS and detect a synchronization signal ID.

[0148] PSBCH (Physical Sidelink Broadcast Channel) may be a (broadcast) channel that transmits basic (system) information that a terminal must know first before transmitting or receiving an SL signal. For example, the basic information may be information related to SLSS, duplex mode (DM), TDD UL / DL (Time Division Duplex Uplink / Downlink) configuration, resource pool-related information, type of application related to SLSS, subframe offset, broadcast information, etc. For example, in order to evaluate PSBCH performance, in NR V2X, the payload size of PSBCH may be 56 bits, including a 24-bit CRC (Cyclic Redundancy Check).

[0149] S-PSS, S-SSS and PSBCH may be included in a block format supporting periodic transmission (e.g., SL SS (Synchronization Signal) / PSBCH block, hereinafter referred to as S-SSB (Sidelink-Synchronization Signal Block)). The S-SSB may have the same numerology (i.e., SCS and CP length) as the PSCCH (Physical Sidelink Control Channel) / PSSCH (Physical Sidelink Shared Channel) in the carrier, and the transmission bandwidth may be within a (pre-)configured SL BWP (Sidelink BWP). For example, the bandwidth of the S-SSB may be 11 RBs (Resource Blocks). For example, the PSBCH may span 11 RBs. And, the frequency location of the S-SSB may be (pre-)configured. Therefore, the terminal does not need to perform hypothesis detection in the frequency to discover the S-SSB in the carrier.

[0150] FIG. 15 illustrates a procedure for a terminal to perform V2X or SL communication according to a resource allocation mode, according to one embodiment of the present disclosure. The embodiment of FIG. 15 may be combined with various embodiments of the present disclosure.

[0151] Referring to (a) of FIG. 15, in resource allocation mode 1, the base station may schedule SL resources to be used by the terminal for SL transmission. For example, in step S1500, the base station may transmit information related to SL resources and / or information related to UL resources to the first terminal. For example, the UL resources may include PUCCH resources and / or PUSCH resources. For example, the UL resources may be resources for reporting SL HARQ feedback to the base station.

[0152] For example, a first terminal may receive information related to a dynamic grant (DG) resource and / or information related to a configured grant (CG) resource from a base station. For example, a CG resource may include a CG type 1 resource or a CG type 2 resource. In this specification, a DG resource may be a resource that a base station configures / allocates to the first terminal via downlink control information (DCI). In this specification, a CG resource may be a (periodic) resource that a base station configures / allocates to the first terminal via DCI and / or an RRC message. For example, in the case of a CG type 1 resource, the base station may transmit an RRC message including information related to the CG resource to the first terminal. For example, in the case of a CG type 2 resource, the base station may transmit an RRC message including information related to the CG resource to the first terminal, and the base station may transmit a DCI related to activation or release of the CG resource to the first terminal.

[0153] In step S1510, the first terminal may transmit a PSCCH (e.g., Sidelink Control Information (SCI) or 1st-stage SCI) to the second terminal based on the resource scheduling. In step S1520, the first terminal may transmit a PSSCH (e.g., 2nd-stage SCI, MAC PDU, data, etc.) related to the PSCCH to the second terminal. In step S1530, the first terminal may receive a PSFCH related to the PSCCH / PSSCH from the second terminal. For example, HARQ feedback information (e.g., NACK information or ACK information) may be received from the second terminal via the PSFCH. In step S1540, the first terminal may transmit / report HARQ feedback information to the base station via a PUCCH or a PUSCH. For example, the HARQ feedback information reported to the base station may be information generated by the first terminal based on the HARQ feedback information received from the second terminal. For example, the HARQ feedback information reported to the base station may be information generated by the first terminal based on a rule set in advance. For example, the DCI may be DCI for scheduling SL.

[0154] Referring to (b) of FIG. 15, in resource allocation mode 2, a terminal can determine SL transmission resources within SL resources set by a base station / network or preset SL resources. For example, the set SL resources or preset SL resources may be a resource pool. For example, the terminal can autonomously select or schedule resources for SL transmission. For example, the terminal can perform SL communication by selecting resources within the set resource pool. For example, the terminal can select resources within a selection window by performing sensing and resource (re)selection procedures. For example, the sensing can be performed on a subchannel basis. For example, in step S1510, a first terminal that has selected resources within the resource pool can transmit a PSCCH (e.g., Sidelink Control Information (SCI) or 1st-stage SCI) to a second terminal using the resources. In step S1520, the first terminal may transmit a PSSCH (e.g., 2nd-stage SCI, MAC PDU, data, etc.) related to the PSCCH to the second terminal. In step S1530, the first terminal may receive a PSFCH related to the PSCCH / PSSCH from the second terminal.

[0155] Referring to (a) or (b) of FIG. 15, for example, a first terminal may transmit an SCI to a second terminal on a PSCCH. Alternatively, for example, the first terminal may transmit two consecutive SCIs (e.g., 2-stage SCIs) to the second terminal on the PSCCH and / or the PSSCH. In this case, the second terminal may decode the two consecutive SCIs (e.g., 2-stage SCIs) to receive the PSSCH from the first terminal. In this specification, an SCI transmitted on a PSCCH may be referred to as a 1st SCI, a 1st SCI, a 1st-stage SCI, or a 1st-stage SCI format, and an SCI transmitted on a PSSCH may be referred to as a 2nd SCI, a 2nd SCI, a 2nd-stage SCI, or a 2nd-stage SCI format.

[0156] Referring to (a) or (b) of FIG. 15, in step S1530, the first terminal may receive a PSFCH. For example, the first terminal and the second terminal may determine PSFCH resources, and the second terminal may use the PSFCH resources to transmit HARQ feedback to the first terminal.

[0157] Referring to (a) of FIG. 15, in step S1540, the first terminal may transmit SL HARQ feedback to the base station via PUCCH and / or PUSCH.

[0158] Meanwhile, the aforementioned sidelink can be defined as terminal-to-terminal communication or direct communication between terminals. In this case, the PSCCH can be defined as a physical control channel for terminal-to-terminal communication, the PSSCH as a physical data channel or physical shared channel for terminal-to-terminal communication, and the PSFCH as a physical feedback transmission channel between terminals.

[0159] Method for acquiring and processing adaptive sensing image data in V2X services

[0160] V2X services play a significant role in ensuring safety, including collision prevention, and controlling efficient traffic flow by allowing road users (vehicles, RSUs, pedestrians, etc.) to communicate with surrounding road users to transmit their status information (location, speed, size, etc.) or environmental information (maps, signal information, etc.). Currently, various V2X services using various communication methods (short-range communication, long-range communication) exist, and service level requirements (SLREs) have been defined to ensure the organic operation of V2X services involving multiple ITS-Stations.

[0161] Among these service structures, infrastructure-based services based on Roadside Units (RSUs) can help improve traffic safety by detecting and predicting collisions through generating, transmitting, and relaying ITS messages between Vulnerable Road Users (VRUs) and vehicles. RSUs can employ AI capabilities to improve the quality of object detection and predict collisions by implementing camera-based solutions. Furthermore, RSUs can detect the locations of unconnected VRUs or objects (via PC5 or Uu links) with relative accuracy. The RSU-based network connection is structured as a camera (or image capture module) - AI sensor (AI sensing module) - analyzer (sensing data analysis module) - message generator (message generation module), and depending on the processing complexity of object recognition, MEC (Multi-access Edge Computing) and cloud can be appropriately utilized.

[0162] Here, object recognition is a computer vision technology that automatically detects and recognizes specific objects in digital images. Typical object recognition technologies available in ITS are deep learning-based object recognition technologies (e.g., DNN, Yolo (You Only Look Once)), which input an image into an artificial neural network model, extract features from the image, and perform object recognition and classification based on the extracted features. Deep learning-based object recognition technologies may include object detection (OD) technology for detecting objects in an input image and object classification (OC) technology for classifying the detected objects. The two technologies can also be processed as a single logical unit. Each technology is described in detail as follows.

[0163] - Input image into an artificial neural network model (trained to detect objects in images) -> The artificial neural network model extracts features from the input image -> The artificial neural network model uses the extracted features to create a bounding box in the area where the object is likely to be in the image -> Input the extracted image into an artificial neural network model (trained to classify the type of object) -> Classify the input image to predict the type of object detected

[0164] For example, YOLO (You Only Look Once) is a deep learning-based model for object detection. It can predict object locations and classes in real time by processing an image with a single forward propagation. Specifically, YOLO divides the input image into multiple grids and predicts multiple bounding boxes and class probabilities in each grid cell to detect objects.

[0165] These diverse object recognition technologies can play a crucial role in recognizing objects within ITS and supporting V2X services. Furthermore, in the fields of V2X, ITS, and autonomous driving, object recognition technology with the speed and accuracy to enable real-time processing can be crucial.

[0166] Meanwhile, the existing object recognition technologies / algorithms described above may suffer from the following object recognition errors. For example, the errors described below may be caused by the characteristics of the image acquisition device, such as the camera that acquires the image.

[0167] ① Small Object Recognition Errors: Object recognition algorithms typically excel at recognizing large objects, but are more likely to encounter errors with small objects. For example, recognition rates for small objects can vary depending on the altitude or height of the camera capturing the image.

[0168] ② Inappropriate brightness or contrast: The brightness or contrast of an object can vary depending on lighting conditions or the shooting location. In this case, the object recognition rate based on object recognition algorithms can be significantly reduced. For example, object recognition rates can be significantly reduced when the road is darkened in the early morning hours.

[0169] ③ Distorted Image Recognition Errors: Distortion or deformation can occur in images due to the object's speed or weather conditions, such as fog. In these cases, the object recognition rate based on the object recognition algorithm can significantly decrease. For example, images of fast-moving objects can exhibit blur distortion due to insufficient shutter speed of the image acquisition device or a mismatch between shutter timing and speed. This distortion can significantly reduce the object recognition rate in the image.

[0170] Although there are image enhancement algorithms that consider the issues related to this in object recognition algorithms, they are not the core technology of object recognition. In addition, the correction function of the image acquisition device (e.g. CCTV or camera) itself may be utilized, but such correction may be for the linear correction of the image acquisition device itself rather than correction that reflects the road conditions. Such improvement algorithm technology is not directly linked to the object recognition technology of the object recognition algorithm, and may be a technology that operates independently. In addition, CCTV, as an image acquisition device, has limitations in that it cannot be directly used to improve the recognition rate of object recognition because it uses continuous computing power without optimization due to static settings, which is low in efficiency.

[0171] To address these issues, it is necessary to directly control the image acquisition device based on the output information / object recognition information from an AI (Artificial Intelligence) sensor / AI-based object recognition model, thereby acquiring images with characteristics optimized for the road conditions / environment. This allows the AI ​​sensor / model to input images with characteristics optimized for the road conditions, significantly improving the object recognition rate of the AI ​​sensor / model.

[0172] To this end, the proposed invention can generate configuration parameters for image correction and / or image acquisition device control based on specific trigger conditions, and operate internally in a restful manner according to a connected interface structure, or generate messages through an external MEC or cloud and transmit them to the reference architecture connection structure(s). In addition, the proposed invention can adaptively acquire and process images by reflecting image processing requirements contained in the configuration / control parameters or generated messages through the generation of new modules of the video capturing module, the post-processing module, and the configuration manager module of the reference architecture. In this case, the proposed invention can control the image acquisition device to acquire an image suitable for the road environment / road situation through the above-mentioned setting parameters, and input the acquired image into an AI sensor / AI model to significantly improve the object recognition rate, and can omit / minimize the operation of additional correction / filtering for the image by acquiring the image itself optimized for object recognition, thereby significantly reducing the computing power consumption for object recognition.

[0173] Below, a method in which a device such as an RSU generates configuration parameters for image correction and / or image acquisition device control based on output information of an AI-based object recognition model, and a method for controlling the image acquisition device based on the configuration parameters are described in detail.

[0174] Figure 16 is a diagram illustrating an architecture capable of processing and acquiring adaptive sensing images.

[0175] Referring to FIG. 16, the reference architecture may include an AI sensing module (310), a sensor data analysis module (320), a message generation module (330), an application module (340), and an infrastructure operation module (400). Here, the AI ​​sensing module (310) may include a learned AI-based object recognition model (or AI model) for object recognition / object classification. In addition, the reference architecture may further include proposed new modules, such as an image capture module (210), an image post-processing module (220), and an image configuration manager module (100).

[0176] The newly proposed image capture module (210), image post-processing module (220), and setting management module (100) can operate based on the satisfaction of at least one of the following trigger conditions.

[0177] - ① When the comprehensive judgment value of the reliability or criterion significant value of the object recognition algorithm is below a preset threshold: For example, the reliability or criterion significant value of the algorithm may be recognition accuracy, recognition precision, recall, F1 score (harmonic mean of precision and recall), etc. For example, the recognition accuracy may be the number of objects (correctly) recognized from the image by the AI ​​model / the total number of objects (number of recognizable objects). For example, when the AI ​​model recognizes / detects 10 objects in the image, classifies 8 out of 10 objects, and fails to classify 2 objects, the accuracy of the AI ​​model may be calculated as 8 / 10*100=80%.

[0178] - ② When active configuration parameter input or a message related thereto is provided according to the road environment: For example, active configuration parameter input may be provided in the case of specific road conditions / road environments such as traffic congestion (determined by vehicle speed), road conditions by time zone (e.g., rush hour), operation of variable signals (traffic lights that turn on in the evening), etc. For example, when the sensor data analysis module (320) and / or the infrastructure operation module (400) detect the above-described situations, they may transmit / input a message including the active configuration parameters described above or information related thereto to the configuration management module (100). In this case, the configuration management module (100) may perform an operation for controlling the image capture module (210) / image post-processing module (220) based on the message.

[0179] - ③ When a specific input is provided for efficient computing use (as a setting management module) from the AI ​​sensing module (310) side

[0180] - ④ When object recognition is difficult due to high RSU or CCTV location

[0181] - ⑤ When the creation of control parameters is instructed by the infrastructure operation module

[0182] - ⑥ When application of control parameters is required due to detection of specific events such as special accidents or weather.

[0183] Meanwhile, the setting management module (100) can generate a control parameter for controlling the image capture module (210) or a message including the control parameter when at least one of the above-described triggering conditions is satisfied. The setting management module (100) can provide the generated control parameter to the image capture module (210) / image post-processing module (220) to adjust / control parameters related to image acquisition or adjust parameters related to performing post-processing on the image. In this way, the setting management module (100) can control the image characteristics itself acquired from the image capture module (210) / image post-processing module (220) to be suitable for road conditions.

[0184] Below, each of the modules described above is described in detail. Each module is intended to be a logical unit, and the functions / operations described below can be implemented as functions at the RSU level, MEC level, and Cloud level.

[0185] (1) Image capture module

[0186] The image capture module (210) may be an image acquisition device (camera, CCTV). The image capture module (210) may acquire / capture raw data for an image that has gone through a general OETF (Optical Electronic Transfer Function), and may transfer the raw data for the captured / acquired image to an image post-processing module (220) / AI sensing module (230) / image encoding module (230). Here, the OETF may be a conversion function defined to convert an optical input signal into an electrical output signal in an image / video system. More specifically, in the image / video system, the OETF (Optical-Electro Transfer Function) may be a conversion function used to convert an original image acquired by a camera sensor into an image in a digital format.

[0187] The image capture module (210) can adjust / change the conversion function related to the OETF based on the control parameters received from the configuration management module (100). For example, the image capture module (210) can adjust / control the conversion ratio / ratio of the electrical output signal to the optical input signal by changing the conversion function of the OETF according to the control parameters received from the configuration management module (100).

[0188] Alternatively, the image capture module (210) may change the capture properties of the original image by adjusting the contrast ratio based on the control parameters received from the setting management module (100). Alternatively, the image capture module (210) may adaptively change the image resolution, frame rate, and / or brightness based on the control parameters received from the setting management module (100) according to the road conditions / environment.

[0189] (2) AI sensing module

[0190] The AI ​​sensing module (230) may include an object recognition model trained to recognize objects from images and classify recognized objects. For example, the AI ​​sensing module (230) may perform object detection and object classification using image object recognition technology. Here, object recognition may be an operation in which an AI model, which is an artificial neural network model, extracts features from an input image (based on CNN, etc.) and then designates / masks an area in which an object is recognized as a bounding box. Object classification may be an operation in which a recognized object is predicted based on features of an image inside the bounding box. For example, the AI ​​sensing module (230) may be trained in advance to be able to recognize objects (pedestrians, bicycles, motorcycles, vehicles, etc.) related to V2X services. For example, the AI ​​sensing module (230) may generate / predict object information (speed, direction of movement, movement case, object shape, object size, etc.) for the recognized object. The AI ​​sensing module (230) can output object recognition information, which is output information including recognized / classified object information.

[0191] The AI ​​sensing module (230) can transmit prediction results or output data regarding objects that are meaningful in relation to the V2X service among the detected objects (i.e., objects that need to be generated as object information about the surrounding environment in relation to the V2X service, such as pedestrians, bicycles, motorcycles, vehicles, etc.) to the sensor data analysis module (320). Alternatively, the AI ​​sensing module (230) can calculate accuracy (or object recognition rate), which is a value obtained by dividing the number of correctly recognized objects (or the number of classified objects) as described above by the total number of recognized objects (or the total number of recognized / detected objects). The calculated object recognition rate / accuracy can be used to evaluate the performance of the object recognition / classification algorithm of the AI ​​sensing module (230). Meanwhile, the AI ​​sensing module (230) can also calculate accuracy in categories such as path prediction and hazard detection as well as object recognition / classification.

[0192] The AI ​​sensing module (230) can transmit output results including object information about recognized objects and / or information about the object recognition rate to the setting management module (100) and / or the sensor data analysis module (320).

[0193] (3) Sensor data analysis module

[0194] The sensor data analysis module (320) can determine whether to generate an object message including object information resulting from detection / recognition of objects related to a V2X service based on the output information / output data of the AI ​​sensing module (230). When determining to generate an object message, the sensor data analysis module (320) can provide information on parameters related to the generation of the message and the object information to the message generation module.

[0195] (4) Message generation module

[0196] The message generation module (330) can generate a message for V2X or ITS based on the parameters and object information transmitted from the sensor data analysis module (320).

[0197] (5) Application module

[0198] The application module (340) can decode a message received from the message generation module (330) and provide the user with object information, etc. included in the message.

[0199] (6) Settings Management Module

[0200] The configuration management module (100) can determine whether to generate at least one control parameter for controlling the image capture module (210) and / or the image post-processing module (220) based on output information / output data (e.g., object recognition information) received from the AI ​​sensing module (230) and / or the sensor data analysis module (320). The configuration management module (100) can transmit the generated at least one control parameter to the image capture module (210) or the image post-processing module (220).

[0201] For example, the setting management module (100) may trigger the generation of at least one control parameter for controlling the image capture module (210) and / or the image post-processing module (220) when the accuracy or object recognition rate included in the output information / output data is below a preset threshold. For example, the object recognition information may further include cause information on a failure factor of object recognition and / or object classification. The cause information may include information on a failure factor of object classification, such as a blurring phenomenon due to object speed, low resolution of the image compared to the object size, increase in object density, lighting conditions, increase in noise due to worsening weather such as snow / rain, etc. In this case, the setting management module (100) may determine at least one control parameter that requires control based on a failure factor of object recognition included in the cause information.

[0202] For example, the setting management module (100) can generate a control parameter to increase the frame rate or increase the shutter speed to remove the blur phenomenon when the failure factor is a blur phenomenon. Alternatively, the setting management module (100) can generate a control parameter to divide the image into a predetermined number of ROIs or increase the resolution in response to an object recognition identification failure due to object density. Alternatively, the setting management module (100) can generate at least one control parameter to adjust the tone mapping function / contrast ratio / OETF function so that more bit values ​​can be allocated to dark areas to resolve an object recognition failure factor due to a decrease in lighting conditions (brightness). The setting management module (100) can control the image capture module (210) and / or the image post-processing module (220) by transmitting a control message including the generated at least one control parameter to the image capture module (210) and / or the image post-processing module (220) so as to acquire an image in which the object recognition failure factor is resolved. In this case, the setting management module (100) can effectively improve the object recognition rate of the AI ​​sensing module (230) by changing the image characteristics to suit the road conditions through the above control parameters.

[0203] Alternatively, the setting management module (100) may collect various road environment information (road traffic conditions, weather, time information, etc.) transmitted from the infrastructure operation module (400), and control the image capture module (210) and / or the image post-processing module (220) so that an image capable of increasing an object recognition rate can be acquired based on the collected road environment information. Alternatively, the setting management module (100) may generate at least one control parameter corresponding to the specific road condition / road environment when information on a specific road condition / road environment is input from the infrastructure operation module (400), and control the image capture module (210) and / or the image post-processing module (220) based on the generated at least one control parameter.

[0204] (7) Image post-processing module

[0205] The image post-processing module (220) can additionally correct an image whose characteristics are adaptively controlled according to the road environment based on the control parameters transmitted from the setting management module (100). The image post-processing module (220) can provide the AI ​​sensing module (230) with image data that has been corrected to be optimized for object recognition. For example, when the at least one control parameter is received, the image post-processing module (220) can correct the image data transmitted from the image capture module (210) based on the at least one control parameter so that a specific area where objects on the road move is clearly revealed, or can perform tone mapping processing to adjust the contrast of an area where bounding boxes frequently appear in the image data.

[0206] Alternatively, the image post-processing module (220) may drop the current frame from the buffer and not transmit it to the AI ​​sensing module (230) if the image change between frames is below a certain level based on the differential image between the previous frame and the current frame. In this case, the computing resources and load associated with AI sensing can be significantly reduced.

[0207] Alternatively, in the proposed invention, the configuration management module (100) may transmit the generated control parameters to another module (such as a video post-processing module or an image capture module (210)) in a restful manner, or may transmit the control parameters through a message having a specific cycle.

[0208] Alternatively, the configuration management module (100) may generate control parameters / messages for controlling the image capture module (210) and / or the image post-processing module (220) based on the parameters defined as in Table 5. Here, the messages may be messages based on the message frames of the Society of Automotive Engineers (SAE) and / or the European Telecommunications Standards Institute (ETSI).

[0209] ElementTyperemark@messageIDintSAE / new message ID defined by ETSI... <sequence>@minimumWidthUnsigned intMinimum resolution width@minimumHeightUnsigned intMinimum resolution Height@toneMappingUnsigned intTone mapping function type defined@transferCharacteristicUnsigned intOETF function type@minimumFrameRateUnsigned intRecommended minimum frame rate@contrastfloat or intContrast adjustment ratio (0~100%)@brightnessfloat or intContrast adjustment ratio (0~100%)@divisionROIUnsigned intNumber of divisions <sequence>@idUnsigned int From the top left to the right, id mappingdivisionROI count according to <sequence>decision@startWidthUnsigned int@startHeightUnsigned int@endWidthUnsigned int@endHeightUnsigned int@toneMappingUnsigned int@contrastfloat or int@brightnessfloat or int

[0210] @minimumWidth, @minimumHeight are parameters defined to adjust the resolution for the image capture module (or image acquisition device such as camera, CCTV, etc.) and / or the image post-processing module. The values ​​of @minimumWidth and @minimumHeight can be determined based on the size of the object in the image, etc. @minimumFrameRate is a parameter for adjusting the capturing frame rate of the image, and the value can be determined based on the density of the object or the moving speed of the object. @contrast is a parameter for adjusting the contrast of the captured image, and can be expressed in % units. For example, if a contrast reduction of about 0.05 is required, the value of @contrast can be set to -0.05 or 5%.

[0211] @divisionROI may be a parameter for controlling the number of divisions of the image. The value of @divisionROI may be determined based on the density of objects in the image, the location / number of unclassified objects, etc. For example, if the value of @divisionROI is 4, the image acquisition device / image capture module may divide the captured / acquired image into 4 ROIs. Here, the image division may not necessarily be division at the same ratio, and the image characteristics may be additionally considered to divide it into ROIs at an appropriate ratio.

[0212] Meanwhile, the object size on the image acquired by the RSU height may become relatively small, and the object recognition rate based on the image may deteriorate. Considering this, @divisionROI can also be used as a parameter for additional expansion of the segmented ROIs. Each segmented ROI can be defined by an ID that extends to the right from the upper left corner.

[0213] @toneMapping can be a parameter for controlling RGB or YUV (Luminance Chrominance, Chrominance) values ​​in a captured image. Based on @toneMapping, the number of bits allocated / mapped for low-level areas or high-level areas can be controlled. For example, more bits can be allocated to a specific area with low levels of the image by applying a gamma function. In this case, dark parts in the image can be expressed more clearly. For example, as defined in Table 6, the value of @toneMapping and the corresponding mapping function can be predefined, and the setting management module (100) can determine the indication value of the tone mapping function to be indicated through the control parameter based on Table 6.

[0214] @transferCharacteristic can be a parameter for a transfer function that converts a light / optical signal into an electrical signal as OETF information. @transferCharacteristic can be defined as in Table 7. For example, when a camera / image acquisition device captures an image for brightness that supports HDR (High Dynamic Range), the camera / image acquisition device can input the expression range into a specific function indicated by @transferCharacteristic to obtain an appropriate RGB output.

[0215] @toneMappingValueLinear (Regular) function0x01Linear functionLow tone based function0x02Tone mapping function that allocates more whole bits to dark areasMiddle tone based function0x03Tone mapping function that allocates more whole bits to medium brightness areasHigh tone based function0x04Tone mapping function that allocates more whole bits to high brightness areasCustomized0x05Administrator's arbitrary customized functionReversed

[0216] @transferCharacteristicValueGamma function0x01Linear functionPQ function0x02Tone mapping function that applies by allocating more whole bits to dark areasCustomized0x03Administrator's arbitrary customized functionReversed

[0217] FIGS. 17 to 19 are diagrams illustrating how a settings management module controls an image capture module or an image post-processing module.

[0218] Referring to Figure 17 (a), during the early morning hours, on dark roads, vehicle traffic may be light and driving speeds may increase. In this case, object recognition in images may be difficult due to the low lighting and / or high driving speeds. Furthermore, due to the low traffic volume, the AI ​​sensing module may perform object recognition / detection even on images with low differential values ​​compared to the previous frame (i.e., images that are nearly identical to the previous frame), which may result in inefficient object recognition / detection.

[0219] In this case, the setting management module can generate control parameters for controlling the contrast and tone mapping functions when the object recognition rate included in the output result / output data of the AI ​​sensing module is below a specific threshold and the cause information related to dark lighting is included, and can provide the generated control parameters to the image capture module. For example, Fig. 17 (a) illustrates an image acquired by the image capture module in an environment where object recognition is difficult due to insufficient lighting, and Fig. 17 (b) illustrates an image acquired from the image capture module controlled by the above-described control parameters. Object recognition may be difficult for the image in Fig. 17 (a) using a general image capture method and image processing method. In contrast, referring to Fig. 17 (b), the image capture module can acquire an image having a characteristic in which dark parts are more clearly highlighted by controlling the contrast and tone mapping functions based on the lighting environment. In this case, relatively accurate object recognition can be guaranteed even with standard image capture and processing methods within the AI ​​sensing module. By directly controlling the characteristics of the captured images from the image capture module based on the road environment, the AI ​​sensing module can achieve a higher object recognition rate than conventional algorithms that only correct previously acquired images.

[0220] Alternatively, referring to FIG. 18, the setting management module can significantly increase the object recognition rate, which is lowered due to a dark environment, by generating a control parameter based on a tone mapping function. For example, as illustrated in FIG. 18, various tone mapping functions (f(x)) can be defined in the relationship between input(x) and output(y). For example, in the case where the tone mapping function is a log-based function, more image bits are allocated to areas corresponding to low tones than to areas corresponding to high tones, thereby increasing the (pixel) expression range for areas with low tones. Alternatively, the setting management module can additionally generate a control parameter for adjusting / controlling an appropriate contrast ratio for a dark environment to control the image capture module. In this case, an image, which can facilitate object recognition, can be acquired, as illustrated in FIG. 17 (b).

[0221] In addition, during the early morning hours when there is little traffic, the driving speed of an object (e.g., a car) may be relatively faster than the driving speed of an object during the day. In this case, a blurring phenomenon may occur in the image acquired by the image capture module. Therefore, the configuration management module can generate control parameters for controlling the frame rate (fps) and shutter speed to remove the blurring shape in the image acquired by the image capture module. For example, an image capture module that supports 30 fps can acquire an image of an object running at a speed of more than 100 km / h (i.e., more than 27.78 m / s) at an intersection. In this case, when the frame rate of the image capture module is 30 fps, one frame can be captured / acquired every 1 / 30 second (in this case, the object's moving distance between frames is 0.926 m), and in this case, a blurring phenomenon may occur. Therefore, the setting management module can minimize blurring in the image by providing the image capture module with a control parameter that adjusts the frame rate (fps) to 60 fps, thereby reducing the acquisition interval of 1 frame to 1 / 60 second (or the object movement distance between frames is 0.463 m).

[0222] Alternatively, the AI ​​sensing module can recognize / detect objects in real time for all input acquired / captured images. However, in the early morning hours, there is little vehicle traffic and the amount of variation (i.e., difference in pixel values) between frames is minimal, making it inefficient to perform object recognition / detection in real time for all images. Therefore, the image post-processing module or the image capture module can transmit only at least one frame among multiple frames to the AI ​​sensing module based on the amount of variation between frames. For example, the image post-processing module or the image capture module can drop the current frame and not transmit it to the AI ​​sensing module if the difference value between the current frame and the immediately preceding frame is the same or less than a specific threshold difference.

[0223] Alternatively, referring to FIG. 19, a small-sized or unclassified obstacle on the road may be detected / recognized. For example, if a sinkhole with a color similar to the road or an obstacle with a color similar to the road is detected / recognized on the road (or, if detection / recognition information about the sinkhole or obstacle is received from the AI ​​sensing module), the setting management module may generate a control parameter for changing the capture / image resolution to a specific level or higher and a control parameter for magnifying a specific ROI on the image. At this time, the specific ROI may be indicated / specified through the parameters of @startWidth, @startHeight, @endWidth, and @endHeight. The setting management module may transmit the generated control parameters to the image capture module and / or the image post-processing module. In this case, the image capture module and / or the image post-processing module may transmit / input an image obtained by capturing the specific ROI from an image with a resolution of a specific level or higher to the AI ​​sensing module based on the control parameters. In this case, the AI ​​sensing module can improve the object recognition rate by performing additional learning on obstacles that are not included in object classification based on an image with a specific ROI enlarged.

[0224] Additionally, RSUs or CCTVs located at high elevations may experience reduced object recognition rates due to their height. In this case, the configuration management module can apply a process to sense specific regions of interest (ROIs) when object recognition rates decline. For example, an embodiment can be provided in which only specific regions of interest, including roads and VRUs, are considered analysis areas, excluding areas that do not require recognition, and input and recognize only those specific areas to the AI ​​sensing module simultaneously.

[0225] FIG. 20 is a diagram illustrating how a device transmits a control message to control an image acquisition device.

[0226] Referring to FIG. 20, the device can obtain object recognition information from an artificial neural network-based object recognition model that recognizes an object from an image of an image acquisition device (S201).

[0227] For example, the image acquisition device may acquire an image for a predetermined geographical area and transmit it to the object recognition model. The object recognition model may extract features from the image, recognize objects existing in the geographical area based on the extracted features, and perform a classification operation on the recognized objects. The device may receive object recognition information about objects recognized in the image from the object recognition model. Meanwhile, the image acquisition device may be a CCTV or a camera, including an image capture module and / or an image post-processing module described with reference to FIGS. 16 to 19. The object recognition model may be an AI model included in the AI ​​sensing module described with reference to FIGS. 16 to 19. The device may include the setting management module described with reference to FIGS. 16 to 19. Alternatively, the device may include at least one of a sensing data analysis module and a message generation module in addition to the setting management module. For example, the device may be an RSU including the setting management module, the sensing data analysis module, and the message generation module.

[0228] Object recognition information may include information (location, size, shape, mobility information, etc.) of objects recognized / detected in the image and / or accuracy / reliability (hereinafter, object recognition rate) for the image. Here, the object recognition rate may be calculated by the object recognition model based on the number of objects recognized / detected in the image and the number of classified objects, as described above. Alternatively, the device may directly calculate the object recognition rate based on the number of recognized objects and the number of classified objects included in the object recognition information. For example, the object recognition rate may be calculated based on a value obtained by dividing the number of classified objects by the number of recognized objects. For example, if the number of objects recognized by the object recognition model in the input image is 20 and the object recognition model completes classification for only 15 objects among the objects, the object recognition rate may be calculated as 15 / 20*100=75%. Here, a classified object may be an object for which the type / type of the object has a predicted probability higher than a predefined predicted probability, and an unclassified object may be an object for which the type / type of the object has a predicted probability lower than a predefined predicted probability. In this case, the object recognition rate may be the number of objects with a predicted probability higher than a predefined predicted probability / the total number of recognized objects.

[0229] Alternatively, the object recognition information may further include factor information regarding the cause of failure in classification of some of the recognized objects. For example, the object recognition model may predict the existence of a specific object, but may not be able to classify the type / classification of the specific object due to brightness, object density, color of the object similar to the color of the surrounding environment, size of the object, or movement speed of the object. In this case, the object recognition model may generate cause information regarding the cause of failure in classification of the specific object, and further include the cause information in the object recognition information.

[0230] Next, the device can transmit an object message to a peripheral device based on object information recognized from the image included in the object recognition information (S203). As described with reference to FIGS. 16 to 19, the device can select objects that need to be transmitted to peripheral terminals / peripheral devices through messages for V2X services, etc., based on the object recognition information of the object recognition model, and transmit an object message including object information about the selected objects to the peripheral terminals / peripheral devices. Alternatively, the device can transmit the object information about the selected objects to a network or an RSU and request the network or an RSU to transmit an object message including the object information.

[0231] Next, the device can determine whether to generate / transmit a control message for controlling the image acquisition device based on the object recognition rate included in the object recognition information (S205). As described with reference to FIGS. 16 to 19, the device can generate at least one control parameter for controlling the image acquisition device when the object recognition rate is less than (or equal to or lower than) a preset threshold recognition rate, and transmit / transmit a control message including the generated at least one control parameter to the image acquisition device.

[0232] Specifically, when the object recognition rate is lower than the preset threshold recognition rate, the device may generate the control parameters based on the cause information included in the object recognition information as described above. For example, the device may generate at least one control parameter for adjusting a tone mapping function, a gamma function, a frame rate, a resolution, at least two ROI divisions, a contrast ratio, a shutter speed, an OETF function, etc. so as to resolve a cause of object recognition failure due to the cause information. For example, the device may generate at least one control parameter based on the values ​​defined in Tables 5 to 7 to control a parameter for image acquisition of the image acquisition device. Through this, the device may adjust / change in advance the characteristics of an image to be acquired by the image acquisition device so as to increase the object recognition rate of the object recognition model based on the cause information. Meanwhile, when the object recognition rate is higher than the preset threshold recognition rate, the device may not generate a control parameter for controlling the image acquisition device.

[0233] Alternatively, based on the fact that the object recognition information further includes unclassified object information about an object having a mobility below a specific threshold, the device may transmit the control message even if the object recognition rate is above the preset threshold. For example, the object recognition model may recognize an object on the road having a mobility below a specific threshold for a certain period of time (or an object that moves little for a certain period of time). The object recognition model may not be able to classify the type and kind of the recognized object due to the size, color, shape, etc. of the recognized object (e.g., a road obstacle such as a sinkhole having a color similar to that of the road). In this case, the object recognition model may further include information about an unclassified object having a mobility below a specific threshold in the object recognition information. The device may transmit the control message to the image acquisition device even if the object recognition rate of the object recognition model is above the preset threshold when the object recognition information further includes information about the unclassified object. Here, the control message may transmit a control parameter including information on the number of ROIs to be segmented for the image and the ROI ID where the unclassified object is located, as illustrated in FIG. 19, to the image acquisition device (or the image post-processing module of the image acquisition device). In this case, the image acquisition device may segment the acquired image into a plurality of ROIs based on the control parameter, and transmit a (magnified) segmented image corresponding to the ROI having the ROI ID to the object recognition model. In this case, the object recognition model may intensively perform object recognition only for a specific ROI, and thus may more effectively predict object characteristics for the unclassified object than when object recognition is performed for the entire image.

[0234] Alternatively, the device may transmit a control message to the image acquisition device for controlling the image acquisition device to drop at least one image among the images transmitted to the object recognition model. For example, the device may transmit the control message when the object recognition information includes object information for objects less than a preset threshold number. In this case, the image acquisition device may drop the acquired image without transmitting it to the object recognition model when the image acquired based on the control message has a difference in pixel value less than a preset threshold difference compared to a previous image. Thereafter, the device may transmit a control message to the image acquisition device for releasing the dropping of the at least one image when the object recognition information includes object information for objects greater than or equal to a preset threshold number. Meanwhile, the preset threshold difference may be determined based on the number of objects included in the object recognition information and transmitted through the control message.

[0235] In this way, the proposed invention can adjust the characteristics of the acquired image itself through direct control of the image acquisition device based on factors that reduce the object recognition rate of the artificial neural network model. Alternatively, the proposed invention can effectively improve the object recognition rate in the image by adjusting the characteristics of the acquired image itself based on the road environment or road conditions. Alternatively, the proposed invention can effectively reduce the processing load and power consumption of object recognition by having the image acquisition device drop images in which the pixel value difference between the images is below a certain threshold without passing them to the object recognition model when it is determined that object recognition for all images in real time is unnecessary.

[0236] Figure 21 is a drawing illustrating how an image acquisition device receives a control message from a device.

[0237] Referring to FIG. 21, an image acquisition device can acquire an image of a specific geographic area (S211). The image acquisition device may be a CCTV or camera fixedly installed to capture the specific geographic area. The image acquisition device may include an image capture module and / or an image post-processing module as described in FIGS. 16 to 20.

[0238] Next, the image acquisition device can transmit / transmit the image to an artificial neural network-based object recognition model trained to recognize objects from the image (S213). The image acquisition device can transmit the acquired image or video to the object recognition model according to a preset cycle.

[0239] Next, the image acquisition device can receive a control message from the first device, which includes at least one control parameter that instructs a change in a parameter related to acquisition of the image (S215). Here, the control message can be received from the first device with transmission triggered based on the object recognition rate included in the object recognition information of the object recognition model being less than a preset threshold as described with reference to FIGS. 16 to 20. In this case, the image acquisition device can adjust parameters related to image acquisition, such as contrast ratio, tone mapping function, gamma function, image segmentation, frame rate, shutter speed, and OETF function, according to the at least one parameter included in the control message, and the characteristics of the image can be changed according to the adjustment of the parameters. For example, the image acquisition device can change / adjust parameters related to image acquisition according to the control message included in the control message based on Tables 5 to 7 described above.

[0240] Alternatively, the image acquisition device may determine whether to drop the acquired image under the control of the first device. For example, if the first device transmits a control message related to dropping some images, the image acquisition device may drop the currently acquired image without transmitting it to the object recognition model if the difference between the pixel values ​​of the currently acquired image and the pixel values ​​of the image acquired immediately before is less than a preset threshold difference (or the threshold difference transmitted in the control message).

[0241] Examples of communication systems to which the invention applies

[0242] Although not limited thereto, the various descriptions, functions, procedures, proposals, methods and / or operational flowcharts of the present invention disclosed in this document may be applied to various fields requiring wireless communication / connection (e.g., 5G) between devices.

[0243] Hereinafter, more specific examples will be provided with reference to the drawings. In the drawings / descriptions below, the same drawing reference numerals may represent identical or corresponding hardware blocks, software blocks, or functional blocks, unless otherwise described.

[0244] Figure 22 illustrates a communication system applied to the present invention.

[0245] Referring to FIG. 22, a communication system (1) applied to the present invention includes a wireless device, a base station, and a network. Here, the wireless device refers to a device that performs communication using a wireless access technology (e.g., 5G NR (New RAT), LTE (Long Term Evolution)) and may be referred to as a communication / wireless / 5G device. Although not limited thereto, the wireless device may include a robot (100a), a vehicle (100b-1, 100b-2), an XR (eXtended Reality) device (100c), a hand-held device (100d), a home appliance (100e), an IoT (Internet of Things) device (100f), and an AI device / server (400). For example, the vehicle may include a vehicle equipped with a wireless communication function, an autonomous vehicle, a vehicle capable of performing vehicle-to-vehicle communication, etc. Here, the vehicle may include an Unmanned Aerial Vehicle (UAV) (e.g., a drone). XR devices include AR (Augmented Reality) / VR (Virtual Reality) / MR (Mixed Reality) devices, and can be implemented in the form of HMD (Head-Mounted Device), HUD (Head-Up Display) installed in a vehicle, television, smartphone, computer, wearable device, home appliance, digital signage, vehicle, robot, etc. Mobile devices can include smartphone, smart pad, wearable device (e.g., smart watch, smart glass), computer (e.g., laptop, etc.), etc. Home appliances can include TV, refrigerator, washing machine, etc. IoT devices can include sensors, smart meters, etc. For example, base stations and networks can also be implemented as wireless devices, and a specific wireless device (200a) can act as a base station / network node to other wireless devices.

[0246] Wireless devices (100a to 100f) can be connected to a network (300) via a base station (200). Artificial Intelligence (AI) technology can be applied to the wireless devices (100a to 100f), and the wireless devices (100a to 100f) can be connected to an AI server (400) via the network (300). The network (300) can be configured using a 3G network, a 4G (e.g., LTE) network, a 5G (e.g., NR) network, etc. The wireless devices (100a to 100f) can communicate with each other via the base station (200) / network (300), but can also communicate directly (e.g., sidelink communication) without going through the base station / network. For example, vehicles (100b-1, 100b-2) can communicate directly (e.g., V2V (Vehicle to Vehicle) / V2X (Vehicle to Everything) communication). In addition, IoT devices (e.g., sensors) can communicate directly with other IoT devices (e.g., sensors) or other wireless devices (100a to 100f).

[0247] Wireless communication / connection (150a, 150b, 150c) can be established between wireless devices (100a~100f) / base stations (200), and base stations (200) / base stations (200). Here, wireless communication / connection can be achieved through various wireless access technologies (e.g., 5G NR) such as uplink / downlink communication (150a), sidelink communication (150b) (or D2D communication), and communication between base stations (150c) (e.g., relay, IAB (Integrated Access Backhaul). Through wireless communication / connection (150a, 150b, 150c), wireless devices and base stations / wireless devices, and base stations and base stations can transmit / receive wireless signals to each other. For example, wireless communication / connection (150a, 150b, 150c) can transmit / receive signals through various physical channels. To this end, at least some of various configuration information setting processes for transmitting / receiving wireless signals, various signal processing processes (e.g., channel encoding / decoding, modulation / demodulation, resource mapping / demapping, etc.), and resource allocation processes can be performed based on various proposals of the present invention.

[0248] Examples of wireless devices to which the present invention is applied

[0249] Figure 23 illustrates a wireless device applicable to the present invention.

[0250] Referring to FIG. 23, the first wireless device (100) and the second wireless device (200) can transmit and receive wireless signals through various wireless access technologies (e.g., LTE, NR). Here, {the first wireless device (100), the second wireless device (200)} can correspond to {the wireless device (100x), the base station (200)} and / or {the wireless device (100x), the wireless device (100x)} of FIG. 22.

[0251] A first wireless device (100) includes one or more processors (102) and one or more memories (104), and may further include one or more transceivers (106) and / or one or more antennas (108). The processor (102) controls the memories (104) and / or the transceivers (106), and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document. For example, the processor (102) may process information in the memory (104) to generate first information / signal, and then transmit a wireless signal including the first information / signal via the transceiver (106). In addition, the processor (102) may receive a wireless signal including second information / signal via the transceiver (106), and then store information obtained from signal processing of the second information / signal in the memory (104). The memory (104) may be connected to the processor (102) and may store various information related to the operation of the processor (102). For example, the memory (104) may perform some or all of the processes controlled by the processor (102), or may store software code including commands for performing the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. Here, the processor (102) and the memory (104) may be part of a communication modem / circuit / chipset designed to implement wireless communication technology (e.g., LTE, NR). The transceiver (106) may be connected to the processor (102) and may transmit and / or receive wireless signals via one or more antennas (108). The transceiver (106) may include a transmitter and / or a receiver. The transceiver (106) may be used interchangeably with an RF (Radio Frequency) unit. In the present invention, a wireless device may also mean a communication modem / circuit / chipset.

[0252] Specifically, the first wireless device or apparatus (100) may include a processor (102) and a memory (104) connected to a transceiver (106). The memory (104) may include at least one program capable of performing operations related to the embodiments described in FIGS. 16 to 21.

[0253] The processor (102) controls the transceiver (106) to obtain object recognition information from an artificial neural network-based object recognition model that recognizes an object from an image of an image acquisition device, transmits an object message to a peripheral device based on object information recognized from the image included in the object recognition information, and determines whether to transmit the control message for controlling the image acquisition device based on an object recognition rate for the image included in the object recognition information.

[0254] Alternatively, a processing device may be configured including a processor (102) and a memory (104) for controlling a device transmitting a control message. In this case, at least one processor; and at least one memory connected to the at least one processor and storing instructions, wherein the instructions, when executed by the at least one processor, cause the device to: acquire object recognition information from an artificial neural network-based object recognition model that recognizes an object from an image of an image acquisition device, transmit an object message to a peripheral device based on object information recognized from the image included in the object recognition information, and determine whether to transmit the control message for controlling the image acquisition device based on an object recognition rate for the image included in the object recognition information.

[0255] The second wireless device (200) includes one or more processors (202), one or more memories (204), and may further include one or more transceivers (206) and / or one or more antennas (208). The processor (202) controls the memories (204) and / or the transceivers (206), and may be configured to implement the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document. For example, the processor (202) may process information in the memory (204) to generate third information / signals, and then transmit a wireless signal including the third information / signals via the transceivers (206). Furthermore, the processor (202) may receive a wireless signal including fourth information / signals via the transceivers (206), and then store information obtained from signal processing of the fourth information / signals in the memory (204). The memory (204) may be connected to the processor (202) and may store various information related to the operation of the processor (202). For example, the memory (204) may perform some or all of the processes controlled by the processor (202), or may store software code including commands for performing the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. Here, the processor (202) and the memory (204) may be part of a communication modem / circuit / chip designed to implement wireless communication technology (e.g., LTE, NR). The transceiver (206) may be connected to the processor (202) and may transmit and / or receive wireless signals via one or more antennas (208). The transceiver (206) may include a transmitter and / or a receiver. The transceiver (206) may be used interchangeably with an RF unit. In the present invention, a wireless device may also mean a communication modem / circuit / chip.

[0256] Specifically, the second wireless device or image acquisition device (200) may include a transceiver or RF transceiver (206), an image sensor (not shown), a processor (202), and a memory (204). The memory (204) may include at least one program capable of performing operations related to the embodiments described in FIGS. 16 to 21.

[0257] The processor (202) controls the image sensor to acquire an image for a specific geographical area, controls the transceiver (206) to transmit the image to an artificial neural network-based object recognition model trained to recognize an object from the image, and can receive a control message from the first device including at least one control parameter that instructs a change in a parameter related to acquisition of the image. Here, the control message can be received from the first device based on an object recognition rate included in object recognition information of the object recognition model being less than a preset threshold.

[0258] Hereinafter, the hardware elements of the wireless device (100, 200) will be described in more detail. Although not limited thereto, one or more protocol layers may be implemented by one or more processors (102, 202). For example, one or more processors (102, 202) may implement one or more layers (e.g., functional layers such as PHY, MAC, RLC, PDCP, RRC, SDAP). One or more processors (102, 202) may generate one or more Protocol Data Units (PDUs) and / or one or more Service Data Units (SDUs) according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. One or more processors (102, 202) may generate messages, control information, data, or information according to the descriptions, functions, procedures, proposals, methods, and / or operation flowcharts disclosed in this document. One or more processors (102, 202) can generate signals (e.g., baseband signals) including PDUs, SDUs, messages, control information, data or information according to the functions, procedures, proposals and / or methods disclosed herein, and provide the signals to one or more transceivers (106, 206). One or more processors (102, 202) can receive signals (e.g., baseband signals) from one or more transceivers (106, 206) and obtain PDUs, SDUs, messages, control information, data or information according to the descriptions, functions, procedures, proposals, methods and / or operational flowcharts disclosed herein.

[0259] One or more processors (102, 202) may be referred to as a controller, a microcontroller, a microprocessor, or a microcomputer. One or more processors (102, 202) may be implemented by hardware, firmware, software, or a combination thereof. For example, one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), one or more Digital Signal Processing Devices (DSPDs), one or more Programmable Logic Devices (PLDs), or one or more Field Programmable Gate Arrays (FPGAs) may be included in one or more processors (102, 202). The descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed in this document may be implemented using firmware or software, and the firmware or software may be implemented to include modules, procedures, functions, etc. The descriptions, functions, procedures, suggestions, methods and / or operation flowcharts disclosed in this document may be implemented using firmware or software configured to perform one or more processors (102, 202) or stored in one or more memories (104, 204) and executed by one or more processors (102, 202). The descriptions, functions, procedures, suggestions, methods and / or operation flowcharts disclosed in this document may be implemented using firmware or software in the form of codes, instructions and / or sets of instructions.

[0260] One or more memories (104, 204) may be coupled to one or more processors (102, 202) and may store various forms of data, signals, messages, information, programs, codes, instructions, and / or commands. The one or more memories (104, 204) may be configured as ROM, RAM, EPROM, flash memory, hard drives, registers, cache memory, computer-readable storage media, and / or combinations thereof. The one or more memories (104, 204) may be located internally and / or externally to the one or more processors (102, 202). Additionally, the one or more memories (104, 204) may be coupled to the one or more processors (102, 202) via various technologies, such as wired or wireless connections.

[0261] One or more transceivers (106, 206) can transmit user data, control information, wireless signals / channels, etc., as mentioned in the methods and / or flowcharts of this document, to one or more other devices. One or more transceivers (106, 206) can receive user data, control information, wireless signals / channels, etc., as mentioned in the descriptions, functions, procedures, proposals, methods and / or flowcharts of this document, from one or more other devices. For example, one or more transceivers (106, 206) can be connected to one or more processors (102, 202) and can transmit and receive wireless signals. For example, one or more processors (102, 202) can control one or more transceivers (106, 206) to transmit user data, control information, or wireless signals to one or more other devices. Additionally, one or more processors (102, 202) may control one or more transceivers (106, 206) to receive user data, control information, or wireless signals from one or more other devices. Additionally, one or more transceivers (106, 206) may be coupled to one or more antennas (108, 208), and one or more transceivers (106, 206) may be configured to transmit and receive user data, control information, wireless signals / channels, or the like, as referred to in the descriptions, functions, procedures, proposals, methods, and / or operational flowcharts disclosed herein, via one or more antennas (108, 208). In this document, one or more antennas may be multiple physical antennas or multiple logical antennas (e.g., antenna ports). One or more transceivers (106, 206) can convert received user data, control information, wireless signals / channels, etc. from RF band signals to baseband signals in order to process the received user data, control information, wireless signals / channels, etc. using one or more processors (102, 202).One or more transceivers (106, 206) may convert user data, control information, wireless signals / channels, etc. processed by one or more processors (102, 202) from baseband signals to RF band signals. For this purpose, one or more transceivers (106, 206) may include an (analog) oscillator and / or filter.

[0262] Examples of wireless devices to which the present invention is applied

[0263] Figure 24 illustrates another example of a wireless device applicable to the present invention. The wireless device may be implemented in various forms depending on the use case / service (see Figure 22).

[0264] Referring to FIG. 24, the wireless device (100, 200) corresponds to the wireless device (100, 200) of FIG. 23 and may be composed of various elements, components, units / units, and / or modules. For example, the wireless device (100, 200) may include a communication unit (110), a control unit (120), a memory unit (130), and additional elements (140). The communication unit may include a communication circuit (112) and a transceiver(s) (114). For example, the communication circuit (112) may include one or more processors (102, 202) and / or one or more memories (104, 204) of FIG. 24. For example, the transceiver(s) (114) may include one or more transceivers (106, 206) and / or one or more antennas (108, 208) of FIG. 23. The control unit (120) is electrically connected to the communication unit (110), the memory unit (130), and the additional elements (140) and controls the overall operation of the wireless device. For example, the control unit (120) may control the electrical / mechanical operation of the wireless device based on the program / code / command / information stored in the memory unit (130). In addition, the control unit (120) may transmit information stored in the memory unit (130) to an external device (e.g., another communication device) via a wireless / wired interface through the communication unit (110), or store information received from an external device (e.g., another communication device) via a wireless / wired interface in the memory unit (130).

[0265] The additional element (140) may be configured in various ways depending on the type of the wireless device. For example, the additional element (140) may include at least one of a power unit / battery, an input / output unit (I / O unit), a driving unit, and a computing unit. Although not limited thereto, the wireless device may be implemented in the form of a robot (Fig. 22, 100a), a vehicle (Fig. 22, 100b-1, 100b-2), an XR device (Fig. 22, 100c), a portable device (Fig. 22, 100d), a home appliance (Fig. 22, 100e), an IoT device (Fig. 22, 100f), a digital broadcasting terminal, a hologram device, a public safety device, an MTC device, a medical device, a fintech device (or a financial device), a security device, a climate / environmental device, an AI server / device (Fig. 22, 400), a base station (Fig. 22, 200), a network node, etc. Wireless devices may be mobile or stationary depending on the use / service.

[0266] In FIG. 24, various elements, components, units / parts, and / or modules within the wireless device (100, 200) may be entirely interconnected via a wired interface, or at least some may be wirelessly connected via a communication unit (110). For example, within the wireless device (100, 200), the control unit (120) and the communication unit (110) may be wired, and the control unit (120) and a first unit (e.g., 130, 140) may be wirelessly connected via the communication unit (110). In addition, each element, component, unit / part, and / or module within the wireless device (100, 200) may further include one or more elements. For example, the control unit (120) may be composed of a set of one or more processors. For example, the control unit (120) may be composed of a set of a communication control processor, an application processor, an electronic control unit (ECU), a graphics processing processor, a memory control processor, etc. As another example, the memory unit (130) may be composed of RAM (Random Access Memory), DRAM (Dynamic RAM), ROM (Read Only Memory), flash memory, volatile memory, non-volatile memory, and / or a combination thereof.

[0267] Examples of vehicles or autonomous vehicles to which the present invention is applied

[0268] Figure 25 illustrates a vehicle or autonomous vehicle applicable to the present invention. The vehicle or autonomous vehicle may be implemented as a mobile robot, a car, a train, a manned or unmanned aerial vehicle (AV), a ship, or the like.

[0269] Referring to FIG. 25, a vehicle or autonomous vehicle (100) may include an antenna unit (108), a communication unit (110), a control unit (120), a driving unit (140a), a power supply unit (140b), a sensor unit (140c), and an autonomous driving unit (140d). The antenna unit (108) may be configured as a part of the communication unit (110). Blocks 110 / 130 / 140a to 140d correspond to blocks 110 / 130 / 140 of FIG. 24, respectively.

[0270] The communication unit (110) can transmit and receive signals (e.g., data, control signals, etc.) with external devices such as other vehicles, base stations (e.g., base stations, road side units, etc.), and servers. The control unit (120) can control elements of the vehicle or autonomous vehicle (100) to perform various operations. The control unit (120) can include an ECU (Electronic Control Unit). The drive unit (140a) can drive the vehicle or autonomous vehicle (100) on the ground. The drive unit (140a) can include an engine, a motor, a power train, wheels, brakes, a steering device, etc. The power supply unit (140b) supplies power to the vehicle or autonomous vehicle (100) and can include a wired / wireless charging circuit, a battery, etc. The sensor unit (140c) can obtain vehicle status, surrounding environment information, user information, etc. The sensor unit (140c) may include an IMU (inertial measurement unit) sensor, a collision sensor, a wheel sensor, a speed sensor, an incline sensor, a weight detection sensor, a heading sensor, a position module, a vehicle forward / backward sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor, a temperature sensor, a humidity sensor, an ultrasonic sensor, an illuminance sensor, a pedal position sensor, etc. The autonomous driving unit (140d) may implement a technology for maintaining a driving lane, a technology for automatically controlling speed such as adaptive cruise control, a technology for automatically driving along a set path, a technology for automatically setting a path and driving when a destination is set, etc.

[0271] For example, the communication unit (110) can receive map data, traffic information data, etc. from an external server. The autonomous driving unit (140d) can generate an autonomous driving route and driving plan based on the acquired data. The control unit (120) can control the drive unit (140a) so that the vehicle or autonomous vehicle (100) moves along the autonomous driving route according to the driving plan (e.g., speed / direction control). During autonomous driving, the communication unit (110) can irregularly / periodically acquire the latest traffic information data from an external server and can acquire surrounding traffic information data from surrounding vehicles. In addition, during autonomous driving, the sensor unit (140c) can acquire vehicle status and surrounding environment information. The autonomous driving unit (140d) can update the autonomous driving route and driving plan based on newly acquired data / information. The communication unit (110) can transmit information regarding the vehicle location, autonomous driving route, driving plan, etc. to the external server. External servers can predict traffic information data in advance using AI technology or other technologies based on information collected from vehicles or autonomous vehicles, and provide the predicted traffic information data to the vehicles or autonomous vehicles.

[0272] Figure 26 illustrates an AI device applied to the present invention.

[0273] AI devices can be implemented as fixed or mobile devices, such as TVs, projectors, smartphones, PCs, laptops, digital broadcasting terminals, tablet PCs, wearable devices, set-top boxes (STBs), radios, washing machines, refrigerators, digital signage, robots, and vehicles.

[0274] Referring to FIG. 26, the AI ​​device (100) may include a communication unit (110), a control unit (120), a memory unit (130), an input / output unit (140a / 140b), a learning processor unit (140c), and a sensor unit (140d). Blocks 110 to 130 / 140a to 140d correspond to blocks 110 to 130 / 140 of FIG. 23, respectively.

[0275] The communication unit (110) can transmit and receive wired and wireless signals (e.g., sensor information, user input, learning models, control signals, etc.) with external devices such as other AI devices (e.g., FIG. 22, 100x, 200, 400) or AI servers (200) using wired and wireless communication technology. To this end, the communication unit (110) can transmit information within the memory unit (130) to the external device or transfer a signal received from the external device to the memory unit (130).

[0276] The control unit (120) may determine at least one executable operation of the AI ​​device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm. In addition, the control unit (120) may control components of the AI ​​device (100) to perform the determined operation. For example, the control unit (120) may request, search, receive, or utilize data from the learning processor unit (140c) or the memory unit (130), and may control components of the AI ​​device (100) to perform a predicted operation or an operation determined to be desirable among at least one executable operation. In addition, the control unit (120) may collect history information including the operation contents of the AI ​​device (100) or user feedback on the operation, and store the collected history information in the memory unit (130) or the learning processor unit (140c), or transmit the collected history information to an external device such as an AI server (FIG. 18, 400). The collected history information may be used to update a learning model.

[0277] The memory unit (130) can store data that supports various functions of the AI ​​device (100). For example, the memory unit (130) can store data obtained from the input unit (140a), data obtained from the communication unit (110), output data of the learning processor unit (140c), and data obtained from the sensing unit (140). In addition, the memory unit (130) can store control information and / or software codes necessary for the operation / execution of the control unit (120).

[0278] The input unit (140a) can obtain various types of data from the outside of the AI ​​device (100). For example, the input unit (120) can obtain learning data for model learning, input data to which the learning model will be applied, etc. The input unit (140a) may include a camera, a microphone, and / or a user input unit. The output unit (140b) may generate output related to sight, hearing, or touch. The output unit (140b) may include a display unit, a speaker, and / or a haptic module, etc. The sensing unit (140) can obtain at least one of internal information of the AI ​​device (100), information about the surrounding environment of the AI ​​device (100), and user information using various sensors. The sensing unit (140) may include a proximity sensor, an illuminance sensor, an acceleration sensor, a magnetic sensor, a gyro sensor, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, and / or a radar, etc.

[0279] The learning processor unit (140c) can train a model composed of an artificial neural network using learning data. The learning processor unit (140c) can perform AI processing together with the learning processor unit of the AI ​​server (Fig. 18, 400). The learning processor unit (140c) can process information received from an external device via the communication unit (110) and / or information stored in the memory unit (130). In addition, the output value of the learning processor unit (140c) can be transmitted to an external device via the communication unit (110) and / or stored in the memory unit (130).

[0280] The wireless communication technology implemented in the wireless device (XXX, YYY) of this specification may include not only LTE, NR, and 6G, but also Narrowband Internet of Things for low-power communication. At this time, for example, NB-IoT technology may be an example of LPWAN (Low Power Wide Area Network) technology, and may be implemented with standards such as LTE Cat NB1 and / or LTE Cat NB2, and is not limited to the above-described names. Additionally or alternatively, the wireless communication technology implemented in the wireless device (XXX, YYY) of this specification may perform communication based on LTE-M technology. At this time, for example, LTE-M technology may be an example of LPWAN technology, and may be called by various names such as eMTC (enhanced Machine Type Communication). For example, LTE-M technology can be implemented by at least one of various standards such as 1) LTE CAT 0, 2) LTE Cat M1, 3) LTE Cat M2, 4) LTE non-BL (non-Bandwidth Limited), 5) LTE-MTC, 6) LTE Machine Type Communication, and / or 7) LTE M, and is not limited to the above-described names. Additionally or alternatively, the wireless communication technology implemented in the wireless device (XXX, YYY) of the present specification can include at least one of ZigBee, Bluetooth, and Low Power Wide Area Network (LPWAN) considering low-power communication, and is not limited to the above-described names. For example, ZigBee technology can create PAN (personal area networks) related to small / low-power digital communication based on various standards such as IEEE 802.15.4, and can be called by various names.

[0281] The embodiments described above are combinations of components and features of the present invention in a predetermined form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented without being combined with other components or features. Furthermore, it is also possible to form an embodiment of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment or may be replaced with corresponding components or features of another embodiment. It is self-evident that claims that do not have an explicit citation relationship in the patent claims may be combined to form an embodiment or may be incorporated as a new claim through a post-application amendment.

[0282] In this document, embodiments of the present invention have been described primarily focusing on the signal transmission and reception relationship between a terminal and a base station. This transmission and reception relationship is equally / similarly extended to signal transmission and reception between a terminal and a relay or a base station and a relay. Certain operations described as being performed by a base station in this document may, in some cases, be performed by its upper node. That is, it is obvious that various operations performed for communication with a terminal in a network composed of multiple network nodes including a base station may be performed by the base station or other network nodes other than the base station. The base station may be replaced by terms such as fixed station, Node B, eNode B (eNB), and access point. In addition, the terminal may be replaced by terms such as UE (User Equipment), MS (Mobile Station), MSS (Mobile Subscriber Station).

[0283] Embodiments of the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, an embodiment of the present invention may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0284] When implemented via firmware or software, an embodiment of the present invention may be implemented in the form of modules, procedures, functions, etc. that perform the functions or operations described above. The software code may be stored in a memory unit and executed by a processor. The memory unit may be located within or outside the processor and may exchange data with the processor via various known means.

[0285] It will be apparent to those skilled in the art that the present invention can be embodied in other specific forms without departing from the scope of the invention. Therefore, the above detailed description should not be construed as limiting in any respect, but rather as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all modifications within the scope of equivalents of the present invention are intended to be included within the scope of the present invention.

[0286] The embodiments of the present invention as described above can be applied to various mobile communication systems.< / sequence> < / sequence> < / sequence>

Claims

1. In a method for transmitting a control message by a device in a wireless communication system, A step of acquiring object recognition information from an artificial neural network-based object recognition model that recognizes and classifies an object from an image of an image acquisition device; A step of transmitting an object message to a peripheral device based on object information recognized from the image included in the object recognition information; and A method comprising a step of determining whether to transmit the control message for controlling the image acquisition device based on the object recognition rate for the image included in the object recognition information.

2. In paragraph 1, A method, characterized in that the control message is transmitted based on the object recognition rate being below a preset threshold.

3. In paragraph 1, The object recognition information further includes factor information about the cause of the object recognition model's failure to classify at least one object from the image, A method, characterized in that the at least one control parameter is determined based on the factor information.

4. In paragraph 3, A method characterized in that the above factor information includes at least one of information on object classification failure according to lighting conditions, object classification failure according to object density, object classification failure according to object size, and object classification failure according to object movement speed.

5. In paragraph 2, A method characterized in that the control message is transmitted even if the object recognition rate is higher than the preset threshold, based on the object recognition information further including unclassified object information for objects having a mobility below a certain threshold.

6. In paragraph 5, A method, characterized in that the control message includes a control parameter that specifies a portion of an area to which an object corresponding to the unclassified object information belongs on the image.

7. In paragraph 1, A method characterized in that the image acquisition device does not transmit at least one image among the images to the object recognition model based on a difference in pixel values ​​between the acquired images.

8. In paragraph 1, A method characterized in that the object recognition rate is calculated based on a value obtained by dividing the number of objects classified by object type among objects recognized by the object recognition module from the image by the number of objects.

9. A computer-readable recording medium recording a program for performing the method described in paragraph 1.

10. In a device for transmitting a control message in a wireless communication system, RF (Radio Frequency) transmitter and receiver; and A processor connected to the RF transceiver, A device in which the processor controls the RF transceiver to obtain object recognition information from an artificial neural network-based object recognition model that recognizes and classifies objects from images of an image acquisition device, transmits an object message to a peripheral device based on object information recognized from the image included in the object recognition information, and determines whether to transmit the control message for controlling the image acquisition device based on an object recognition rate for the image included in the object recognition information.

11. In a processing device that controls a device that transmits a control message in a wireless communication system, at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor, wherein said device causes: A processing device that acquires object recognition information from an artificial neural network-based object recognition model that recognizes and classifies objects from images of an image acquisition device, transmits an object message to a peripheral device based on object information recognized from the image included in the object recognition information, and determines whether to transmit the control message for controlling the image acquisition device based on an object recognition rate for the image included in the object recognition information.

12. In a method for an image acquisition device to receive a control message in a wireless communication system, A step of acquiring an image for a specific geographic area; A step of transferring the image to an artificial neural network-based object recognition model trained to recognize and classify objects from the image; and comprising the step of receiving a control message from a first device, the control message including at least one control parameter instructing a change of a parameter related to acquisition of the image; A method wherein the control message is received from the first device based on the object recognition rate included in the object recognition information of the object recognition model being less than a preset threshold.

13. A computer-readable recording medium recording a program for performing the method described in Article 12.

14. In an image acquisition device that receives a control message in a wireless communication system, An image sensor that acquires images; RF (Radio Frequency) transmitter and receiver; and A processor connected to the RF transceiver and the image sensor, The processor controls the image sensor to acquire an image for a specific geographical area, controls the RF transceiver to transmit the image to an artificial neural network-based object recognition model trained to recognize and classify an object from the image, and receives a control message from the first device including at least one control parameter that instructs a change in a parameter related to acquisition of the image. An image acquisition device, wherein the control message is received from the first device based on the object recognition rate included in the object recognition information of the object recognition model being less than a preset threshold.

15. In a processing device that controls an image acquisition device that receives a control message in a wireless communication system, at least one processor; and At least one memory connected to said at least one processor and storing instructions, said instructions being executed by said at least one processor to cause said image acquisition device to: Acquire an image for a specific geographical area, transmit the image to an artificial neural network-based object recognition model trained to recognize and classify objects from the image, and receive a control message from a first device including at least one control parameter that instructs a change in a parameter related to acquisition of the image; A processing device, wherein the control message is received from the first device based on the object recognition rate included in the object recognition information of the object recognition model being less than a preset threshold.

Citation Information

Patent Citations

  • Composition for treating Sarcopenia including alginate oligosaccharide as an active ingredient

    KR1020200123009A

  • Culture media for in vitro maturation of oocyte containing halibut egg and in vitro embryo production using there of

    KR1020220001455A

  • System for analyzing pot hole data of road pavement using AI and for the same

    KR102094341B1

  • Method and system for object recognition based on environment variation data learning

    KR102511315B1

  • KR20190098091A